From ericsnowcurrently at gmail.com  Tue Apr  1 00:37:32 2014
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Mon, 31 Mar 2014 16:37:32 -0600
Subject: [Python-Dev] Treating tokenize.Untokenizer as private
In-Reply-To: <lhcglp$t2k$1@ger.gmane.org>
References: <le0i6q$soa$1@ger.gmane.org>
 <CALFfu7Ai0NhNFYwckP=FehYASWFocBgi8mHfqzygs3+73aFt4A@mail.gmail.com>
 <lhcglp$t2k$1@ger.gmane.org>
Message-ID: <CALFfu7A8j_0qKHCre62QYomaMNUB1hmNviDG1qNcfE_5TtNxKQ@mail.gmail.com>

On Mon, Mar 31, 2014 at 1:45 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 3/31/2014 2:30 PM, Eric Snow wrote:
>> Is this still an open question, Terry?
>
>
> It is not currently for #9974 because after consideration of two proposed
> patches (one part of another issue), I decided that the conditions being
> guarded against either could not occur or could be checked in the caller, so
> that no api change was needed.
>
> However, I believe the fix for another bug,
> http://bugs.python.org/issue20387
> will require copying the code that correctly formats indents in the old
> compat method to the newer method, which incorrectly assumes that indents
> are spaces only.  I might end up wishing I could refactor the code. So this
> may well become a live issue again, and I would still like to know what
> people think.

It simply depends on the utility of customizing the default behavior
there.  I seem to remember subclassing Untokenize for something in a
personal project, but I expect doing so wasn't necessary.

>
> How do we do code searches (as for use of "Untokenize") these days?

I believe there was one for which Guido was advocating, but I don't
recall it's name.  Such a search would be useful though.

-eric

From ncoghlan at gmail.com  Tue Apr  1 00:51:35 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 1 Apr 2014 08:51:35 +1000
Subject: [Python-Dev] libpython added to ABI tracker
In-Reply-To: <CAMpsgwZo7kYDT0-DZy+yKpac0E=iKikhz-MHjqP-8H_Qq55bpA@mail.gmail.com>
References: <5335337F.8000303@rosalab.ru>
 <CAMpsgwYf-KfY4+6=nLxm52L_NYcq_Tj3T9tAdJSmtf2K3GFL=A@mail.gmail.com>
 <533953B9.2040506@rosalab.ru>
 <CAMpsgwZo7kYDT0-DZy+yKpac0E=iKikhz-MHjqP-8H_Qq55bpA@mail.gmail.com>
Message-ID: <CADiSq7fChg8wXgoiqMab=tL7ipO9y8ODeh0+o4zaVKBVSk2n0g@mail.gmail.com>

On 1 Apr 2014 01:38, "Victor Stinner" <victor.stinner at gmail.com> wrote:
>
> 2014-03-31 13:38 GMT+02:00 Andrey Ponomarenko <aponomarenko at rosalab.ru>:
> > The public libpython API changes will be tracked here:
> > http://upstream-tracker.org/versions/python_public_api.html
> >
> > For now I've excluded only symbols starting with an underscore. What
other
> > symbols should be excluded?
>
> It's not a matter of underscore. You should define Py_LIMITED_API to
> 0x03020000 to test the stable ABI of Python 3.2.
>
> http://docs.python.org/dev/c-api/stable.html

Well, we have more than one ABI, with different guarantees. The "no leading
underscore" one we promise not to change in maintenance releases, but we
only preserve *API* compatibility in feature releases (mostly due to
structs changing size).

The "stable ABI" (aka Py_LIMITED_API) is the one where we promise to hide
all the memory layout details and treat it as "additive only" so that
binaries built with previous releases keep working. That should never break
ABI compatibility, and only get new additions if the macro definition is
bumped up to match the newer release.

Cheers,
Nick.

P.S. I understand it was Anatoly that put the process in motion to get this
set up. Thanks for doing that Anatoly, it's a genuinely good idea.

>
> Victor
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140401/15d7fa67/attachment.html>

From Nikolaus at rath.org  Tue Apr  1 02:15:35 2014
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Mon, 31 Mar 2014 17:15:35 -0700
Subject: [Python-Dev] Adding a readinto1 method to BufferedReader
Message-ID: <871txhucpk.fsf@vostro.rath.org>

Hello,

The BufferedReader (and BufferedRWPair) classes both have a read1()
method in addition to the regular read() method to bypass the internal
buffer. This is quite useful if you need to do some buffered reading
(e.g. to parse a header) followed by a lot of bulk data that you want to
process as it streams in.

However, the readinto() method does not have a corresponding readinto1()
method. I would like to add this method. I have proposed a patch in
http://bugs.python.org/issue20578.

Are there any comments, objections, encouragements...?


Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             ?Time flies like an arrow, fruit flies like a Banana.?

From ncoghlan at gmail.com  Tue Apr  1 13:45:08 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 1 Apr 2014 21:45:08 +1000
Subject: [Python-Dev] libpython added to ABI tracker
In-Reply-To: <533AA1A0.9020505@rosalab.ru>
References: <5335337F.8000303@rosalab.ru>
 <CAMpsgwYf-KfY4+6=nLxm52L_NYcq_Tj3T9tAdJSmtf2K3GFL=A@mail.gmail.com>
 <533953B9.2040506@rosalab.ru>
 <CAMpsgwZo7kYDT0-DZy+yKpac0E=iKikhz-MHjqP-8H_Qq55bpA@mail.gmail.com>
 <CADiSq7fChg8wXgoiqMab=tL7ipO9y8ODeh0+o4zaVKBVSk2n0g@mail.gmail.com>
 <533AA1A0.9020505@rosalab.ru>
Message-ID: <CADiSq7egsPKkq9qxG+kX4wfFvGz9-VVEvKXRzntk5VL9oh+F_w@mail.gmail.com>

On 1 April 2014 21:23, Andrey Ponomarenko <aponomarenko at rosalab.ru> wrote:
>
> Nick Coghlan wrote:
>>
>> On 1 Apr 2014 01:38, "Victor Stinner" <victor.stinner at gmail.com> wrote:
>>>
>>> 2014-03-31 13:38 GMT+02:00 Andrey Ponomarenko <aponomarenko at rosalab.ru>:
>>>>
>>>> The public libpython API changes will be tracked here:
>>>> http://upstream-tracker.org/versions/python_public_api.html
>>>>
>>>> For now I've excluded only symbols starting with an underscore. What
>>
>> other
>>>>
>>>> symbols should be excluded?
>>>
>>> It's not a matter of underscore. You should define Py_LIMITED_API to
>>> 0x03020000 to test the stable ABI of Python 3.2.
>>>
>>> http://docs.python.org/dev/c-api/stable.html
>>
>> Well, we have more than one ABI, with different guarantees. The "no
>> leading
>> underscore" one we promise not to change in maintenance releases, but we
>> only preserve *API* compatibility in feature releases (mostly due to
>> structs changing size).
>>
>> The "stable ABI" (aka Py_LIMITED_API) is the one where we promise to hide
>> all the memory layout details and treat it as "additive only" so that
>> binaries built with previous releases keep working. That should never
>> break
>> ABI compatibility, and only get new additions if the macro definition is
>> bumped up to match the newer release.
>>
>> Cheers,
>> Nick.
>
>
> The stable libpython ABI with Py_LIMITED_API=0x03020000 will be tracked at
> http://upstream-tracker.org/versions/python_stable_api.html

Thanks! The "leading underscore means private" convention is also
applicable here (that's a general guideline for Python related APIs).

Interesting to see the UCS2 removal there for 3.3. That's a genuine
removal from the public ABI as part of PEP 393. I guess the reason
nobody complained is because most 3.2 Linux builds used the UCS4 ABI
instead, and the stable ABI hadn't seen broad adoption on Windows in
the 3.2->3.3 time frame.

Regarding the warnings for this one - is there a way for the checker
to warn if data structures are exposed directly, rather than as opaque
types? It's fine if there isn't, it would just be cool if there was -
one of the premises of the stable ABI is that it *doesn't* expose the
type definitions directly to consuming code, just the pointers to them
(hence allowing the struct size to change without actually breaking
compatibility with the defined ABI).

Regardless, this service already shows we've made some mistakes with
the stable ABI in previous releases - it is indicating there are new
symbols in the stable ABI for 3.3 and 3.4 that aren't properly guarded
with version constraints. That means it is currently possible to set
Py_LIMITED_API=0x03020000 and get something that won't actually run
properly on 3.2.

Georg, Larry, Benjamin - should checking these be added to PEP 101, so
we don't get the same thing happening for 3.5?

> I also added source-compatibility reports to the "public" API tracker:
> http://upstream-tracker.org/versions/python_public_api.html

Thanks again for setting these up!

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From benjamin at python.org  Tue Apr  1 16:54:01 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 01 Apr 2014 07:54:01 -0700
Subject: [Python-Dev] libpython added to ABI tracker
In-Reply-To: <CADiSq7egsPKkq9qxG+kX4wfFvGz9-VVEvKXRzntk5VL9oh+F_w@mail.gmail.com>
References: <5335337F.8000303@rosalab.ru>
 <CAMpsgwYf-KfY4+6=nLxm52L_NYcq_Tj3T9tAdJSmtf2K3GFL=A@mail.gmail.com>
 <533953B9.2040506@rosalab.ru>
 <CAMpsgwZo7kYDT0-DZy+yKpac0E=iKikhz-MHjqP-8H_Qq55bpA@mail.gmail.com>
 <CADiSq7fChg8wXgoiqMab=tL7ipO9y8ODeh0+o4zaVKBVSk2n0g@mail.gmail.com>
 <533AA1A0.9020505@rosalab.ru>
 <CADiSq7egsPKkq9qxG+kX4wfFvGz9-VVEvKXRzntk5VL9oh+F_w@mail.gmail.com>
Message-ID: <1396364041.26514.101452685.3DAB706F@webmail.messagingengine.com>

On Tue, Apr 1, 2014, at 4:45, Nick Coghlan wrote:
> Georg, Larry, Benjamin - should checking these be added to PEP 101, so
> we don't get the same thing happening for 3.5?

I would like it if we could put this in the testsuite somehow.

From aponomarenko at rosalab.ru  Tue Apr  1 13:23:12 2014
From: aponomarenko at rosalab.ru (Andrey Ponomarenko)
Date: Tue, 01 Apr 2014 15:23:12 +0400
Subject: [Python-Dev] libpython added to ABI tracker
In-Reply-To: <CADiSq7fChg8wXgoiqMab=tL7ipO9y8ODeh0+o4zaVKBVSk2n0g@mail.gmail.com>
References: <5335337F.8000303@rosalab.ru>
 <CAMpsgwYf-KfY4+6=nLxm52L_NYcq_Tj3T9tAdJSmtf2K3GFL=A@mail.gmail.com>
 <533953B9.2040506@rosalab.ru>
 <CAMpsgwZo7kYDT0-DZy+yKpac0E=iKikhz-MHjqP-8H_Qq55bpA@mail.gmail.com>
 <CADiSq7fChg8wXgoiqMab=tL7ipO9y8ODeh0+o4zaVKBVSk2n0g@mail.gmail.com>
Message-ID: <533AA1A0.9020505@rosalab.ru>


Nick Coghlan wrote:
> On 1 Apr 2014 01:38, "Victor Stinner" <victor.stinner at gmail.com> wrote:
>> 2014-03-31 13:38 GMT+02:00 Andrey Ponomarenko <aponomarenko at rosalab.ru>:
>>> The public libpython API changes will be tracked here:
>>> http://upstream-tracker.org/versions/python_public_api.html
>>>
>>> For now I've excluded only symbols starting with an underscore. What
> other
>>> symbols should be excluded?
>> It's not a matter of underscore. You should define Py_LIMITED_API to
>> 0x03020000 to test the stable ABI of Python 3.2.
>>
>> http://docs.python.org/dev/c-api/stable.html
> Well, we have more than one ABI, with different guarantees. The "no leading
> underscore" one we promise not to change in maintenance releases, but we
> only preserve *API* compatibility in feature releases (mostly due to
> structs changing size).
>
> The "stable ABI" (aka Py_LIMITED_API) is the one where we promise to hide
> all the memory layout details and treat it as "additive only" so that
> binaries built with previous releases keep working. That should never break
> ABI compatibility, and only get new additions if the macro definition is
> bumped up to match the newer release.
>
> Cheers,
> Nick.
>
> P.S. I understand it was Anatoly that put the process in motion to get this
> set up. Thanks for doing that Anatoly, it's a genuinely good idea.

The stable libpython ABI with Py_LIMITED_API=0x03020000 will be tracked 
at http://upstream-tracker.org/versions/python_stable_api.html

I also added source-compatibility reports to the "public" API tracker: 
http://upstream-tracker.org/versions/python_public_api.html

Thanks.

-- 
Andrey Ponomarenko, NTC IT ROSA.


From zachary.ware+pydev at gmail.com  Tue Apr  1 20:28:53 2014
From: zachary.ware+pydev at gmail.com (Zachary Ware)
Date: Tue, 1 Apr 2014 13:28:53 -0500
Subject: [Python-Dev] [Python-checkins] cpython (3.4): simplify check,
 since now there are only new-style classes
In-Reply-To: <3fyzPf5WxCz7LjT@mail.python.org>
References: <3fyzPf5WxCz7LjT@mail.python.org>
Message-ID: <CAKJDb-OPFE6Mu0z-kBJ0KTmCrk-q-k8kt1ub6kPHC6pGbp-Nrg@mail.gmail.com>

On Tue, Apr 1, 2014 at 1:21 PM, benjamin.peterson
<python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/abb85902ce79
> changeset:   90090:abb85902ce79
> branch:      3.4
> parent:      90088:4a2dabac976d
> user:        Benjamin Peterson <benjamin at python.org>
> date:        Tue Apr 01 14:20:56 2014 -0400
> summary:
>   simplify check, since now there are only new-style classes
>
> files:
>   Lib/urllib/request.py |  7 ++-----
>   1 files changed, 2 insertions(+), 5 deletions(-)
>
>
> diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py
> --- a/Lib/urllib/request.py
> +++ b/Lib/urllib/request.py
> @@ -511,9 +511,6 @@
>      If any of the handlers passed as arguments are subclasses of the
>      default handlers, the default handlers will not be used.
>      """
> -    def isclass(obj):
> -        return isinstance(obj, type) or hasattr(obj, "__bases__")
> -
>      opener = OpenerDirector()
>      default_classes = [ProxyHandler, UnknownHandler, HTTPHandler,
>                         HTTPDefaultErrorHandler, HTTPRedirectHandler,
> @@ -524,7 +521,7 @@
>      skip = set()
>      for klass in default_classes:
>          for check in handlers:
> -            if isclass(check):
> +            if instance(check, type):

Should be isinstance, should it not?

-- 
Zach

From cf.natali at gmail.com  Tue Apr  1 20:29:38 2014
From: cf.natali at gmail.com (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=)
Date: Tue, 1 Apr 2014 19:29:38 +0100
Subject: [Python-Dev] pickle self-delimiting
Message-ID: <CAH_1eM1QbAeqjiUekoLepb2i_CuHkcLHfwRYyrUV-cv+boqO9g@mail.gmail.com>

Hi,

Unless I'm mistaken, pickle's documentation doesn't mention that the pickle
wire-format is self-delimiting. Is there any reason why it's not documented?

The reason I'm asking is because I've seen some code out there doing its
own ad-hoc length-prefix framing.

Cheers,

cf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140401/c8da25e3/attachment.html>

From solipsis at pitrou.net  Tue Apr  1 20:35:41 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 1 Apr 2014 20:35:41 +0200
Subject: [Python-Dev] pickle self-delimiting
References: <CAH_1eM1QbAeqjiUekoLepb2i_CuHkcLHfwRYyrUV-cv+boqO9g@mail.gmail.com>
Message-ID: <20140401203541.4999968c@fsol>

On Tue, 1 Apr 2014 19:29:38 +0100
Charles-Fran?ois Natali <cf.natali at gmail.com> wrote:
> Hi,
> 
> Unless I'm mistaken, pickle's documentation doesn't mention that the pickle
> wire-format is self-delimiting. Is there any reason why it's not documented?

No reason AFAIK. However, the fact that it is self-delimited is implicit
in the fact that "Bytes past the pickled object?s representation are
ignored": https://docs.python.org/dev/library/pickle.html#pickle.load

Also, note that protocol 4 now features a length-prefix framing to
improve buffering performance with arbitrary streams.

Regards

Antoine.



From solipsis at pitrou.net  Tue Apr  1 20:36:24 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 1 Apr 2014 20:36:24 +0200
Subject: [Python-Dev] cpython (3.4): simplify check,
 since now there are only new-style classes
References: <3fyzPf5WxCz7LjT@mail.python.org>
 <CAKJDb-OPFE6Mu0z-kBJ0KTmCrk-q-k8kt1ub6kPHC6pGbp-Nrg@mail.gmail.com>
Message-ID: <20140401203624.0860bb40@fsol>

On Tue, 1 Apr 2014 13:28:53 -0500
Zachary Ware <zachary.ware+pydev at gmail.com> wrote:
> > @@ -524,7 +521,7 @@
> >      skip = set()
> >      for klass in default_classes:
> >          for check in handlers:
> > -            if isclass(check):
> > +            if instance(check, type):
> 
> Should be isinstance, should it not?

Sounds like a well-tested code path :-)

Regards

Antoine.



From ncoghlan at gmail.com  Tue Apr  1 22:59:24 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 2 Apr 2014 06:59:24 +1000
Subject: [Python-Dev] libpython added to ABI tracker
In-Reply-To: <1396364041.26514.101452685.3DAB706F@webmail.messagingengine.com>
References: <5335337F.8000303@rosalab.ru>
 <CAMpsgwYf-KfY4+6=nLxm52L_NYcq_Tj3T9tAdJSmtf2K3GFL=A@mail.gmail.com>
 <533953B9.2040506@rosalab.ru>
 <CAMpsgwZo7kYDT0-DZy+yKpac0E=iKikhz-MHjqP-8H_Qq55bpA@mail.gmail.com>
 <CADiSq7fChg8wXgoiqMab=tL7ipO9y8ODeh0+o4zaVKBVSk2n0g@mail.gmail.com>
 <533AA1A0.9020505@rosalab.ru>
 <CADiSq7egsPKkq9qxG+kX4wfFvGz9-VVEvKXRzntk5VL9oh+F_w@mail.gmail.com>
 <1396364041.26514.101452685.3DAB706F@webmail.messagingengine.com>
Message-ID: <CADiSq7c8DcfV71O-nj5HeN4QqKNAz4qkbAi=8SiXRkwff7=73A@mail.gmail.com>

On 2 Apr 2014 00:54, "Benjamin Peterson" <benjamin at python.org> wrote:
>
> On Tue, Apr 1, 2014, at 4:45, Nick Coghlan wrote:
> > Georg, Larry, Benjamin - should checking these be added to PEP 101, so
> > we don't get the same thing happening for 3.5?
>
> I would like it if we could put this in the testsuite somehow.

Perhaps an independent automated daily or weekly check like Antoine's daily
refleak hunter would be better?

Cheers,
Nick.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140402/392d41de/attachment.html>

From benjamin at python.org  Tue Apr  1 23:31:33 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 01 Apr 2014 14:31:33 -0700
Subject: [Python-Dev] libpython added to ABI tracker
In-Reply-To: <CADiSq7c8DcfV71O-nj5HeN4QqKNAz4qkbAi=8SiXRkwff7=73A@mail.gmail.com>
References: <5335337F.8000303@rosalab.ru>
 <CAMpsgwYf-KfY4+6=nLxm52L_NYcq_Tj3T9tAdJSmtf2K3GFL=A@mail.gmail.com>
 <533953B9.2040506@rosalab.ru>
 <CAMpsgwZo7kYDT0-DZy+yKpac0E=iKikhz-MHjqP-8H_Qq55bpA@mail.gmail.com>
 <CADiSq7fChg8wXgoiqMab=tL7ipO9y8ODeh0+o4zaVKBVSk2n0g@mail.gmail.com>
 <533AA1A0.9020505@rosalab.ru>
 <CADiSq7egsPKkq9qxG+kX4wfFvGz9-VVEvKXRzntk5VL9oh+F_w@mail.gmail.com>
 <1396364041.26514.101452685.3DAB706F@webmail.messagingengine.com>
 <CADiSq7c8DcfV71O-nj5HeN4QqKNAz4qkbAi=8SiXRkwff7=73A@mail.gmail.com>
Message-ID: <1396387893.4194.101611537.57F058EC@webmail.messagingengine.com>



On Tue, Apr 1, 2014, at 13:59, Nick Coghlan wrote:
> On 2 Apr 2014 00:54, "Benjamin Peterson" <benjamin at python.org> wrote:
> >
> > On Tue, Apr 1, 2014, at 4:45, Nick Coghlan wrote:
> > > Georg, Larry, Benjamin - should checking these be added to PEP 101, so
> > > we don't get the same thing happening for 3.5?
> >
> > I would like it if we could put this in the testsuite somehow.
> 
> Perhaps an independent automated daily or weekly check like Antoine's
> daily
> refleak hunter would be better?

As long as it fails loudly.

From cf.natali at gmail.com  Tue Apr  1 23:56:20 2014
From: cf.natali at gmail.com (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=)
Date: Tue, 1 Apr 2014 22:56:20 +0100
Subject: [Python-Dev] pickle self-delimiting
In-Reply-To: <20140401203541.4999968c@fsol>
References: <CAH_1eM1QbAeqjiUekoLepb2i_CuHkcLHfwRYyrUV-cv+boqO9g@mail.gmail.com>
 <20140401203541.4999968c@fsol>
Message-ID: <CAH_1eM12caKkhpBRQY5X2e=R_qXxawtEZUQWPHBwc2euDGA3eA@mail.gmail.com>

> No reason AFAIK. However, the fact that it is self-delimited is implicit
> in the fact that "Bytes past the pickled object's representation are
> ignored": https://docs.python.org/dev/library/pickle.html#pickle.load

I find this sentence worrying: it could lead one to think that load() could
read more bytes than the expected object representation size: this would
make pickle actually non self-delimiting, and could lead to problems when
reading e.g. from a socket, since an extraneous read() could block.

I think it's worth making it clear in the doc, I'll open an issue on the
tracker.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140401/0d429978/attachment.html>

From mal at egenix.com  Wed Apr  2 14:52:01 2014
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 02 Apr 2014 14:52:01 +0200
Subject: [Python-Dev] Negative timedelta strings
In-Reply-To: <CALGmxEJzym2R-fo=qrWy9+fzmd1p1T1BkV4NgYokxp8J69uU7g@mail.gmail.com>
References: <20140328110342.GI16526@ando>	<CAP7h-xa4p7eoDsyh_iaTqQoYQyu-Ab_kgj2Yoz+QS2ZT-K6-bQ@mail.gmail.com>	<CANc-5UxAksGP4ddbS=apwyqdLwgTvs30cxuuibzHCNb+k_Zg=Q@mail.gmail.com>	<CAP7h-xaA91vWp3kxnpYs0e99T+x77zUTpwDj5KL7wtBXDcjQvw@mail.gmail.com>	<5335E754.4010305@canterbury.ac.nz>	<CAFT4OTGSLtvrCRNthXCPsb7w7NvDXGUGzOU4C9zVAfGEyWfrCA@mail.gmail.com>
 <CALGmxEJzym2R-fo=qrWy9+fzmd1p1T1BkV4NgYokxp8J69uU7g@mail.gmail.com>
Message-ID: <533C07F1.2060302@egenix.com>

n 31.03.2014 21:09, Chris Barker wrote:
> On Fri, Mar 28, 2014 at 2:52 PM, Fred Drake <fred at fdrake.net> wrote:
> 
>> On Fri, Mar 28, 2014 at 5:19 PM, Greg Ewing <greg.ewing at canterbury.ac.nz>
>> wrote:
>>> ISO 8601 doesn't seem to define a representation for
>>> negative durations, though, so it wouldn't solve the
>>> original problem.
>>
>> Aside from the horribleness of the ISO 8601 notation for a duration, it's
>> best not to confuse the notions of duration and delta.  Notionally, a delta
>> contains more information than a duration.
> 
> 
> and less -- really it's different.
> 
> A duration would be really useful actually, for things like "next month",
> etc,. IIRC, mxDateTime has something for this, but it's NOT the same as a
> timedelta.

mxDateTime has DateTimeDelta objects which represent a time delta
(in the mathematical sense) between two points in DateTime and
RelativeDateTime objects which allow defining deltas in terms
of qualifiers which are applied to the left hand side of an operation.

With RelativeDateTime you can do things like e.g.

first_of_next_month = now() + RelativeDateTime(months=+1, day=1)

There are some other concepts you can emulate with these, like e.g.
a specific time frame (DateTime + one of the above deltas), a
reoccurring time (start_time + one of the deltas + number
occurrences + exceptions), an age concept (difference between two
DateTime values expressed in RelativeDateTime terms), etc.

Some examples:

>>> from mx.DateTime import *

>>> print RelativeDateTime(months=+1, day=1)
YYYY-(+01)-01 HH:MM:SS

>>> print now() + RelativeDateTime(months=+1, day=1)
2014-05-01 14:49:05.83

>>> print Age(now(), Date(1969,4,6))
(+0044)-(+11)-(+27) (+14):(+49):(+02)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 02 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2014-04-09: PyCon 2014, Montreal, Canada ...                7 days to go
2014-04-29: Python Meeting Duesseldorf ...                 27 days to go

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From skip at pobox.com  Wed Apr  2 15:04:51 2014
From: skip at pobox.com (Skip Montanaro)
Date: Wed, 2 Apr 2014 08:04:51 -0500
Subject: [Python-Dev] Negative timedelta strings
In-Reply-To: <533C07F1.2060302@egenix.com>
References: <20140328110342.GI16526@ando>
 <CAP7h-xa4p7eoDsyh_iaTqQoYQyu-Ab_kgj2Yoz+QS2ZT-K6-bQ@mail.gmail.com>
 <CANc-5UxAksGP4ddbS=apwyqdLwgTvs30cxuuibzHCNb+k_Zg=Q@mail.gmail.com>
 <CAP7h-xaA91vWp3kxnpYs0e99T+x77zUTpwDj5KL7wtBXDcjQvw@mail.gmail.com>
 <5335E754.4010305@canterbury.ac.nz>
 <CAFT4OTGSLtvrCRNthXCPsb7w7NvDXGUGzOU4C9zVAfGEyWfrCA@mail.gmail.com>
 <CALGmxEJzym2R-fo=qrWy9+fzmd1p1T1BkV4NgYokxp8J69uU7g@mail.gmail.com>
 <533C07F1.2060302@egenix.com>
Message-ID: <CANc-5Ux9CtdS5vctuOtSRupME72gv1__-jSwnuEYpOJ+7UTR-Q@mail.gmail.com>

On Wed, Apr 2, 2014 at 7:52 AM, M.-A. Lemburg <mal at egenix.com> wrote:
> >>> print now() + RelativeDateTime(months=+1, day=1)
> 2014-05-01 14:49:05.83

I find this sort date arithmetic unintuitive, though I'm at a loss to
come up with better logic than you have:

>>> d = Date(2014, 2, 28)
>>> d + RelativeDateTime(months=+1)
<mx.DateTime.DateTime object for '2014-03-28 00:00:00.00' at 1eda8c8>
>>> d = Date(2014, 1, 31)
>>> d + RelativeDateTime(months=+1)
<mx.DateTime.DateTime object for '2014-03-03 00:00:00.00' at 1eda870>

I guess the assumption is that one month is the length in days of the
current month, though, you wind up with situations where shorter
months can be skipped altogether. Is there a way to talk in terms of
"months" but not have short months get skipped?

Thx,

Skip

From python-dev at masklinn.net  Wed Apr  2 15:12:14 2014
From: python-dev at masklinn.net (Xavier Morel)
Date: Wed, 2 Apr 2014 15:12:14 +0200
Subject: [Python-Dev] Negative timedelta strings
In-Reply-To: <CANc-5Ux9CtdS5vctuOtSRupME72gv1__-jSwnuEYpOJ+7UTR-Q@mail.gmail.com>
References: <20140328110342.GI16526@ando>
 <CAP7h-xa4p7eoDsyh_iaTqQoYQyu-Ab_kgj2Yoz+QS2ZT-K6-bQ@mail.gmail.com>
 <CANc-5UxAksGP4ddbS=apwyqdLwgTvs30cxuuibzHCNb+k_Zg=Q@mail.gmail.com>
 <CAP7h-xaA91vWp3kxnpYs0e99T+x77zUTpwDj5KL7wtBXDcjQvw@mail.gmail.com>
 <5335E754.4010305@canterbury.ac.nz>
 <CAFT4OTGSLtvrCRNthXCPsb7w7NvDXGUGzOU4C9zVAfGEyWfrCA@mail.gmail.com>
 <CALGmxEJzym2R-fo=qrWy9+fzmd1p1T1BkV4NgYokxp8J69uU7g@mail.gmail.com>
 <533C07F1.2060302@egenix.com>
 <CANc-5Ux9CtdS5vctuOtSRupME72gv1__-jSwnuEYpOJ+7UTR-Q@mail.gmail.com>
Message-ID: <A9F769BC-938D-45AE-84F0-980B90FC118B@masklinn.net>


On 2014-04-02, at 15:04 , Skip Montanaro <skip at pobox.com> wrote:

> On Wed, Apr 2, 2014 at 7:52 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>>>>> print now() + RelativeDateTime(months=+1, day=1)
>> 2014-05-01 14:49:05.83
> 
> I find this sort date arithmetic unintuitive, though I'm at a loss to
> come up with better logic than you have:
> 
>>>> d = Date(2014, 2, 28)
>>>> d + RelativeDateTime(months=+1)
> <mx.DateTime.DateTime object for '2014-03-28 00:00:00.00' at 1eda8c8>
>>>> d = Date(2014, 1, 31)
>>>> d + RelativeDateTime(months=+1)
> <mx.DateTime.DateTime object for '2014-03-03 00:00:00.00' at 1eda870>
> 
> I guess the assumption is that one month is the length in days of the
> current month, though, you wind up with situations where shorter
> months can be skipped altogether. Is there a way to talk in terms of
> "months" but not have short months get skipped?

FWIW dateutil has a slightly different logic there:

>>> date(2014, 2, 28) + relativedelta(months=+1)
datetime.date(2014, 3, 28)
>>> date(2014, 1, 31) + relativedelta(months=+1)
datetime.date(2014, 2, 28)


From mal at egenix.com  Wed Apr  2 15:18:23 2014
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 02 Apr 2014 15:18:23 +0200
Subject: [Python-Dev] Negative timedelta strings
In-Reply-To: <CANc-5Ux9CtdS5vctuOtSRupME72gv1__-jSwnuEYpOJ+7UTR-Q@mail.gmail.com>
References: <20140328110342.GI16526@ando>	<CAP7h-xa4p7eoDsyh_iaTqQoYQyu-Ab_kgj2Yoz+QS2ZT-K6-bQ@mail.gmail.com>	<CANc-5UxAksGP4ddbS=apwyqdLwgTvs30cxuuibzHCNb+k_Zg=Q@mail.gmail.com>	<CAP7h-xaA91vWp3kxnpYs0e99T+x77zUTpwDj5KL7wtBXDcjQvw@mail.gmail.com>	<5335E754.4010305@canterbury.ac.nz>	<CAFT4OTGSLtvrCRNthXCPsb7w7NvDXGUGzOU4C9zVAfGEyWfrCA@mail.gmail.com>	<CALGmxEJzym2R-fo=qrWy9+fzmd1p1T1BkV4NgYokxp8J69uU7g@mail.gmail.com>	<533C07F1.2060302@egenix.com>
 <CANc-5Ux9CtdS5vctuOtSRupME72gv1__-jSwnuEYpOJ+7UTR-Q@mail.gmail.com>
Message-ID: <533C0E1F.5070905@egenix.com>

On 02.04.2014 15:04, Skip Montanaro wrote:
> On Wed, Apr 2, 2014 at 7:52 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>>>>> print now() + RelativeDateTime(months=+1, day=1)
>> 2014-05-01 14:49:05.83
> 
> I find this sort date arithmetic unintuitive, though I'm at a loss to
> come up with better logic than you have:
> 
>>>> d = Date(2014, 2, 28)
>>>> d + RelativeDateTime(months=+1)
> <mx.DateTime.DateTime object for '2014-03-28 00:00:00.00' at 1eda8c8>
>>>> d = Date(2014, 1, 31)
>>>> d + RelativeDateTime(months=+1)
> <mx.DateTime.DateTime object for '2014-03-03 00:00:00.00' at 1eda870>
> 
> I guess the assumption is that one month is the length in days of the
> current month, though, you wind up with situations where shorter
> months can be skipped altogether. Is there a way to talk in terms of
> "months" but not have short months get skipped?

I'm not really satisfied with this solution either.

The problem is that the qualifier "+ 1 month" is not defined for
dates with don't have a corresponding day in the following month.

This should probably either raise an exception or use some
parameter to "solve" the problem, by e.g. choosing the last day
of the month or using the current scheme:

mxDateTime skips to the first of the next month and then advances
the number of days defined in the left hand DateTime value,
i.e. Date(2014, 2, 1) + 30 * oneDay.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 02 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2014-04-09: PyCon 2014, Montreal, Canada ...                7 days to go
2014-04-29: Python Meeting Duesseldorf ...                 27 days to go

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From aponomarenko at rosalab.ru  Wed Apr  2 16:18:53 2014
From: aponomarenko at rosalab.ru (Andrey Ponomarenko)
Date: Wed, 02 Apr 2014 18:18:53 +0400
Subject: [Python-Dev] libpython added to ABI tracker
In-Reply-To: <CADiSq7egsPKkq9qxG+kX4wfFvGz9-VVEvKXRzntk5VL9oh+F_w@mail.gmail.com>
References: <5335337F.8000303@rosalab.ru>
 <CAMpsgwYf-KfY4+6=nLxm52L_NYcq_Tj3T9tAdJSmtf2K3GFL=A@mail.gmail.com>
 <533953B9.2040506@rosalab.ru>
 <CAMpsgwZo7kYDT0-DZy+yKpac0E=iKikhz-MHjqP-8H_Qq55bpA@mail.gmail.com>
 <CADiSq7fChg8wXgoiqMab=tL7ipO9y8ODeh0+o4zaVKBVSk2n0g@mail.gmail.com>
 <533AA1A0.9020505@rosalab.ru>
 <CADiSq7egsPKkq9qxG+kX4wfFvGz9-VVEvKXRzntk5VL9oh+F_w@mail.gmail.com>
Message-ID: <533C1C4D.507@rosalab.ru>


Nick Coghlan wrote:
> Regarding the warnings for this one - is there a way for the checker
> to warn if data structures are exposed directly, rather than as opaque
> types? It's fine if there isn't, it would just be cool if there was -
> one of the premises of the stable ABI is that it *doesn't* expose the
> type definitions directly to consuming code, just the pointers to them
> (hence allowing the struct size to change without actually breaking
> compatibility with the defined ABI).

The ABI of the library can be dumped by the abi-compliance-checker basic 
tool to a text file in the human readable format, so anyone can analyse 
it in order to find problems of any kind.

Homepage of the tool: https://github.com/lvc/abi-compliance-checker

To dump the libpython ABI type:

$> abi-compliance-checker -l libpython -dump descriptor.xml

The descriptor.xml input file ({RELPATH} - path to the python install 
tree, i.e. installation "prefix"):

<version>
     3.4.0
</version>

<headers>
     {RELPATH}/include
</headers>

<libs>
     {RELPATH}/lib/libpython3.4m.so.1.0
</libs>

<skip_including>
     ast.h
     Python-ast.h
     asdl.h
     pyexpat.h
     pymactoolbox.h
</skip_including>

<gcc_options>
     -DPy_LIMITED_API=0x03020000
</gcc_options>

I've already created the sample dump of the libpython-3.4.0 stable ABI 
here: http://upstream-tracker.org/downloads/libpython_3.4.0.abi.tar.gz

In order to analyse data types in the ABI please read 'TypeInfo' section 
of the dump.

I see several structures with exposed definition in the stable ABI v3.4.0:

struct PyStructSequence_Desc
struct grammar
struct PyStructSequence_Field
struct _node
struct cchar_t
struct PyType_Spec
struct PyType_Slot
struct dfa
struct labellist
struct PyMethodDef
struct _Py_Identifier
struct state
struct PyVarObject
struct arc
struct label
struct PyModuleDef
struct PyModuleDef_Base
struct PyMemberDef
struct PyGetSetDef
struct _object
struct PyCursesWindowObject


Thanks.

-- 
Andrey Ponomarenko, NTC IT ROSA.


From martin at v.loewis.de  Wed Apr  2 22:55:57 2014
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 02 Apr 2014 22:55:57 +0200
Subject: [Python-Dev] libpython added to ABI tracker
In-Reply-To: <CADiSq7egsPKkq9qxG+kX4wfFvGz9-VVEvKXRzntk5VL9oh+F_w@mail.gmail.com>
References: <5335337F.8000303@rosalab.ru>
 <CAMpsgwYf-KfY4+6=nLxm52L_NYcq_Tj3T9tAdJSmtf2K3GFL=A@mail.gmail.com>
 <533953B9.2040506@rosalab.ru>
 <CAMpsgwZo7kYDT0-DZy+yKpac0E=iKikhz-MHjqP-8H_Qq55bpA@mail.gmail.com>
 <CADiSq7fChg8wXgoiqMab=tL7ipO9y8ODeh0+o4zaVKBVSk2n0g@mail.gmail.com>
 <533AA1A0.9020505@rosalab.ru>
 <CADiSq7egsPKkq9qxG+kX4wfFvGz9-VVEvKXRzntk5VL9oh+F_w@mail.gmail.com>
Message-ID: <533C795D.4080308@v.loewis.de>

Am 01.04.14 13:45, schrieb Nick Coghlan:
> Interesting to see the UCS2 removal there for 3.3. That's a genuine
> removal from the public ABI as part of PEP 393. I guess the reason
> nobody complained is because most 3.2 Linux builds used the UCS4 ABI
> instead, and the stable ABI hadn't seen broad adoption on Windows in
> the 3.2->3.3 time frame.

Not really. The intention was that the stable ABI wouldn't have any
UCS2/UCS4 denotation in the function names, see

http://hg.python.org/cpython/file/9186f4a18584/PC/python3.def

Functions that explicitly referred to Py_UNICODE were banned from
the ABI; functions that were mapped but shouldn't have been mapped
were meant to be unmapped.

However, it seems that this wasn't properly implemented, see

http://bugs.python.org/issue17432

> Regardless, this service already shows we've made some mistakes with
> the stable ABI in previous releases - it is indicating there are new
> symbols in the stable ABI for 3.3 and 3.4 that aren't properly guarded
> with version constraints. That means it is currently possible to set
> Py_LIMITED_API=0x03020000 and get something that won't actually run
> properly on 3.2.

Depends on the operating system. On Windows, the import library has
much less additions; anything declared in the header files that
is not in python3.def is a bug in the header files (by default).

Regards,
Martin


From tjreedy at udel.edu  Thu Apr  3 01:50:12 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 02 Apr 2014 19:50:12 -0400
Subject: [Python-Dev] Gmane not functioning.
Message-ID: <533CA234.30203@udel.edu>

Just to let those who read the list as a list (or via mail.python.org, 
as I just did as a backup) know, news.gmane.org appears to have 
stoppedreceiving new messages from mailing listsat about 0:30 this 
morning (apr 2). (I am judging this from the last recorded post on the 
super-busy linux kernallist 
http://news.gmane.org/gmane.linux.kernel/cutoff=1676398). So there will 
be some unknown number of people not reading and posting who might 
otherwise. Ditto for other lists.


-- Terry Jan Reedy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140402/66b43b4f/attachment.html>

From ncoghlan at gmail.com  Thu Apr  3 10:52:27 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 3 Apr 2014 18:52:27 +1000
Subject: [Python-Dev] libpython added to ABI tracker
In-Reply-To: <533C1C4D.507@rosalab.ru>
References: <5335337F.8000303@rosalab.ru>
 <CAMpsgwYf-KfY4+6=nLxm52L_NYcq_Tj3T9tAdJSmtf2K3GFL=A@mail.gmail.com>
 <533953B9.2040506@rosalab.ru>
 <CAMpsgwZo7kYDT0-DZy+yKpac0E=iKikhz-MHjqP-8H_Qq55bpA@mail.gmail.com>
 <CADiSq7fChg8wXgoiqMab=tL7ipO9y8ODeh0+o4zaVKBVSk2n0g@mail.gmail.com>
 <533AA1A0.9020505@rosalab.ru>
 <CADiSq7egsPKkq9qxG+kX4wfFvGz9-VVEvKXRzntk5VL9oh+F_w@mail.gmail.com>
 <533C1C4D.507@rosalab.ru>
Message-ID: <CADiSq7dCkHfNaHQvEerxu6wW8gx=z851cNd9F+zNyexKUjYh5A@mail.gmail.com>

On 3 April 2014 00:18, Andrey Ponomarenko <aponomarenko at rosalab.ru> wrote:
>
> Nick Coghlan wrote:
>>
>> Regarding the warnings for this one - is there a way for the checker
>> to warn if data structures are exposed directly, rather than as opaque
>> types? It's fine if there isn't, it would just be cool if there was -
>> one of the premises of the stable ABI is that it *doesn't* expose the
>> type definitions directly to consuming code, just the pointers to them
>> (hence allowing the struct size to change without actually breaking
>> compatibility with the defined ABI).
>
>
> The ABI of the library can be dumped by the abi-compliance-checker basic
> tool to a text file in the human readable format, so anyone can analyse it
> in order to find problems of any kind.

Thanks! I've now filed http://bugs.python.org/issue21142 as an RFE -
I'll definitely be looking into setting up a regular run of this to
help us avoid any new issues with managing the evolution of the stable
ABI (and the public ABI in general).

Cheers,
Nick.



-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From skip at pobox.com  Thu Apr  3 15:51:09 2014
From: skip at pobox.com (Skip Montanaro)
Date: Thu, 3 Apr 2014 08:51:09 -0500
Subject: [Python-Dev] Python 4?
Message-ID: <CANc-5UwY5EAAOhO741biNnmMJCQKsk5x1RpKiNkA7mB+X0PATQ@mail.gmail.com>

I saw mention recently of Python 4 and assumed all such references
were either April Fool's jokes or pie-in-the-sky dreams for a new
version of Python which may never arrive.

Then I saw this message to webmaster. I would have guessed he meant
3.4 except he mentions having a Python3 (well, actually, a Phython3)
directory.

WTH is Python 4?

Skip

---------- Forwarded message ----------
From: C. Reese <careese at vcu.org>
Date: Thu, Apr 3, 2014 at 8:03 AM
Subject: Updated phython
To: webmaster at python.org


just updated to Python 4.  (installed to C: drive) ... now i notice
there are separate Python2 & Python3 directories as well.  i am not a
programmer in any sense (just a basic computer user).

1. do i need to have python on my computer at all for basic use?
(word, excel, power point, etc.)'
2. can i safely remove the Python 2 & Phython3 directories? (to save
hard disk space)

thanks
CR

From brett at python.org  Thu Apr  3 15:59:55 2014
From: brett at python.org (Brett Cannon)
Date: Thu, 3 Apr 2014 09:59:55 -0400
Subject: [Python-Dev] Python 4?
In-Reply-To: <CANc-5UwY5EAAOhO741biNnmMJCQKsk5x1RpKiNkA7mB+X0PATQ@mail.gmail.com>
References: <CANc-5UwY5EAAOhO741biNnmMJCQKsk5x1RpKiNkA7mB+X0PATQ@mail.gmail.com>
Message-ID: <CAP1=2W5PvbrtFP0+f+Q490dLNKHp2p6K44WiXFwP+4ZvtkUsEg@mail.gmail.com>

On Thu, Apr 3, 2014 at 9:51 AM, Skip Montanaro <skip at pobox.com> wrote:

> I saw mention recently of Python 4 and assumed all such references
> were either April Fool's jokes or pie-in-the-sky dreams for a new
> version of Python which may never arrive.
>

It's the latter. Some of us have been tossing around "Python 4" like we did
"Py3k" before we starting Python 3 in earnest; maybe we should make sure to
say Py4k to be perfectly clear there are no Python 4 plans ATM.


>
> Then I saw this message to webmaster. I would have guessed he meant
> 3.4 except he mentions having a Python3 (well, actually, a Phython3)
> directory.
>
> WTH is Python 4?
>

Who the heck knows what the person specifically means, although it sounds
like he did mean Python 3.4 which would explain why he know has a Python3
directory.

-Brett


> Skip
>
> ---------- Forwarded message ----------
> From: C. Reese <careese at vcu.org>
> Date: Thu, Apr 3, 2014 at 8:03 AM
> Subject: Updated phython
> To: webmaster at python.org
>
>
> just updated to Python 4.  (installed to C: drive) ... now i notice
> there are separate Python2 & Python3 directories as well.  i am not a
> programmer in any sense (just a basic computer user).
>
> 1. do i need to have python on my computer at all for basic use?
> (word, excel, power point, etc.)'
> 2. can i safely remove the Python 2 & Phython3 directories? (to save
> hard disk space)
>
> thanks
> CR
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140403/6abc4c05/attachment.html>

From skip at pobox.com  Thu Apr  3 16:02:45 2014
From: skip at pobox.com (Skip Montanaro)
Date: Thu, 3 Apr 2014 09:02:45 -0500
Subject: [Python-Dev] Python 4?
In-Reply-To: <CAP1=2W5PvbrtFP0+f+Q490dLNKHp2p6K44WiXFwP+4ZvtkUsEg@mail.gmail.com>
References: <CANc-5UwY5EAAOhO741biNnmMJCQKsk5x1RpKiNkA7mB+X0PATQ@mail.gmail.com>
 <CAP1=2W5PvbrtFP0+f+Q490dLNKHp2p6K44WiXFwP+4ZvtkUsEg@mail.gmail.com>
Message-ID: <CANc-5UyUPOzHUGStGiSsMfSAnQgnsd7LzDBy+ZkRUqyiX7gmnw@mail.gmail.com>

On Thu, Apr 3, 2014 at 8:59 AM, Brett Cannon <brett at python.org> wrote:
>
> Who the heck knows what the person specifically means, although it sounds like he did mean Python 3.4 which would explain why he know has a Python3 directory.

Thanks. I'll try and see what he really has. I was worried that an
April Fool's prank went so far as to put a tarted up version of Python
3.x out in the wild where unsuspecting folks would find and install
it.

Skip

From rosuav at gmail.com  Thu Apr  3 16:22:07 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 4 Apr 2014 01:22:07 +1100
Subject: [Python-Dev] Python 4?
In-Reply-To: <CANc-5UyUPOzHUGStGiSsMfSAnQgnsd7LzDBy+ZkRUqyiX7gmnw@mail.gmail.com>
References: <CANc-5UwY5EAAOhO741biNnmMJCQKsk5x1RpKiNkA7mB+X0PATQ@mail.gmail.com>
 <CAP1=2W5PvbrtFP0+f+Q490dLNKHp2p6K44WiXFwP+4ZvtkUsEg@mail.gmail.com>
 <CANc-5UyUPOzHUGStGiSsMfSAnQgnsd7LzDBy+ZkRUqyiX7gmnw@mail.gmail.com>
Message-ID: <CAPTjJmqFjL+j7ZzswgDfZSbyS3eqzf4+wR=J3LMk_sE+pK7yJQ@mail.gmail.com>

On Fri, Apr 4, 2014 at 1:02 AM, Skip Montanaro <skip at pobox.com> wrote:
> On Thu, Apr 3, 2014 at 8:59 AM, Brett Cannon <brett at python.org> wrote:
>>
>> Who the heck knows what the person specifically means, although it sounds like he did mean Python 3.4 which would explain why he know has a Python3 directory.
>
> Thanks. I'll try and see what he really has. I was worried that an
> April Fool's prank went so far as to put a tarted up version of Python
> 3.x out in the wild where unsuspecting folks would find and install
> it.
>

I doubt it, from the sound of the rest of the email. Two likely
possibilities: One, he actually just meant "Python 3" and typo'd it;
two, he's abbreviating the version numbers in Java style - Python 2 =
3.2, Python 3 = 3.3, and he now has a Python 4 = 3.4.

Either way, I'd advise him to post to python-list, where people will
be happy to help out. Exact path names would help.

ChrisA

From rdmurray at bitdance.com  Thu Apr  3 16:36:57 2014
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 03 Apr 2014 10:36:57 -0400
Subject: [Python-Dev] Python 4?
In-Reply-To: <CAP1=2W5PvbrtFP0+f+Q490dLNKHp2p6K44WiXFwP+4ZvtkUsEg@mail.gmail.com>
References: <CANc-5UwY5EAAOhO741biNnmMJCQKsk5x1RpKiNkA7mB+X0PATQ@mail.gmail.com>
 <CAP1=2W5PvbrtFP0+f+Q490dLNKHp2p6K44WiXFwP+4ZvtkUsEg@mail.gmail.com>
Message-ID: <20140403143702.8D2E1250002@webabinitio.net>

On Thu, 03 Apr 2014 09:59:55 -0400, Brett Cannon <brett at python.org> wrote:
> On Thu, Apr 3, 2014 at 9:51 AM, Skip Montanaro <skip at pobox.com> wrote:
> 
> > I saw mention recently of Python 4 and assumed all such references
> > were either April Fool's jokes or pie-in-the-sky dreams for a new
> > version of Python which may never arrive.
> >
> 
> It's the latter. Some of us have been tossing around "Python 4" like we did
> "Py3k" before we starting Python 3 in earnest; maybe we should make sure to
> say Py4k to be perfectly clear there are no Python 4 plans ATM.

I think Python4 is just what follows python 3.9, myself :).

More seriously, I don't believe there should ever be a Py4k the way there
was a Py3k, and would prefer not to feed any rumours that there might be.

--David

From tseaver at palladion.com  Thu Apr  3 16:46:18 2014
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 03 Apr 2014 10:46:18 -0400
Subject: [Python-Dev] Python 4?
In-Reply-To: <20140403143702.8D2E1250002@webabinitio.net>
References: <CANc-5UwY5EAAOhO741biNnmMJCQKsk5x1RpKiNkA7mB+X0PATQ@mail.gmail.com>
 <CAP1=2W5PvbrtFP0+f+Q490dLNKHp2p6K44WiXFwP+4ZvtkUsEg@mail.gmail.com>
 <20140403143702.8D2E1250002@webabinitio.net>
Message-ID: <lhjs7g$f3i$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 04/03/2014 10:36 AM, R. David Murray wrote:

> More seriously, I don't believe there should ever be a Py4k the way
> there was a Py3k, and would prefer not to feed any rumours that there
> might be.

Amen!


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlM9dDoACgkQ+gerLs4ltQ6ikACgrneAqvKimcm64/AKCbFmp4pn
zEoAoK0q6nFXDDRL+UY3zI/PPFDvr3cg
=gTu1
-----END PGP SIGNATURE-----


From jtyree at enthought.com  Wed Apr  2 22:08:35 2014
From: jtyree at enthought.com (John Tyree)
Date: Wed, 2 Apr 2014 15:08:35 -0500
Subject: [Python-Dev] Incorrect behavior in str.format() method when padding
	with '\x00'
Message-ID: <CAMryXbG1N40WMFF3Bf9SghBtHDtFZi6BUftrSSO+bp4owhpkfw@mail.gmail.com>

Hi all,

Is there any particularly reason for the following behavior on both 2.7.6
and 3.4.0 ?

>>> "{:\x00<5}".format(2)
> '2    '
> >>>
> "{:\x20<5}".format(2)
>
> '2    '
> >>>
> "{:\x01<5}".format(2)
>
> '2\x01\x01\x01\x01'
> >>> "{:\x00<5}".format(2) ==
> "{:\x20<5}".format(2)
>
> True
>

The docs say "If a valid *align* value is specified, it can be preceded by
a *fill* character that can be any character and defaults to a space if
omitted," so I'm inclined to call this a good old fashioned bug.

-- 
John Tyree
Scientific Software Developer
Enthought, Inc.
www.enthought.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140402/533ac582/attachment.html>

From aponomarenko at rosalab.ru  Thu Apr  3 13:02:08 2014
From: aponomarenko at rosalab.ru (Andrey Ponomarenko)
Date: Thu, 03 Apr 2014 15:02:08 +0400
Subject: [Python-Dev] libpython added to ABI tracker
In-Reply-To: <CADiSq7dCkHfNaHQvEerxu6wW8gx=z851cNd9F+zNyexKUjYh5A@mail.gmail.com>
References: <5335337F.8000303@rosalab.ru>
 <CAMpsgwYf-KfY4+6=nLxm52L_NYcq_Tj3T9tAdJSmtf2K3GFL=A@mail.gmail.com>
 <533953B9.2040506@rosalab.ru>
 <CAMpsgwZo7kYDT0-DZy+yKpac0E=iKikhz-MHjqP-8H_Qq55bpA@mail.gmail.com>
 <CADiSq7fChg8wXgoiqMab=tL7ipO9y8ODeh0+o4zaVKBVSk2n0g@mail.gmail.com>
 <533AA1A0.9020505@rosalab.ru>
 <CADiSq7egsPKkq9qxG+kX4wfFvGz9-VVEvKXRzntk5VL9oh+F_w@mail.gmail.com>
 <533C1C4D.507@rosalab.ru>
 <CADiSq7dCkHfNaHQvEerxu6wW8gx=z851cNd9F+zNyexKUjYh5A@mail.gmail.com>
Message-ID: <533D3FB0.2060809@rosalab.ru>

Nick Coghlan wrote:
> On 3 April 2014 00:18, Andrey Ponomarenko <aponomarenko at rosalab.ru> wrote:
>> Nick Coghlan wrote:
>>> Regarding the warnings for this one - is there a way for the checker
>>> to warn if data structures are exposed directly, rather than as opaque
>>> types? It's fine if there isn't, it would just be cool if there was -
>>> one of the premises of the stable ABI is that it *doesn't* expose the
>>> type definitions directly to consuming code, just the pointers to them
>>> (hence allowing the struct size to change without actually breaking
>>> compatibility with the defined ABI).
>>
>> The ABI of the library can be dumped by the abi-compliance-checker basic
>> tool to a text file in the human readable format, so anyone can analyse it
>> in order to find problems of any kind.
> Thanks! I've now filed http://bugs.python.org/issue21142 as an RFE -
> I'll definitely be looking into setting up a regular run of this to
> help us avoid any new issues with managing the evolution of the stable
> ABI (and the public ABI in general).

I wrote some instructions on how to setup local ABI checker in the 
comment to issue 21142.

The main idea is to create ABI dump of the reference libpython version 
(say, 3.2) and compare it with the ABI dump of the current version. The 
ABI dump of the reference version can be stored somewhere in the source 
tree.

Thanks.

-- 
Andrey Ponomarenko, NTC IT ROSA.


From eric at trueblade.com  Thu Apr  3 17:20:20 2014
From: eric at trueblade.com (Eric V. Smith)
Date: Thu, 03 Apr 2014 11:20:20 -0400
Subject: [Python-Dev] Incorrect behavior in str.format() method when
 padding with '\x00'
In-Reply-To: <CAMryXbG1N40WMFF3Bf9SghBtHDtFZi6BUftrSSO+bp4owhpkfw@mail.gmail.com>
References: <CAMryXbG1N40WMFF3Bf9SghBtHDtFZi6BUftrSSO+bp4owhpkfw@mail.gmail.com>
Message-ID: <533D7C34.5050308@trueblade.com>

On 04/02/2014 04:08 PM, John Tyree wrote:
> 
> Hi all,
> 
> Is there any particularly reason for the following behavior on both
> 2.7.6 and 3.4.0 ?
> 
>     >>> "{:\x00<5}".format(2)
>     '2    '
>     >>>
>     "{:\x20<5}".format(2)                                                                                                
> 
>     '2    '
>     >>>
>     "{:\x01<5}".format(2)                                                                                                
> 
>     '2\x01\x01\x01\x01'
>     >>> "{:\x00<5}".format(2) ==
>     "{:\x20<5}".format(2)                                                                       
> 
>     True
> 
> 
> The docs say "If a valid /align/ value is specified, it can be preceded
> by a /fill/ character that can be any character and defaults to a space
> if omitted," so I'm inclined to call this a good old fashioned bug.

I'd say it's a bug. Please open a bug, assign it to me (eric.smith), and
I'll comment on it there.

Eric.



From rdmurray at bitdance.com  Thu Apr  3 17:37:49 2014
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 03 Apr 2014 11:37:49 -0400
Subject: [Python-Dev] Incorrect behavior in str.format() method when
	padding with '\x00'
In-Reply-To: <533D7C34.5050308@trueblade.com>
References: <CAMryXbG1N40WMFF3Bf9SghBtHDtFZi6BUftrSSO+bp4owhpkfw@mail.gmail.com>
 <533D7C34.5050308@trueblade.com>
Message-ID: <20140403153750.33A97250049@webabinitio.net>

On Thu, 03 Apr 2014 11:20:20 -0400, "Eric V. Smith" <eric at trueblade.com> wrote:
> On 04/02/2014 04:08 PM, John Tyree wrote:
> > 
> > Hi all,
> > 
> > Is there any particularly reason for the following behavior on both
> > 2.7.6 and 3.4.0 ?
> > 
> >     >>> "{:\x00<5}".format(2)
> >     '2    '
> >     >>>
> >     "{:\x20<5}".format(2)                                                                                                
> > 
> >     '2    '
> >     >>>
> >     "{:\x01<5}".format(2)                                                                                                
> > 
> >     '2\x01\x01\x01\x01'
> >     >>> "{:\x00<5}".format(2) ==
> >     "{:\x20<5}".format(2)                                                                       
> > 
> >     True
> > 
> > 
> > The docs say "If a valid /align/ value is specified, it can be preceded
> > by a /fill/ character that can be any character and defaults to a space
> > if omitted," so I'm inclined to call this a good old fashioned bug.
> 
> I'd say it's a bug. Please open a bug, assign it to me (eric.smith), and
> I'll comment on it there.

There is an existing bug report (assigned to you :), with a partial patch:
http://bugs.python.org/issue12546

--David

From python at mrabarnett.plus.com  Thu Apr  3 17:49:32 2014
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 03 Apr 2014 16:49:32 +0100
Subject: [Python-Dev] Incorrect behavior in str.format() method when
 padding with '\x00'
In-Reply-To: <533D7C34.5050308@trueblade.com>
References: <CAMryXbG1N40WMFF3Bf9SghBtHDtFZi6BUftrSSO+bp4owhpkfw@mail.gmail.com>
 <533D7C34.5050308@trueblade.com>
Message-ID: <533D830C.1000505@mrabarnett.plus.com>

On 2014-04-03 16:20, Eric V. Smith wrote:
> On 04/02/2014 04:08 PM, John Tyree wrote:
>>
>> Hi all,
>>
>> Is there any particularly reason for the following behavior on both
>> 2.7.6 and 3.4.0 ?
>>
>>     >>> "{:\x00<5}".format(2)
>>     '2    '
>>     >>>
>>     "{:\x20<5}".format(2)
>>
>>     '2    '
>>     >>>
>>     "{:\x01<5}".format(2)
>>
>>     '2\x01\x01\x01\x01'
>>     >>> "{:\x00<5}".format(2) ==
>>     "{:\x20<5}".format(2)
>>
>>     True
>>
>>
>> The docs say "If a valid /align/ value is specified, it can be preceded
>> by a /fill/ character that can be any character and defaults to a space
>> if omitted," so I'm inclined to call this a good old fashioned bug.
>
> I'd say it's a bug. Please open a bug, assign it to me (eric.smith), and
> I'll comment on it there.
>
I'd agree. It looks like '\x00' is being confused with a NUL terminator
in the C code. I've seen such a thing before (now fixed).

From eric at trueblade.com  Thu Apr  3 19:12:20 2014
From: eric at trueblade.com (Eric V. Smith)
Date: Thu, 03 Apr 2014 13:12:20 -0400
Subject: [Python-Dev] Incorrect behavior in str.format() method when
 padding with '\x00'
In-Reply-To: <20140403153750.33A97250049@webabinitio.net>
References: <CAMryXbG1N40WMFF3Bf9SghBtHDtFZi6BUftrSSO+bp4owhpkfw@mail.gmail.com>
 <533D7C34.5050308@trueblade.com> <20140403153750.33A97250049@webabinitio.net>
Message-ID: <533D9674.7030805@trueblade.com>

On 04/03/2014 11:37 AM, R. David Murray wrote:
> On Thu, 03 Apr 2014 11:20:20 -0400, "Eric V. Smith" <eric at trueblade.com> wrote:
>> On 04/02/2014 04:08 PM, John Tyree wrote:
>>>
>>> Hi all,
>>>
>>> Is there any particularly reason for the following behavior on both
>>> 2.7.6 and 3.4.0 ?
>>>
>>>     >>> "{:\x00<5}".format(2)
>>>     '2    '
>>>     >>>
>>>     "{:\x20<5}".format(2)                                                                                                
>>>
>>>     '2    '
>>>     >>>
>>>     "{:\x01<5}".format(2)                                                                                                
>>>
>>>     '2\x01\x01\x01\x01'
>>>     >>> "{:\x00<5}".format(2) ==
>>>     "{:\x20<5}".format(2)                                                                       
>>>
>>>     True
>>>
>>>
>>> The docs say "If a valid /align/ value is specified, it can be preceded
>>> by a /fill/ character that can be any character and defaults to a space
>>> if omitted," so I'm inclined to call this a good old fashioned bug.
>>
>> I'd say it's a bug. Please open a bug, assign it to me (eric.smith), and
>> I'll comment on it there.
> 
> There is an existing bug report (assigned to you :), with a partial patch:
> http://bugs.python.org/issue12546

Heh. I'll add this to my list of things to do at PyCon sprints.

Eric.



From eric at trueblade.com  Thu Apr  3 19:13:17 2014
From: eric at trueblade.com (Eric V. Smith)
Date: Thu, 03 Apr 2014 13:13:17 -0400
Subject: [Python-Dev] Incorrect behavior in str.format() method when
 padding with '\x00'
In-Reply-To: <533D830C.1000505@mrabarnett.plus.com>
References: <CAMryXbG1N40WMFF3Bf9SghBtHDtFZi6BUftrSSO+bp4owhpkfw@mail.gmail.com>
 <533D7C34.5050308@trueblade.com> <533D830C.1000505@mrabarnett.plus.com>
Message-ID: <533D96AD.60504@trueblade.com>

On 04/03/2014 11:49 AM, MRAB wrote:
> On 2014-04-03 16:20, Eric V. Smith wrote:
>> On 04/02/2014 04:08 PM, John Tyree wrote:
>>>
>>> Hi all,
>>>
>>> Is there any particularly reason for the following behavior on both
>>> 2.7.6 and 3.4.0 ?
>>>
>>>     >>> "{:\x00<5}".format(2)
>>>     '2    '
>>>     >>>
>>>     "{:\x20<5}".format(2)
>>>
>>>     '2    '
>>>     >>>
>>>     "{:\x01<5}".format(2)
>>>
>>>     '2\x01\x01\x01\x01'
>>>     >>> "{:\x00<5}".format(2) ==
>>>     "{:\x20<5}".format(2)
>>>
>>>     True
>>>
>>>
>>> The docs say "If a valid /align/ value is specified, it can be preceded
>>> by a /fill/ character that can be any character and defaults to a space
>>> if omitted," so I'm inclined to call this a good old fashioned bug.
>>
>> I'd say it's a bug. Please open a bug, assign it to me (eric.smith), and
>> I'll comment on it there.
>>
> I'd agree. It looks like '\x00' is being confused with a NUL terminator
> in the C code. I've seen such a thing before (now fixed).

Now that I recall the other issue (that I should have fixed already, but
let drop), it's because 0 is used as a flag to mean "use the default".

Eric.



From tjreedy at udel.edu  Thu Apr  3 20:04:18 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 03 Apr 2014 14:04:18 -0400
Subject: [Python-Dev] Gmane not functioning.
In-Reply-To: <533CA234.30203@udel.edu>
References: <533CA234.30203@udel.edu>
Message-ID: <lhk7r8$64h$1@ger.gmane.org>

It is again now.

-- 
Terry Jan Reedy


From kmod at dropbox.com  Thu Apr  3 20:19:31 2014
From: kmod at dropbox.com (Kevin Modzelewski)
Date: Thu, 3 Apr 2014 11:19:31 -0700
Subject: [Python-Dev] Pyston: a Python JIT on LLVM
Message-ID: <CAO=oM6s92ee0oeChdpmtNvmb4Av=iKGoxVdXGvKodYroMYQavQ@mail.gmail.com>

Hi all,

I'm excited to announce Pyston, a Python JIT under development at Dropbox,
built on top of LLVM.  You can read more about it at the introductory blog
post<https://tech.dropbox.com/2014/04/introducing-pyston-an-upcoming-jit-based-python-implementation/>,
or check out the code on github <https://github.com/dropbox/pyston>.

Since it's the question that I think most people will inevitably (and
rightly) ask, why do we think there's a place for Pyston when there's PyPy
and (previously) Unladen Swallow?

Compared to PyPy, Pyston makes a number of different technical choices,
such as using a method-at-a-time JIT vs PyPy's tracing JIT.  The
method-at-a-time approach seems to be being validated in the JavaScript
world, so there's reason to think there might be room to improve over PyPy,
though their extremely impressive performance makes that an admittedly
very, very high bar.  We also think that extension module support is a
first-class issue and have some ideas on how to try to make that fast.

As for Unladen Swallow, there are some reasons to think that LLVM has
matured greatly in the past few years, particularly in the JIT engine which
has been completely replaced.  I'm not sure if that's the only part to the
story; I'd be interested in talking with any of the people who were
involved or knowledgeable about the project.


It's definitely a very tall order to try to stand out among the existing
implementations, but in time we think we can do it.

I'll be at the language summit and PyCon sprints, and hope to see some of
you there!

kmod
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140403/b1bbc65f/attachment.html>

From sturla.molden at gmail.com  Fri Apr  4 00:42:36 2014
From: sturla.molden at gmail.com (Sturla Molden)
Date: Thu, 3 Apr 2014 22:42:36 +0000 (UTC)
Subject: [Python-Dev] Pyston: a Python JIT on LLVM
References: <CAO=oM6s92ee0oeChdpmtNvmb4Av=iKGoxVdXGvKodYroMYQavQ@mail.gmail.com>
Message-ID: <212815941418257280.135044sturla.molden-gmail.com@news.gmane.org>

Kevin Modzelewski <kmod at dropbox.com> wrote:

> Since it's the question that I think most people will inevitably (and
> rightly) ask, why do we think there's a place for Pyston when there's PyPy
> and (previously) Unladen Swallow?

Have you seen Numba, the Python JIT that integrates with NumPy?

http://numba.pydata.org

It uses LLVM to compile Python bytecode. When I have tried it I tend to get
speed comparable to -O2 in C for numerical and algorithmic code.

Here is an example, giving a 150 times speed boost to Python:

http://stackoverflow.com/questions/21811381/how-to-shove-this-loop-into-numpy/21818591#21818591


Sturla


From emile at fenx.com  Fri Apr  4 01:02:57 2014
From: emile at fenx.com (Emile van Sebille)
Date: Thu, 03 Apr 2014 16:02:57 -0700
Subject: [Python-Dev] Python 4?
In-Reply-To: <lhjs7g$f3i$1@ger.gmane.org>
References: <CANc-5UwY5EAAOhO741biNnmMJCQKsk5x1RpKiNkA7mB+X0PATQ@mail.gmail.com>
 <CAP1=2W5PvbrtFP0+f+Q490dLNKHp2p6K44WiXFwP+4ZvtkUsEg@mail.gmail.com>
 <20140403143702.8D2E1250002@webabinitio.net> <lhjs7g$f3i$1@ger.gmane.org>
Message-ID: <lhkpb3$p3q$1@ger.gmane.org>

On 4/3/2014 7:46 AM, Tres Seaver wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 04/03/2014 10:36 AM, R. David Murray wrote:
>
>> More seriously, I don't believe there should ever be a Py4k the way
>> there was a Py3k, and would prefer not to feed any rumours that there
>> might be.
>
> Amen!

+1

Can't we just refer to PySky and leave it at that?

:)

Emile




From v+python at g.nevcal.com  Fri Apr  4 01:32:17 2014
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Thu, 03 Apr 2014 16:32:17 -0700
Subject: [Python-Dev] Python 4?
In-Reply-To: <lhkpb3$p3q$1@ger.gmane.org>
References: <CANc-5UwY5EAAOhO741biNnmMJCQKsk5x1RpKiNkA7mB+X0PATQ@mail.gmail.com>
 <CAP1=2W5PvbrtFP0+f+Q490dLNKHp2p6K44WiXFwP+4ZvtkUsEg@mail.gmail.com>
 <20140403143702.8D2E1250002@webabinitio.net> <lhjs7g$f3i$1@ger.gmane.org>
 <lhkpb3$p3q$1@ger.gmane.org>
Message-ID: <533DEF81.9060400@g.nevcal.com>

On 4/3/2014 4:02 PM, Emile van Sebille wrote:
>
> Can't we just refer to PySky and leave it at that? 

Hey! That's an excellent idea! And it avoids the version number 
prediction/expectation too.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140403/230b765c/attachment.html>

From solipsis at pitrou.net  Fri Apr  4 10:59:06 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 04 Apr 2014 10:59:06 +0200
Subject: [Python-Dev] Pyston: a Python JIT on LLVM
In-Reply-To: <CAO=oM6s92ee0oeChdpmtNvmb4Av=iKGoxVdXGvKodYroMYQavQ@mail.gmail.com>
References: <CAO=oM6s92ee0oeChdpmtNvmb4Av=iKGoxVdXGvKodYroMYQavQ@mail.gmail.com>
Message-ID: <lhls8q$qf9$1@ger.gmane.org>


Hello,

Le 03/04/2014 20:19, Kevin Modzelewski a ?crit :
> I'm excited to announce Pyston, a Python JIT under development at
> Dropbox, built on top of LLVM.  You can read more about it at the
> introductory blog post
> <https://tech.dropbox.com/2014/04/introducing-pyston-an-upcoming-jit-based-python-implementation/>,
> or check out the code on github <https://github.com/dropbox/pyston>.

I'm a bit surprised by the approach. Why don't you simply process 
CPython bytecode, rather than strive to reimplement Python fully?

Also, I wonder if it's worthwhile to use a conservative GC, rather than 
reuse the original refcounting scheme (especially since you want to 
support existing extension modules).

Regards

Antoine.



From d.s.seljebotn at astro.uio.no  Fri Apr  4 12:13:53 2014
From: d.s.seljebotn at astro.uio.no (Dag Sverre Seljebotn)
Date: Fri, 04 Apr 2014 12:13:53 +0200
Subject: [Python-Dev] Pyston: a Python JIT on LLVM
In-Reply-To: <212815941418257280.135044sturla.molden-gmail.com@news.gmane.org>
References: <CAO=oM6s92ee0oeChdpmtNvmb4Av=iKGoxVdXGvKodYroMYQavQ@mail.gmail.com>
 <212815941418257280.135044sturla.molden-gmail.com@news.gmane.org>
Message-ID: <533E85E1.7050001@astro.uio.no>

On 04/04/2014 12:42 AM, Sturla Molden wrote:
> Kevin Modzelewski <kmod at dropbox.com> wrote:
>
>> Since it's the question that I think most people will inevitably (and
>> rightly) ask, why do we think there's a place for Pyston when there's PyPy
>> and (previously) Unladen Swallow?
>
> Have you seen Numba, the Python JIT that integrates with NumPy?
>
> http://numba.pydata.org

Specifically, Numba compiles to LLVM too, and tries to be somewhat 
general-purpose although it's tuned to numerical code. And their reason 
for not using PyPy is the same: C extensions. So while their "market 
segment" is different from yours, the technology may not be.

Dag Sverre


>
> It uses LLVM to compile Python bytecode. When I have tried it I tend to get
> speed comparable to -O2 in C for numerical and algorithmic code.
>
> Here is an example, giving a 150 times speed boost to Python:
>
> http://stackoverflow.com/questions/21811381/how-to-shove-this-loop-into-numpy/21818591#21818591
>
>
> Sturla
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/d.s.seljebotn%40astro.uio.no
>


From brett at python.org  Fri Apr  4 16:12:23 2014
From: brett at python.org (Brett Cannon)
Date: Fri, 4 Apr 2014 10:12:23 -0400
Subject: [Python-Dev] [Python-checkins] cpython: fix #21076: turn signal
 module constants into enums
In-Reply-To: <3g0hvS5h5Wz7LjM@mail.python.org>
References: <3g0hvS5h5Wz7LjM@mail.python.org>
Message-ID: <CAP1=2W7Qif5qhND=VA+rOnt1+fGFM75Rp-K9aN0T3oGZweXMUA@mail.gmail.com>

This broke compilation on at least OS X, but I'm willing to bet for all
UNIX-based systems. I have a fix in the works.


On Fri, Apr 4, 2014 at 9:34 AM, giampaolo.rodola <python-checkins at python.org
> wrote:

> http://hg.python.org/cpython/rev/c9239171e429
> changeset:   90128:c9239171e429
> user:        Giampaolo Rodola' <g.rodola at gmail.com>
> date:        Fri Apr 04 15:34:17 2014 +0200
> summary:
>   fix #21076: turn signal module constants into enums
>
> files:
>   Doc/library/signal.rst   |  10 +++
>   Doc/whatsnew/3.5.rst     |   5 +
>   Lib/signal.py            |  84 ++++++++++++++++++++++++++++
>   Lib/test/test_doctest.py |   2 +-
>   Lib/test/test_signal.py  |  39 +++++++++++-
>   Modules/signalmodule.c   |   4 +-
>   PC/config.c              |   2 +-
>   7 files changed, 138 insertions(+), 8 deletions(-)
>
>
> diff --git a/Doc/library/signal.rst b/Doc/library/signal.rst
> --- a/Doc/library/signal.rst
> +++ b/Doc/library/signal.rst
> @@ -65,6 +65,16 @@
>  Module contents
>  ---------------
>
> +.. versionchanged:: 3.5
> +   signal (SIG*), handler (:const:`SIG_DFL`, :const:`SIG_IGN`) and sigmask
> +   (:const:`SIG_BLOCK`, :const:`SIG_UNBLOCK`, :const:`SIG_SETMASK`)
> +   related constants listed below were turned into
> +   :class:`enums <enum.IntEnum>`.
> +   :func:`getsignal`, :func:`pthread_sigmask`, :func:`sigpending` and
> +   :func:`sigwait` functions return human-readable
> +   :class:`enums <enum.IntEnum>`.
> +
> +
>  The variables defined in the :mod:`signal` module are:
>
>
> diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst
> --- a/Doc/whatsnew/3.5.rst
> +++ b/Doc/whatsnew/3.5.rst
> @@ -134,6 +134,11 @@
>  Improved Modules
>  ================
>
> +* Different constants of :mod:`signal` module are now enumeration values
> using
> +  the :mod:`enum` module. This allows meaningful names to be printed
> during
> +  debugging, instead of integer ?magic numbers?. (contribute by Giampaolo
> +  Rodola' in :issue:`21076`)
> +
>  * :class:`xmlrpc.client.ServerProxy` is now a :term:`context manager`
>    (contributed by Claudiu Popa in :issue:`20627`).
>
> diff --git a/Lib/signal.py b/Lib/signal.py
> new file mode 100644
> --- /dev/null
> +++ b/Lib/signal.py
> @@ -0,0 +1,84 @@
> +import _signal
> +from _signal import *
> +from functools import wraps as _wraps
> +from enum import IntEnum as _IntEnum
> +
> +_globals = globals()
> +
> +Signals = _IntEnum(
> +    'Signals',
> +    {name: value for name, value in _globals.items()
> +     if name.isupper()
> +        and (name.startswith('SIG') and not name.startswith('SIG_'))
> +        or name.startswith('CTRL_')})
> +
> +class Handlers(_IntEnum):
> +    SIG_DFL = _signal.SIG_DFL
> +    SIG_IGN = _signal.SIG_IGN
> +
> +_globals.update(Signals.__members__)
> +_globals.update(Handlers.__members__)
> +
> +if 'pthread_sigmask' in _globals:
> +    class Sigmasks(_IntEnum):
> +        SIG_BLOCK = _signal.SIG_BLOCK
> +        SIG_UNBLOCK = _signal.SIG_UNBLOCK
> +        SIG_SETMASK = _signal.SIG_SETMASK
> +
> +    _globals.update(Sigmasks.__members__)
> +
> +
> +def _int_to_enum(value, enum_klass):
> +    """Convert a numeric value to an IntEnum member.
> +    If it's not a known member, return the numeric value itself.
> +    """
> +    try:
> +        return enum_klass(value)
> +    except ValueError:
> +        return value
> +
> +
> +def _enum_to_int(value):
> +    """Convert an IntEnum member to a numeric value.
> +    If it's not a IntEnum member return the value itself.
> +    """
> +    try:
> +        return int(value)
> +    except (ValueError, TypeError):
> +        return value
> +
> +
> + at _wraps(_signal.signal)
> +def signal(signalnum, handler):
> +    handler = _signal.signal(_enum_to_int(signalnum),
> _enum_to_int(handler))
> +    return _int_to_enum(handler, Handlers)
> +
> +
> + at _wraps(_signal.getsignal)
> +def getsignal(signalnum):
> +    handler = _signal.getsignal(signalnum)
> +    return _int_to_enum(handler, Handlers)
> +
> +
> +if 'pthread_sigmask' in _globals:
> +    @_wraps(_signal.pthread_sigmask)
> +    def pthread_sigmask(how, mask):
> +        sigs_set = _signal.pthread_sigmask(how, mask)
> +        return set(_int_to_enum(x, Signals) for x in sigs_set)
> +    pthread_sigmask.__doc__ = _signal.pthread_sigmask.__doc__
> +
> +
> + at _wraps(_signal.sigpending)
> +def sigpending():
> +    sigs = _signal.sigpending()
> +    return set(_int_to_enum(x, Signals) for x in sigs)
> +
> +
> +if 'sigwait' in _globals:
> +    @_wraps(_signal.sigwait)
> +    def sigwait(sigset):
> +        retsig = _signal.sigwait(sigset)
> +        return _int_to_enum(retsig, Signals)
> +    sigwait.__doc__ = _signal.sigwait
> +
> +del _globals, _wraps
> diff --git a/Lib/test/test_doctest.py b/Lib/test/test_doctest.py
> --- a/Lib/test/test_doctest.py
> +++ b/Lib/test/test_doctest.py
> @@ -2897,7 +2897,7 @@
>
>  def test_main():
>      # Check the doctest cases in doctest itself:
> -    support.run_doctest(doctest, verbosity=True)
> +    ret = support.run_doctest(doctest, verbosity=True)
>      # Check the doctest cases defined here:
>      from test import test_doctest
>      support.run_doctest(test_doctest, verbosity=True)
> diff --git a/Lib/test/test_signal.py b/Lib/test/test_signal.py
> --- a/Lib/test/test_signal.py
> +++ b/Lib/test/test_signal.py
> @@ -1,6 +1,7 @@
>  import unittest
>  from test import support
>  from contextlib import closing
> +import enum
>  import gc
>  import pickle
>  import select
> @@ -39,6 +40,22 @@
>          return None
>
>
> +class GenericTests(unittest.TestCase):
> +
> +    def test_enums(self):
> +        for name in dir(signal):
> +            sig = getattr(signal, name)
> +            if name in {'SIG_DFL', 'SIG_IGN'}:
> +                self.assertIsInstance(sig, signal.Handlers)
> +            elif name in {'SIG_BLOCK', 'SIG_UNBLOCK', 'SIG_SETMASK'}:
> +                self.assertIsInstance(sig, signal.Sigmasks)
> +            elif name.startswith('SIG') and not name.startswith('SIG_'):
> +                self.assertIsInstance(sig, signal.Signals)
> +            elif name.startswith('CTRL_'):
> +                self.assertIsInstance(sig, signal.Signals)
> +                self.assertEqual(sys.platform, "win32")
> +
> +
>  @unittest.skipIf(sys.platform == "win32", "Not valid on Windows")
>  class InterProcessSignalTests(unittest.TestCase):
>      MAX_DURATION = 20   # Entire test should last at most 20 sec.
> @@ -195,6 +212,7 @@
>
>      def test_getsignal(self):
>          hup = signal.signal(signal.SIGHUP, self.trivial_signal_handler)
> +        self.assertIsInstance(hup, signal.Handlers)
>          self.assertEqual(signal.getsignal(signal.SIGHUP),
>                           self.trivial_signal_handler)
>          signal.signal(signal.SIGHUP, hup)
> @@ -271,7 +289,7 @@
>
>          os.close(read)
>          os.close(write)
> -        """.format(signals, ordered, test_body)
> +        """.format(tuple(map(int, signals)), ordered, test_body)
>
>          assert_python_ok('-c', code)
>
> @@ -604,6 +622,8 @@
>              signal.pthread_sigmask(signal.SIG_BLOCK, [signum])
>              os.kill(os.getpid(), signum)
>              pending = signal.sigpending()
> +            for sig in pending:
> +                assert isinstance(sig, signal.Signals), repr(pending)
>              if pending != {signum}:
>                  raise Exception('%s != {%s}' % (pending, signum))
>              try:
> @@ -660,6 +680,7 @@
>          code = '''if 1:
>          import signal
>          import sys
> +        from signal import Signals
>
>          def handler(signum, frame):
>              1/0
> @@ -702,6 +723,7 @@
>          def test(signum):
>              signal.alarm(1)
>              received = signal.sigwait([signum])
> +            assert isinstance(received, signal.Signals), received
>              if received != signum:
>                  raise Exception('received %s, not %s' % (received,
> signum))
>          ''')
> @@ -842,8 +864,14 @@
>          def kill(signum):
>              os.kill(os.getpid(), signum)
>
> +        def check_mask(mask):
> +            for sig in mask:
> +                assert isinstance(sig, signal.Signals), repr(sig)
> +
>          def read_sigmask():
> -            return signal.pthread_sigmask(signal.SIG_BLOCK, [])
> +            sigmask = signal.pthread_sigmask(signal.SIG_BLOCK, [])
> +            check_mask(sigmask)
> +            return sigmask
>
>          signum = signal.SIGUSR1
>
> @@ -852,6 +880,7 @@
>
>          # Unblock SIGUSR1 (and copy the old mask) to test our signal
> handler
>          old_mask = signal.pthread_sigmask(signal.SIG_UNBLOCK, [signum])
> +        check_mask(old_mask)
>          try:
>              kill(signum)
>          except ZeroDivisionError:
> @@ -861,11 +890,13 @@
>
>          # Block and then raise SIGUSR1. The signal is blocked: the signal
>          # handler is not called, and the signal is now pending
> -        signal.pthread_sigmask(signal.SIG_BLOCK, [signum])
> +        mask = signal.pthread_sigmask(signal.SIG_BLOCK, [signum])
> +        check_mask(mask)
>          kill(signum)
>
>          # Check the new mask
>          blocked = read_sigmask()
> +        check_mask(blocked)
>          if signum not in blocked:
>              raise Exception("%s not in %s" % (signum, blocked))
>          if old_mask ^ blocked != {signum}:
> @@ -928,7 +959,7 @@
>
>  def test_main():
>      try:
> -        support.run_unittest(PosixTests, InterProcessSignalTests,
> +        support.run_unittest(GenericTests, PosixTests,
> InterProcessSignalTests,
>                               WakeupFDTests, WakeupSignalTests,
>                               SiginterruptTest, ItimerTest,
> WindowsSignalTests,
>                               PendingSignalsTests)
> diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c
> --- a/Modules/signalmodule.c
> +++ b/Modules/signalmodule.c
> @@ -967,7 +967,7 @@
>  };
>
>  PyMODINIT_FUNC
> -PyInit_signal(void)
> +PyInit__signal(void)
>  {
>      PyObject *m, *d, *x;
>      int i;
> @@ -1380,7 +1380,7 @@
>  void
>  PyOS_InitInterrupts(void)
>  {
> -    PyObject *m = PyImport_ImportModule("signal");
> +    PyObject *m = PyImport_ImportModule("_signal");
>      if (m) {
>          Py_DECREF(m);
>      }
> diff --git a/PC/config.c b/PC/config.c
> --- a/PC/config.c
> +++ b/PC/config.c
> @@ -19,7 +19,7 @@
>  extern PyObject* PyInit__md5(void);
>  extern PyObject* PyInit_nt(void);
>  extern PyObject* PyInit__operator(void);
> -extern PyObject* PyInit_signal(void);
> +extern PyObject* PyInit__signal(void);
>  extern PyObject* PyInit__sha1(void);
>  extern PyObject* PyInit__sha256(void);
>  extern PyObject* PyInit__sha512(void);
>
> --
> Repository URL: http://hg.python.org/cpython
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> https://mail.python.org/mailman/listinfo/python-checkins
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140404/ec3c9b19/attachment.html>

From brett at python.org  Fri Apr  4 16:21:39 2014
From: brett at python.org (Brett Cannon)
Date: Fri, 4 Apr 2014 10:21:39 -0400
Subject: [Python-Dev] [Python-checkins] cpython: fix #21076: turn signal
 module constants into enums
In-Reply-To: <CAP1=2W7Qif5qhND=VA+rOnt1+fGFM75Rp-K9aN0T3oGZweXMUA@mail.gmail.com>
References: <3g0hvS5h5Wz7LjM@mail.python.org>
 <CAP1=2W7Qif5qhND=VA+rOnt1+fGFM75Rp-K9aN0T3oGZweXMUA@mail.gmail.com>
Message-ID: <CAP1=2W4wfsa9xtOaL6VdcLpk_vGtBFd-2dd7YGrn8aSpZoChbA@mail.gmail.com>

On Fri, Apr 4, 2014 at 10:12 AM, Brett Cannon <brett at python.org> wrote:

> This broke compilation on at least OS X, but I'm willing to bet for all
> UNIX-based systems. I have a fix in the works.
>

Fix is in rev c6e63bb132fb <http://hg.python.org/cpython/rev/c6e63bb132fb>.


>
>
> On Fri, Apr 4, 2014 at 9:34 AM, giampaolo.rodola <
> python-checkins at python.org> wrote:
>
>> http://hg.python.org/cpython/rev/c9239171e429
>> changeset:   90128:c9239171e429
>> user:        Giampaolo Rodola' <g.rodola at gmail.com>
>> date:        Fri Apr 04 15:34:17 2014 +0200
>> summary:
>>   fix #21076: turn signal module constants into enums
>>
>> files:
>>   Doc/library/signal.rst   |  10 +++
>>   Doc/whatsnew/3.5.rst     |   5 +
>>   Lib/signal.py            |  84 ++++++++++++++++++++++++++++
>>   Lib/test/test_doctest.py |   2 +-
>>   Lib/test/test_signal.py  |  39 +++++++++++-
>>   Modules/signalmodule.c   |   4 +-
>>   PC/config.c              |   2 +-
>>   7 files changed, 138 insertions(+), 8 deletions(-)
>>
>>
>> diff --git a/Doc/library/signal.rst b/Doc/library/signal.rst
>> --- a/Doc/library/signal.rst
>> +++ b/Doc/library/signal.rst
>> @@ -65,6 +65,16 @@
>>  Module contents
>>  ---------------
>>
>> +.. versionchanged:: 3.5
>> +   signal (SIG*), handler (:const:`SIG_DFL`, :const:`SIG_IGN`) and
>> sigmask
>> +   (:const:`SIG_BLOCK`, :const:`SIG_UNBLOCK`, :const:`SIG_SETMASK`)
>> +   related constants listed below were turned into
>> +   :class:`enums <enum.IntEnum>`.
>> +   :func:`getsignal`, :func:`pthread_sigmask`, :func:`sigpending` and
>> +   :func:`sigwait` functions return human-readable
>> +   :class:`enums <enum.IntEnum>`.
>> +
>> +
>>  The variables defined in the :mod:`signal` module are:
>>
>>
>> diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst
>> --- a/Doc/whatsnew/3.5.rst
>> +++ b/Doc/whatsnew/3.5.rst
>> @@ -134,6 +134,11 @@
>>  Improved Modules
>>  ================
>>
>> +* Different constants of :mod:`signal` module are now enumeration values
>> using
>> +  the :mod:`enum` module. This allows meaningful names to be printed
>> during
>> +  debugging, instead of integer ?magic numbers?. (contribute by Giampaolo
>> +  Rodola' in :issue:`21076`)
>> +
>>  * :class:`xmlrpc.client.ServerProxy` is now a :term:`context manager`
>>    (contributed by Claudiu Popa in :issue:`20627`).
>>
>> diff --git a/Lib/signal.py b/Lib/signal.py
>> new file mode 100644
>> --- /dev/null
>> +++ b/Lib/signal.py
>> @@ -0,0 +1,84 @@
>> +import _signal
>> +from _signal import *
>> +from functools import wraps as _wraps
>> +from enum import IntEnum as _IntEnum
>> +
>> +_globals = globals()
>> +
>> +Signals = _IntEnum(
>> +    'Signals',
>> +    {name: value for name, value in _globals.items()
>> +     if name.isupper()
>> +        and (name.startswith('SIG') and not name.startswith('SIG_'))
>> +        or name.startswith('CTRL_')})
>> +
>> +class Handlers(_IntEnum):
>> +    SIG_DFL = _signal.SIG_DFL
>> +    SIG_IGN = _signal.SIG_IGN
>> +
>> +_globals.update(Signals.__members__)
>> +_globals.update(Handlers.__members__)
>> +
>> +if 'pthread_sigmask' in _globals:
>> +    class Sigmasks(_IntEnum):
>> +        SIG_BLOCK = _signal.SIG_BLOCK
>> +        SIG_UNBLOCK = _signal.SIG_UNBLOCK
>> +        SIG_SETMASK = _signal.SIG_SETMASK
>> +
>> +    _globals.update(Sigmasks.__members__)
>> +
>> +
>> +def _int_to_enum(value, enum_klass):
>> +    """Convert a numeric value to an IntEnum member.
>> +    If it's not a known member, return the numeric value itself.
>> +    """
>> +    try:
>> +        return enum_klass(value)
>> +    except ValueError:
>> +        return value
>> +
>> +
>> +def _enum_to_int(value):
>> +    """Convert an IntEnum member to a numeric value.
>> +    If it's not a IntEnum member return the value itself.
>> +    """
>> +    try:
>> +        return int(value)
>> +    except (ValueError, TypeError):
>> +        return value
>> +
>> +
>> + at _wraps(_signal.signal)
>> +def signal(signalnum, handler):
>> +    handler = _signal.signal(_enum_to_int(signalnum),
>> _enum_to_int(handler))
>> +    return _int_to_enum(handler, Handlers)
>> +
>> +
>> + at _wraps(_signal.getsignal)
>> +def getsignal(signalnum):
>> +    handler = _signal.getsignal(signalnum)
>> +    return _int_to_enum(handler, Handlers)
>> +
>> +
>> +if 'pthread_sigmask' in _globals:
>> +    @_wraps(_signal.pthread_sigmask)
>> +    def pthread_sigmask(how, mask):
>> +        sigs_set = _signal.pthread_sigmask(how, mask)
>> +        return set(_int_to_enum(x, Signals) for x in sigs_set)
>> +    pthread_sigmask.__doc__ = _signal.pthread_sigmask.__doc__
>> +
>> +
>> + at _wraps(_signal.sigpending)
>> +def sigpending():
>> +    sigs = _signal.sigpending()
>> +    return set(_int_to_enum(x, Signals) for x in sigs)
>> +
>> +
>> +if 'sigwait' in _globals:
>> +    @_wraps(_signal.sigwait)
>> +    def sigwait(sigset):
>> +        retsig = _signal.sigwait(sigset)
>> +        return _int_to_enum(retsig, Signals)
>> +    sigwait.__doc__ = _signal.sigwait
>> +
>> +del _globals, _wraps
>> diff --git a/Lib/test/test_doctest.py b/Lib/test/test_doctest.py
>> --- a/Lib/test/test_doctest.py
>> +++ b/Lib/test/test_doctest.py
>> @@ -2897,7 +2897,7 @@
>>
>>  def test_main():
>>      # Check the doctest cases in doctest itself:
>> -    support.run_doctest(doctest, verbosity=True)
>> +    ret = support.run_doctest(doctest, verbosity=True)
>>      # Check the doctest cases defined here:
>>      from test import test_doctest
>>      support.run_doctest(test_doctest, verbosity=True)
>> diff --git a/Lib/test/test_signal.py b/Lib/test/test_signal.py
>> --- a/Lib/test/test_signal.py
>> +++ b/Lib/test/test_signal.py
>> @@ -1,6 +1,7 @@
>>  import unittest
>>  from test import support
>>  from contextlib import closing
>> +import enum
>>  import gc
>>  import pickle
>>  import select
>> @@ -39,6 +40,22 @@
>>          return None
>>
>>
>> +class GenericTests(unittest.TestCase):
>> +
>> +    def test_enums(self):
>> +        for name in dir(signal):
>> +            sig = getattr(signal, name)
>> +            if name in {'SIG_DFL', 'SIG_IGN'}:
>> +                self.assertIsInstance(sig, signal.Handlers)
>> +            elif name in {'SIG_BLOCK', 'SIG_UNBLOCK', 'SIG_SETMASK'}:
>> +                self.assertIsInstance(sig, signal.Sigmasks)
>> +            elif name.startswith('SIG') and not name.startswith('SIG_'):
>> +                self.assertIsInstance(sig, signal.Signals)
>> +            elif name.startswith('CTRL_'):
>> +                self.assertIsInstance(sig, signal.Signals)
>> +                self.assertEqual(sys.platform, "win32")
>> +
>> +
>>  @unittest.skipIf(sys.platform == "win32", "Not valid on Windows")
>>  class InterProcessSignalTests(unittest.TestCase):
>>      MAX_DURATION = 20   # Entire test should last at most 20 sec.
>> @@ -195,6 +212,7 @@
>>
>>      def test_getsignal(self):
>>          hup = signal.signal(signal.SIGHUP, self.trivial_signal_handler)
>> +        self.assertIsInstance(hup, signal.Handlers)
>>          self.assertEqual(signal.getsignal(signal.SIGHUP),
>>                           self.trivial_signal_handler)
>>          signal.signal(signal.SIGHUP, hup)
>> @@ -271,7 +289,7 @@
>>
>>          os.close(read)
>>          os.close(write)
>> -        """.format(signals, ordered, test_body)
>> +        """.format(tuple(map(int, signals)), ordered, test_body)
>>
>>          assert_python_ok('-c', code)
>>
>> @@ -604,6 +622,8 @@
>>              signal.pthread_sigmask(signal.SIG_BLOCK, [signum])
>>              os.kill(os.getpid(), signum)
>>              pending = signal.sigpending()
>> +            for sig in pending:
>> +                assert isinstance(sig, signal.Signals), repr(pending)
>>              if pending != {signum}:
>>                  raise Exception('%s != {%s}' % (pending, signum))
>>              try:
>> @@ -660,6 +680,7 @@
>>          code = '''if 1:
>>          import signal
>>          import sys
>> +        from signal import Signals
>>
>>          def handler(signum, frame):
>>              1/0
>> @@ -702,6 +723,7 @@
>>          def test(signum):
>>              signal.alarm(1)
>>              received = signal.sigwait([signum])
>> +            assert isinstance(received, signal.Signals), received
>>              if received != signum:
>>                  raise Exception('received %s, not %s' % (received,
>> signum))
>>          ''')
>> @@ -842,8 +864,14 @@
>>          def kill(signum):
>>              os.kill(os.getpid(), signum)
>>
>> +        def check_mask(mask):
>> +            for sig in mask:
>> +                assert isinstance(sig, signal.Signals), repr(sig)
>> +
>>          def read_sigmask():
>> -            return signal.pthread_sigmask(signal.SIG_BLOCK, [])
>> +            sigmask = signal.pthread_sigmask(signal.SIG_BLOCK, [])
>> +            check_mask(sigmask)
>> +            return sigmask
>>
>>          signum = signal.SIGUSR1
>>
>> @@ -852,6 +880,7 @@
>>
>>          # Unblock SIGUSR1 (and copy the old mask) to test our signal
>> handler
>>          old_mask = signal.pthread_sigmask(signal.SIG_UNBLOCK, [signum])
>> +        check_mask(old_mask)
>>          try:
>>              kill(signum)
>>          except ZeroDivisionError:
>> @@ -861,11 +890,13 @@
>>
>>          # Block and then raise SIGUSR1. The signal is blocked: the signal
>>          # handler is not called, and the signal is now pending
>> -        signal.pthread_sigmask(signal.SIG_BLOCK, [signum])
>> +        mask = signal.pthread_sigmask(signal.SIG_BLOCK, [signum])
>> +        check_mask(mask)
>>          kill(signum)
>>
>>          # Check the new mask
>>          blocked = read_sigmask()
>> +        check_mask(blocked)
>>          if signum not in blocked:
>>              raise Exception("%s not in %s" % (signum, blocked))
>>          if old_mask ^ blocked != {signum}:
>> @@ -928,7 +959,7 @@
>>
>>  def test_main():
>>      try:
>> -        support.run_unittest(PosixTests, InterProcessSignalTests,
>> +        support.run_unittest(GenericTests, PosixTests,
>> InterProcessSignalTests,
>>                               WakeupFDTests, WakeupSignalTests,
>>                               SiginterruptTest, ItimerTest,
>> WindowsSignalTests,
>>                               PendingSignalsTests)
>> diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c
>> --- a/Modules/signalmodule.c
>> +++ b/Modules/signalmodule.c
>> @@ -967,7 +967,7 @@
>>  };
>>
>>  PyMODINIT_FUNC
>> -PyInit_signal(void)
>> +PyInit__signal(void)
>>  {
>>      PyObject *m, *d, *x;
>>      int i;
>> @@ -1380,7 +1380,7 @@
>>  void
>>  PyOS_InitInterrupts(void)
>>  {
>> -    PyObject *m = PyImport_ImportModule("signal");
>> +    PyObject *m = PyImport_ImportModule("_signal");
>>      if (m) {
>>          Py_DECREF(m);
>>      }
>> diff --git a/PC/config.c b/PC/config.c
>> --- a/PC/config.c
>> +++ b/PC/config.c
>> @@ -19,7 +19,7 @@
>>  extern PyObject* PyInit__md5(void);
>>  extern PyObject* PyInit_nt(void);
>>  extern PyObject* PyInit__operator(void);
>> -extern PyObject* PyInit_signal(void);
>> +extern PyObject* PyInit__signal(void);
>>  extern PyObject* PyInit__sha1(void);
>>  extern PyObject* PyInit__sha256(void);
>>  extern PyObject* PyInit__sha512(void);
>>
>> --
>> Repository URL: http://hg.python.org/cpython
>>
>> _______________________________________________
>> Python-checkins mailing list
>> Python-checkins at python.org
>> https://mail.python.org/mailman/listinfo/python-checkins
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140404/ff31ca9e/attachment-0001.html>

From victor.stinner at gmail.com  Fri Apr  4 16:33:37 2014
From: victor.stinner at gmail.com (Victor Stinner)
Date: Fri, 4 Apr 2014 16:33:37 +0200
Subject: [Python-Dev] [Python-checkins] cpython: fix #21076: turn signal
 module constants into enums
In-Reply-To: <CAP1=2W4wfsa9xtOaL6VdcLpk_vGtBFd-2dd7YGrn8aSpZoChbA@mail.gmail.com>
References: <3g0hvS5h5Wz7LjM@mail.python.org>
 <CAP1=2W7Qif5qhND=VA+rOnt1+fGFM75Rp-K9aN0T3oGZweXMUA@mail.gmail.com>
 <CAP1=2W4wfsa9xtOaL6VdcLpk_vGtBFd-2dd7YGrn8aSpZoChbA@mail.gmail.com>
Message-ID: <CAMpsgwapZaN6hSHt3UMEtu_sEP3i+K24+pWoFVR6T45nkGzLGg@mail.gmail.com>

2014-04-04 16:21 GMT+02:00 Brett Cannon <brett at python.org>:
> Fix is in rev c6e63bb132fb.

Hum, this one was not enough for me. I also modified Modules/Setup.config.in:

changeset:   90137:df5120efb86e
tag:         tip
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Fri Apr 04 16:30:04 2014 +0200
files:       Modules/Setup.config.in
description:
Issue #21076: the C signal module has been renamed to _signal


diff -r c6e63bb132fb -r df5120efb86e Modules/Setup.config.in
--- a/Modules/Setup.config.in   Fri Apr 04 10:20:28 2014 -0400
+++ b/Modules/Setup.config.in   Fri Apr 04 16:30:04 2014 +0200
@@ -7,7 +7,7 @@
 @USE_THREAD_MODULE at _thread _threadmodule.c

 # The signal module
- at USE_SIGNAL_MODULE@signal signalmodule.c
+ at USE_SIGNAL_MODULE@_signal signalmodule.c

 # The rest of the modules previously listed in this file are built
 # by the setup.py script in Python 2.1 and later.

Victor

From bcannon at gmail.com  Fri Apr  4 16:43:56 2014
From: bcannon at gmail.com (Brett Cannon)
Date: Fri, 04 Apr 2014 14:43:56 +0000
Subject: [Python-Dev] [Python-checkins] cpython: fix #21076: turn signal
 module constants into enums
References: <3g0hvS5h5Wz7LjM@mail.python.org>
 <CAP1=2W7Qif5qhND=VA+rOnt1+fGFM75Rp-K9aN0T3oGZweXMUA@mail.gmail.com>
 <CAP1=2W4wfsa9xtOaL6VdcLpk_vGtBFd-2dd7YGrn8aSpZoChbA@mail.gmail.com>
 <CAMpsgwapZaN6hSHt3UMEtu_sEP3i+K24+pWoFVR6T45nkGzLGg@mail.gmail.com>
Message-ID: <CAP1=2W7Jp9ebFbKcqp-Z1OHTptSQiWetgH2nu=zK18gVz+tgow@mail.gmail.com>

On Fri Apr 04 2014 at 10:34:06 AM, Victor Stinner <victor.stinner at gmail.com>
wrote:

> 2014-04-04 16:21 GMT+02:00 Brett Cannon <brett at python.org>:
> > Fix is in rev c6e63bb132fb.
>
> Hum, this one was not enough for me. I also modified Modules/
> Setup.config.in:
>

Wasn't for me either in the end as it failed when I did a distclean. The
unfortunately bit me in the middle of trying to get a 3.4->default merge so
I just tried to do the best I could. This might take a little while to
clean up as the Windows 7 buildbot now builds -- which it has been doing
for a while -- but claims it can't find _signal.sigpending which I can at
least see on OS X so that bit of code my need tweaking.

-Brett


>
> changeset:   90137:df5120efb86e
> tag:         tip
> user:        Victor Stinner <victor.stinner at gmail.com>
> date:        Fri Apr 04 16:30:04 2014 +0200
> files:       Modules/Setup.config.in
> description:
> Issue #21076: the C signal module has been renamed to _signal
>
>
> diff -r c6e63bb132fb -r df5120efb86e Modules/Setup.config.in
> --- a/Modules/Setup.config.in   Fri Apr 04 10:20:28 2014 -0400
> +++ b/Modules/Setup.config.in   Fri Apr 04 16:30:04 2014 +0200
> @@ -7,7 +7,7 @@
>  @USE_THREAD_MODULE at _thread _threadmodule.c
>
>  # The signal module
> - at USE_SIGNAL_MODULE@signal signalmodule.c
> + at USE_SIGNAL_MODULE@_signal signalmodule.c
>
>  # The rest of the modules previously listed in this file are built
>  # by the setup.py script in Python 2.1 and later.
>
> Victor
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140404/cbdcc865/attachment.html>

From g.rodola at gmail.com  Fri Apr  4 17:08:37 2014
From: g.rodola at gmail.com (Giampaolo Rodola')
Date: Fri, 4 Apr 2014 17:08:37 +0200
Subject: [Python-Dev] [Python-checkins] cpython: fix #21076: turn signal
 module constants into enums
In-Reply-To: <CAP1=2W7Jp9ebFbKcqp-Z1OHTptSQiWetgH2nu=zK18gVz+tgow@mail.gmail.com>
References: <3g0hvS5h5Wz7LjM@mail.python.org>
 <CAP1=2W7Qif5qhND=VA+rOnt1+fGFM75Rp-K9aN0T3oGZweXMUA@mail.gmail.com>
 <CAP1=2W4wfsa9xtOaL6VdcLpk_vGtBFd-2dd7YGrn8aSpZoChbA@mail.gmail.com>
 <CAMpsgwapZaN6hSHt3UMEtu_sEP3i+K24+pWoFVR6T45nkGzLGg@mail.gmail.com>
 <CAP1=2W7Jp9ebFbKcqp-Z1OHTptSQiWetgH2nu=zK18gVz+tgow@mail.gmail.com>
Message-ID: <CAFYqXL-E99QjaoDn4uGPMCtSeZb3JxUn5yBENNpfCydfUJnb1w@mail.gmail.com>

Sorry for the troubles. :(
I committed this because it worked on my local copy of Python 3.5 but after
I tried a brand new "hg clone" it didnt.


On Fri, Apr 4, 2014 at 4:43 PM, Brett Cannon <bcannon at gmail.com> wrote:

>
>
> On Fri Apr 04 2014 at 10:34:06 AM, Victor Stinner <
> victor.stinner at gmail.com> wrote:
>
>> 2014-04-04 16:21 GMT+02:00 Brett Cannon <brett at python.org>:
>> > Fix is in rev c6e63bb132fb.
>>
>> Hum, this one was not enough for me. I also modified Modules/
>> Setup.config.in:
>>
>
> Wasn't for me either in the end as it failed when I did a distclean. The
> unfortunately bit me in the middle of trying to get a 3.4->default merge so
> I just tried to do the best I could. This might take a little while to
> clean up as the Windows 7 buildbot now builds -- which it has been doing
> for a while -- but claims it can't find _signal.sigpending which I can at
> least see on OS X so that bit of code my need tweaking.
>
> -Brett
>
>
>>
>> changeset:   90137:df5120efb86e
>> tag:         tip
>> user:        Victor Stinner <victor.stinner at gmail.com>
>> date:        Fri Apr 04 16:30:04 2014 +0200
>> files:       Modules/Setup.config.in
>> description:
>> Issue #21076: the C signal module has been renamed to _signal
>>
>>
>> diff -r c6e63bb132fb -r df5120efb86e Modules/Setup.config.in
>> --- a/Modules/Setup.config.in   Fri Apr 04 10:20:28 2014 -0400
>> +++ b/Modules/Setup.config.in   Fri Apr 04 16:30:04 2014 +0200
>> @@ -7,7 +7,7 @@
>>  @USE_THREAD_MODULE at _thread _threadmodule.c
>>
>>  # The signal module
>> - at USE_SIGNAL_MODULE@signal signalmodule.c
>> + at USE_SIGNAL_MODULE@_signal signalmodule.c
>>
>>  # The rest of the modules previously listed in this file are built
>>  # by the setup.py script in Python 2.1 and later.
>>
>> Victor
>>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com
>
>


-- 
Giampaolo - http://grodola.blogspot.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140404/dde0fdf5/attachment.html>

From phd at phdru.name  Fri Apr  4 17:16:11 2014
From: phd at phdru.name (Oleg Broytman)
Date: Fri, 4 Apr 2014 17:16:11 +0200
Subject: [Python-Dev] [Python-checkins] cpython: fix #21076: turn signal
 module constants into enums
In-Reply-To: <CAFYqXL-E99QjaoDn4uGPMCtSeZb3JxUn5yBENNpfCydfUJnb1w@mail.gmail.com>
References: <3g0hvS5h5Wz7LjM@mail.python.org>
 <CAP1=2W7Qif5qhND=VA+rOnt1+fGFM75Rp-K9aN0T3oGZweXMUA@mail.gmail.com>
 <CAP1=2W4wfsa9xtOaL6VdcLpk_vGtBFd-2dd7YGrn8aSpZoChbA@mail.gmail.com>
 <CAMpsgwapZaN6hSHt3UMEtu_sEP3i+K24+pWoFVR6T45nkGzLGg@mail.gmail.com>
 <CAP1=2W7Jp9ebFbKcqp-Z1OHTptSQiWetgH2nu=zK18gVz+tgow@mail.gmail.com>
 <CAFYqXL-E99QjaoDn4uGPMCtSeZb3JxUn5yBENNpfCydfUJnb1w@mail.gmail.com>
Message-ID: <20140404151611.GA22629@phdru.name>

On Fri, Apr 04, 2014 at 05:08:37PM +0200, Giampaolo Rodola' <g.rodola at gmail.com> wrote:
> Sorry for the troubles. :(
> I committed this because it worked on my local copy of Python 3.5 but after
> I tried a brand new "hg clone" it didnt.

   make distclean was not enough?

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From techtonik at gmail.com  Fri Apr  4 17:21:07 2014
From: techtonik at gmail.com (anatoly techtonik)
Date: Fri, 4 Apr 2014 18:21:07 +0300
Subject: [Python-Dev] New absolute __file__ in Python 3.4
Message-ID: <CAPkN8xJtVdrC1=idg=QSwTLoOVJ0s31YgAaYrGghA4tk+SCUjQ@mail.gmail.com>

https://docs.python.org/3.4/whatsnew/3.4.html#other-language-changes

1. Is this absolute name with symlinks resolved?
2. Why there is a special case for __main__?
(i.e. Special cases aren't special enough to break the rules.)
3. What link should I click in Python reference to read
    about standard globals like __file__ and this change?
    https://docs.python.org/3.4/library/index.html
    something like this in PHP
    http://www.php.net/manual/en/language.constants.predefined.php
-- 
anatoly t.

From status at bugs.python.org  Fri Apr  4 18:07:41 2014
From: status at bugs.python.org (Python tracker)
Date: Fri,  4 Apr 2014 18:07:41 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20140404160741.CF77456A1A@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2014-03-28 - 2014-04-04)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    4550 (+16)
  closed 28376 (+52)
  total  32926 (+68)

Open issues with patches: 2093 


Issues opened (48)
==================

#21087: imp.frozen_init() incorrectly removes module during reload
http://bugs.python.org/issue21087  opened by eric.snow

#21088: curses addch() argument position reverses in Python3.4.0
http://bugs.python.org/issue21088  opened by masamoto

#21090: File read silently stops after EIO I/O error
http://bugs.python.org/issue21090  opened by ivank

#21091: EmailMessage.is_attachment should be a method
http://bugs.python.org/issue21091  opened by brandon-rhodes

#21095: EmailMessage should support Header objects
http://bugs.python.org/issue21095  opened by brandon-rhodes

#21099: Switch applicable importlib tests to use PEP 451 API
http://bugs.python.org/issue21099  opened by eric.snow

#21101: Extend the PyDict C API to handle cases where the hash value i
http://bugs.python.org/issue21101  opened by rhettinger

#21103: Encoding str to IDNA with ellipsis decomposes to empty labels
http://bugs.python.org/issue21103  opened by chfoo

#21106: Updated Mac folder icon
http://bugs.python.org/issue21106  opened by viveksjain

#21107: Add pgen.vcxproj to allow regenerating grammar files on Window
http://bugs.python.org/issue21107  opened by zach.ware

#21108: Add examples for importlib in doc
http://bugs.python.org/issue21108  opened by sahutd

#21109: tarfile: Traversal attack vulnerability
http://bugs.python.org/issue21109  opened by Daniel.Garcia

#21110: Slowdown and high memory usage when adding a new module in emb
http://bugs.python.org/issue21110  opened by MrValdez

#21111: PyLong_AsUnsignedLongAndOverflow does not exist
http://bugs.python.org/issue21111  opened by h.venev

#21112: 3.4 regression: unittest.expectedFailure no longer works on Te
http://bugs.python.org/issue21112  opened by William.Schwartz

#21114: wsgiref.simple_server doesn't handle multi-line headers correc
http://bugs.python.org/issue21114  opened by Alan.Braithwaite

#21116: Failure to create multiprocessing shared arrays larger than 50
http://bugs.python.org/issue21116  opened by mboquien

#21117: inspect.signature: inaccuracies for partial functions
http://bugs.python.org/issue21117  opened by yselivanov

#21118: str.translate is absurdly slow in majority of use cases (takes
http://bugs.python.org/issue21118  opened by josh.rosenberg

#21119: asyncio create_connection resource warning
http://bugs.python.org/issue21119  opened by landersson

#21120: PyArena type is used in headers from the limited API
http://bugs.python.org/issue21120  opened by aponomarenko

#21121: -Werror=declaration-after-statement is added even for extensio
http://bugs.python.org/issue21121  opened by nilsge

#21122: CPython fails to build modules with LLVM LTO on Mac OS X
http://bugs.python.org/issue21122  opened by Sjlver

#21124: _struct module compilation error under Cygwin 1.7.17 on Python
http://bugs.python.org/issue21124  opened by dellair.jie

#21126: Integrate doctest.run_docstring_examples() with unittest
http://bugs.python.org/issue21126  opened by giampaolo.rodola

#21127: Path objects cannot be constructed from str subclasses
http://bugs.python.org/issue21127  opened by Antony.Lee

#21128: testing stdlib and compatibility with pypy
http://bugs.python.org/issue21128  opened by mattip

#21130: equivalent functools.partial instances should compare equal
http://bugs.python.org/issue21130  opened by theller

#21131: test_faulthandler.test_register_chain fails on 64bit ppc/arm w
http://bugs.python.org/issue21131  opened by bkabrda

#21132: Failure to import win32api
http://bugs.python.org/issue21132  opened by woakesd

#21133: unittest discover should allow option to run each package sepa
http://bugs.python.org/issue21133  opened by the.mulhern

#21134: Segfault when stringifying UnicodeEncodeError (or UnicodeDecod
http://bugs.python.org/issue21134  opened by Marek.Kowalski

#21136: fractions.Fraction.__pow__ does unneeded renormalization
http://bugs.python.org/issue21136  opened by Orborde

#21137: Better repr for threading.Lock()
http://bugs.python.org/issue21137  opened by rhettinger

#21138: mimetypes.MimeType UnicodeDecodeError
http://bugs.python.org/issue21138  opened by tanbro

#21139: Idle: change default reformat width from 70 to 72
http://bugs.python.org/issue21139  opened by terry.reedy

#21140: Idle: saving an OutputWindow should default to .txt
http://bugs.python.org/issue21140  opened by terry.reedy

#21141: Don't mention Perl in Windows build output
http://bugs.python.org/issue21141  opened by zach.ware

#21142: Daily/weekly ABI scan?
http://bugs.python.org/issue21142  opened by ncoghlan

#21144: ensurepip "AssertionError: Multiple .dist-info directories"
http://bugs.python.org/issue21144  opened by demmeln

#21145: Add the @cached_property decorator
http://bugs.python.org/issue21145  opened by Omer.Katz

#21146: update gzip usage examples in docs
http://bugs.python.org/issue21146  opened by wolma

#21147: sqlite3 doesn't complain if the request contains a null charac
http://bugs.python.org/issue21147  opened by haypo

#21148: avoid memset in small tuple creation
http://bugs.python.org/issue21148  opened by jtaylor

#21150: Add quick links table to argparse docs
http://bugs.python.org/issue21150  opened by rhettinger

#21151: winreg.SetValueEx causes crash if value = None
http://bugs.python.org/issue21151  opened by dmo2118

#21152: Idle: timed autosave for shell (and maybe editor) window
http://bugs.python.org/issue21152  opened by terry.reedy

#21153: bdist_rpm fails if project contains files with spaces in the n
http://bugs.python.org/issue21153  opened by jason.coombs



Most recent 15 issues with no replies (15)
==========================================

#21152: Idle: timed autosave for shell (and maybe editor) window
http://bugs.python.org/issue21152

#21150: Add quick links table to argparse docs
http://bugs.python.org/issue21150

#21147: sqlite3 doesn't complain if the request contains a null charac
http://bugs.python.org/issue21147

#21144: ensurepip "AssertionError: Multiple .dist-info directories"
http://bugs.python.org/issue21144

#21139: Idle: change default reformat width from 70 to 72
http://bugs.python.org/issue21139

#21133: unittest discover should allow option to run each package sepa
http://bugs.python.org/issue21133

#21132: Failure to import win32api
http://bugs.python.org/issue21132

#21127: Path objects cannot be constructed from str subclasses
http://bugs.python.org/issue21127

#21126: Integrate doctest.run_docstring_examples() with unittest
http://bugs.python.org/issue21126

#21120: PyArena type is used in headers from the limited API
http://bugs.python.org/issue21120

#21119: asyncio create_connection resource warning
http://bugs.python.org/issue21119

#21114: wsgiref.simple_server doesn't handle multi-line headers correc
http://bugs.python.org/issue21114

#21112: 3.4 regression: unittest.expectedFailure no longer works on Te
http://bugs.python.org/issue21112

#21110: Slowdown and high memory usage when adding a new module in emb
http://bugs.python.org/issue21110

#21107: Add pgen.vcxproj to allow regenerating grammar files on Window
http://bugs.python.org/issue21107



Most recent 15 issues waiting for review (15)
=============================================

#21148: avoid memset in small tuple creation
http://bugs.python.org/issue21148

#21141: Don't mention Perl in Windows build output
http://bugs.python.org/issue21141

#21140: Idle: saving an OutputWindow should default to .txt
http://bugs.python.org/issue21140

#21137: Better repr for threading.Lock()
http://bugs.python.org/issue21137

#21136: fractions.Fraction.__pow__ does unneeded renormalization
http://bugs.python.org/issue21136

#21128: testing stdlib and compatibility with pypy
http://bugs.python.org/issue21128

#21126: Integrate doctest.run_docstring_examples() with unittest
http://bugs.python.org/issue21126

#21124: _struct module compilation error under Cygwin 1.7.17 on Python
http://bugs.python.org/issue21124

#21118: str.translate is absurdly slow in majority of use cases (takes
http://bugs.python.org/issue21118

#21117: inspect.signature: inaccuracies for partial functions
http://bugs.python.org/issue21117

#21116: Failure to create multiprocessing shared arrays larger than 50
http://bugs.python.org/issue21116

#21114: wsgiref.simple_server doesn't handle multi-line headers correc
http://bugs.python.org/issue21114

#21112: 3.4 regression: unittest.expectedFailure no longer works on Te
http://bugs.python.org/issue21112

#21109: tarfile: Traversal attack vulnerability
http://bugs.python.org/issue21109

#21108: Add examples for importlib in doc
http://bugs.python.org/issue21108



Top 10 most discussed issues (10)
=================================

#21117: inspect.signature: inaccuracies for partial functions
http://bugs.python.org/issue21117  21 msgs

#21085: compile error Python3.3 on Cygwin
http://bugs.python.org/issue21085  16 msgs

#21116: Failure to create multiprocessing shared arrays larger than 50
http://bugs.python.org/issue21116  12 msgs

#20904: HAVE_PY_SET_53BIT_PRECISION for m68k
http://bugs.python.org/issue20904  11 msgs

#17390: display python version on idle title bar
http://bugs.python.org/issue17390   8 msgs

#21106: Updated Mac folder icon
http://bugs.python.org/issue21106   8 msgs

#21122: CPython fails to build modules with LLVM LTO on Mac OS X
http://bugs.python.org/issue21122   8 msgs

#19655: Replace the ASDL parser carried with CPython
http://bugs.python.org/issue19655   7 msgs

#21028: ElementTree objects should support all the same methods as Ele
http://bugs.python.org/issue21028   7 msgs

#21076: Turn signal.SIG* constants into enums
http://bugs.python.org/issue21076   7 msgs



Issues closed (50)
==================

#8410: Fix emulated lock to be 'fair'
http://bugs.python.org/issue8410  closed by kristjan.jonsson

#11704: functools.partial doesn't create bound methods
http://bugs.python.org/issue11704  closed by ncoghlan

#11824: freeze.py broken due to ABI flags
http://bugs.python.org/issue11824  closed by loewis

#15044: _dbm not building on Fedora 17
http://bugs.python.org/issue15044  closed by ncoghlan

#15067: Clean up the sqlite3 docs
http://bugs.python.org/issue15067  closed by python-dev

#15331: Codecs docs should explain that the bytes-bytes shorthand alia
http://bugs.python.org/issue15331  closed by ncoghlan

#16475: Support object instancing and recursion in marshal
http://bugs.python.org/issue16475  closed by kristjan.jonsson

#16716: Deprecate OSError aliases in the doc
http://bugs.python.org/issue16716  closed by asvetlov

#17522: Add api PyGILState_Check
http://bugs.python.org/issue17522  closed by kristjan.jonsson

#17705: Fill Character cannot be \0
http://bugs.python.org/issue17705  closed by skrah

#17862: itertools.chunks(iterable, size, fill=None)
http://bugs.python.org/issue17862  closed by rhettinger

#17969: multiprocessing crash on exit
http://bugs.python.org/issue17969  closed by kristjan.jonsson

#18652: Add itertools.first_true (return first true item in iterable)
http://bugs.python.org/issue18652  closed by rhettinger

#19491: Python Crashing When Saving Documents
http://bugs.python.org/issue19491  closed by terry.reedy

#19501: Buildbot testing of 3.2 broken
http://bugs.python.org/issue19501  closed by zach.ware

#19505: OrderedDict views don't implement __reversed__
http://bugs.python.org/issue19505  closed by serhiy.storchaka

#19640: Dynamically generate the _source attribute of namedtuple to sa
http://bugs.python.org/issue19640  closed by rhettinger

#19697: Document the possible values for __main__.__spec__
http://bugs.python.org/issue19697  closed by eric.snow

#20022: Eliminate the use of the deprecated bundlebuilder.py script in
http://bugs.python.org/issue20022  closed by ned.deily

#20375: ElementTree: Document handling processing instructions
http://bugs.python.org/issue20375  closed by nikratio

#20550: Use specific asserts in collections tests
http://bugs.python.org/issue20550  closed by rhettinger

#20887: stdlib compatibility with pypy, test_zipfile.py
http://bugs.python.org/issue20887  closed by benjamin.peterson

#20942: _frozen_importlib should not have a __file__ attribute
http://bugs.python.org/issue20942  closed by brett.cannon

#21014: `1` => `True`; for tutorial docs
http://bugs.python.org/issue21014  closed by rhettinger

#21029: IDLE mis-coloring "print"
http://bugs.python.org/issue21029  closed by rhettinger

#21066: The separate download version for the documentation doesn't wo
http://bugs.python.org/issue21066  closed by loewis

#21073: Py_ReprEnter potentially misbehaves during malformed thread st
http://bugs.python.org/issue21073  closed by pitrou

#21074: Too aggressive constant folding
http://bugs.python.org/issue21074  closed by ncoghlan

#21082: os.makedirs(exist_ok=True) is not thread-safe: umask is set te
http://bugs.python.org/issue21082  closed by python-dev

#21086: Source with "# -*- coding: ASCII -*-" and UNIX EOL (\n) result
http://bugs.python.org/issue21086  closed by Arfrever

#21089: macro SET_SYS_FROM_STRING_BORROW doesn't get unset
http://bugs.python.org/issue21089  closed by python-dev

#21092: json serializer implicitly stringifies non-string keys
http://bugs.python.org/issue21092  closed by brett.cannon

#21093: ctypes test_macholib fails if libz is not installed in /usr/li
http://bugs.python.org/issue21093  closed by ned.deily

#21094: sqlite3 CheckOpcodeCount fails with AssertionError: 5 not grea
http://bugs.python.org/issue21094  closed by benjamin.peterson

#21096: Python icon hardcoded
http://bugs.python.org/issue21096  closed by ned.deily

#21097: Move test_namespace_pkgs into test_importlib.
http://bugs.python.org/issue21097  closed by eric.snow

#21098: Address remaining PEP 451-related to-do comments.
http://bugs.python.org/issue21098  closed by eric.snow

#21100: Micro-optimization for tuple comparison
http://bugs.python.org/issue21100  closed by rhettinger

#21102: 3.4.0 PDF (a4,letter) and EPUB documentation missing
http://bugs.python.org/issue21102  closed by benjamin.peterson

#21104: Read from file aborted
http://bugs.python.org/issue21104  closed by Alex.Grinko

#21105: functools.partialmethod example doesn't actually work
http://bugs.python.org/issue21105  closed by python-dev

#21113: Error usage of class.__bases__
http://bugs.python.org/issue21113  closed by ZealotuS

#21115: python.org server change causing failures in test_urllib2net t
http://bugs.python.org/issue21115  closed by benjamin.peterson

#21123: Compilation error on _struct module on Python 3.4
http://bugs.python.org/issue21123  closed by dellair.jie

#21125: traceback.extract_tb says "quadruple" when it means "tuple"
http://bugs.python.org/issue21125  closed by rhettinger

#21129: Segfault in python interpreter
http://bugs.python.org/issue21129  closed by zach.ware

#21135: Remove useless _vb_pattern parameter in cgi.valid_boundary
http://bugs.python.org/issue21135  closed by python-dev

#21143: Copy-paste error in documentation of __builtin__.max
http://bugs.python.org/issue21143  closed by rhettinger

#21149: logging._removeHandlerRef is not threadsafe during interpreter
http://bugs.python.org/issue21149  closed by python-dev

#21154: Small fix in 2.7.6 lang ref
http://bugs.python.org/issue21154  closed by python-dev

From tjreedy at udel.edu  Fri Apr  4 21:26:49 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 04 Apr 2014 15:26:49 -0400
Subject: [Python-Dev] New absolute __file__ in Python 3.4
In-Reply-To: <CAPkN8xJtVdrC1=idg=QSwTLoOVJ0s31YgAaYrGghA4tk+SCUjQ@mail.gmail.com>
References: <CAPkN8xJtVdrC1=idg=QSwTLoOVJ0s31YgAaYrGghA4tk+SCUjQ@mail.gmail.com>
Message-ID: <lhn120$cas$1@ger.gmane.org>

On 4/4/2014 11:21 AM, anatoly techtonik wrote:
> https://docs.python.org/3.4/whatsnew/3.4.html#other-language-changes
>
> 1. Is this absolute name with symlinks resolved?
> 2. Why there is a special case for __main__?
> (i.e. Special cases aren't special enough to break the rules.)

Did you read the linked issue?
http://bugs.python.org/issue18416

> 3. What link should I click in Python reference to read
>      about standard globals like __file__ and this change?
>      https://docs.python.org/3.4/library/index.html

I don't know how you got that link. Learn to use the general index 
listed on the contents page and top right of every page. For underscore 
names, https://docs.python.org/3/genindex-_.html. __file__ will lead you 
to the module entry in Chapter 3 Data model, section 2 The standard type 
hierarchy. Looking up module/namespace gets you to the same place. It is 
possible that the discussion of '__file__' should be slightly revised in 
light of the What's new entry.

-- 
Terry Jan Reedy


From kmod at dropbox.com  Fri Apr  4 22:54:04 2014
From: kmod at dropbox.com (Kevin Modzelewski)
Date: Fri, 4 Apr 2014 13:54:04 -0700
Subject: [Python-Dev] Pyston: a Python JIT on LLVM
In-Reply-To: <212815941418257280.135044sturla.molden-gmail.com@news.gmane.org>
References: <CAO=oM6s92ee0oeChdpmtNvmb4Av=iKGoxVdXGvKodYroMYQavQ@mail.gmail.com>
 <212815941418257280.135044sturla.molden-gmail.com@news.gmane.org>
Message-ID: <CAO=oM6s3NHF1XUEdRhEBLwQgACpFZxtOJYv27EGm9h_EOvQWGw@mail.gmail.com>

Using optional type annotations is a really promising strategy and may
eventually be added to Pyston, but our primary target right now is
unmodified and untyped Python code.  I think there's room for both
approaches -- I think the "using type annotations to achieve near-native
performance" can be very useful ex. in a numerical computing context, but
might not apply as well to a "large web application" case.

On Thu, Apr 3, 2014 at 3:42 PM, Sturla Molden <sturla.molden at gmail.com>wrote:

> Kevin Modzelewski <kmod at dropbox.com> wrote:
>
> > Since it's the question that I think most people will inevitably (and
> > rightly) ask, why do we think there's a place for Pyston when there's
> PyPy
> > and (previously) Unladen Swallow?
>
> Have you seen Numba, the Python JIT that integrates with NumPy?
>
> http://numba.pydata.org
>
> It uses LLVM to compile Python bytecode. When I have tried it I tend to get
> speed comparable to -O2 in C for numerical and algorithmic code.
>
> Here is an example, giving a 150 times speed boost to Python:
>
>
> http://stackoverflow.com/questions/21811381/how-to-shove-this-loop-into-numpy/21818591#21818591
>
>
> Sturla
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/kmod%40dropbox.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140404/030a7e1e/attachment-0001.html>

From kmod at dropbox.com  Fri Apr  4 23:57:32 2014
From: kmod at dropbox.com (Kevin Modzelewski)
Date: Fri, 4 Apr 2014 14:57:32 -0700
Subject: [Python-Dev] Pyston: a Python JIT on LLVM
In-Reply-To: <lhls8q$qf9$1@ger.gmane.org>
References: <CAO=oM6s92ee0oeChdpmtNvmb4Av=iKGoxVdXGvKodYroMYQavQ@mail.gmail.com>
 <lhls8q$qf9$1@ger.gmane.org>
Message-ID: <CAO=oM6vqXAxN9f8Yai7wBsnxdH=pv72jNJmoPyg3iay3BQRywQ@mail.gmail.com>

On Fri, Apr 4, 2014 at 1:59 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:

>
> I'm a bit surprised by the approach. Why don't you simply process CPython
> bytecode, rather than strive to reimplement Python fully?
>

The original choice to operate on Python AST rather than bytecode was made
somewhat arbitrarily, but I think I still support that decision since it
avoids having to do a potentially performance-degrading translation between
a stack language and a register language.  It means we lose the ability to
execute pyc-only distributions, but I suppose that support for that could
be added if it becomes important.

As for why we're building our own runtime as part of all of this (which I
think is what you're getting at), I think that a lot of the performance of
an implementation is caught up in the runtime and isn't just about the AST-
or bytecode- execution.  There are a couple systems out there that will
compile Python to C modules, where all the behavior is implemented using
calls back through the C API.  I haven't tried them, but I'd suspect that
without type information, the gains from doing this aren't that great,
since while you can get rid of bytecode dispatch and perhaps get a better
view of control flow, it doesn't address anything about the dynamic nature
of Python.  For example, in Pyston the fast path for an instance attribute
lookup (with no __getattribute__) will be just two loads: one to lookup the
attribute array and one to load the appropriate offset.  I'd say that it's
necessary to have a different runtime to support that, because it has to be
cooperative and 1) use a different object representation everywhere and 2)
know how to backpatch attribute-lookups to fully take advantage of it.
 That said, we definitely try to not reimplement something if we don't need
to.


> Also, I wonder if it's worthwhile to use a conservative GC, rather than
> reuse the original refcounting scheme (especially since you want to support
> existing extension modules).


I wonder that too :)  The only way to know for sure will be to get it
working on real code, but I feel comfortable with the approach since I
trust that everyone else using GCs are happy and for a reason, and I think
it's possible any GC-related advantages can mask the related
extension-compatibility cost.

I was pretty happy when we switched from refcounting to a tracing GC; the
refcounting was already somewhat optimized (didn't emit any
obviously-duplicate increfs/decrefs), but removing the refcounting
operations opened up a number of other optimizations.  As a simple example,
when refcounting you can't typically do tail call elimination because you
have to do some decrefs() at the end of the function, and those decrefs
will also typically keep the variables live even if they didn't otherwise
need to be.  It was also very hard to tell that certain operations were
no-ops, since even if something is a no-op at the Python level, it can
still do a bunch of refcounting.  You can (and we almost did) write an
optimizer to try to match up all the incref's and decref's, but then you
have to start proving that certain variables remain the same after a
function call, etc....  I'm sure it's possible, but using a GC instead made
all of these optimizations much more natural.



Pyston is definitely far on one side of the effort-vs-potential-payoff
spectrum, and it's certainly fair to say that there are other approaches
that would be less work to implement.  I think that with the wealth of very
impressive existing options, though, it makes sense to take the risky path
and to shoot very high, and I'm fortunate to be in a situation where we can
make a risky bet like that.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140404/b88b889e/attachment.html>

From sturla.molden at gmail.com  Sat Apr  5 23:15:28 2014
From: sturla.molden at gmail.com (Sturla Molden)
Date: Sat, 5 Apr 2014 21:15:28 +0000 (UTC)
Subject: [Python-Dev] Pyston: a Python JIT on LLVM
References: <CAO=oM6s92ee0oeChdpmtNvmb4Av=iKGoxVdXGvKodYroMYQavQ@mail.gmail.com>
 <212815941418257280.135044sturla.molden-gmail.com@news.gmane.org>
 <CAO=oM6s3NHF1XUEdRhEBLwQgACpFZxtOJYv27EGm9h_EOvQWGw@mail.gmail.com>
Message-ID: <1910254775418424738.164225sturla.molden-gmail.com@news.gmane.org>

Kevin Modzelewski <kmod at dropbox.com> wrote:
> Using optional type annotations is a really promising strategy and may
> eventually be added to Pyston, but our primary target right now is
> unmodified and untyped Python code

What I meant to say is that Numba already has done the boiler-plate coding.
Even if you use no type annotations, it is already a Python bytecode
JIT-compiler based on LLVM that is hooked up with CPython. You might have
to add optimizations to it, yes, but it has the skeleton for a CPython
LLVM-based JIT compiler set up and running.

If you provide no type annotations, Numba's autojit decorator will do a
data-guided specialization. The types will be inferred from running the
code through the CPython interpreter, and then Numba will generate a
specialization. This is somewhat similar to the information-gathering that
GCC does when we run profile-guided optimizations.

Sturla


From benjamin at python.org  Sun Apr  6 20:35:35 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 06 Apr 2014 11:35:35 -0700
Subject: [Python-Dev] [Python-checkins] cpython (3.3): minor docfix
 (reported at docs@python.org) trace.py argument is --count not
In-Reply-To: <3g22kC1YG5z7LjT@mail.python.org>
References: <3g22kC1YG5z7LjT@mail.python.org>
Message-ID: <1396809335.22219.103363933.6CF1B945@webmail.messagingengine.com>

On Sun, Apr 6, 2014, at 11:01, senthil.kumaran wrote:
> http://hg.python.org/cpython/rev/b49d990aaa9d
> changeset:   90160:b49d990aaa9d
> branch:      3.3
> parent:      90130:63e6afd3ff1f
> user:        Senthil Kumaran <senthil at uthcode.com>
> date:        Sun Apr 06 10:59:47 2014 -0700
> summary:
>   minor docfix (reported at docs at python.org) trace.py argument is --count
>   not --counts

This change is of course not harmful, but remember that 3.3 is in
security-fix only mode.

From njs at pobox.com  Mon Apr  7 03:41:29 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Mon, 7 Apr 2014 02:41:29 +0100
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
Message-ID: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>

Hi all,

I've just finished updating PEP 465 with resolutions to the various
issues that were raised during the python-ideas thread about it. (The
main changes since that thread are that @@ has been removed, and we
now definitely propose that @ have the same precedence and
associativity as *.)

I believe this leaves only one open question, which is where exactly
to stick the new matmul slots into PyTypeObject. This is the kind of
fiddly detail that can easily be settled later if the PEP is accepted,
though.

So, I guess as far as I'm concerned, this is ready to go. Feedback welcome:
  http://legacy.python.org/dev/peps/pep-0465/

-n

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

From benjamin at python.org  Mon Apr  7 05:13:26 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 06 Apr 2014 20:13:26 -0700
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
 multiplication
In-Reply-To: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
Message-ID: <1396840406.21603.103475673.36331ED6@webmail.messagingengine.com>

On Sun, Apr 6, 2014, at 18:41, Nathaniel Smith wrote:
> Hi all,
> 
> I've just finished updating PEP 465 with resolutions to the various
> issues that were raised during the python-ideas thread about it. (The
> main changes since that thread are that @@ has been removed, and we
> now definitely propose that @ have the same precedence and
> associativity as *.)
> 
> I believe this leaves only one open question, which is where exactly
> to stick the new matmul slots into PyTypeObject. This is the kind of
> fiddly detail that can easily be settled later if the PEP is accepted,
> though.

I don't see what it shouldn't be in PyNumberMethods. Surely, we're not
going to get a flood of requests for more matrix operators, are we? :)

From victor.stinner at gmail.com  Mon Apr  7 11:37:13 2014
From: victor.stinner at gmail.com (Victor Stinner)
Date: Mon, 7 Apr 2014 11:37:13 +0200
Subject: [Python-Dev] Windows buildbots are red: test_idlelib
Message-ID: <CAMpsgwbatGMsXqET_8FZH9qST9+NUdW+TQJY6QhdAXmWhbZzQQ@mail.gmail.com>

Hi,

Unit tests are failing on Windows because of this issue:
http://bugs.python.org/issue21059

It looks like a regression in test_idlelib introduced with this issue:
http://bugs.python.org/issue15968

Zachary Ware wrote a fix:
http://bugs.python.org/issue20035

Can someone please review Zachary's patch? If not, I suggest to revert
changes of issue #15968 to have working Windows buildbots.

Victor

From nombizile at gmail.com  Mon Apr  7 09:15:48 2014
From: nombizile at gmail.com (Kells Pablo)
Date: Mon, 7 Apr 2014 09:15:48 +0200
Subject: [Python-Dev] Python-Dev Digest, Vol 129, Issue 6
In-Reply-To: <mailman.45731.1396621327.18129.python-dev@python.org>
References: <mailman.45731.1396621327.18129.python-dev@python.org>
Message-ID: <CAE0cYs7X8PN=PhoN9JUSLG_3HHFs0ohp74jGARqUqroA_O2Now@mail.gmail.com>

HELLO...

!thank you for all the cooperation and emails send. i would like that you
now stop sending them..

thank you in advance

On Fri, Apr 4, 2014 at 4:22 PM, <python-dev-request at python.org> wrote:

> Send Python-Dev mailing list submissions to
>         python-dev at python.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         https://mail.python.org/mailman/listinfo/python-dev
> or, via email, send a message with subject or body 'help' to
>         python-dev-request at python.org
>
> You can reach the person managing the list at
>         python-dev-owner at python.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Python-Dev digest..."
>
>
> Today's Topics:
>
>    1. Re: Pyston: a Python JIT on LLVM (Dag Sverre Seljebotn)
>    2. Re: [Python-checkins] cpython: fix #21076: turn signal module
>       constants into enums (Brett Cannon)
>    3. Re: [Python-checkins] cpython: fix #21076: turn signal module
>       constants into enums (Brett Cannon)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Fri, 04 Apr 2014 12:13:53 +0200
> From: Dag Sverre Seljebotn <d.s.seljebotn at astro.uio.no>
> To: python-dev at python.org, kmod at dropbox.com
> Subject: Re: [Python-Dev] Pyston: a Python JIT on LLVM
> Message-ID: <533E85E1.7050001 at astro.uio.no>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> On 04/04/2014 12:42 AM, Sturla Molden wrote:
> > Kevin Modzelewski <kmod at dropbox.com> wrote:
> >
> >> Since it's the question that I think most people will inevitably (and
> >> rightly) ask, why do we think there's a place for Pyston when there's
> PyPy
> >> and (previously) Unladen Swallow?
> >
> > Have you seen Numba, the Python JIT that integrates with NumPy?
> >
> > http://numba.pydata.org
>
> Specifically, Numba compiles to LLVM too, and tries to be somewhat
> general-purpose although it's tuned to numerical code. And their reason
> for not using PyPy is the same: C extensions. So while their "market
> segment" is different from yours, the technology may not be.
>
> Dag Sverre
>
>
> >
> > It uses LLVM to compile Python bytecode. When I have tried it I tend to
> get
> > speed comparable to -O2 in C for numerical and algorithmic code.
> >
> > Here is an example, giving a 150 times speed boost to Python:
> >
> >
> http://stackoverflow.com/questions/21811381/how-to-shove-this-loop-into-numpy/21818591#21818591
> >
> >
> > Sturla
> >
> > _______________________________________________
> > Python-Dev mailing list
> > Python-Dev at python.org
> > https://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/d.s.seljebotn%40astro.uio.no
> >
>
>
>
> ------------------------------
>
> Message: 2
> Date: Fri, 4 Apr 2014 10:12:23 -0400
> From: Brett Cannon <brett at python.org>
> To: python-dev <python-dev at python.org>
> Cc: python-checkins <python-checkins at python.org>
> Subject: Re: [Python-Dev] [Python-checkins] cpython: fix #21076: turn
>         signal module constants into enums
> Message-ID:
>         <CAP1=2W7Qif5qhND=
> VA+rOnt1+fGFM75Rp-K9aN0T3oGZweXMUA at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> This broke compilation on at least OS X, but I'm willing to bet for all
> UNIX-based systems. I have a fix in the works.
>
>
> On Fri, Apr 4, 2014 at 9:34 AM, giampaolo.rodola <
> python-checkins at python.org
> > wrote:
>
> > http://hg.python.org/cpython/rev/c9239171e429
> > changeset:   90128:c9239171e429
> > user:        Giampaolo Rodola' <g.rodola at gmail.com>
> > date:        Fri Apr 04 15:34:17 2014 +0200
> > summary:
> >   fix #21076: turn signal module constants into enums
> >
> > files:
> >   Doc/library/signal.rst   |  10 +++
> >   Doc/whatsnew/3.5.rst     |   5 +
> >   Lib/signal.py            |  84 ++++++++++++++++++++++++++++
> >   Lib/test/test_doctest.py |   2 +-
> >   Lib/test/test_signal.py  |  39 +++++++++++-
> >   Modules/signalmodule.c   |   4 +-
> >   PC/config.c              |   2 +-
> >   7 files changed, 138 insertions(+), 8 deletions(-)
> >
> >
> > diff --git a/Doc/library/signal.rst b/Doc/library/signal.rst
> > --- a/Doc/library/signal.rst
> > +++ b/Doc/library/signal.rst
> > @@ -65,6 +65,16 @@
> >  Module contents
> >  ---------------
> >
> > +.. versionchanged:: 3.5
> > +   signal (SIG*), handler (:const:`SIG_DFL`, :const:`SIG_IGN`) and
> sigmask
> > +   (:const:`SIG_BLOCK`, :const:`SIG_UNBLOCK`, :const:`SIG_SETMASK`)
> > +   related constants listed below were turned into
> > +   :class:`enums <enum.IntEnum>`.
> > +   :func:`getsignal`, :func:`pthread_sigmask`, :func:`sigpending` and
> > +   :func:`sigwait` functions return human-readable
> > +   :class:`enums <enum.IntEnum>`.
> > +
> > +
> >  The variables defined in the :mod:`signal` module are:
> >
> >
> > diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst
> > --- a/Doc/whatsnew/3.5.rst
> > +++ b/Doc/whatsnew/3.5.rst
> > @@ -134,6 +134,11 @@
> >  Improved Modules
> >  ================
> >
> > +* Different constants of :mod:`signal` module are now enumeration values
> > using
> > +  the :mod:`enum` module. This allows meaningful names to be printed
> > during
> > +  debugging, instead of integer ?magic numbers?. (contribute by
> Giampaolo
> > +  Rodola' in :issue:`21076`)
> > +
> >  * :class:`xmlrpc.client.ServerProxy` is now a :term:`context manager`
> >    (contributed by Claudiu Popa in :issue:`20627`).
> >
> > diff --git a/Lib/signal.py b/Lib/signal.py
> > new file mode 100644
> > --- /dev/null
> > +++ b/Lib/signal.py
> > @@ -0,0 +1,84 @@
> > +import _signal
> > +from _signal import *
> > +from functools import wraps as _wraps
> > +from enum import IntEnum as _IntEnum
> > +
> > +_globals = globals()
> > +
> > +Signals = _IntEnum(
> > +    'Signals',
> > +    {name: value for name, value in _globals.items()
> > +     if name.isupper()
> > +        and (name.startswith('SIG') and not name.startswith('SIG_'))
> > +        or name.startswith('CTRL_')})
> > +
> > +class Handlers(_IntEnum):
> > +    SIG_DFL = _signal.SIG_DFL
> > +    SIG_IGN = _signal.SIG_IGN
> > +
> > +_globals.update(Signals.__members__)
> > +_globals.update(Handlers.__members__)
> > +
> > +if 'pthread_sigmask' in _globals:
> > +    class Sigmasks(_IntEnum):
> > +        SIG_BLOCK = _signal.SIG_BLOCK
> > +        SIG_UNBLOCK = _signal.SIG_UNBLOCK
> > +        SIG_SETMASK = _signal.SIG_SETMASK
> > +
> > +    _globals.update(Sigmasks.__members__)
> > +
> > +
> > +def _int_to_enum(value, enum_klass):
> > +    """Convert a numeric value to an IntEnum member.
> > +    If it's not a known member, return the numeric value itself.
> > +    """
> > +    try:
> > +        return enum_klass(value)
> > +    except ValueError:
> > +        return value
> > +
> > +
> > +def _enum_to_int(value):
> > +    """Convert an IntEnum member to a numeric value.
> > +    If it's not a IntEnum member return the value itself.
> > +    """
> > +    try:
> > +        return int(value)
> > +    except (ValueError, TypeError):
> > +        return value
> > +
> > +
> > + at _wraps(_signal.signal)
> > +def signal(signalnum, handler):
> > +    handler = _signal.signal(_enum_to_int(signalnum),
> > _enum_to_int(handler))
> > +    return _int_to_enum(handler, Handlers)
> > +
> > +
> > + at _wraps(_signal.getsignal)
> > +def getsignal(signalnum):
> > +    handler = _signal.getsignal(signalnum)
> > +    return _int_to_enum(handler, Handlers)
> > +
> > +
> > +if 'pthread_sigmask' in _globals:
> > +    @_wraps(_signal.pthread_sigmask)
> > +    def pthread_sigmask(how, mask):
> > +        sigs_set = _signal.pthread_sigmask(how, mask)
> > +        return set(_int_to_enum(x, Signals) for x in sigs_set)
> > +    pthread_sigmask.__doc__ = _signal.pthread_sigmask.__doc__
> > +
> > +
> > + at _wraps(_signal.sigpending)
> > +def sigpending():
> > +    sigs = _signal.sigpending()
> > +    return set(_int_to_enum(x, Signals) for x in sigs)
> > +
> > +
> > +if 'sigwait' in _globals:
> > +    @_wraps(_signal.sigwait)
> > +    def sigwait(sigset):
> > +        retsig = _signal.sigwait(sigset)
> > +        return _int_to_enum(retsig, Signals)
> > +    sigwait.__doc__ = _signal.sigwait
> > +
> > +del _globals, _wraps
> > diff --git a/Lib/test/test_doctest.py b/Lib/test/test_doctest.py
> > --- a/Lib/test/test_doctest.py
> > +++ b/Lib/test/test_doctest.py
> > @@ -2897,7 +2897,7 @@
> >
> >  def test_main():
> >      # Check the doctest cases in doctest itself:
> > -    support.run_doctest(doctest, verbosity=True)
> > +    ret = support.run_doctest(doctest, verbosity=True)
> >      # Check the doctest cases defined here:
> >      from test import test_doctest
> >      support.run_doctest(test_doctest, verbosity=True)
> > diff --git a/Lib/test/test_signal.py b/Lib/test/test_signal.py
> > --- a/Lib/test/test_signal.py
> > +++ b/Lib/test/test_signal.py
> > @@ -1,6 +1,7 @@
> >  import unittest
> >  from test import support
> >  from contextlib import closing
> > +import enum
> >  import gc
> >  import pickle
> >  import select
> > @@ -39,6 +40,22 @@
> >          return None
> >
> >
> > +class GenericTests(unittest.TestCase):
> > +
> > +    def test_enums(self):
> > +        for name in dir(signal):
> > +            sig = getattr(signal, name)
> > +            if name in {'SIG_DFL', 'SIG_IGN'}:
> > +                self.assertIsInstance(sig, signal.Handlers)
> > +            elif name in {'SIG_BLOCK', 'SIG_UNBLOCK', 'SIG_SETMASK'}:
> > +                self.assertIsInstance(sig, signal.Sigmasks)
> > +            elif name.startswith('SIG') and not name.startswith('SIG_'):
> > +                self.assertIsInstance(sig, signal.Signals)
> > +            elif name.startswith('CTRL_'):
> > +                self.assertIsInstance(sig, signal.Signals)
> > +                self.assertEqual(sys.platform, "win32")
> > +
> > +
> >  @unittest.skipIf(sys.platform == "win32", "Not valid on Windows")
> >  class InterProcessSignalTests(unittest.TestCase):
> >      MAX_DURATION = 20   # Entire test should last at most 20 sec.
> > @@ -195,6 +212,7 @@
> >
> >      def test_getsignal(self):
> >          hup = signal.signal(signal.SIGHUP, self.trivial_signal_handler)
> > +        self.assertIsInstance(hup, signal.Handlers)
> >          self.assertEqual(signal.getsignal(signal.SIGHUP),
> >                           self.trivial_signal_handler)
> >          signal.signal(signal.SIGHUP, hup)
> > @@ -271,7 +289,7 @@
> >
> >          os.close(read)
> >          os.close(write)
> > -        """.format(signals, ordered, test_body)
> > +        """.format(tuple(map(int, signals)), ordered, test_body)
> >
> >          assert_python_ok('-c', code)
> >
> > @@ -604,6 +622,8 @@
> >              signal.pthread_sigmask(signal.SIG_BLOCK, [signum])
> >              os.kill(os.getpid(), signum)
> >              pending = signal.sigpending()
> > +            for sig in pending:
> > +                assert isinstance(sig, signal.Signals), repr(pending)
> >              if pending != {signum}:
> >                  raise Exception('%s != {%s}' % (pending, signum))
> >              try:
> > @@ -660,6 +680,7 @@
> >          code = '''if 1:
> >          import signal
> >          import sys
> > +        from signal import Signals
> >
> >          def handler(signum, frame):
> >              1/0
> > @@ -702,6 +723,7 @@
> >          def test(signum):
> >              signal.alarm(1)
> >              received = signal.sigwait([signum])
> > +            assert isinstance(received, signal.Signals), received
> >              if received != signum:
> >                  raise Exception('received %s, not %s' % (received,
> > signum))
> >          ''')
> > @@ -842,8 +864,14 @@
> >          def kill(signum):
> >              os.kill(os.getpid(), signum)
> >
> > +        def check_mask(mask):
> > +            for sig in mask:
> > +                assert isinstance(sig, signal.Signals), repr(sig)
> > +
> >          def read_sigmask():
> > -            return signal.pthread_sigmask(signal.SIG_BLOCK, [])
> > +            sigmask = signal.pthread_sigmask(signal.SIG_BLOCK, [])
> > +            check_mask(sigmask)
> > +            return sigmask
> >
> >          signum = signal.SIGUSR1
> >
> > @@ -852,6 +880,7 @@
> >
> >          # Unblock SIGUSR1 (and copy the old mask) to test our signal
> > handler
> >          old_mask = signal.pthread_sigmask(signal.SIG_UNBLOCK, [signum])
> > +        check_mask(old_mask)
> >          try:
> >              kill(signum)
> >          except ZeroDivisionError:
> > @@ -861,11 +890,13 @@
> >
> >          # Block and then raise SIGUSR1. The signal is blocked: the
> signal
> >          # handler is not called, and the signal is now pending
> > -        signal.pthread_sigmask(signal.SIG_BLOCK, [signum])
> > +        mask = signal.pthread_sigmask(signal.SIG_BLOCK, [signum])
> > +        check_mask(mask)
> >          kill(signum)
> >
> >          # Check the new mask
> >          blocked = read_sigmask()
> > +        check_mask(blocked)
> >          if signum not in blocked:
> >              raise Exception("%s not in %s" % (signum, blocked))
> >          if old_mask ^ blocked != {signum}:
> > @@ -928,7 +959,7 @@
> >
> >  def test_main():
> >      try:
> > -        support.run_unittest(PosixTests, InterProcessSignalTests,
> > +        support.run_unittest(GenericTests, PosixTests,
> > InterProcessSignalTests,
> >                               WakeupFDTests, WakeupSignalTests,
> >                               SiginterruptTest, ItimerTest,
> > WindowsSignalTests,
> >                               PendingSignalsTests)
> > diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c
> > --- a/Modules/signalmodule.c
> > +++ b/Modules/signalmodule.c
> > @@ -967,7 +967,7 @@
> >  };
> >
> >  PyMODINIT_FUNC
> > -PyInit_signal(void)
> > +PyInit__signal(void)
> >  {
> >      PyObject *m, *d, *x;
> >      int i;
> > @@ -1380,7 +1380,7 @@
> >  void
> >  PyOS_InitInterrupts(void)
> >  {
> > -    PyObject *m = PyImport_ImportModule("signal");
> > +    PyObject *m = PyImport_ImportModule("_signal");
> >      if (m) {
> >          Py_DECREF(m);
> >      }
> > diff --git a/PC/config.c b/PC/config.c
> > --- a/PC/config.c
> > +++ b/PC/config.c
> > @@ -19,7 +19,7 @@
> >  extern PyObject* PyInit__md5(void);
> >  extern PyObject* PyInit_nt(void);
> >  extern PyObject* PyInit__operator(void);
> > -extern PyObject* PyInit_signal(void);
> > +extern PyObject* PyInit__signal(void);
> >  extern PyObject* PyInit__sha1(void);
> >  extern PyObject* PyInit__sha256(void);
> >  extern PyObject* PyInit__sha512(void);
> >
> > --
> > Repository URL: http://hg.python.org/cpython
> >
> > _______________________________________________
> > Python-checkins mailing list
> > Python-checkins at python.org
> > https://mail.python.org/mailman/listinfo/python-checkins
> >
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://mail.python.org/pipermail/python-dev/attachments/20140404/ec3c9b19/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 3
> Date: Fri, 4 Apr 2014 10:21:39 -0400
> From: Brett Cannon <brett at python.org>
> To: python-dev <python-dev at python.org>
> Cc: python-checkins <python-checkins at python.org>
> Subject: Re: [Python-Dev] [Python-checkins] cpython: fix #21076: turn
>         signal module constants into enums
> Message-ID:
>         <CAP1=
> 2W4wfsa9xtOaL6VdcLpk_vGtBFd-2dd7YGrn8aSpZoChbA at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> On Fri, Apr 4, 2014 at 10:12 AM, Brett Cannon <brett at python.org> wrote:
>
> > This broke compilation on at least OS X, but I'm willing to bet for all
> > UNIX-based systems. I have a fix in the works.
> >
>
> Fix is in rev c6e63bb132fb <http://hg.python.org/cpython/rev/c6e63bb132fb
> >.
>
>
> >
> >
> > On Fri, Apr 4, 2014 at 9:34 AM, giampaolo.rodola <
> > python-checkins at python.org> wrote:
> >
> >> http://hg.python.org/cpython/rev/c9239171e429
> >> changeset:   90128:c9239171e429
> >> user:        Giampaolo Rodola' <g.rodola at gmail.com>
> >> date:        Fri Apr 04 15:34:17 2014 +0200
> >> summary:
> >>   fix #21076: turn signal module constants into enums
> >>
> >> files:
> >>   Doc/library/signal.rst   |  10 +++
> >>   Doc/whatsnew/3.5.rst     |   5 +
> >>   Lib/signal.py            |  84 ++++++++++++++++++++++++++++
> >>   Lib/test/test_doctest.py |   2 +-
> >>   Lib/test/test_signal.py  |  39 +++++++++++-
> >>   Modules/signalmodule.c   |   4 +-
> >>   PC/config.c              |   2 +-
> >>   7 files changed, 138 insertions(+), 8 deletions(-)
> >>
> >>
> >> diff --git a/Doc/library/signal.rst b/Doc/library/signal.rst
> >> --- a/Doc/library/signal.rst
> >> +++ b/Doc/library/signal.rst
> >> @@ -65,6 +65,16 @@
> >>  Module contents
> >>  ---------------
> >>
> >> +.. versionchanged:: 3.5
> >> +   signal (SIG*), handler (:const:`SIG_DFL`, :const:`SIG_IGN`) and
> >> sigmask
> >> +   (:const:`SIG_BLOCK`, :const:`SIG_UNBLOCK`, :const:`SIG_SETMASK`)
> >> +   related constants listed below were turned into
> >> +   :class:`enums <enum.IntEnum>`.
> >> +   :func:`getsignal`, :func:`pthread_sigmask`, :func:`sigpending` and
> >> +   :func:`sigwait` functions return human-readable
> >> +   :class:`enums <enum.IntEnum>`.
> >> +
> >> +
> >>  The variables defined in the :mod:`signal` module are:
> >>
> >>
> >> diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst
> >> --- a/Doc/whatsnew/3.5.rst
> >> +++ b/Doc/whatsnew/3.5.rst
> >> @@ -134,6 +134,11 @@
> >>  Improved Modules
> >>  ================
> >>
> >> +* Different constants of :mod:`signal` module are now enumeration
> values
> >> using
> >> +  the :mod:`enum` module. This allows meaningful names to be printed
> >> during
> >> +  debugging, instead of integer ?magic numbers?. (contribute by
> Giampaolo
> >> +  Rodola' in :issue:`21076`)
> >> +
> >>  * :class:`xmlrpc.client.ServerProxy` is now a :term:`context manager`
> >>    (contributed by Claudiu Popa in :issue:`20627`).
> >>
> >> diff --git a/Lib/signal.py b/Lib/signal.py
> >> new file mode 100644
> >> --- /dev/null
> >> +++ b/Lib/signal.py
> >> @@ -0,0 +1,84 @@
> >> +import _signal
> >> +from _signal import *
> >> +from functools import wraps as _wraps
> >> +from enum import IntEnum as _IntEnum
> >> +
> >> +_globals = globals()
> >> +
> >> +Signals = _IntEnum(
> >> +    'Signals',
> >> +    {name: value for name, value in _globals.items()
> >> +     if name.isupper()
> >> +        and (name.startswith('SIG') and not name.startswith('SIG_'))
> >> +        or name.startswith('CTRL_')})
> >> +
> >> +class Handlers(_IntEnum):
> >> +    SIG_DFL = _signal.SIG_DFL
> >> +    SIG_IGN = _signal.SIG_IGN
> >> +
> >> +_globals.update(Signals.__members__)
> >> +_globals.update(Handlers.__members__)
> >> +
> >> +if 'pthread_sigmask' in _globals:
> >> +    class Sigmasks(_IntEnum):
> >> +        SIG_BLOCK = _signal.SIG_BLOCK
> >> +        SIG_UNBLOCK = _signal.SIG_UNBLOCK
> >> +        SIG_SETMASK = _signal.SIG_SETMASK
> >> +
> >> +    _globals.update(Sigmasks.__members__)
> >> +
> >> +
> >> +def _int_to_enum(value, enum_klass):
> >> +    """Convert a numeric value to an IntEnum member.
> >> +    If it's not a known member, return the numeric value itself.
> >> +    """
> >> +    try:
> >> +        return enum_klass(value)
> >> +    except ValueError:
> >> +        return value
> >> +
> >> +
> >> +def _enum_to_int(value):
> >> +    """Convert an IntEnum member to a numeric value.
> >> +    If it's not a IntEnum member return the value itself.
> >> +    """
> >> +    try:
> >> +        return int(value)
> >> +    except (ValueError, TypeError):
> >> +        return value
> >> +
> >> +
> >> + at _wraps(_signal.signal)
> >> +def signal(signalnum, handler):
> >> +    handler = _signal.signal(_enum_to_int(signalnum),
> >> _enum_to_int(handler))
> >> +    return _int_to_enum(handler, Handlers)
> >> +
> >> +
> >> + at _wraps(_signal.getsignal)
> >> +def getsignal(signalnum):
> >> +    handler = _signal.getsignal(signalnum)
> >> +    return _int_to_enum(handler, Handlers)
> >> +
> >> +
> >> +if 'pthread_sigmask' in _globals:
> >> +    @_wraps(_signal.pthread_sigmask)
> >> +    def pthread_sigmask(how, mask):
> >> +        sigs_set = _signal.pthread_sigmask(how, mask)
> >> +        return set(_int_to_enum(x, Signals) for x in sigs_set)
> >> +    pthread_sigmask.__doc__ = _signal.pthread_sigmask.__doc__
> >> +
> >> +
> >> + at _wraps(_signal.sigpending)
> >> +def sigpending():
> >> +    sigs = _signal.sigpending()
> >> +    return set(_int_to_enum(x, Signals) for x in sigs)
> >> +
> >> +
> >> +if 'sigwait' in _globals:
> >> +    @_wraps(_signal.sigwait)
> >> +    def sigwait(sigset):
> >> +        retsig = _signal.sigwait(sigset)
> >> +        return _int_to_enum(retsig, Signals)
> >> +    sigwait.__doc__ = _signal.sigwait
> >> +
> >> +del _globals, _wraps
> >> diff --git a/Lib/test/test_doctest.py b/Lib/test/test_doctest.py
> >> --- a/Lib/test/test_doctest.py
> >> +++ b/Lib/test/test_doctest.py
> >> @@ -2897,7 +2897,7 @@
> >>
> >>  def test_main():
> >>      # Check the doctest cases in doctest itself:
> >> -    support.run_doctest(doctest, verbosity=True)
> >> +    ret = support.run_doctest(doctest, verbosity=True)
> >>      # Check the doctest cases defined here:
> >>      from test import test_doctest
> >>      support.run_doctest(test_doctest, verbosity=True)
> >> diff --git a/Lib/test/test_signal.py b/Lib/test/test_signal.py
> >> --- a/Lib/test/test_signal.py
> >> +++ b/Lib/test/test_signal.py
> >> @@ -1,6 +1,7 @@
> >>  import unittest
> >>  from test import support
> >>  from contextlib import closing
> >> +import enum
> >>  import gc
> >>  import pickle
> >>  import select
> >> @@ -39,6 +40,22 @@
> >>          return None
> >>
> >>
> >> +class GenericTests(unittest.TestCase):
> >> +
> >> +    def test_enums(self):
> >> +        for name in dir(signal):
> >> +            sig = getattr(signal, name)
> >> +            if name in {'SIG_DFL', 'SIG_IGN'}:
> >> +                self.assertIsInstance(sig, signal.Handlers)
> >> +            elif name in {'SIG_BLOCK', 'SIG_UNBLOCK', 'SIG_SETMASK'}:
> >> +                self.assertIsInstance(sig, signal.Sigmasks)
> >> +            elif name.startswith('SIG') and not
> name.startswith('SIG_'):
> >> +                self.assertIsInstance(sig, signal.Signals)
> >> +            elif name.startswith('CTRL_'):
> >> +                self.assertIsInstance(sig, signal.Signals)
> >> +                self.assertEqual(sys.platform, "win32")
> >> +
> >> +
> >>  @unittest.skipIf(sys.platform == "win32", "Not valid on Windows")
> >>  class InterProcessSignalTests(unittest.TestCase):
> >>      MAX_DURATION = 20   # Entire test should last at most 20 sec.
> >> @@ -195,6 +212,7 @@
> >>
> >>      def test_getsignal(self):
> >>          hup = signal.signal(signal.SIGHUP, self.trivial_signal_handler)
> >> +        self.assertIsInstance(hup, signal.Handlers)
> >>          self.assertEqual(signal.getsignal(signal.SIGHUP),
> >>                           self.trivial_signal_handler)
> >>          signal.signal(signal.SIGHUP, hup)
> >> @@ -271,7 +289,7 @@
> >>
> >>          os.close(read)
> >>          os.close(write)
> >> -        """.format(signals, ordered, test_body)
> >> +        """.format(tuple(map(int, signals)), ordered, test_body)
> >>
> >>          assert_python_ok('-c', code)
> >>
> >> @@ -604,6 +622,8 @@
> >>              signal.pthread_sigmask(signal.SIG_BLOCK, [signum])
> >>              os.kill(os.getpid(), signum)
> >>              pending = signal.sigpending()
> >> +            for sig in pending:
> >> +                assert isinstance(sig, signal.Signals), repr(pending)
> >>              if pending != {signum}:
> >>                  raise Exception('%s != {%s}' % (pending, signum))
> >>              try:
> >> @@ -660,6 +680,7 @@
> >>          code = '''if 1:
> >>          import signal
> >>          import sys
> >> +        from signal import Signals
> >>
> >>          def handler(signum, frame):
> >>              1/0
> >> @@ -702,6 +723,7 @@
> >>          def test(signum):
> >>              signal.alarm(1)
> >>              received = signal.sigwait([signum])
> >> +            assert isinstance(received, signal.Signals), received
> >>              if received != signum:
> >>                  raise Exception('received %s, not %s' % (received,
> >> signum))
> >>          ''')
> >> @@ -842,8 +864,14 @@
> >>          def kill(signum):
> >>              os.kill(os.getpid(), signum)
> >>
> >> +        def check_mask(mask):
> >> +            for sig in mask:
> >> +                assert isinstance(sig, signal.Signals), repr(sig)
> >> +
> >>          def read_sigmask():
> >> -            return signal.pthread_sigmask(signal.SIG_BLOCK, [])
> >> +            sigmask = signal.pthread_sigmask(signal.SIG_BLOCK, [])
> >> +            check_mask(sigmask)
> >> +            return sigmask
> >>
> >>          signum = signal.SIGUSR1
> >>
> >> @@ -852,6 +880,7 @@
> >>
> >>          # Unblock SIGUSR1 (and copy the old mask) to test our signal
> >> handler
> >>          old_mask = signal.pthread_sigmask(signal.SIG_UNBLOCK, [signum])
> >> +        check_mask(old_mask)
> >>          try:
> >>              kill(signum)
> >>          except ZeroDivisionError:
> >> @@ -861,11 +890,13 @@
> >>
> >>          # Block and then raise SIGUSR1. The signal is blocked: the
> signal
> >>          # handler is not called, and the signal is now pending
> >> -        signal.pthread_sigmask(signal.SIG_BLOCK, [signum])
> >> +        mask = signal.pthread_sigmask(signal.SIG_BLOCK, [signum])
> >> +        check_mask(mask)
> >>          kill(signum)
> >>
> >>          # Check the new mask
> >>          blocked = read_sigmask()
> >> +        check_mask(blocked)
> >>          if signum not in blocked:
> >>              raise Exception("%s not in %s" % (signum, blocked))
> >>          if old_mask ^ blocked != {signum}:
> >> @@ -928,7 +959,7 @@
> >>
> >>  def test_main():
> >>      try:
> >> -        support.run_unittest(PosixTests, InterProcessSignalTests,
> >> +        support.run_unittest(GenericTests, PosixTests,
> >> InterProcessSignalTests,
> >>                               WakeupFDTests, WakeupSignalTests,
> >>                               SiginterruptTest, ItimerTest,
> >> WindowsSignalTests,
> >>                               PendingSignalsTests)
> >> diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c
> >> --- a/Modules/signalmodule.c
> >> +++ b/Modules/signalmodule.c
> >> @@ -967,7 +967,7 @@
> >>  };
> >>
> >>  PyMODINIT_FUNC
> >> -PyInit_signal(void)
> >> +PyInit__signal(void)
> >>  {
> >>      PyObject *m, *d, *x;
> >>      int i;
> >> @@ -1380,7 +1380,7 @@
> >>  void
> >>  PyOS_InitInterrupts(void)
> >>  {
> >> -    PyObject *m = PyImport_ImportModule("signal");
> >> +    PyObject *m = PyImport_ImportModule("_signal");
> >>      if (m) {
> >>          Py_DECREF(m);
> >>      }
> >> diff --git a/PC/config.c b/PC/config.c
> >> --- a/PC/config.c
> >> +++ b/PC/config.c
> >> @@ -19,7 +19,7 @@
> >>  extern PyObject* PyInit__md5(void);
> >>  extern PyObject* PyInit_nt(void);
> >>  extern PyObject* PyInit__operator(void);
> >> -extern PyObject* PyInit_signal(void);
> >> +extern PyObject* PyInit__signal(void);
> >>  extern PyObject* PyInit__sha1(void);
> >>  extern PyObject* PyInit__sha256(void);
> >>  extern PyObject* PyInit__sha512(void);
> >>
> >> --
> >> Repository URL: http://hg.python.org/cpython
> >>
> >> _______________________________________________
> >> Python-checkins mailing list
> >> Python-checkins at python.org
> >> https://mail.python.org/mailman/listinfo/python-checkins
> >>
> >>
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://mail.python.org/pipermail/python-dev/attachments/20140404/ff31ca9e/attachment.html
> >
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
>
>
> ------------------------------
>
> End of Python-Dev Digest, Vol 129, Issue 6
> ******************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140407/76702981/attachment-0001.html>

From rosuav at gmail.com  Mon Apr  7 17:49:08 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 8 Apr 2014 01:49:08 +1000
Subject: [Python-Dev] Python-Dev Digest, Vol 129, Issue 6
In-Reply-To: <CAE0cYs7X8PN=PhoN9JUSLG_3HHFs0ohp74jGARqUqroA_O2Now@mail.gmail.com>
References: <mailman.45731.1396621327.18129.python-dev@python.org>
 <CAE0cYs7X8PN=PhoN9JUSLG_3HHFs0ohp74jGARqUqroA_O2Now@mail.gmail.com>
Message-ID: <CAPTjJmoL2u0b+rVfDcq_nDsZYTCbUHCF-ut7G+FvDO5ya39w=w@mail.gmail.com>

On Mon, Apr 7, 2014 at 5:15 PM, Kells Pablo <nombizile at gmail.com> wrote:
> HELLO...
>
> !thank you for all the cooperation and emails send. i would like that you
> now stop sending them..
>
> thank you in advance
>
> On Fri, Apr 4, 2014 at 4:22 PM, <python-dev-request at python.org> wrote:
>>
>> Send Python-Dev mailing list submissions to
>>         python-dev at python.org
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>>         https://mail.python.org/mailman/listinfo/python-dev
>> or, via email, send a message with subject or body 'help' to
>>         python-dev-request at python.org
>>
>

If you'd have a look at the email you just quoted (in its entirety)
back to the list, you would see how to unsubscribe. We here on the
list are unable to unsubscribe you, and the software that runs the
list doesn't listen to these sorts of posts.

ChrisA

From alexander.belopolsky at gmail.com  Mon Apr  7 18:52:58 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Mon, 7 Apr 2014 12:52:58 -0400
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <1396840406.21603.103475673.36331ED6@webmail.messagingengine.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <1396840406.21603.103475673.36331ED6@webmail.messagingengine.com>
Message-ID: <CAP7h-xayw4yw_82+Ncr3ykCJmJu3oah1RxBGFVAUxZdvm8Fg6Q@mail.gmail.com>

On Sun, Apr 6, 2014 at 11:13 PM, Benjamin Peterson <benjamin at python.org>wrote:

> > I believe this leaves only one open question, which is where exactly
> > to stick the new matmul slots into PyTypeObject. This is the kind of
> > fiddly detail that can easily be settled later if the PEP is accepted,
> > though.
>
> I don't see what it shouldn't be in PyNumberMethods. Surely, we're not
> going to get a flood of requests for more matrix operators, are we? :)


We may want to introduce say  PyArrayMethods even if we don't introduce more
array operators.  We can populate that struct with array-specific
alternatives for
PySequence/PyMappingMethods and eliminate the need for dynamically created
array types to allocate those.

There is also a way to introduce PyArrayMethods at no cost to current
implementation:
we can rename tp_reserved (formerly known as tp_compare) to tp_as_array.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140407/ea51913b/attachment.html>

From benjamin at python.org  Mon Apr  7 19:11:34 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 07 Apr 2014 10:11:34 -0700
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
 multiplication
In-Reply-To: <CAP7h-xayw4yw_82+Ncr3ykCJmJu3oah1RxBGFVAUxZdvm8Fg6Q@mail.gmail.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <1396840406.21603.103475673.36331ED6@webmail.messagingengine.com>
 <CAP7h-xayw4yw_82+Ncr3ykCJmJu3oah1RxBGFVAUxZdvm8Fg6Q@mail.gmail.com>
Message-ID: <1396890694.21608.103740329.5D0332F8@webmail.messagingengine.com>



On Mon, Apr 7, 2014, at 9:52, Alexander Belopolsky wrote:
> On Sun, Apr 6, 2014 at 11:13 PM, Benjamin Peterson
> <benjamin at python.org>wrote:
> 
> > > I believe this leaves only one open question, which is where exactly
> > > to stick the new matmul slots into PyTypeObject. This is the kind of
> > > fiddly detail that can easily be settled later if the PEP is accepted,
> > > though.
> >
> > I don't see what it shouldn't be in PyNumberMethods. Surely, we're not
> > going to get a flood of requests for more matrix operators, are we? :)
> 
> 
> We may want to introduce say  PyArrayMethods even if we don't introduce
> more
> array operators.  We can populate that struct with array-specific
> alternatives for
> PySequence/PyMappingMethods and eliminate the need for dynamically
> created
> array types to allocate those.

Why would we want to do that?

From tjreedy at udel.edu  Mon Apr  7 18:39:57 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 07 Apr 2014 12:39:57 -0400
Subject: [Python-Dev] [Python-checkins] cpython (3.4): asyncio: Document
 Task.cancel() properly.
In-Reply-To: <3g2R9K30xfz7LjN@mail.python.org>
References: <3g2R9K30xfz7LjN@mail.python.org>
Message-ID: <5342D4DD.1020505@udel.edu>

On 4/7/2014 5:22 AM, victor.stinner wrote:

>       def cancel(self):
> +        """Request that a task to cancel itself.

For proper English, this should be one of these:
"Request that a task cancel itself."
"Request a task to cancel itself."

I think the first is slightly better.

TJR


From francismb at email.de  Mon Apr  7 20:54:49 2014
From: francismb at email.de (francis)
Date: Mon, 07 Apr 2014 20:54:49 +0200
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
Message-ID: <5342F479.8040108@email.de>


>
> So, I guess as far as I'm concerned, this is ready to go. Feedback
welcome:
>   http://legacy.python.org/dev/peps/pep-0465/
>

Hi,
just curiosity: why is the second parameter 'o2' in:

PyObject* PyObject_MatrixMultiply(PyObject *o1, PyObject o2)

not a pointer to PyObject?

Thanks in advance!

francis



From robert.kern at gmail.com  Mon Apr  7 22:03:13 2014
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 07 Apr 2014 21:03:13 +0100
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <5342F479.8040108@email.de>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <5342F479.8040108@email.de>
Message-ID: <lhv0a1$bi9$1@ger.gmane.org>

On 2014-04-07 19:54, francis wrote:
>
>>
>> So, I guess as far as I'm concerned, this is ready to go. Feedback
> welcome:
>>    http://legacy.python.org/dev/peps/pep-0465/
>>
>
> Hi,
> just curiosity: why is the second parameter 'o2' in:
>
> PyObject* PyObject_MatrixMultiply(PyObject *o1, PyObject o2)
>
> not a pointer to PyObject?

Typo, I'm fairly certain.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From njs at pobox.com  Mon Apr  7 21:21:02 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Mon, 7 Apr 2014 20:21:02 +0100
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <5342F479.8040108@email.de>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <5342F479.8040108@email.de>
Message-ID: <CAPJVwBkT9jnT6qb0465HcL8rXuaWkOEnZWjUPVHRKLctso7_Aw@mail.gmail.com>

On Mon, Apr 7, 2014 at 7:54 PM, francis <francismb at email.de> wrote:
>
>>
>> So, I guess as far as I'm concerned, this is ready to go. Feedback
> welcome:
>>   http://legacy.python.org/dev/peps/pep-0465/
>>
>
> Hi,
> just curiosity: why is the second parameter 'o2' in:
>
> PyObject* PyObject_MatrixMultiply(PyObject *o1, PyObject o2)
>
> not a pointer to PyObject?

Because typo. Thanks for the catch :-)

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

From victor.stinner at gmail.com  Mon Apr  7 22:38:03 2014
From: victor.stinner at gmail.com (Victor Stinner)
Date: Mon, 7 Apr 2014 22:38:03 +0200
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
Message-ID: <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>

Hi,

2014-04-07 3:41 GMT+02:00 Nathaniel Smith <njs at pobox.com>:
> So, I guess as far as I'm concerned, this is ready to go. Feedback welcome:
>   http://legacy.python.org/dev/peps/pep-0465/

I'm not convinced yet that there is enough usage of Python in
mathematical world to modify the Python language to add a new
operator. Python is used for a lot of different use cases, in a lot of
domains. I'm not sure that it's a good thing to modify the *language*
for a specific domain. But you can do a lot without modify the
language :-)

I'm a little bit surprised by the "Count of Python source files on
Github matching given search terms" table, it's very different from
these statistics:
http://python3wos.appspot.com/

Where are six, pytz,  mock, webob, etc. in your table? (all modules
which come before "numpy" in the "Python 3 Wall of Superpowers")

> But isn't it weird to add an operator with no stdlib uses?

I agree that it sounds weird :-) Maybe we should start by putting some
parts of numpy/scipy/sage/pylab/panda into the stdlib? (I'm not sure
that the new statistics module is such beginning.)

--

It would be nice to support A ? B too, because it's much more
readable. You can configure a keyword to write arbitrary characters.
For example, on Linux you can write ? using "Compose x x" if you
configured the Compose key. Or sometimes, you can replace "@" with "?"
using your favorite text editor (copy-paste from another script, from
a webpage, or something else).

You may mention Perl 6 meta operators, but it's not directly related:
http://en.wikibooks.org/wiki/Perl_6_Programming/Meta_Operators

Victor

From solipsis at pitrou.net  Mon Apr  7 22:46:04 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 07 Apr 2014 22:46:04 +0200
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
Message-ID: <lhv2qc$hkj$1@ger.gmane.org>

Le 07/04/2014 22:38, Victor Stinner a ?crit :
>
> It would be nice to support A ? B too, because it's much more
> readable. You can configure a keyword to write arbitrary characters.

Well, IMHO Python code should be writable without having to "configure 
your keyboard".

Regards

Antoine.



From victor.stinner at gmail.com  Mon Apr  7 22:55:29 2014
From: victor.stinner at gmail.com (Victor Stinner)
Date: Mon, 7 Apr 2014 22:55:29 +0200
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <lhv2qc$hkj$1@ger.gmane.org>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
 <lhv2qc$hkj$1@ger.gmane.org>
Message-ID: <CAMpsgwb9azgO5T=u9-65+_DtjrR6_WTh7xFPTCgeUyf7wqWePw@mail.gmail.com>

2014-04-07 22:46 GMT+02:00 Antoine Pitrou <solipsis at pitrou.net>:
> Le 07/04/2014 22:38, Victor Stinner a ?crit :
>> It would be nice to support A ? B too, because it's much more
>> readable. You can configure a keyword to write arbitrary characters.
>
> Well, IMHO Python code should be writable without having to "configure your
> keyboard".

I proposed to support both syntaxes, so you can write "@" if you are
unable to write ?.

It's not because I'm unable to write chinese that Python should no
allow chinese characters in Python identifiers :-)

Victor

From alexander.belopolsky at gmail.com  Mon Apr  7 23:06:20 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Mon, 7 Apr 2014 17:06:20 -0400
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <CAMpsgwb9azgO5T=u9-65+_DtjrR6_WTh7xFPTCgeUyf7wqWePw@mail.gmail.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
 <lhv2qc$hkj$1@ger.gmane.org>
 <CAMpsgwb9azgO5T=u9-65+_DtjrR6_WTh7xFPTCgeUyf7wqWePw@mail.gmail.com>
Message-ID: <CAP7h-xZm6RBvKQdkKUQCZ+4pWxFgEeQtJmK+HwZia0JRadiw5g@mail.gmail.com>

On Mon, Apr 7, 2014 at 4:55 PM, Victor Stinner <victor.stinner at gmail.com>wrote:

> I proposed to support both syntaxes, so you can write "@" if you are
> unable to write ?.
>

It won't be obvious for the readers of the code whether ? stands for @ or
for *. Both * and @ are ASCII approximations to proper mathematical
typesetting.   It may be added to the PEP that ? is almost never used for
dot or scalar product.  In vector context it is commonly used for vector
product a.k.a. cross product.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140407/2ed843ca/attachment.html>

From njs at pobox.com  Mon Apr  7 23:03:57 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Mon, 7 Apr 2014 22:03:57 +0100
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
Message-ID: <CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>

On Mon, Apr 7, 2014 at 9:38 PM, Victor Stinner <victor.stinner at gmail.com> wrote:
> Hi,
>
> 2014-04-07 3:41 GMT+02:00 Nathaniel Smith <njs at pobox.com>:
>> So, I guess as far as I'm concerned, this is ready to go. Feedback welcome:
>>   http://legacy.python.org/dev/peps/pep-0465/
>
> I'm not convinced yet that there is enough usage of Python in
> mathematical world to modify the Python language to add a new
> operator. Python is used for a lot of different use cases, in a lot of
> domains. I'm not sure that it's a good thing to modify the *language*
> for a specific domain. But you can do a lot without modify the
> language :-)
>
> I'm a little bit surprised by the "Count of Python source files on
> Github matching given search terms" table, it's very different from
> these statistics:
> http://python3wos.appspot.com/
>
> Where are six, pytz,  mock, webob, etc. in your table? (all modules
> which come before "numpy" in the "Python 3 Wall of Superpowers")

They'd be down in bottom half, with ~30-50k total. (You can check
easily by running the search yourself :-).)

PyPI downloads are not a great proxy for usage, for a number of
reasons. The way to get really big on PyPI downloads is to be depended
on by a lot of projects get deployed often :-). This is very different
from being used directly in lots of different files. Consider also
that probably a majority of numpy users get numpy (and python, etc.)
by using one of the many specialized scientific python distributions
that different companies and people maintain:
  http://www.scipy.org/install.html#scientific-python-distributions
and also that using pip to install scientific packages basically never
works, so no-one uses the pip -r requirements.txt system for
deployment...

>> But isn't it weird to add an operator with no stdlib uses?
>
> I agree that it sounds weird :-) Maybe we should start by putting some
> parts of numpy/scipy/sage/pylab/panda into the stdlib? (I'm not sure
> that the new statistics module is such beginning.)

There are many reasons why this is not a great idea in the short term
-- including the problem mentioned a few sentences after the one you
quoted, which is that @ seems to be a precondition to getting
consensus on a numeric array duck type, which in turn would be a
precondition to putting an array type into the stdlib ;-). So while
putting numpy into the stdlib is probably a bad idea regardless, even
if we wanted to do it there's a chicken-and-egg problem.

> --
>
> It would be nice to support A ? B too, because it's much more
> readable. You can configure a keyword to write arbitrary characters.
> For example, on Linux you can write ? using "Compose x x" if you
> configured the Compose key. Or sometimes, you can replace "@" with "?"
> using your favorite text editor (copy-paste from another script, from
> a webpage, or something else).

Sounds like a pretty major violation of TOOWTDI...

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

From alexander.belopolsky at gmail.com  Mon Apr  7 23:22:29 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Mon, 7 Apr 2014 17:22:29 -0400
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <1396890694.21608.103740329.5D0332F8@webmail.messagingengine.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <1396840406.21603.103475673.36331ED6@webmail.messagingengine.com>
 <CAP7h-xayw4yw_82+Ncr3ykCJmJu3oah1RxBGFVAUxZdvm8Fg6Q@mail.gmail.com>
 <1396890694.21608.103740329.5D0332F8@webmail.messagingengine.com>
Message-ID: <CAP7h-xatixic=Y8aZOwHmfW_EAy71Ax_1MeA4cE+CTu3SSc=kw@mail.gmail.com>

On Mon, Apr 7, 2014 at 1:11 PM, Benjamin Peterson <benjamin at python.org>
wrote:
>
> > We can populate that struct with array-specific alternatives for
> > PySequence/PyMappingMethods and eliminate the need for dynamically
> > created array types to allocate those.
>
> Why would we want to do that?

I assume "that" means "create array types dynamically."  There are many
reasons to do that in modern array implementations.  See for example
ctypes.  A more practical reason however is that I
believe PySequence/PyMappingMethods are cloned whenever a subclass of a
sequence/mapping type is created.  For better or worse, subclassing of
numpy arrays is a popular sport these days.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140407/126ce55b/attachment-0001.html>

From benjamin at python.org  Mon Apr  7 23:23:23 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 07 Apr 2014 14:23:23 -0700
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
 multiplication
In-Reply-To: <CAP7h-xatixic=Y8aZOwHmfW_EAy71Ax_1MeA4cE+CTu3SSc=kw@mail.gmail.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <1396840406.21603.103475673.36331ED6@webmail.messagingengine.com>
 <CAP7h-xayw4yw_82+Ncr3ykCJmJu3oah1RxBGFVAUxZdvm8Fg6Q@mail.gmail.com>
 <1396890694.21608.103740329.5D0332F8@webmail.messagingengine.com>
 <CAP7h-xatixic=Y8aZOwHmfW_EAy71Ax_1MeA4cE+CTu3SSc=kw@mail.gmail.com>
Message-ID: <1396905803.23121.103839313.31DB324E@webmail.messagingengine.com>



On Mon, Apr 7, 2014, at 14:22, Alexander Belopolsky wrote:
> On Mon, Apr 7, 2014 at 1:11 PM, Benjamin Peterson <benjamin at python.org>
> wrote:
> >
> > > We can populate that struct with array-specific alternatives for
> > > PySequence/PyMappingMethods and eliminate the need for dynamically
> > > created array types to allocate those.
> >
> > Why would we want to do that?
> 
> I assume "that" means "create array types dynamically."  There are many
> reasons to do that in modern array implementations.  See for example
> ctypes.  A more practical reason however is that I
> believe PySequence/PyMappingMethods are cloned whenever a subclass of a
> sequence/mapping type is created.  For better or worse, subclassing of
> numpy arrays is a popular sport these days.

I can understand why creating new array types is good fun, but how is
creating a new struct helpful?

From alexander.belopolsky at gmail.com  Mon Apr  7 23:33:25 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Mon, 7 Apr 2014 17:33:25 -0400
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <1396905803.23121.103839313.31DB324E@webmail.messagingengine.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <1396840406.21603.103475673.36331ED6@webmail.messagingengine.com>
 <CAP7h-xayw4yw_82+Ncr3ykCJmJu3oah1RxBGFVAUxZdvm8Fg6Q@mail.gmail.com>
 <1396890694.21608.103740329.5D0332F8@webmail.messagingengine.com>
 <CAP7h-xatixic=Y8aZOwHmfW_EAy71Ax_1MeA4cE+CTu3SSc=kw@mail.gmail.com>
 <1396905803.23121.103839313.31DB324E@webmail.messagingengine.com>
Message-ID: <CAP7h-xZPs7F5Vfw5gdvnCp6x96_wUVsO1ys7CvkiRqticXVBqg@mail.gmail.com>

On Mon, Apr 7, 2014 at 5:23 PM, Benjamin Peterson <benjamin at python.org>wrote:

> I can understand why creating new array types is good fun, but how is
> creating a new struct helpful?
>

We can start by reviewing the reasons for having separate
PyNumber/PySequence/PyMappingMethods
structures.  I believe that one of the reasons is that many types need to
allocate only one of the three.  Numpy arrays, IIRC, allocate all three.  A
dedicated PyArrayMethods struct can replace some if not all of these
allocations.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140407/5661612c/attachment.html>

From benjamin at python.org  Mon Apr  7 23:34:32 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 07 Apr 2014 14:34:32 -0700
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
 multiplication
In-Reply-To: <CAP7h-xZPs7F5Vfw5gdvnCp6x96_wUVsO1ys7CvkiRqticXVBqg@mail.gmail.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <1396840406.21603.103475673.36331ED6@webmail.messagingengine.com>
 <CAP7h-xayw4yw_82+Ncr3ykCJmJu3oah1RxBGFVAUxZdvm8Fg6Q@mail.gmail.com>
 <1396890694.21608.103740329.5D0332F8@webmail.messagingengine.com>
 <CAP7h-xatixic=Y8aZOwHmfW_EAy71Ax_1MeA4cE+CTu3SSc=kw@mail.gmail.com>
 <1396905803.23121.103839313.31DB324E@webmail.messagingengine.com>
 <CAP7h-xZPs7F5Vfw5gdvnCp6x96_wUVsO1ys7CvkiRqticXVBqg@mail.gmail.com>
Message-ID: <1396906472.25643.103842761.70D66176@webmail.messagingengine.com>



On Mon, Apr 7, 2014, at 14:33, Alexander Belopolsky wrote:
> On Mon, Apr 7, 2014 at 5:23 PM, Benjamin Peterson
> <benjamin at python.org>wrote:
> 
> > I can understand why creating new array types is good fun, but how is
> > creating a new struct helpful?
> >
> 
> We can start by reviewing the reasons for having separate
> PyNumber/PySequence/PyMappingMethods
> structures.  I believe that one of the reasons is that many types need to
> allocate only one of the three.  Numpy arrays, IIRC, allocate all three. 
> A
> dedicated PyArrayMethods struct can replace some if not all of these
> allocations.

I can't say it seems like a terrible important thing to optimize to me.

From ethan at stoneleaf.us  Mon Apr  7 22:49:49 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 07 Apr 2014 13:49:49 -0700
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
Message-ID: <53430F6D.2020006@stoneleaf.us>

On 04/07/2014 01:38 PM, Victor Stinner wrote:
>
> I'm not sure that it's a good thing to modify the *language*
> for a specific domain. But you can do a lot without modify the
> language :-)

That ship has already sailed.  Features have already been added at the behest of the numerical community.

--
~Ethan~

From alexander.belopolsky at gmail.com  Mon Apr  7 23:43:52 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Mon, 7 Apr 2014 17:43:52 -0400
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
 <CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>
Message-ID: <CAP7h-xZPkJycFxyi4q8uc-HucM=tgAVDVXPmcGdJPH0G8-ZgiA@mail.gmail.com>

On Mon, Apr 7, 2014 at 5:03 PM, Nathaniel Smith <njs at pobox.com> wrote:

> no-one uses the pip -r requirements.txt system for
> deployment...
>

I must be among "no-one" then. :-) Yet my systems don't leave much of a
footprint on PyPI because we use PIP_DOWNLOAD_CACHE and internal PyPI
mirrors.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140407/f22d10e8/attachment.html>

From alexander.belopolsky at gmail.com  Mon Apr  7 23:47:54 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Mon, 7 Apr 2014 17:47:54 -0400
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
 <CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>
Message-ID: <CAP7h-xY_tq8WzkY+U4Fcno9ttnoBwHnY+sY6L7V5rfxmoamaNQ@mail.gmail.com>

On Mon, Apr 7, 2014 at 5:03 PM, Nathaniel Smith <njs at pobox.com> wrote:

> > It would be nice to support A ? B too, because it's much more
> > readable. You can configure a keyword to write arbitrary characters.
> > For example, on Linux you can write ? using "Compose x x" if you
> > configured the Compose key. Or sometimes, you can replace "@" with "?"
> > using your favorite text editor (copy-paste from another script, from
> > a webpage, or something else).
>
> Sounds like a pretty major violation of TOOWTDI...


Python used to have an alias <> for != and I for one miss <> in 3.x.  I
don't think TOOWTDI should be the last word in this debate.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140407/4c75babe/attachment.html>

From larry at hastings.org  Mon Apr  7 23:58:18 2014
From: larry at hastings.org (Larry Hastings)
Date: Mon, 07 Apr 2014 14:58:18 -0700
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <CAP7h-xY_tq8WzkY+U4Fcno9ttnoBwHnY+sY6L7V5rfxmoamaNQ@mail.gmail.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
 <CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>
 <CAP7h-xY_tq8WzkY+U4Fcno9ttnoBwHnY+sY6L7V5rfxmoamaNQ@mail.gmail.com>
Message-ID: <53431F7A.6040807@hastings.org>


On 04/07/2014 02:47 PM, Alexander Belopolsky wrote:
>
> On Mon, Apr 7, 2014 at 5:03 PM, Nathaniel Smith <njs at pobox.com 
> <mailto:njs at pobox.com>> wrote:
>
>     > It would be nice to support A ? B too, because it's much more
>     > readable. You can configure a keyword to write arbitrary characters.
>     > For example, on Linux you can write ? using "Compose x x" if you
>     > configured the Compose key. Or sometimes, you can replace "@"
>     with "?"
>     > using your favorite text editor (copy-paste from another script,
>     from
>     > a webpage, or something else).
>
>     Sounds like a pretty major violation of TOOWTDI...
>
>
> Python used to have an alias <> for != and I for one miss <> in 3.x. 
>  I don't think TOOWTDI should be the last word in this debate.

Right, and <> was removed because TOOWTDI.  I am -1**3001 on adding 
redundant non-ASCII operators to the language.  Python != APL.


/arry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140407/fe03454f/attachment.html>

From barry at python.org  Mon Apr  7 23:58:44 2014
From: barry at python.org (Barry Warsaw)
Date: Mon, 7 Apr 2014 17:58:44 -0400
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
 multiplication
In-Reply-To: <CAP7h-xY_tq8WzkY+U4Fcno9ttnoBwHnY+sY6L7V5rfxmoamaNQ@mail.gmail.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
 <CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>
 <CAP7h-xY_tq8WzkY+U4Fcno9ttnoBwHnY+sY6L7V5rfxmoamaNQ@mail.gmail.com>
Message-ID: <20140407175844.00f92cfc@anarchist.wooz.org>

On Apr 07, 2014, at 05:47 PM, Alexander Belopolsky wrote:

>Python used to have an alias <> for != and I for one miss <> in 3.x.  I
>don't think TOOWTDI should be the last word in this debate.

PEP 401 to the rescue:

% python3
Python 3.4.0 (default, Mar 22 2014, 22:51:25) 
[GCC 4.8.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from __future__ import barry_as_FLUFL
>>> 'flufl' <> 'bdfl'
True
>>> 'flufl' != 'bdfl'
  File "<stdin>", line 1
    'flufl' != 'bdfl'
             ^
SyntaxError: invalid syntax


(And no, I am not recommending you actually *use* that in live code. :)

flufl-ly y'rs,
-Barry

From benjamin at python.org  Tue Apr  8 00:04:18 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 07 Apr 2014 15:04:18 -0700
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
 multiplication
In-Reply-To: <20140407175844.00f92cfc@anarchist.wooz.org>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
 <CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>
 <CAP7h-xY_tq8WzkY+U4Fcno9ttnoBwHnY+sY6L7V5rfxmoamaNQ@mail.gmail.com>
 <20140407175844.00f92cfc@anarchist.wooz.org>
Message-ID: <1396908258.9798.103852961.532425F7@webmail.messagingengine.com>

On Mon, Apr 7, 2014, at 14:58, Barry Warsaw wrote:
> On Apr 07, 2014, at 05:47 PM, Alexander Belopolsky wrote:
> 
> >Python used to have an alias <> for != and I for one miss <> in 3.x.  I
> >don't think TOOWTDI should be the last word in this debate.
> 
> PEP 401 to the rescue:

It occurs to me that since that Aprils' Fools joke is many years old
now, we should remove it.

From alexander.belopolsky at gmail.com  Tue Apr  8 00:16:48 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Mon, 7 Apr 2014 18:16:48 -0400
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <53431F7A.6040807@hastings.org>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
 <CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>
 <CAP7h-xY_tq8WzkY+U4Fcno9ttnoBwHnY+sY6L7V5rfxmoamaNQ@mail.gmail.com>
 <53431F7A.6040807@hastings.org>
Message-ID: <CAP7h-xb8oJrD7A1ZF=6fm9XVumx8e7z6NVSBah9Le1qdPtH=eQ@mail.gmail.com>

On Mon, Apr 7, 2014 at 5:58 PM, Larry Hastings <larry at hastings.org> wrote:

> I am -1**3001 on adding redundant non-ASCII operators to the language.


>>> -1**3001
-1

:-)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140407/d9ef4a97/attachment.html>

From larry at hastings.org  Tue Apr  8 00:37:42 2014
From: larry at hastings.org (Larry Hastings)
Date: Mon, 07 Apr 2014 15:37:42 -0700
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <CAP7h-xb8oJrD7A1ZF=6fm9XVumx8e7z6NVSBah9Le1qdPtH=eQ@mail.gmail.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>	<CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>	<CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>	<CAP7h-xY_tq8WzkY+U4Fcno9ttnoBwHnY+sY6L7V5rfxmoamaNQ@mail.gmail.com>	<53431F7A.6040807@hastings.org>
 <CAP7h-xb8oJrD7A1ZF=6fm9XVumx8e7z6NVSBah9Le1qdPtH=eQ@mail.gmail.com>
Message-ID: <534328B6.5030806@hastings.org>


On 04/07/2014 03:16 PM, Alexander Belopolsky wrote:
>
> On Mon, Apr 7, 2014 at 5:58 PM, Larry Hastings <larry at hastings.org 
> <mailto:larry at hastings.org>> wrote:
>
>     I am -1**3001 on adding redundant non-ASCII operators to the language.
>
>
> >>> -1**3001
> -1
>
> :-)

http://www.quickmeme.com/img/9c/9cb11f91cfda4d161c44e5b2c18c242c60411ac42dc8debc58b2e6a4c17efb0c.jpg


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140407/c134af72/attachment.html>

From guido at python.org  Tue Apr  8 00:41:08 2014
From: guido at python.org (Guido van Rossum)
Date: Mon, 7 Apr 2014 15:41:08 -0700
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <534328B6.5030806@hastings.org>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
 <CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>
 <CAP7h-xY_tq8WzkY+U4Fcno9ttnoBwHnY+sY6L7V5rfxmoamaNQ@mail.gmail.com>
 <53431F7A.6040807@hastings.org>
 <CAP7h-xb8oJrD7A1ZF=6fm9XVumx8e7z6NVSBah9Le1qdPtH=eQ@mail.gmail.com>
 <534328B6.5030806@hastings.org>
Message-ID: <CAP7+vJKOP+7JZx5nZ5R=1AiucxUtYMigPK1gPENnfyNYoDGTtg@mail.gmail.com>

I'm now accepting the PEP, so you all can stop joking around.


On Mon, Apr 7, 2014 at 3:37 PM, Larry Hastings <larry at hastings.org> wrote:

>
> On 04/07/2014 03:16 PM, Alexander Belopolsky wrote:
>
>
> On Mon, Apr 7, 2014 at 5:58 PM, Larry Hastings <larry at hastings.org> wrote:
>
>> I am -1**3001 on adding redundant non-ASCII operators to the language.
>
>
> >>> -1**3001
> -1
>
>  :-)
>
>
>
> http://www.quickmeme.com/img/9c/9cb11f91cfda4d161c44e5b2c18c242c60411ac42dc8debc58b2e6a4c17efb0c.jpg
>
>
> */arry*
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140407/c108b314/attachment-0001.html>

From greg.ewing at canterbury.ac.nz  Tue Apr  8 00:19:09 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 08 Apr 2014 10:19:09 +1200
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
 multiplication
In-Reply-To: <CAP7h-xZPs7F5Vfw5gdvnCp6x96_wUVsO1ys7CvkiRqticXVBqg@mail.gmail.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <1396840406.21603.103475673.36331ED6@webmail.messagingengine.com>
 <CAP7h-xayw4yw_82+Ncr3ykCJmJu3oah1RxBGFVAUxZdvm8Fg6Q@mail.gmail.com>
 <1396890694.21608.103740329.5D0332F8@webmail.messagingengine.com>
 <CAP7h-xatixic=Y8aZOwHmfW_EAy71Ax_1MeA4cE+CTu3SSc=kw@mail.gmail.com>
 <1396905803.23121.103839313.31DB324E@webmail.messagingengine.com>
 <CAP7h-xZPs7F5Vfw5gdvnCp6x96_wUVsO1ys7CvkiRqticXVBqg@mail.gmail.com>
Message-ID: <5343245D.4010609@canterbury.ac.nz>

Alexander Belopolsky wrote:
> We can start by reviewing the reasons for having separate 
> PyNumber/PySequence/PyMappingMethods structures.  I believe that one of 
> the reasons is that many types need to allocate only one of the three.

That much is probably true.

>  Numpy arrays, IIRC, allocate all three.  A dedicated PyArrayMethods 
> struct can replace some if not all of these allocations.

I don't see how. NumPy arrays allocate all three because
they override just about every method in existence. Adding
another struct isn't going to eliminate the need for the
existing ones.

-- 
Greg

From steve at pearwood.info  Tue Apr  8 03:04:29 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 8 Apr 2014 11:04:29 +1000
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <1396908258.9798.103852961.532425F7@webmail.messagingengine.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
 <CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>
 <CAP7h-xY_tq8WzkY+U4Fcno9ttnoBwHnY+sY6L7V5rfxmoamaNQ@mail.gmail.com>
 <20140407175844.00f92cfc@anarchist.wooz.org>
 <1396908258.9798.103852961.532425F7@webmail.messagingengine.com>
Message-ID: <20140408010429.GH16466@ando>

On Mon, Apr 07, 2014 at 03:04:18PM -0700, Benjamin Peterson wrote:
> On Mon, Apr 7, 2014, at 14:58, Barry Warsaw wrote:
> > On Apr 07, 2014, at 05:47 PM, Alexander Belopolsky wrote:
> > 
> > >Python used to have an alias <> for != and I for one miss <> in 3.x.  I
> > >don't think TOOWTDI should be the last word in this debate.
> > 
> > PEP 401 to the rescue:
> 
> It occurs to me that since that Aprils' Fools joke is many years old
> now, we should remove it.

-1 on removal.

It makes a nice Easter Egg, especially now that "import this" has become 
less of an Easter Egg and more of a standard Python module :-)



-- 
Steven

From benjamin at python.org  Tue Apr  8 03:06:17 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 07 Apr 2014 18:06:17 -0700
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
 multiplication
In-Reply-To: <20140408010429.GH16466@ando>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
 <CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>
 <CAP7h-xY_tq8WzkY+U4Fcno9ttnoBwHnY+sY6L7V5rfxmoamaNQ@mail.gmail.com>
 <20140407175844.00f92cfc@anarchist.wooz.org>
 <1396908258.9798.103852961.532425F7@webmail.messagingengine.com>
 <20140408010429.GH16466@ando>
Message-ID: <1396919177.7953.103901257.70C90989@webmail.messagingengine.com>

On Mon, Apr 7, 2014, at 18:04, Steven D'Aprano wrote:
> On Mon, Apr 07, 2014 at 03:04:18PM -0700, Benjamin Peterson wrote:
> > On Mon, Apr 7, 2014, at 14:58, Barry Warsaw wrote:
> > > On Apr 07, 2014, at 05:47 PM, Alexander Belopolsky wrote:
> > > 
> > > >Python used to have an alias <> for != and I for one miss <> in 3.x.  I
> > > >don't think TOOWTDI should be the last word in this debate.
> > > 
> > > PEP 401 to the rescue:
> > 
> > It occurs to me that since that Aprils' Fools joke is many years old
> > now, we should remove it.
> 
> -1 on removal.

You can't be serious.

> 
> It makes a nice Easter Egg, especially now that "import this" has become 
> less of an Easter Egg and more of a standard Python module :-)

It's a terrible Easter Egg because it's basically a CPython core
developer in-joke.

From barry at python.org  Tue Apr  8 03:11:56 2014
From: barry at python.org (Barry Warsaw)
Date: Mon, 7 Apr 2014 21:11:56 -0400
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
 multiplication
In-Reply-To: <1396919177.7953.103901257.70C90989@webmail.messagingengine.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
 <CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>
 <CAP7h-xY_tq8WzkY+U4Fcno9ttnoBwHnY+sY6L7V5rfxmoamaNQ@mail.gmail.com>
 <20140407175844.00f92cfc@anarchist.wooz.org>
 <1396908258.9798.103852961.532425F7@webmail.messagingengine.com>
 <20140408010429.GH16466@ando>
 <1396919177.7953.103901257.70C90989@webmail.messagingengine.com>
Message-ID: <20140407211156.701663f3@limelight.wooz.org>

On Apr 07, 2014, at 06:06 PM, Benjamin Peterson wrote:

>> > It occurs to me that since that Aprils' Fools joke is many years old
>> > now, we should remove it.
>> 
>> -1 on removal.
>
>You can't be serious.

Hey man, don't break all my code!  <wink>

-Barry

From benjamin at python.org  Tue Apr  8 03:13:51 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 07 Apr 2014 18:13:51 -0700
Subject: [Python-Dev] stupid jokes (was PEP 465: A dedicated infix operator
 for matrix multiplication)
In-Reply-To: <20140407211156.701663f3@limelight.wooz.org>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
 <CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>
 <CAP7h-xY_tq8WzkY+U4Fcno9ttnoBwHnY+sY6L7V5rfxmoamaNQ@mail.gmail.com>
 <20140407175844.00f92cfc@anarchist.wooz.org>
 <1396908258.9798.103852961.532425F7@webmail.messagingengine.com>
 <20140408010429.GH16466@ando>
 <1396919177.7953.103901257.70C90989@webmail.messagingengine.com>
 <20140407211156.701663f3@limelight.wooz.org>
Message-ID: <1396919631.9227.103902661.12892955@webmail.messagingengine.com>

On Mon, Apr 7, 2014, at 18:11, Barry Warsaw wrote:
> On Apr 07, 2014, at 06:06 PM, Benjamin Peterson wrote:
> 
> >> > It occurs to me that since that Aprils' Fools joke is many years old
> >> > now, we should remove it.
> >> 
> >> -1 on removal.
> >
> >You can't be serious.
> 
> Hey man, don't break all my code!  <wink>

Surely, you mean <>wink<>?

From barry at python.org  Tue Apr  8 03:15:47 2014
From: barry at python.org (Barry Warsaw)
Date: Mon, 7 Apr 2014 21:15:47 -0400
Subject: [Python-Dev] stupid jokes (was PEP 465: A dedicated infix
 operator for matrix multiplication)
In-Reply-To: <1396919631.9227.103902661.12892955@webmail.messagingengine.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
 <CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>
 <CAP7h-xY_tq8WzkY+U4Fcno9ttnoBwHnY+sY6L7V5rfxmoamaNQ@mail.gmail.com>
 <20140407175844.00f92cfc@anarchist.wooz.org>
 <1396908258.9798.103852961.532425F7@webmail.messagingengine.com>
 <20140408010429.GH16466@ando>
 <1396919177.7953.103901257.70C90989@webmail.messagingengine.com>
 <20140407211156.701663f3@limelight.wooz.org>
 <1396919631.9227.103902661.12892955@webmail.messagingengine.com>
Message-ID: <20140407211547.198487d7@anarchist.wooz.org>

On Apr 07, 2014, at 06:13 PM, Benjamin Peterson wrote:

>On Mon, Apr 7, 2014, at 18:11, Barry Warsaw wrote:
>> On Apr 07, 2014, at 06:06 PM, Benjamin Peterson wrote:
>> 
>> >> > It occurs to me that since that Aprils' Fools joke is many years old
>> >> > now, we should remove it.
>> >> 
>> >> -1 on removal.
>> >
>> >You can't be serious.
>> 
>> Hey man, don't break all my code!  <wink>
>
>Surely, you mean <>wink<>?

Now you get it! :)

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140407/cddabb93/attachment.sig>

From guido at python.org  Tue Apr  8 03:45:37 2014
From: guido at python.org (Guido van Rossum)
Date: Mon, 7 Apr 2014 18:45:37 -0700
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <1396919177.7953.103901257.70C90989@webmail.messagingengine.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
 <CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>
 <CAP7h-xY_tq8WzkY+U4Fcno9ttnoBwHnY+sY6L7V5rfxmoamaNQ@mail.gmail.com>
 <20140407175844.00f92cfc@anarchist.wooz.org>
 <1396908258.9798.103852961.532425F7@webmail.messagingengine.com>
 <20140408010429.GH16466@ando>
 <1396919177.7953.103901257.70C90989@webmail.messagingengine.com>
Message-ID: <CAP7+vJKJQx_OJP2M2iVDJK_FKqDj+xWMQYK1HNSLgLH7nB6cOg@mail.gmail.com>

So what? Aren't we allowed to have fun? :-)


On Mon, Apr 7, 2014 at 6:06 PM, Benjamin Peterson <benjamin at python.org>wrote:

> On Mon, Apr 7, 2014, at 18:04, Steven D'Aprano wrote:
> > On Mon, Apr 07, 2014 at 03:04:18PM -0700, Benjamin Peterson wrote:
> > > On Mon, Apr 7, 2014, at 14:58, Barry Warsaw wrote:
> > > > On Apr 07, 2014, at 05:47 PM, Alexander Belopolsky wrote:
> > > >
> > > > >Python used to have an alias <> for != and I for one miss <> in
> 3.x.  I
> > > > >don't think TOOWTDI should be the last word in this debate.
> > > >
> > > > PEP 401 to the rescue:
> > >
> > > It occurs to me that since that Aprils' Fools joke is many years old
> > > now, we should remove it.
> >
> > -1 on removal.
>
> You can't be serious.
>
> >
> > It makes a nice Easter Egg, especially now that "import this" has become
> > less of an Easter Egg and more of a standard Python module :-)
>
> It's a terrible Easter Egg because it's basically a CPython core
> developer in-joke.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140407/70a312ab/attachment.html>

From python at mrabarnett.plus.com  Tue Apr  8 03:56:52 2014
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 08 Apr 2014 02:56:52 +0100
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <CAP7+vJKJQx_OJP2M2iVDJK_FKqDj+xWMQYK1HNSLgLH7nB6cOg@mail.gmail.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
 <CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>
 <CAP7h-xY_tq8WzkY+U4Fcno9ttnoBwHnY+sY6L7V5rfxmoamaNQ@mail.gmail.com>
 <20140407175844.00f92cfc@anarchist.wooz.org>
 <1396908258.9798.103852961.532425F7@webmail.messagingengine.com>
 <20140408010429.GH16466@ando>
 <1396919177.7953.103901257.70C90989@webmail.messagingengine.com>
 <CAP7+vJKJQx_OJP2M2iVDJK_FKqDj+xWMQYK1HNSLgLH7nB6cOg@mail.gmail.com>
Message-ID: <53435764.4080003@mrabarnett.plus.com>

On 2014-04-08 02:45, Guido van Rossum wrote:
> So what? Aren't we allowed to have fun? :-)
>
Next thing you know, he'll be threatening people with The Comfy Chair!
>
> On Mon, Apr 7, 2014 at 6:06 PM, Benjamin Peterson <benjamin at python.org
> <mailto:benjamin at python.org>> wrote:
>
>     On Mon, Apr 7, 2014, at 18:04, Steven D'Aprano wrote:
>      > On Mon, Apr 07, 2014 at 03:04:18PM -0700, Benjamin Peterson wrote:
>      > > On Mon, Apr 7, 2014, at 14:58, Barry Warsaw wrote:
>      > > > On Apr 07, 2014, at 05:47 PM, Alexander Belopolsky wrote:
>      > > >
>      > > > >Python used to have an alias <> for != and I for one miss <>
>     in 3.x.  I
>      > > > >don't think TOOWTDI should be the last word in this debate.
>      > > >
>      > > > PEP 401 to the rescue:
>      > >
>      > > It occurs to me that since that Aprils' Fools joke is many
>     years old
>      > > now, we should remove it.
>      >
>      > -1 on removal.
>
>     You can't be serious.
>
>      >
>      > It makes a nice Easter Egg, especially now that "import this" has
>     become
>      > less of an Easter Egg and more of a standard Python module :-)
>
>     It's a terrible Easter Egg because it's basically a CPython core
>     developer in-joke.


From steve at pearwood.info  Tue Apr  8 04:02:34 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 8 Apr 2014 12:02:34 +1000
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <1396919177.7953.103901257.70C90989@webmail.messagingengine.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
 <CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>
 <CAP7h-xY_tq8WzkY+U4Fcno9ttnoBwHnY+sY6L7V5rfxmoamaNQ@mail.gmail.com>
 <20140407175844.00f92cfc@anarchist.wooz.org>
 <1396908258.9798.103852961.532425F7@webmail.messagingengine.com>
 <20140408010429.GH16466@ando>
 <1396919177.7953.103901257.70C90989@webmail.messagingengine.com>
Message-ID: <20140408020234.GI16466@ando>

On Mon, Apr 07, 2014 at 06:06:17PM -0700, Benjamin Peterson wrote:
> On Mon, Apr 7, 2014, at 18:04, Steven D'Aprano wrote:
> > On Mon, Apr 07, 2014 at 03:04:18PM -0700, Benjamin Peterson wrote:
> > > On Mon, Apr 7, 2014, at 14:58, Barry Warsaw wrote:
> > > > On Apr 07, 2014, at 05:47 PM, Alexander Belopolsky wrote:
> > > > 
> > > > >Python used to have an alias <> for != and I for one miss <> in 3.x.  I
> > > > >don't think TOOWTDI should be the last word in this debate.
> > > > 
> > > > PEP 401 to the rescue:
> > > 
> > > It occurs to me that since that Aprils' Fools joke is many years old
> > > now, we should remove it.
> > 
> > -1 on removal.
> 
> You can't be serious.

I can't? Would it help if I sprinkle smileys and *winks* throughout my 
post?


> > It makes a nice Easter Egg, especially now that "import this" has become 
> > less of an Easter Egg and more of a standard Python module :-)
> 
> It's a terrible Easter Egg because it's basically a CPython core
> developer in-joke.

Are we really going to start arguing about humour and what makes a good 
easter egg? I suppose next you're going to tell me that Monty Python 
isn't very funny.

It is precisely because it is a subtle in-joke that makes it a good 
easter egg. It's not difficult to find, just import __future__ as a 
regular module and call dir(__future__), so the fun is not in 
discovering the egg, but in working out what it does and what it means.

Many, many more people take part in the CPython core developer culture 
than just the core developers themselves. Look at the readership of this 
mailing list, which is open to the public and has regular posters who 
aren't core developers. In-jokes like Guido as the BDFL and Tim Peter's 
"adverb-phrase-ly 'yrs" signatures have become widely known throughout 
the Python community. Barry as FLUFL is less well known, but still 
accessible to anyone willing to put the effort in. These days, when 
almost any in-joke is only a quick google search away from being 
explained, that effort is trivial.

So yes, I am serious.


-- 
Steven

From rosuav at gmail.com  Tue Apr  8 05:05:29 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 8 Apr 2014 13:05:29 +1000
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <20140408020234.GI16466@ando>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
 <CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>
 <CAP7h-xY_tq8WzkY+U4Fcno9ttnoBwHnY+sY6L7V5rfxmoamaNQ@mail.gmail.com>
 <20140407175844.00f92cfc@anarchist.wooz.org>
 <1396908258.9798.103852961.532425F7@webmail.messagingengine.com>
 <20140408010429.GH16466@ando>
 <1396919177.7953.103901257.70C90989@webmail.messagingengine.com>
 <20140408020234.GI16466@ando>
Message-ID: <CAPTjJmoqBhTEZ8fTuPb90Hgy=X6veFshpm0g-NUhzimaaRuskg@mail.gmail.com>

On Tue, Apr 8, 2014 at 12:02 PM, Steven D'Aprano <steve at pearwood.info> wrote:
>> You can't be serious.
>
> I can't? Would it help if I sprinkle smileys and *winks* throughout my
> post?

You can be serious, Steven, but it's more likely to happen if you
*don't* use smileys...

*not very serious*

ChrisA

From ncoghlan at gmail.com  Tue Apr  8 06:08:16 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 8 Apr 2014 14:08:16 +1000
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <53435764.4080003@mrabarnett.plus.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
 <CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>
 <CAP7h-xY_tq8WzkY+U4Fcno9ttnoBwHnY+sY6L7V5rfxmoamaNQ@mail.gmail.com>
 <20140407175844.00f92cfc@anarchist.wooz.org>
 <1396908258.9798.103852961.532425F7@webmail.messagingengine.com>
 <20140408010429.GH16466@ando>
 <1396919177.7953.103901257.70C90989@webmail.messagingengine.com>
 <CAP7+vJKJQx_OJP2M2iVDJK_FKqDj+xWMQYK1HNSLgLH7nB6cOg@mail.gmail.com>
 <53435764.4080003@mrabarnett.plus.com>
Message-ID: <CADiSq7cN7-i_2=4zrDkib5zUXH5YtxhW4jm9_TJa+hmAGSrfxQ@mail.gmail.com>

On 7 Apr 2014 21:58, "MRAB" <python at mrabarnett.plus.com> wrote:
>
> On 2014-04-08 02:45, Guido van Rossum wrote:
>>
>> So what? Aren't we allowed to have fun? :-)
>>
> Next thing you know, he'll be threatening people with The Comfy Chair!

You may want to take a look at the packaging metadata 2.0 spec ;)

I was also going to add a +1 for the actual topic of this thread, but
Guido's acceptance of the PEP rendered that point rather moot :)

Cheers,
Nick.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140408/0b635f23/attachment.html>

From victor.stinner at gmail.com  Tue Apr  8 09:11:40 2014
From: victor.stinner at gmail.com (Victor Stinner)
Date: Tue, 8 Apr 2014 09:11:40 +0200
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <20140408010429.GH16466@ando>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
 <CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>
 <CAP7h-xY_tq8WzkY+U4Fcno9ttnoBwHnY+sY6L7V5rfxmoamaNQ@mail.gmail.com>
 <20140407175844.00f92cfc@anarchist.wooz.org>
 <1396908258.9798.103852961.532425F7@webmail.messagingengine.com>
 <20140408010429.GH16466@ando>
Message-ID: <CAMpsgwYGBdWUomzS_1Doh=WxE6j87Sdtkv8SmQ6qN_AS9MMhmw@mail.gmail.com>

2014-04-08 3:04 GMT+02:00 Steven D'Aprano <steve at pearwood.info>:
>> > >Python used to have an alias <> for != and I for one miss <> in 3.x.  I
>> > >don't think TOOWTDI should be the last word in this debate.
>> >
>> > PEP 401 to the rescue:
>>
>> It occurs to me that since that Aprils' Fools joke is many years old
>> now, we should remove it.
>
> -1 on removal.
>
> It makes a nice Easter Egg, especially now that "import this" has become
> less of an Easter Egg and more of a standard Python module :-)

I'm also against the removal of jokes! Removing the antigravity module
would break the backward compatibility! from __futures__ import
braces!

Ten years ago, we asked me to add the "IPv6" feature in a firewall. I
started to implement the RFC 1924 to have a full support. In the C
language, handling the base 85 is not simple because you need
arithmetic operations on large integers (128 bits).
https://tools.ietf.org/html/rfc1924

3 days later, when my code was working, I saw the date of the RFC... I
was young and naive :-)

Victor

From bjourne at gmail.com  Tue Apr  8 10:58:52 2014
From: bjourne at gmail.com (=?ISO-8859-1?Q?Bj=F6rn_Lindqvist?=)
Date: Tue, 8 Apr 2014 10:58:52 +0200
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
Message-ID: <CALG+76cEoo92AecUNAVdK8QOsZvQ+SovJo7rRvFGJKnxj=uEOA@mail.gmail.com>

2014-04-07 3:41 GMT+02:00 Nathaniel Smith <njs at pobox.com>:
> So, I guess as far as I'm concerned, this is ready to go. Feedback welcome:
>   http://legacy.python.org/dev/peps/pep-0465/

Couldn't you please have made your motivation example actually runnable?

import numpy as np
from numpy.linalg import inv, solve

# Using dot function:
S = np.dot((np.dot(H, beta) - r).T,
           np.dot(inv(np.dot(np.dot(H, V), H.T)), np.dot(H, beta) - r))

# Using dot method:
S = (H.dot(beta) - r).T.dot(inv(H.dot(V).dot(H.T))).dot(H.dot(beta) - r)

Don't keep your reader hanging! Tell us what the magical variables H,
beta, r and V are. And why import solve when you aren't using it?
Curious readers that aren't very good at matrix math, like me, should
still be able to follow your logic. Even if it is just random data,
it's better than nothing!


-- 
mvh/best regards Bj?rn Lindqvist

From sturla.molden at gmail.com  Tue Apr  8 12:23:03 2014
From: sturla.molden at gmail.com (Sturla Molden)
Date: Tue, 8 Apr 2014 10:23:03 +0000 (UTC)
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CALG+76cEoo92AecUNAVdK8QOsZvQ+SovJo7rRvFGJKnxj=uEOA@mail.gmail.com>
Message-ID: <2109428694418644466.649566sturla.molden-gmail.com@news.gmane.org>

Bj?rn Lindqvist <bjourne at gmail.com> wrote:

> import numpy as np
> from numpy.linalg import inv, solve
> 
> # Using dot function:
> S = np.dot((np.dot(H, beta) - r).T,
>            np.dot(inv(np.dot(np.dot(H, V), H.T)), np.dot(H, beta) - r))
> 
> # Using dot method:
> S = (H.dot(beta) - r).T.dot(inv(H.dot(V).dot(H.T))).dot(H.dot(beta) - r)
> 
> Don't keep your reader hanging! Tell us what the magical variables H,
> beta, r and V are. And why import solve when you aren't using it?
> Curious readers that aren't very good at matrix math, like me, should
> still be able to follow your logic. Even if it is just random data,
> it's better than nothing!

Perhaps. But you don't need to know matrix multiplication to see that those
expressions are not readable. And by extension, you can still imagine that
bugs can easily hide in unreadable code.  

Matrix multiplications are used extensively in anything from engineering to
statistics to computer graphics (2D and 3D). This operator will be a good
thing for a lot of us.

Sturla


From bjourne at gmail.com  Tue Apr  8 13:24:40 2014
From: bjourne at gmail.com (=?ISO-8859-1?Q?Bj=F6rn_Lindqvist?=)
Date: Tue, 8 Apr 2014 13:24:40 +0200
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <2109428694418644466.649566sturla.molden-gmail.com@news.gmane.org>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CALG+76cEoo92AecUNAVdK8QOsZvQ+SovJo7rRvFGJKnxj=uEOA@mail.gmail.com>
 <2109428694418644466.649566sturla.molden-gmail.com@news.gmane.org>
Message-ID: <CALG+76f8JEVy29-S12OkWU08jxOY5_Vzjj6SWLG7khcvY8acPg@mail.gmail.com>

2014-04-08 12:23 GMT+02:00 Sturla Molden <sturla.molden at gmail.com>:
> Bj?rn Lindqvist <bjourne at gmail.com> wrote:
>
>> import numpy as np
>> from numpy.linalg import inv, solve
>>
>> # Using dot function:
>> S = np.dot((np.dot(H, beta) - r).T,
>>            np.dot(inv(np.dot(np.dot(H, V), H.T)), np.dot(H, beta) - r))
>>
>> # Using dot method:
>> S = (H.dot(beta) - r).T.dot(inv(H.dot(V).dot(H.T))).dot(H.dot(beta) - r)
>>
>> Don't keep your reader hanging! Tell us what the magical variables H,
>> beta, r and V are. And why import solve when you aren't using it?
>> Curious readers that aren't very good at matrix math, like me, should
>> still be able to follow your logic. Even if it is just random data,
>> it's better than nothing!
>
> Perhaps. But you don't need to know matrix multiplication to see that those
> expressions are not readable. And by extension, you can still imagine that
> bugs can easily hide in unreadable code.
>
> Matrix multiplications are used extensively in anything from engineering to
> statistics to computer graphics (2D and 3D). This operator will be a good
> thing for a lot of us.

All I ask for is to be able to see that with my own eyes. Maybe there
is some drastic improvement I can invent to make the algorithm much
more readable? Then the PEP:s motivation falls down. I don't want to
have to believe that the code the pep author came up with is the most
optimal. I want to prove that for myself.


-- 
mvh/best regards Bj?rn Lindqvist

From dholth at gmail.com  Tue Apr  8 14:12:19 2014
From: dholth at gmail.com (Daniel Holth)
Date: Tue, 8 Apr 2014 08:12:19 -0400
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <CADiSq7cN7-i_2=4zrDkib5zUXH5YtxhW4jm9_TJa+hmAGSrfxQ@mail.gmail.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
 <CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>
 <CAP7h-xY_tq8WzkY+U4Fcno9ttnoBwHnY+sY6L7V5rfxmoamaNQ@mail.gmail.com>
 <20140407175844.00f92cfc@anarchist.wooz.org>
 <1396908258.9798.103852961.532425F7@webmail.messagingengine.com>
 <20140408010429.GH16466@ando>
 <1396919177.7953.103901257.70C90989@webmail.messagingengine.com>
 <CAP7+vJKJQx_OJP2M2iVDJK_FKqDj+xWMQYK1HNSLgLH7nB6cOg@mail.gmail.com>
 <53435764.4080003@mrabarnett.plus.com>
 <CADiSq7cN7-i_2=4zrDkib5zUXH5YtxhW4jm9_TJa+hmAGSrfxQ@mail.gmail.com>
Message-ID: <CAG8k2+7fLhALKK8nGhqHLUnwrC5tkvKxTk=q6ob-n6+fj7Mcrw@mail.gmail.com>

On Tue, Apr 8, 2014 at 12:08 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
> On 7 Apr 2014 21:58, "MRAB" <python at mrabarnett.plus.com> wrote:
>>
>> On 2014-04-08 02:45, Guido van Rossum wrote:
>>>
>>> So what? Aren't we allowed to have fun? :-)
>>>
>> Next thing you know, he'll be threatening people with The Comfy Chair!
>
> You may want to take a look at the packaging metadata 2.0 spec ;)
>
> I was also going to add a +1 for the actual topic of this thread, but
> Guido's acceptance of the PEP rendered that point rather moot :)

@ [1]

From ncoghlan at gmail.com  Tue Apr  8 14:32:00 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 8 Apr 2014 22:32:00 +1000
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <CALG+76f8JEVy29-S12OkWU08jxOY5_Vzjj6SWLG7khcvY8acPg@mail.gmail.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CALG+76cEoo92AecUNAVdK8QOsZvQ+SovJo7rRvFGJKnxj=uEOA@mail.gmail.com>
 <2109428694418644466.649566sturla.molden-gmail.com@news.gmane.org>
 <CALG+76f8JEVy29-S12OkWU08jxOY5_Vzjj6SWLG7khcvY8acPg@mail.gmail.com>
Message-ID: <CADiSq7eu6WCpTkA6xGuHFCskivpMz9aj-ELQ8-seg4w9vwUPQQ@mail.gmail.com>

On 8 April 2014 21:24, Bj?rn Lindqvist <bjourne at gmail.com> wrote:
> 2014-04-08 12:23 GMT+02:00 Sturla Molden <sturla.molden at gmail.com>:
>> Bj?rn Lindqvist <bjourne at gmail.com> wrote:
>>
>>> import numpy as np
>>> from numpy.linalg import inv, solve
>>>
>>> # Using dot function:
>>> S = np.dot((np.dot(H, beta) - r).T,
>>>            np.dot(inv(np.dot(np.dot(H, V), H.T)), np.dot(H, beta) - r))
>>>
>>> # Using dot method:
>>> S = (H.dot(beta) - r).T.dot(inv(H.dot(V).dot(H.T))).dot(H.dot(beta) - r)
>>>
>>> Don't keep your reader hanging! Tell us what the magical variables H,
>>> beta, r and V are. And why import solve when you aren't using it?
>>> Curious readers that aren't very good at matrix math, like me, should
>>> still be able to follow your logic. Even if it is just random data,
>>> it's better than nothing!
>>
>> Perhaps. But you don't need to know matrix multiplication to see that those
>> expressions are not readable. And by extension, you can still imagine that
>> bugs can easily hide in unreadable code.
>>
>> Matrix multiplications are used extensively in anything from engineering to
>> statistics to computer graphics (2D and 3D). This operator will be a good
>> thing for a lot of us.
>
> All I ask for is to be able to see that with my own eyes. Maybe there
> is some drastic improvement I can invent to make the algorithm much
> more readable? Then the PEP:s motivation falls down. I don't want to
> have to believe that the code the pep author came up with is the most
> optimal. I want to prove that for myself.

Note that the relationship between the CPython core development team
and the Numeric Python community is based heavily on trust - we don't
expect them to teach us to become numeric experts, we just expect them
to explain themselves well enough to persuade us that a core language
or interpreter change is the only realistic way to achieve a
particular goal. This does occasionally result in quirky patterns of
feature adoption, as things like extended slicing, full rich
comparison support, ellipsis support, rich buffer API support, and now
matrix multiplication support, were added for the numeric community's
benefit without necessarily offering any immediately obvious benefit
for those not using the numeric Python stack - it was only later that
they became pervasively adopted throughout the standard library (with
matmul, for example, a follow on proposal to allow tuples, lists and
arrays to handle vector dot products may make sense).

This particular problem has been kicking around long enough, and is
sufficiently familiar to several of us, that what's in the PEP already
presents a compelling rationale for the *folks that need to be
convinced* (which is primarily Guido, but if enough of the other core
devs think something is a questionable idea, we can often talk him out
of it - that's not the case here though).

Attempting to condense that preceding 10+ years of history *into the
PEP itself* wouldn't be a good return on investment - the links to the
earlier PEPs are there, as are the links to these discussion threads.

Cheers,
Nick.

P.S. We've been relatively successful in getting a similar trust based
dynamic off the ground for the packaging and distribution community
over the past year or so. The next big challenge in trust based
delegation for the core development team will likely be around a
Python 3.5+ only WSGI 2.0 (that can assume the Python 3 text model,
the restoration of binary interpolation, the availability of asyncio,
etc), but most of the likely principals there are still burned out
from the WSGI 1.1 debate and the Python 3 transition in general :(

>
>
> --
> mvh/best regards Bj?rn Lindqvist
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com



-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From thomas at python.org  Tue Apr  8 15:11:11 2014
From: thomas at python.org (Thomas Wouters)
Date: Tue, 8 Apr 2014 06:11:11 -0700
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <CADiSq7eu6WCpTkA6xGuHFCskivpMz9aj-ELQ8-seg4w9vwUPQQ@mail.gmail.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CALG+76cEoo92AecUNAVdK8QOsZvQ+SovJo7rRvFGJKnxj=uEOA@mail.gmail.com>
 <2109428694418644466.649566sturla.molden-gmail.com@news.gmane.org>
 <CALG+76f8JEVy29-S12OkWU08jxOY5_Vzjj6SWLG7khcvY8acPg@mail.gmail.com>
 <CADiSq7eu6WCpTkA6xGuHFCskivpMz9aj-ELQ8-seg4w9vwUPQQ@mail.gmail.com>
Message-ID: <CAPdQG2o+mFQ5B+HN1t62=Y+26ccE1yGyLrtqx46fZZjPnNqs2g@mail.gmail.com>

On Apr 8, 2014 2:39 PM, "Nick Coghlan" <ncoghlan at gmail.com> wrote:
>
> On 8 April 2014 21:24, Bj?rn Lindqvist <bjourne at gmail.com> wrote:
> > 2014-04-08 12:23 GMT+02:00 Sturla Molden <sturla.molden at gmail.com>:
> >> Bj?rn Lindqvist <bjourne at gmail.com> wrote:
> >>
> >>> import numpy as np
> >>> from numpy.linalg import inv, solve
> >>>
> >>> # Using dot function:
> >>> S = np.dot((np.dot(H, beta) - r).T,
> >>>            np.dot(inv(np.dot(np.dot(H, V), H.T)), np.dot(H, beta) -
r))
> >>>
> >>> # Using dot method:
> >>> S = (H.dot(beta) - r).T.dot(inv(H.dot(V).dot(H.T))).dot(H.dot(beta) -
r)
> >>>
> >>> Don't keep your reader hanging! Tell us what the magical variables H,
> >>> beta, r and V are. And why import solve when you aren't using it?
> >>> Curious readers that aren't very good at matrix math, like me, should
> >>> still be able to follow your logic. Even if it is just random data,
> >>> it's better than nothing!
> >>
> >> Perhaps. But you don't need to know matrix multiplication to see that
those
> >> expressions are not readable. And by extension, you can still imagine
that
> >> bugs can easily hide in unreadable code.
> >>
> >> Matrix multiplications are used extensively in anything from
engineering to
> >> statistics to computer graphics (2D and 3D). This operator will be a
good
> >> thing for a lot of us.
> >
> > All I ask for is to be able to see that with my own eyes. Maybe there
> > is some drastic improvement I can invent to make the algorithm much
> > more readable? Then the PEP:s motivation falls down. I don't want to
> > have to believe that the code the pep author came up with is the most
> > optimal. I want to prove that for myself.
>
> Note that the relationship between the CPython core development team
> and the Numeric Python community is based heavily on trust - we don't
> expect them to teach us to become numeric experts, we just expect them
> to explain themselves well enough to persuade us that a core language
> or interpreter change is the only realistic way to achieve a
> particular goal. This does occasionally result in quirky patterns of
> feature adoption, as things like extended slicing, full rich
> comparison support, ellipsis support, rich buffer API support, and now
> matrix multiplication support, were added for the numeric community's
> benefit without necessarily offering any immediately obvious benefit
> for those not using the numeric Python stack - it was only later that
> they became pervasively adopted throughout the standard library (with
> matmul, for example, a follow on proposal to allow tuples, lists and
> arrays to handle vector dot products may make sense).
>
> This particular problem has been kicking around long enough, and is

Just to give an indication how long, this came up in the discussion around
PEP 203, too. Fourteen years ago. In fact, augmented assignment could be
listed as one of the features added for the benefit of numeric array folks.

At the time, the discussions were not as focused and the proposals much
more pie-in-the-sky (the idea to have all of @*, @+, @-, @%, etc, as
individual operators specifically stuck in my mind.) This is a much more
grounded proposal.

> sufficiently familiar to several of us, that what's in the PEP already
> presents a compelling rationale for the *folks that need to be
> convinced* (which is primarily Guido, but if enough of the other core
> devs think something is a questionable idea, we can often talk him out
> of it - that's not the case here though).
>
> Attempting to condense that preceding 10+ years of history *into the
> PEP itself* wouldn't be a good return on investment - the links to the
> earlier PEPs are there, as are the links to these discussion threads.
>
> Cheers,
> Nick.
>
> P.S. We've been relatively successful in getting a similar trust based
> dynamic off the ground for the packaging and distribution community
> over the past year or so. The next big challenge in trust based
> delegation for the core development team will likely be around a
> Python 3.5+ only WSGI 2.0 (that can assume the Python 3 text model,
> the restoration of binary interpolation, the availability of asyncio,
> etc), but most of the likely principals there are still burned out
> from the WSGI 1.1 debate and the Python 3 transition in general :(
>
> >
> >
> > --
> > mvh/best regards Bj?rn Lindqvist
> > _______________________________________________
> > Python-Dev mailing list
> > Python-Dev at python.org
> > https://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
>
>
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/thomas%40python.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140408/cda50401/attachment.html>

From njs at pobox.com  Tue Apr  8 14:52:59 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Tue, 8 Apr 2014 13:52:59 +0100
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <CALG+76cEoo92AecUNAVdK8QOsZvQ+SovJo7rRvFGJKnxj=uEOA@mail.gmail.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CALG+76cEoo92AecUNAVdK8QOsZvQ+SovJo7rRvFGJKnxj=uEOA@mail.gmail.com>
Message-ID: <CAPJVwB=JP4Ck01g2zcovUrH3Po-TLPcuZcnqi+M=zCmCp3x+dA@mail.gmail.com>

On Tue, Apr 8, 2014 at 9:58 AM, Bj?rn Lindqvist <bjourne at gmail.com> wrote:
> 2014-04-07 3:41 GMT+02:00 Nathaniel Smith <njs at pobox.com>:
>> So, I guess as far as I'm concerned, this is ready to go. Feedback welcome:
>>   http://legacy.python.org/dev/peps/pep-0465/
>
> Couldn't you please have made your motivation example actually runnable?
>
> import numpy as np
> from numpy.linalg import inv, solve
>
> # Using dot function:
> S = np.dot((np.dot(H, beta) - r).T,
>            np.dot(inv(np.dot(np.dot(H, V), H.T)), np.dot(H, beta) - r))
>
> # Using dot method:
> S = (H.dot(beta) - r).T.dot(inv(H.dot(V).dot(H.T))).dot(H.dot(beta) - r)
>
> Don't keep your reader hanging! Tell us what the magical variables H,
> beta, r and V are. And why import solve when you aren't using it?
> Curious readers that aren't very good at matrix math, like me, should
> still be able to follow your logic. Even if it is just random data,
> it's better than nothing!

There's a footnote that explains the math in more detail and links to
the real code this was adapted from. And solve is used further down in
the section. But running it is really what you want, just insert:

beta = np.random.randn(10)
H = np.random.randn(2, 10)
r = np.random.randn(2)
V = np.random.randn(10, 10)

Does that help? ;-)

See also:
    https://mail.python.org/pipermail/python-ideas/2014-March/027077.html

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

From kir_per at rambler.ru  Tue Apr  8 16:49:13 2014
From: kir_per at rambler.ru (Kirill)
Date: Tue, 8 Apr 2014 18:49:13 +0400
Subject: [Python-Dev] issue with itertools leads the crash
In-Reply-To: <1396721260.939007.13044.8221@mail.rambler.ru>
References: <1396721260.939007.13044.8221@mail.rambler.ru>
Message-ID: <1396968553.65579.26301.58970@mail.rambler.ru>

  issue with itertools leads the crash


  The following code leads to system failure and crash on Ubuntu 12.04.3 with
  Python 2.7.6.

  import itertools

  r = range (1,10)

  r[1:9:2] = itertools.cycle([0])

  Kirill .
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140408/13256317/attachment.html>

From bcannon at gmail.com  Tue Apr  8 17:31:35 2014
From: bcannon at gmail.com (Brett Cannon)
Date: Tue, 08 Apr 2014 15:31:35 +0000
Subject: [Python-Dev]  issue with itertools leads the crash
References: <1396721260.939007.13044.8221@mail.rambler.ru>
 <1396968553.65579.26301.58970@mail.rambler.ru>
Message-ID: <CAP1=2W7Nzr12hpVNVyuORNAqC8TbyN0r+Vpv4WbkDNx0GsP1Hw@mail.gmail.com>

Please a file a bug at bugs.python.org, otherwise this is most likely going
to get lost track of.

On Tue Apr 08 2014 at 11:30:15 AM, Kirill <kir_per at rambler.ru> wrote:

>  issue with itertools leads the crash
>
>   The following code leads to system failure and crash on Ubuntu 12.04.3
> with Python 2.7.6.
>
> import itertools
>
> r = range (1,10)
>
> r[1:9:2] = itertools.cycle([0])
>
>  Kirill .
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140408/d27dc6ad/attachment.html>

From rymg19 at gmail.com  Tue Apr  8 17:55:19 2014
From: rymg19 at gmail.com (Ryan)
Date: Tue, 08 Apr 2014 10:55:19 -0500
Subject: [Python-Dev] PEP 465: A dedicated infix operator for
	matrix	multiplication
In-Reply-To: <CAMpsgwYGBdWUomzS_1Doh=WxE6j87Sdtkv8SmQ6qN_AS9MMhmw@mail.gmail.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
 <CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>
 <CAP7h-xY_tq8WzkY+U4Fcno9ttnoBwHnY+sY6L7V5rfxmoamaNQ@mail.gmail.com>
 <20140407175844.00f92cfc@anarchist.wooz.org>
 <1396908258.9798.103852961.532425F7@webmail.messagingengine.com>
 <20140408010429.GH16466@ando>
 <CAMpsgwYGBdWUomzS_1Doh=WxE6j87Sdtkv8SmQ6qN_AS9MMhmw@mail.gmail.com>
Message-ID: <4e2a1615-a305-4a27-854a-32363f3e86b9@email.android.com>

Ooooh...that stings.

Victor Stinner <victor.stinner at gmail.com> wrote:
>2014-04-08 3:04 GMT+02:00 Steven D'Aprano <steve at pearwood.info>:
>>> > >Python used to have an alias <> for != and I for one miss <> in
>3.x.  I
>>> > >don't think TOOWTDI should be the last word in this debate.
>>> >
>>> > PEP 401 to the rescue:
>>>
>>> It occurs to me that since that Aprils' Fools joke is many years old
>>> now, we should remove it.
>>
>> -1 on removal.
>>
>> It makes a nice Easter Egg, especially now that "import this" has
>become
>> less of an Easter Egg and more of a standard Python module :-)
>
>I'm also against the removal of jokes! Removing the antigravity module
>would break the backward compatibility! from __futures__ import
>braces!
>
>Ten years ago, we asked me to add the "IPv6" feature in a firewall. I
>started to implement the RFC 1924 to have a full support. In the C
>language, handling the base 85 is not simple because you need
>arithmetic operations on large integers (128 bits).
>https://tools.ietf.org/html/rfc1924
>
>3 days later, when my code was working, I saw the date of the RFC... I
>was young and naive :-)
>
>Victor
>_______________________________________________
>Python-Dev mailing list
>Python-Dev at python.org
>https://mail.python.org/mailman/listinfo/python-dev
>Unsubscribe:
>https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com

-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140408/e7e5ea13/attachment-0001.html>

From python at mrabarnett.plus.com  Tue Apr  8 18:30:18 2014
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 08 Apr 2014 17:30:18 +0100
Subject: [Python-Dev] issue with itertools leads the crash
In-Reply-To: <CAP1=2W7Nzr12hpVNVyuORNAqC8TbyN0r+Vpv4WbkDNx0GsP1Hw@mail.gmail.com>
References: <1396721260.939007.13044.8221@mail.rambler.ru>
 <1396968553.65579.26301.58970@mail.rambler.ru>
 <CAP1=2W7Nzr12hpVNVyuORNAqC8TbyN0r+Vpv4WbkDNx0GsP1Hw@mail.gmail.com>
Message-ID: <5344241A.6040208@mrabarnett.plus.com>

On 2014-04-08 16:31, Brett Cannon wrote:
> Please a file a bug at bugs.python.org <http://bugs.python.org>,
> otherwise this is most likely going to get lost track of.
>
> On Tue Apr 08 2014 at 11:30:15 AM, Kirill <kir_per at rambler.ru
> <mailto:kir_per at rambler.ru>> wrote:
>
>         issue with itertools leads the crash
>
>
>           The following code leads to system failure and crash on Ubuntu
>         12.04.3 with Python 2.7.6.
>
>         import itertools
>
>         r = range (1,10)
>
>         r[1:9:2] = itertools.cycle([0])
>
If the RHS yields too few, e.g. 3, you'll get:

ValueError: attempt to assign sequence of size 3 to extended slice of size 4

If it yields too many, e.g. 10, you'll get:

ValueError: attempt to assign sequence of size 10 to extended slice of 
size 4

It'll try to get all the values from the generator, which, of course,
will never happen with 'cycle'. It'll just consume more and more
memory, until it fails.

The fix would be to complain of raise StopIteration doesn't occur after
N+1 yields, where N is the number of values needed by the LHS.

Something for Python 3.5, maybe? :-)

It's not going to happen in Python 2.7; that's the end of the Python 2
series, and it's getting security fixes only.

From ericsnowcurrently at gmail.com  Tue Apr  8 18:40:23 2014
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Tue, 8 Apr 2014 10:40:23 -0600
Subject: [Python-Dev] issue with itertools leads the crash
In-Reply-To: <5344241A.6040208@mrabarnett.plus.com>
References: <1396721260.939007.13044.8221@mail.rambler.ru>
 <1396968553.65579.26301.58970@mail.rambler.ru>
 <CAP1=2W7Nzr12hpVNVyuORNAqC8TbyN0r+Vpv4WbkDNx0GsP1Hw@mail.gmail.com>
 <5344241A.6040208@mrabarnett.plus.com>
Message-ID: <CALFfu7Bvey0M8=CvGFist7AOhp6p6k0_YEjE3T0jLpJXZ6t9qg@mail.gmail.com>

On Apr 8, 2014 10:31 AM, "MRAB" <python at mrabarnett.plus.com> wrote:
> If the RHS yields too few, e.g. 3, you'll get:
>
> ValueError: attempt to assign sequence of size 3 to extended slice of size 4
>
> If it yields too many, e.g. 10, you'll get:
>
> ValueError: attempt to assign sequence of size 10 to extended slice of size 4
[snip]
> The fix would be to complain of raise StopIteration doesn't occur after
> N+1 yields, where N is the number of values needed by the LHS.

Perhaps N+2 would be better:

  raise RuntimeError('{} is right out'.format(n+2))

-eric

From solipsis at pitrou.net  Tue Apr  8 18:49:13 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 08 Apr 2014 18:49:13 +0200
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <20140408020234.GI16466@ando>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
 <CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>
 <CAP7h-xY_tq8WzkY+U4Fcno9ttnoBwHnY+sY6L7V5rfxmoamaNQ@mail.gmail.com>
 <20140407175844.00f92cfc@anarchist.wooz.org>
 <1396908258.9798.103852961.532425F7@webmail.messagingengine.com>
 <20140408010429.GH16466@ando>
 <1396919177.7953.103901257.70C90989@webmail.messagingengine.com>
 <20140408020234.GI16466@ando>
Message-ID: <li19a9$b9j$1@ger.gmane.org>

Le 08/04/2014 04:02, Steven D'Aprano a ?crit :
>
> Many, many more people take part in the CPython core developer culture
> than just the core developers themselves. Look at the readership of this
> mailing list, which is open to the public and has regular posters who
> aren't core developers. In-jokes like Guido as the BDFL

Is it a joke? I thought Guido was the BDFL for real :-o

Regards

Antoine.



From rosuav at gmail.com  Tue Apr  8 18:50:14 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 9 Apr 2014 02:50:14 +1000
Subject: [Python-Dev] issue with itertools leads the crash
In-Reply-To: <CALFfu7Bvey0M8=CvGFist7AOhp6p6k0_YEjE3T0jLpJXZ6t9qg@mail.gmail.com>
References: <1396721260.939007.13044.8221@mail.rambler.ru>
 <1396968553.65579.26301.58970@mail.rambler.ru>
 <CAP1=2W7Nzr12hpVNVyuORNAqC8TbyN0r+Vpv4WbkDNx0GsP1Hw@mail.gmail.com>
 <5344241A.6040208@mrabarnett.plus.com>
 <CALFfu7Bvey0M8=CvGFist7AOhp6p6k0_YEjE3T0jLpJXZ6t9qg@mail.gmail.com>
Message-ID: <CAPTjJmoqpLDE1g5U623BbgemWtdU5aWuHge0g9_+Yn8aEopmjg@mail.gmail.com>

On Wed, Apr 9, 2014 at 2:40 AM, Eric Snow <ericsnowcurrently at gmail.com> wrote:
> On Apr 8, 2014 10:31 AM, "MRAB" <python at mrabarnett.plus.com> wrote:
>> If the RHS yields too few, e.g. 3, you'll get:
>>
>> ValueError: attempt to assign sequence of size 3 to extended slice of size 4
>>
>> If it yields too many, e.g. 10, you'll get:
>>
>> ValueError: attempt to assign sequence of size 10 to extended slice of size 4
> [snip]
>> The fix would be to complain of raise StopIteration doesn't occur after
>> N+1 yields, where N is the number of values needed by the LHS.
>
> Perhaps N+2 would be better:
>
>   raise RuntimeError('{} is right out'.format(n+2))

It would be nice to have a simple notation that fetches what it needs
and ignores any extras.

a, b, c, * = x.split("-")

Bomb if there aren't two hyphens in x, but if there are more, just
take the first three and ignore the rest. Would work with infinite
iterators quite happily.

ChrisA

From rosuav at gmail.com  Tue Apr  8 19:01:07 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 9 Apr 2014 03:01:07 +1000
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <li19a9$b9j$1@ger.gmane.org>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
 <CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>
 <CAP7h-xY_tq8WzkY+U4Fcno9ttnoBwHnY+sY6L7V5rfxmoamaNQ@mail.gmail.com>
 <20140407175844.00f92cfc@anarchist.wooz.org>
 <1396908258.9798.103852961.532425F7@webmail.messagingengine.com>
 <20140408010429.GH16466@ando>
 <1396919177.7953.103901257.70C90989@webmail.messagingengine.com>
 <20140408020234.GI16466@ando> <li19a9$b9j$1@ger.gmane.org>
Message-ID: <CAPTjJmpWOeWc8eMUAsjkPwewJpW=dRFwqeb9So12hjAOm6ibAQ@mail.gmail.com>

On Wed, Apr 9, 2014 at 2:49 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le 08/04/2014 04:02, Steven D'Aprano a ?crit :
>
>>
>> Many, many more people take part in the CPython core developer culture
>> than just the core developers themselves. Look at the readership of this
>> mailing list, which is open to the public and has regular posters who
>> aren't core developers. In-jokes like Guido as the BDFL
>
>
> Is it a joke? I thought Guido was the BDFL for real :-o
>

Is it a joke, or is it serious? Where do you draw the line between wit
for the purpose of making a point, and a joke that bears a kernel of
truth?

               I've wisdom from the East and from the West,
                    That's subject to no academic rule;
               You may find it in the jeering of a jest,
                    Or distil it from the folly of a fool.
               I can teach you with a quip, if I've a mind;
               I can trick you into learning with a laugh;
               Oh, winnow all my folly, and you'll find
                    A grain or two of truth among the chaff!

-- WS Gilbert, via Jack Point the jester, in "Yeomen of the Guard"

To many people, the concept of a benevolent dictator for life whom
nobody has to obey is a joke. And yet that's exactly the truth; Guido
*is* dictator, because Python is his project. But on the other hand
(I'm running out of hands here), he wants his project to be useful to
people, so he has to follow the paradoxical plan of Jim Hacker: "I'm
their leader! I must follow them!". Joke? Truth? Neither? Both? It's
really hard to say...

It's even harder to draw the line when you have, for instance, Monty
Python references being used as metasyntactic variables. A perfectly
serious document needs to explain how to split a string into words:

>>> "Nobody expects the Spanish Inquisition!".split()
['Nobody', 'expects', 'the', 'Spanish', 'Inquisition!']

Is that a joke, or a serious example of an important string operation?
Or perchance a brilliant combination of both... and there I go quoting
another jester, in this case "The Court Jester" starring Danny Kaye.
Am I joking around because I'm citing four different comedies, or am I
making a completely serious point?

ChrisA

From steve at pearwood.info  Tue Apr  8 19:22:19 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 9 Apr 2014 03:22:19 +1000
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <li19a9$b9j$1@ger.gmane.org>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
 <CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>
 <CAP7h-xY_tq8WzkY+U4Fcno9ttnoBwHnY+sY6L7V5rfxmoamaNQ@mail.gmail.com>
 <20140407175844.00f92cfc@anarchist.wooz.org>
 <1396908258.9798.103852961.532425F7@webmail.messagingengine.com>
 <20140408010429.GH16466@ando>
 <1396919177.7953.103901257.70C90989@webmail.messagingengine.com>
 <20140408020234.GI16466@ando> <li19a9$b9j$1@ger.gmane.org>
Message-ID: <20140408172219.GM16466@ando>

On Tue, Apr 08, 2014 at 06:49:13PM +0200, Antoine Pitrou wrote:
> Le 08/04/2014 04:02, Steven D'Aprano a ?crit :
> >
> >Many, many more people take part in the CPython core developer culture
> >than just the core developers themselves. Look at the readership of this
> >mailing list, which is open to the public and has regular posters who
> >aren't core developers. In-jokes like Guido as the BDFL
> 
> Is it a joke? I thought Guido was the BDFL for real :-o

The joke is the title, not the role.

https://www.artima.com/weblogs/viewpost.jsp?thread=235725




-- 
Steven

From larry at hastings.org  Tue Apr  8 19:30:02 2014
From: larry at hastings.org (Larry Hastings)
Date: Tue, 08 Apr 2014 13:30:02 -0400
Subject: [Python-Dev] issue with itertools leads the crash
In-Reply-To: <CAPTjJmoqpLDE1g5U623BbgemWtdU5aWuHge0g9_+Yn8aEopmjg@mail.gmail.com>
References: <1396721260.939007.13044.8221@mail.rambler.ru>
 <1396968553.65579.26301.58970@mail.rambler.ru>
 <CAP1=2W7Nzr12hpVNVyuORNAqC8TbyN0r+Vpv4WbkDNx0GsP1Hw@mail.gmail.com>
 <5344241A.6040208@mrabarnett.plus.com>
 <CALFfu7Bvey0M8=CvGFist7AOhp6p6k0_YEjE3T0jLpJXZ6t9qg@mail.gmail.com>
 <CAPTjJmoqpLDE1g5U623BbgemWtdU5aWuHge0g9_+Yn8aEopmjg@mail.gmail.com>
Message-ID: <5344321A.60302@hastings.org>


On 04/08/2014 12:50 PM, Chris Angelico wrote:
> It would be nice to have a simple notation that fetches what it needs
> and ignores any extras.
>
> a, b, c, * = x.split("-")
>
> Bomb if there aren't two hyphens in x, but if there are more, just
> take the first three and ignore the rest. Would work with infinite
> iterators quite happily.

http://legacy.python.org/dev/peps/pep-0448/


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140408/8ad0b4ec/attachment.html>

From rosuav at gmail.com  Tue Apr  8 19:39:23 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 9 Apr 2014 03:39:23 +1000
Subject: [Python-Dev] issue with itertools leads the crash
In-Reply-To: <5344321A.60302@hastings.org>
References: <1396721260.939007.13044.8221@mail.rambler.ru>
 <1396968553.65579.26301.58970@mail.rambler.ru>
 <CAP1=2W7Nzr12hpVNVyuORNAqC8TbyN0r+Vpv4WbkDNx0GsP1Hw@mail.gmail.com>
 <5344241A.6040208@mrabarnett.plus.com>
 <CALFfu7Bvey0M8=CvGFist7AOhp6p6k0_YEjE3T0jLpJXZ6t9qg@mail.gmail.com>
 <CAPTjJmoqpLDE1g5U623BbgemWtdU5aWuHge0g9_+Yn8aEopmjg@mail.gmail.com>
 <5344321A.60302@hastings.org>
Message-ID: <CAPTjJmoEkiDqF-nRxM-GUjAtqH_P_GM2FdOBVNg5DDsJZ7d0hQ@mail.gmail.com>

On Wed, Apr 9, 2014 at 3:30 AM, Larry Hastings <larry at hastings.org> wrote:
> On 04/08/2014 12:50 PM, Chris Angelico wrote:
>
> It would be nice to have a simple notation that fetches what it needs
> and ignores any extras.
>
> a, b, c, * = x.split("-")
>
> Bomb if there aren't two hyphens in x, but if there are more, just
> take the first three and ignore the rest. Would work with infinite
> iterators quite happily.
>
>
> http://legacy.python.org/dev/peps/pep-0448/
>

Yes, there've been various proposals like that. The exact notation I'm
referring to here isn't mentioned in that PEP, but I know it's been
mentioned before.

ChrisA

From ethan at stoneleaf.us  Tue Apr  8 19:57:36 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Tue, 08 Apr 2014 10:57:36 -0700
Subject: [Python-Dev] [Python-checkins] cpython (3.4): docs: Document
 __objclass__. Closes #19281.
In-Reply-To: <534424DA.5080702@stoneleaf.us>
References: <3g3D6C20xQz7Lnb@mail.python.org> <534424DA.5080702@stoneleaf.us>
Message-ID: <53443890.9060301@stoneleaf.us>

Sorry, posted to wrong list the first time.

On 04/08/2014 09:33 AM, Ethan Furman wrote:
> On 04/08/2014 09:07 AM, yury.selivanov wrote:
>> http://hg.python.org/cpython/rev/0973d45197cc
>  >
>> +The :attr:`__objclass__` is interpreted by the :mod:`inspect` module as
>> +specifying the class where this object was defined (setting this
>> appropriately
>> +can assist in runtime introspection of dynamic class attributes). For
>> callables,
>> +it may indicate that an instance of the given type (or a subclass) is
>> expected
>> +or required as the first positional argument (for example, CPython
>> sets this
>> +attribute for unbound methods that are implemented in C).
>
> This would read better as
>
> The attribute :attr:`__objclass__` . . .
>
> --
> ~Ethan~
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> https://mail.python.org/mailman/listinfo/python-checkins


From yselivanov.ml at gmail.com  Tue Apr  8 20:01:33 2014
From: yselivanov.ml at gmail.com (Yury Selivanov)
Date: Tue, 08 Apr 2014 14:01:33 -0400
Subject: [Python-Dev] [Python-checkins] cpython (3.4): docs: Document
 __objclass__. Closes #19281.
In-Reply-To: <53443890.9060301@stoneleaf.us>
References: <3g3D6C20xQz7Lnb@mail.python.org> <534424DA.5080702@stoneleaf.us>
 <53443890.9060301@stoneleaf.us>
Message-ID: <5344397D.9080905@gmail.com>

Thank you, Ethan.
Fixed.

On 2014-04-08, 1:57 PM, Ethan Furman wrote:
> Sorry, posted to wrong list the first time.
>
> On 04/08/2014 09:33 AM, Ethan Furman wrote:
>> On 04/08/2014 09:07 AM, yury.selivanov wrote:
>>> http://hg.python.org/cpython/rev/0973d45197cc
>>  >
>>> +The :attr:`__objclass__` is interpreted by the :mod:`inspect` 
>>> module as
>>> +specifying the class where this object was defined (setting this
>>> appropriately
>>> +can assist in runtime introspection of dynamic class attributes). For
>>> callables,
>>> +it may indicate that an instance of the given type (or a subclass) is
>>> expected
>>> +or required as the first positional argument (for example, CPython
>>> sets this
>>> +attribute for unbound methods that are implemented in C).
>>
>> This would read better as
>>
>> The attribute :attr:`__objclass__` . . .
>>
>> -- 
>> ~Ethan~
>>
>> _______________________________________________
>> Python-checkins mailing list
>> Python-checkins at python.org
>> https://mail.python.org/mailman/listinfo/python-checkins
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/yselivanov.ml%40gmail.com


From zachary.ware+pydev at gmail.com  Tue Apr  8 22:39:47 2014
From: zachary.ware+pydev at gmail.com (Zachary Ware)
Date: Tue, 8 Apr 2014 15:39:47 -0500
Subject: [Python-Dev] Windows buildbots are red: test_idlelib
In-Reply-To: <CAMpsgwbatGMsXqET_8FZH9qST9+NUdW+TQJY6QhdAXmWhbZzQQ@mail.gmail.com>
References: <CAMpsgwbatGMsXqET_8FZH9qST9+NUdW+TQJY6QhdAXmWhbZzQQ@mail.gmail.com>
Message-ID: <CAKJDb-Pc2RfRSPFFsKaki6DoKC0jUmPnpPa4=vSRspRDbkLdWg@mail.gmail.com>

On Mon, Apr 7, 2014 at 4:37 AM, Victor Stinner <victor.stinner at gmail.com> wrote:
> Hi,
>
> Unit tests are failing on Windows because of this issue:
> http://bugs.python.org/issue21059
>
> It looks like a regression in test_idlelib introduced with this issue:
> http://bugs.python.org/issue15968
>
> Zachary Ware wrote a fix:
> http://bugs.python.org/issue20035
>
> Can someone please review Zachary's patch? If not, I suggest to revert
> changes of issue #15968 to have working Windows buildbots.

Rather than revert the change, I committed what should be a temporary
fix last night; Tools/buildbot/test[-amd64].bat now sets TCL_LIBRARY
before calling the test script.  It seems to be keeping the buildbots
happy until #20035 can reach a satisfactory conclusion.

-- 
Zach

From greg.ewing at canterbury.ac.nz  Wed Apr  9 01:13:38 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 09 Apr 2014 11:13:38 +1200
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
 multiplication
In-Reply-To: <CAMpsgwYGBdWUomzS_1Doh=WxE6j87Sdtkv8SmQ6qN_AS9MMhmw@mail.gmail.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
 <CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>
 <CAP7h-xY_tq8WzkY+U4Fcno9ttnoBwHnY+sY6L7V5rfxmoamaNQ@mail.gmail.com>
 <20140407175844.00f92cfc@anarchist.wooz.org>
 <1396908258.9798.103852961.532425F7@webmail.messagingengine.com>
 <20140408010429.GH16466@ando>
 <CAMpsgwYGBdWUomzS_1Doh=WxE6j87Sdtkv8SmQ6qN_AS9MMhmw@mail.gmail.com>
Message-ID: <534482A2.1080507@canterbury.ac.nz>

Victor Stinner wrote:
> I started to implement the RFC 1924 to have a full support.
> 
> 3 days later, when my code was working, I saw the date of the RFC...

Do you still have the code? It needn't go to waste -- this
would make a fine addition to Python's easter egg basket!

-- 
Greg


From victor.stinner at gmail.com  Wed Apr  9 01:24:03 2014
From: victor.stinner at gmail.com (Victor Stinner)
Date: Wed, 9 Apr 2014 01:24:03 +0200
Subject: [Python-Dev] Windows buildbots are red: test_idlelib
In-Reply-To: <CAKJDb-Pc2RfRSPFFsKaki6DoKC0jUmPnpPa4=vSRspRDbkLdWg@mail.gmail.com>
References: <CAMpsgwbatGMsXqET_8FZH9qST9+NUdW+TQJY6QhdAXmWhbZzQQ@mail.gmail.com>
 <CAKJDb-Pc2RfRSPFFsKaki6DoKC0jUmPnpPa4=vSRspRDbkLdWg@mail.gmail.com>
Message-ID: <CAMpsgwYGNaBh-VnT9Peoa+WbqhyNe3hoRYNZKm-iGP4Gb9Cxxg@mail.gmail.com>

Thanks !

2014-04-08 22:39 GMT+02:00 Zachary Ware <zachary.ware+pydev at gmail.com>:
> On Mon, Apr 7, 2014 at 4:37 AM, Victor Stinner <victor.stinner at gmail.com> wrote:
>> Hi,
>>
>> Unit tests are failing on Windows because of this issue:
>> http://bugs.python.org/issue21059
>>
>> It looks like a regression in test_idlelib introduced with this issue:
>> http://bugs.python.org/issue15968
>>
>> Zachary Ware wrote a fix:
>> http://bugs.python.org/issue20035
>>
>> Can someone please review Zachary's patch? If not, I suggest to revert
>> changes of issue #15968 to have working Windows buildbots.
>
> Rather than revert the change, I committed what should be a temporary
> fix last night; Tools/buildbot/test[-amd64].bat now sets TCL_LIBRARY
> before calling the test script.  It seems to be keeping the buildbots
> happy until #20035 can reach a satisfactory conclusion.
>
> --
> Zach
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com

From victor.stinner at gmail.com  Wed Apr  9 01:28:34 2014
From: victor.stinner at gmail.com (Victor Stinner)
Date: Wed, 9 Apr 2014 01:28:34 +0200
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <534482A2.1080507@canterbury.ac.nz>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
 <CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>
 <CAP7h-xY_tq8WzkY+U4Fcno9ttnoBwHnY+sY6L7V5rfxmoamaNQ@mail.gmail.com>
 <20140407175844.00f92cfc@anarchist.wooz.org>
 <1396908258.9798.103852961.532425F7@webmail.messagingengine.com>
 <20140408010429.GH16466@ando>
 <CAMpsgwYGBdWUomzS_1Doh=WxE6j87Sdtkv8SmQ6qN_AS9MMhmw@mail.gmail.com>
 <534482A2.1080507@canterbury.ac.nz>
Message-ID: <CAMpsgwYb++HgSi4=PaG2OPa5Fpmq7Aufu3f9H5ikzj5-ZquykQ@mail.gmail.com>

2014-04-09 1:13 GMT+02:00 Greg Ewing <greg.ewing at canterbury.ac.nz>:
> Victor Stinner wrote:
>>
>> I started to implement the RFC 1924 to have a full support.
>>
>> 3 days later, when my code was working, I saw the date of the RFC...
>
>
> Do you still have the code? It needn't go to waste -- this
> would make a fine addition to Python's easter egg basket!

Oh, apparently you are not aware. It's no more an april fool, Python
3.4 has support for base85!
https://docs.python.org/3.4/library/base64.html#base64.b85encode

Victor

PS: No, I didn't kept my base85 codec for IPv6 written in C.

From fn681 at ncf.ca  Wed Apr  9 00:32:49 2014
From: fn681 at ncf.ca (cjw)
Date: Tue, 08 Apr 2014 18:32:49 -0400
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <CAP7+vJKOP+7JZx5nZ5R=1AiucxUtYMigPK1gPENnfyNYoDGTtg@mail.gmail.com>
References: <CAP7+vJKOP+7JZx5nZ5R=1AiucxUtYMigPK1gPENnfyNYoDGTtg@mail.gmail.com>
Message-ID: <53447911.9010809@ncf.ca>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140408/937ee129/attachment.html>

From njs at pobox.com  Wed Apr  9 01:26:08 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Wed, 9 Apr 2014 00:26:08 +0100
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <534482A2.1080507@canterbury.ac.nz>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CAMpsgwZ7JcUp=7yQ728kSu69xM9YkaNt10SX-bSSjPiSrQNu5A@mail.gmail.com>
 <CAPJVwBkLmOYUgACdMbGF3aku8tQi6eRkKp_pZj2XhGNMkuzXqg@mail.gmail.com>
 <CAP7h-xY_tq8WzkY+U4Fcno9ttnoBwHnY+sY6L7V5rfxmoamaNQ@mail.gmail.com>
 <20140407175844.00f92cfc@anarchist.wooz.org>
 <1396908258.9798.103852961.532425F7@webmail.messagingengine.com>
 <20140408010429.GH16466@ando>
 <CAMpsgwYGBdWUomzS_1Doh=WxE6j87Sdtkv8SmQ6qN_AS9MMhmw@mail.gmail.com>
 <534482A2.1080507@canterbury.ac.nz>
Message-ID: <CAPJVwBmx55TEpFO9vNoOgN2N_3AjFtN0Eev0Dw=69rJxMK114w@mail.gmail.com>

On 9 Apr 2014 00:15, "Greg Ewing" <greg.ewing at canterbury.ac.nz> wrote:
>
> Victor Stinner wrote:
>>
>> I started to implement the RFC 1924 to have a full support.
>>
>> 3 days later, when my code was working, I saw the date of the RFC...
>
>
> Do you still have the code? It needn't go to waste -- this
> would make a fine addition to Python's easter egg basket!

Even if not it would be pretty easy to reimplement - maybe 10-20 loc. If
you look the joke is they represent 128 bit ipaddrs in a base that's
relatively prime to 2, necessitating a full bignum library just for io. But
python has bignums for free...

-n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140409/8f644170/attachment.html>

From ncoghlan at gmail.com  Wed Apr  9 13:12:42 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 9 Apr 2014 07:12:42 -0400
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <53447911.9010809@ncf.ca>
References: <CAP7+vJKOP+7JZx5nZ5R=1AiucxUtYMigPK1gPENnfyNYoDGTtg@mail.gmail.com>
 <53447911.9010809@ncf.ca>
Message-ID: <CADiSq7cXJArHDJ-24umYz=qDA2wrYLvD+LDJtHry7SHmAK08BQ@mail.gmail.com>

On 8 April 2014 18:32, cjw <fn681 at ncf.ca> wrote:
> Guido,
>
> I am sorry to read this.
>
> I shall be responding more completely in a day or two.
>
> In my view, @ and @@ are completely redundant.  Both operations are  already
> provided, * and **, in numpy.matrix.
>
> PEP 465 provides no clear indication as to how the standard operators fail.

Note that numpy.matrix is specifically discussed in
http://www.python.org/dev/peps/pep-0465/#rejected-alternatives-to-adding-a-new-operator
(it's the first rejected alternative listed).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From robert.kern at gmail.com  Wed Apr  9 13:31:06 2014
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 09 Apr 2014 12:31:06 +0100
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <CADiSq7cXJArHDJ-24umYz=qDA2wrYLvD+LDJtHry7SHmAK08BQ@mail.gmail.com>
References: <CAP7+vJKOP+7JZx5nZ5R=1AiucxUtYMigPK1gPENnfyNYoDGTtg@mail.gmail.com>
 <53447911.9010809@ncf.ca>
 <CADiSq7cXJArHDJ-24umYz=qDA2wrYLvD+LDJtHry7SHmAK08BQ@mail.gmail.com>
Message-ID: <li3b1q$j4p$1@ger.gmane.org>

On 2014-04-09 12:12, Nick Coghlan wrote:
> On 8 April 2014 18:32, cjw <fn681 at ncf.ca> wrote:
>> Guido,
>>
>> I am sorry to read this.
>>
>> I shall be responding more completely in a day or two.
>>
>> In my view, @ and @@ are completely redundant.  Both operations are  already
>> provided, * and **, in numpy.matrix.
>>
>> PEP 465 provides no clear indication as to how the standard operators fail.
>
> Note that numpy.matrix is specifically discussed in
> http://www.python.org/dev/peps/pep-0465/#rejected-alternatives-to-adding-a-new-operator
> (it's the first rejected alternative listed).

To be fair to Colin, the PEP asserts that the community at large would prefer an 
operator to the status quo but only alludes to the reason why it does so rather 
than explaining it fully. Personally, I think that's a reasonable allocation of 
Nathaniel's time, but then I happen to have agreed with the PEP's position 
before it was written, and I personally witnessed all of the history myself so I 
don't need it repeated back to me.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From breamoreboy at yahoo.co.uk  Wed Apr  9 15:26:29 2014
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 09 Apr 2014 14:26:29 +0100
Subject: [Python-Dev] issue with itertools leads the crash
In-Reply-To: <5344241A.6040208@mrabarnett.plus.com>
References: <1396721260.939007.13044.8221@mail.rambler.ru>
 <1396968553.65579.26301.58970@mail.rambler.ru>
 <CAP1=2W7Nzr12hpVNVyuORNAqC8TbyN0r+Vpv4WbkDNx0GsP1Hw@mail.gmail.com>
 <5344241A.6040208@mrabarnett.plus.com>
Message-ID: <li3hq7$krs$1@ger.gmane.org>

On 08/04/2014 17:30, MRAB wrote:
> On 2014-04-08 16:31, Brett Cannon wrote:
>
> Something for Python 3.5, maybe? :-)
>
> It's not going to happen in Python 2.7; that's the end of the Python 2
> series, and it's getting security fixes only.

According to http://legacy.python.org/dev/peps/pep-0373/ the final 
release of 2.7 is scheduled to be 2.7.9 in May 2015.  Did you mean to 
say that 2.7 isn't getting new features?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.

Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com



From adnanumer95 at gmail.com  Wed Apr  9 06:25:50 2014
From: adnanumer95 at gmail.com (adnanumer95 at gmail.com)
Date: Wed, 9 Apr 2014 04:25:50 +0000
Subject: [Python-Dev] =?utf-8?q?A_Friendly_IDLE?=
Message-ID: <5344cec2.6152c20a.0bb9.ffffce81@mx.google.com>

Greeting Everyone. First of all I want to introduce my self Adnan Umer as a student of bachelors in Information Technology.


I?ve few suggestions on improving IDLE. Here are few:

On windows we can open any python file from context menu because IDLE is not a application. I recommends to create a simple executable that  just calls ?idle.pyw? module in lib\idlelib.


On executing python script with IDLE we can?t determine which file is executed. I recommends to print file name before executing. I made a little try to do that and I succeed.


In Python Shell Save & Save As menus are enable and using them we can save shell text as python script (.py) that never executes again on IDLE. I recommends to either disable this option or save shell text as plain text. I made a little try to disable this and succeed.


There is almost no difference on displayed result of these two command




>>> print (1)

1

>>> 1

1


there must be some difference as this creates a lot of confusion for beginners to understand purpose of print statement.


Adnan Umer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140409/6106b662/attachment.html>

From njs at pobox.com  Wed Apr  9 13:47:10 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Wed, 9 Apr 2014 12:47:10 +0100
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <li3b1q$j4p$1@ger.gmane.org>
References: <CAP7+vJKOP+7JZx5nZ5R=1AiucxUtYMigPK1gPENnfyNYoDGTtg@mail.gmail.com>
 <53447911.9010809@ncf.ca>
 <CADiSq7cXJArHDJ-24umYz=qDA2wrYLvD+LDJtHry7SHmAK08BQ@mail.gmail.com>
 <li3b1q$j4p$1@ger.gmane.org>
Message-ID: <CAPJVwBkEX85nbaPGayZ1cUvOg0Ui5pKVi+dqcq1e+ndPq_6rmQ@mail.gmail.com>

On 9 Apr 2014 12:34, "Robert Kern" <robert.kern at gmail.com> wrote:
>
> On 2014-04-09 12:12, Nick Coghlan wrote:
>>
>> On 8 April 2014 18:32, cjw <fn681 at ncf.ca> wrote:
>>>
>>> Guido,
>>>
>>> I am sorry to read this.
>>>
>>> I shall be responding more completely in a day or two.
>>>
>>> In my view, @ and @@ are completely redundant.  Both operations are
 already
>>> provided, * and **, in numpy.matrix.
>>>
>>> PEP 465 provides no clear indication as to how the standard operators
fail.
>>
>>
>> Note that numpy.matrix is specifically discussed in
>>
http://www.python.org/dev/peps/pep-0465/#rejected-alternatives-to-adding-a-new-operator
>> (it's the first rejected alternative listed).
>
>
> To be fair to Colin, the PEP asserts that the community at large would
prefer an operator to the status quo but only alludes to the reason why it
does so rather than explaining it fully. Personally, I think that's a
reasonable allocation of Nathaniel's time, but then I happen to have agreed
with the PEP's position before it was written, and I personally witnessed
all of the history myself so I don't need it repeated back to me.

It could doubtless be clearer or signposted better, but the most explicit
explanation of why the two type solution doesn't work is in the first
section, search for "network effects".

-n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140409/bf2d63b1/attachment.html>

From bjourne at gmail.com  Wed Apr  9 17:25:25 2014
From: bjourne at gmail.com (=?ISO-8859-1?Q?Bj=F6rn_Lindqvist?=)
Date: Wed, 9 Apr 2014 17:25:25 +0200
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <CAPJVwB=JP4Ck01g2zcovUrH3Po-TLPcuZcnqi+M=zCmCp3x+dA@mail.gmail.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CALG+76cEoo92AecUNAVdK8QOsZvQ+SovJo7rRvFGJKnxj=uEOA@mail.gmail.com>
 <CAPJVwB=JP4Ck01g2zcovUrH3Po-TLPcuZcnqi+M=zCmCp3x+dA@mail.gmail.com>
Message-ID: <CALG+76e0ZLq1YSHnjAzBveR6F7tvSDzGUZwwMe=ePK4-BW7phg@mail.gmail.com>

2014-04-08 14:52 GMT+02:00 Nathaniel Smith <njs at pobox.com>:
> On Tue, Apr 8, 2014 at 9:58 AM, Bj?rn Lindqvist <bjourne at gmail.com> wrote:
>> 2014-04-07 3:41 GMT+02:00 Nathaniel Smith <njs at pobox.com>:
>>> So, I guess as far as I'm concerned, this is ready to go. Feedback welcome:
>>>   http://legacy.python.org/dev/peps/pep-0465/
>>
>> Couldn't you please have made your motivation example actually runnable?
>>
>> import numpy as np
>> from numpy.linalg import inv, solve
>>
>> # Using dot function:
>> S = np.dot((np.dot(H, beta) - r).T,
>>            np.dot(inv(np.dot(np.dot(H, V), H.T)), np.dot(H, beta) - r))
>>
>> # Using dot method:
>> S = (H.dot(beta) - r).T.dot(inv(H.dot(V).dot(H.T))).dot(H.dot(beta) - r)
>>
>> Don't keep your reader hanging! Tell us what the magical variables H,
>> beta, r and V are. And why import solve when you aren't using it?
>> Curious readers that aren't very good at matrix math, like me, should
>> still be able to follow your logic. Even if it is just random data,
>> it's better than nothing!
>
> There's a footnote that explains the math in more detail and links to
> the real code this was adapted from. And solve is used further down in
> the section. But running it is really what you want, just insert:
>
> beta = np.random.randn(10)
> H = np.random.randn(2, 10)
> r = np.random.randn(2)
> V = np.random.randn(10, 10)
>
> Does that help? ;-)

Thanks! Yes it does help. Then I can see that this expression:

  np.dot(H, beta) - r

Evaluates to a vector. And a vector transposed is the vector itself.
So the .T part in this expression np.dot(H, beta) - r).T is
unnecessary, isn't it?


-- 
mvh/best regards Bj?rn Lindqvist

From tjreedy at udel.edu  Wed Apr  9 17:25:50 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 09 Apr 2014 11:25:50 -0400
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <53447911.9010809@ncf.ca>
References: <CAP7+vJKOP+7JZx5nZ5R=1AiucxUtYMigPK1gPENnfyNYoDGTtg@mail.gmail.com>
 <53447911.9010809@ncf.ca>
Message-ID: <li3oqb$2oj$1@ger.gmane.org>

On 4/8/2014 6:32 PM, cjw wrote:
> Larry Hastings
> <https://mail.python.org/pipermail/python-dev/2014-April/133818.html>
> wasn't far from the truth.

Larry's note was about adding (redundant) *NON-ascii* unicode symbols, 
in particular ? == \xd7, as in A ? B, as a synonym for '@'. Various 
people have suggested adding various symbol synonyms, such as the Greek 
letter lambda for 'lambda', or a middle dot for '*'.

-- 
Terry Jan Reedy



From benjamin at python.org  Wed Apr  9 17:30:07 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 09 Apr 2014 08:30:07 -0700
Subject: [Python-Dev] A Friendly IDLE
In-Reply-To: <5344cec2.6152c20a.0bb9.ffffce81@mx.google.com>
References: <5344cec2.6152c20a.0bb9.ffffce81@mx.google.com>
Message-ID: <1397057407.1372.104596445.6BE6CB87@webmail.messagingengine.com>

On Tue, Apr 8, 2014, at 21:25, adnanumer95 at gmail.com wrote:
> Greeting Everyone. First of all I want to introduce my self Adnan Umer as
> a student of bachelors in Information Technology.
> 
> 
> I?ve few suggestions on improving IDLE. Here are few:
> 
> On windows we can open any python file from context menu because IDLE is
> not a application. I recommends to create a simple executable that  just
> calls ?idle.pyw? module in lib\idlelib.
> 
> 
> On executing python script with IDLE we can?t determine which file is
> executed. I recommends to print file name before executing. I made a
> little try to do that and I succeed.
> 
> 
> In Python Shell Save & Save As menus are enable and using them we can
> save shell text as python script (.py) that never executes again on IDLE.
> I recommends to either disable this option or save shell text as plain
> text. I made a little try to disable this and succeed.
> 
> 
> There is almost no difference on displayed result of these two command
> 
> 
> 
> 
> >>> print (1)
> 
> 1
> 
> >>> 1
> 
> 1
> 
> 
> there must be some difference as this creates a lot of confusion for
> beginners to understand purpose of print statement.

Python 3.3.3 (default, Jan 19 2014, 01:10:27) 
[GCC 4.7.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> print("hello")
hello
>>> "hello"
'hello'

From njs at pobox.com  Wed Apr  9 17:37:05 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Wed, 9 Apr 2014 16:37:05 +0100
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <CALG+76e0ZLq1YSHnjAzBveR6F7tvSDzGUZwwMe=ePK4-BW7phg@mail.gmail.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CALG+76cEoo92AecUNAVdK8QOsZvQ+SovJo7rRvFGJKnxj=uEOA@mail.gmail.com>
 <CAPJVwB=JP4Ck01g2zcovUrH3Po-TLPcuZcnqi+M=zCmCp3x+dA@mail.gmail.com>
 <CALG+76e0ZLq1YSHnjAzBveR6F7tvSDzGUZwwMe=ePK4-BW7phg@mail.gmail.com>
Message-ID: <CAPJVwBkKLh39G4KV9LeABtBsuT6HSWt-4SN-v7JFjcoRXKLLww@mail.gmail.com>

On Wed, Apr 9, 2014 at 4:25 PM, Bj?rn Lindqvist <bjourne at gmail.com> wrote:
> 2014-04-08 14:52 GMT+02:00 Nathaniel Smith <njs at pobox.com>:
>> On Tue, Apr 8, 2014 at 9:58 AM, Bj?rn Lindqvist <bjourne at gmail.com> wrote:
>>> 2014-04-07 3:41 GMT+02:00 Nathaniel Smith <njs at pobox.com>:
>>>> So, I guess as far as I'm concerned, this is ready to go. Feedback welcome:
>>>>   http://legacy.python.org/dev/peps/pep-0465/
>>>
>>> Couldn't you please have made your motivation example actually runnable?
>>>
>>> import numpy as np
>>> from numpy.linalg import inv, solve
>>>
>>> # Using dot function:
>>> S = np.dot((np.dot(H, beta) - r).T,
>>>            np.dot(inv(np.dot(np.dot(H, V), H.T)), np.dot(H, beta) - r))
>>>
>>> # Using dot method:
>>> S = (H.dot(beta) - r).T.dot(inv(H.dot(V).dot(H.T))).dot(H.dot(beta) - r)
>>>
>>> Don't keep your reader hanging! Tell us what the magical variables H,
>>> beta, r and V are. And why import solve when you aren't using it?
>>> Curious readers that aren't very good at matrix math, like me, should
>>> still be able to follow your logic. Even if it is just random data,
>>> it's better than nothing!
>>
>> There's a footnote that explains the math in more detail and links to
>> the real code this was adapted from. And solve is used further down in
>> the section. But running it is really what you want, just insert:
>>
>> beta = np.random.randn(10)
>> H = np.random.randn(2, 10)
>> r = np.random.randn(2)
>> V = np.random.randn(10, 10)
>>
>> Does that help? ;-)
>
> Thanks! Yes it does help. Then I can see that this expression:
>
>   np.dot(H, beta) - r
>
> Evaluates to a vector. And a vector transposed is the vector itself.
> So the .T part in this expression np.dot(H, beta) - r).T is
> unnecessary, isn't it?

In univariate regressions r and beta are vectors, and the .T is a
no-op. The formula also works for multivariate regression, in which
case r and beta become matrices; in this case the .T becomes
necessary.

-n

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

From fn681 at ncf.ca  Wed Apr  9 13:38:13 2014
From: fn681 at ncf.ca (cjw)
Date: Wed, 09 Apr 2014 07:38:13 -0400
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <CAP7+vJKOP+7JZx5nZ5R=1AiucxUtYMigPK1gPENnfyNYoDGTtg@mail.gmail.com>
References: <CAP7+vJKOP+7JZx5nZ5R=1AiucxUtYMigPK1gPENnfyNYoDGTtg@mail.gmail.com>
Message-ID: <53453125.80003@ncf.ca>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140409/1711f92f/attachment.html>

From python at mrabarnett.plus.com  Wed Apr  9 19:53:19 2014
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 09 Apr 2014 18:53:19 +0100
Subject: [Python-Dev] issue with itertools leads the crash
In-Reply-To: <li3hq7$krs$1@ger.gmane.org>
References: <1396721260.939007.13044.8221@mail.rambler.ru>
 <1396968553.65579.26301.58970@mail.rambler.ru>
 <CAP1=2W7Nzr12hpVNVyuORNAqC8TbyN0r+Vpv4WbkDNx0GsP1Hw@mail.gmail.com>
 <5344241A.6040208@mrabarnett.plus.com> <li3hq7$krs$1@ger.gmane.org>
Message-ID: <5345890F.50405@mrabarnett.plus.com>

On 2014-04-09 14:26, Mark Lawrence wrote:
> On 08/04/2014 17:30, MRAB wrote:
>> On 2014-04-08 16:31, Brett Cannon wrote:
>>
>> Something for Python 3.5, maybe? :-)
>>
>> It's not going to happen in Python 2.7; that's the end of the Python 2
>> series, and it's getting security fixes only.
>
> According to http://legacy.python.org/dev/peps/pep-0373/ the final
> release of 2.7 is scheduled to be 2.7.9 in May 2015.  Did you mean to
> say that 2.7 isn't getting new features?
>
Err, probably... :-(

From rymg19 at gmail.com  Wed Apr  9 20:30:35 2014
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Wed, 9 Apr 2014 13:30:35 -0500
Subject: [Python-Dev] issue with itertools leads the crash
In-Reply-To: <li3hq7$krs$1@ger.gmane.org>
References: <1396721260.939007.13044.8221@mail.rambler.ru>
 <1396968553.65579.26301.58970@mail.rambler.ru>
 <CAP1=2W7Nzr12hpVNVyuORNAqC8TbyN0r+Vpv4WbkDNx0GsP1Hw@mail.gmail.com>
 <5344241A.6040208@mrabarnett.plus.com> <li3hq7$krs$1@ger.gmane.org>
Message-ID: <CAO41-mOtsg7bPDwXG1Ts3ZsTTiRf4JB+SNqyZYi2+CBXTBZxvA@mail.gmail.com>

2015!?!? I was hoping it was a tad further off...the PyPy team is going to
have to start freaking out in about 12 months.


On Wed, Apr 9, 2014 at 8:26 AM, Mark Lawrence <breamoreboy at yahoo.co.uk>wrote:

> On 08/04/2014 17:30, MRAB wrote:
>
>> On 2014-04-08 16:31, Brett Cannon wrote:
>>
>> Something for Python 3.5, maybe? :-)
>>
>> It's not going to happen in Python 2.7; that's the end of the Python 2
>> series, and it's getting security fixes only.
>>
>
> According to http://legacy.python.org/dev/peps/pep-0373/ the final
> release of 2.7 is scheduled to be 2.7.9 in May 2015.  Did you mean to say
> that 2.7 isn't getting new features?
>
> --
> My fellow Pythonistas, ask not what our language can do for you, ask what
> you can do for our language.
>
> Mark Lawrence
>
> ---
> This email is free from viruses and malware because avast! Antivirus
> protection is active.
> http://www.avast.com
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> rymg19%40gmail.com
>



-- 
Ryan
If anybody ever asks me why I prefer C++ to C, my answer will be simple:
"It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
nul-terminated."
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140409/95adcf18/attachment.html>

From geoffspear at gmail.com  Wed Apr  9 20:40:21 2014
From: geoffspear at gmail.com (Geoffrey Spear)
Date: Wed, 9 Apr 2014 14:40:21 -0400
Subject: [Python-Dev] issue with itertools leads the crash
In-Reply-To: <5345890F.50405@mrabarnett.plus.com>
References: <1396721260.939007.13044.8221@mail.rambler.ru>
 <1396968553.65579.26301.58970@mail.rambler.ru>
 <CAP1=2W7Nzr12hpVNVyuORNAqC8TbyN0r+Vpv4WbkDNx0GsP1Hw@mail.gmail.com>
 <5344241A.6040208@mrabarnett.plus.com> <li3hq7$krs$1@ger.gmane.org>
 <5345890F.50405@mrabarnett.plus.com>
Message-ID: <CAGifb9H0fiRWOcDmLktmwtz8Fi4ieXcEcSH6K65P7Jx5OdnZ+Q@mail.gmail.com>

On Wed, Apr 9, 2014 at 1:53 PM, MRAB <python at mrabarnett.plus.com> wrote:
> On 2014-04-09 14:26, Mark Lawrence wrote:
>>
>> On 08/04/2014 17:30, MRAB wrote:
>>>
>>> On 2014-04-08 16:31, Brett Cannon wrote:
>>>
>>> Something for Python 3.5, maybe? :-)
>>>
>>> It's not going to happen in Python 2.7; that's the end of the Python 2
>>> series, and it's getting security fixes only.
>>
>>
>> According to http://legacy.python.org/dev/peps/pep-0373/ the final
>> release of 2.7 is scheduled to be 2.7.9 in May 2015.  Did you mean to
>> say that 2.7 isn't getting new features?
>>
> Err, probably... :-(

Of course, this raises the question of whether making slice assignment
not go into an infinite loop when the programmer asks it to is a
bugfix or a new feature.

Calling:

    list(itertools.cycle([0]))

exhibits the same behavior for the same reason, and I don't think
anyone would call that a bug in Python.

From tjreedy at udel.edu  Wed Apr  9 23:42:03 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 09 Apr 2014 17:42:03 -0400
Subject: [Python-Dev] A Friendly IDLE
In-Reply-To: <5344cec2.6152c20a.0bb9.ffffce81@mx.google.com>
References: <5344cec2.6152c20a.0bb9.ffffce81@mx.google.com>
Message-ID: <li4erp$6e0$1@ger.gmane.org>

On 4/9/2014 12:25 AM, adnanumer95 at gmail.com wrote:
> Greeting Everyone. First of all I want to introduce my self Adnan Umer
> as a student of bachelors in Information Technology.
>
> I?ve few suggestions on improving IDLE. Here are few:

Python-list, python-ideas, or idle-dev lists might have been better 
places to put this, but here are my responses.

>  1.
>     On windows we can open any python file from context menu because
>     IDLE is not a application. I recommends to create a simple
>     executable that  just calls ?idle.pyw? module in lib\idlelib.

I do not understand this. Idle is an application, and there already is 
an idle.pyw. On windows, there is a Start Menu entry that calls 
idle.pyw. In Win7, one can pin the icon to the task bar. I presume one 
could make a desktop shortcut also.

>  2.
>     On executing python script with IDLE we can?t determine which file
>     is executed. I recommends to print file name before executing. I
>     made a little try to do that and I succeed.

I created http://bugs.python.org/issue21192. Post a patch there if you 
have one.

>  3.
>     In Python Shell Save & Save As menus are enable and using them we
>     can save shell text as python script (.py) that never executes again
>     on IDLE. I recommends to either disable this option or save shell
>     text as plain text. I made a little try to disable this and succeed.

We will not disable being able to save the shell window.
http://bugs.python.org/issue11838 is about saving in runnable form.

-- 
Terry Jan Reedy



From steve at pearwood.info  Thu Apr 10 02:55:01 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 10 Apr 2014 10:55:01 +1000
Subject: [Python-Dev] issue with itertools leads the crash
In-Reply-To: <CAGifb9H0fiRWOcDmLktmwtz8Fi4ieXcEcSH6K65P7Jx5OdnZ+Q@mail.gmail.com>
References: <1396721260.939007.13044.8221@mail.rambler.ru>
 <1396968553.65579.26301.58970@mail.rambler.ru>
 <CAP1=2W7Nzr12hpVNVyuORNAqC8TbyN0r+Vpv4WbkDNx0GsP1Hw@mail.gmail.com>
 <5344241A.6040208@mrabarnett.plus.com> <li3hq7$krs$1@ger.gmane.org>
 <5345890F.50405@mrabarnett.plus.com>
 <CAGifb9H0fiRWOcDmLktmwtz8Fi4ieXcEcSH6K65P7Jx5OdnZ+Q@mail.gmail.com>
Message-ID: <20140410005501.GD11385@ando>

On Wed, Apr 09, 2014 at 02:40:21PM -0400, Geoffrey Spear wrote:

> Of course, this raises the question of whether making slice assignment
> not go into an infinite loop when the programmer asks it to is a
> bugfix or a new feature.

Definitely a new feature.

> Calling:
> 
>     list(itertools.cycle([0]))
> 
> exhibits the same behavior for the same reason, and I don't think
> anyone would call that a bug in Python.

It's not a bug, nevertheless it is a problem that can be amiliated. 
Some objects now have a __length_hint__  method. cycle could use that to 
indicate that it was infinitely long, and list could raise an exception 
*before* consuming all available memory.

http://legacy.python.org/dev/peps/pep-0424/


-- 
Steven

From guido at python.org  Thu Apr 10 03:08:04 2014
From: guido at python.org (Guido van Rossum)
Date: Wed, 9 Apr 2014 21:08:04 -0400
Subject: [Python-Dev] Language Summit notes
Message-ID: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>

To anyone who took notes at the language summit at PyCon today, even if you
took them just for yourself, would you mind posting them here? It would be
good to have some kind of (informal!) as soon as possible, before we
collectively forget. You won't be held responsible for correctness.

Here are some of my own recollections (I didn't take notes but I have a
decent memory):

- Packaging sucks, but we're improving, and we're actually doing better
than other dynamic languages.

- Kevin Modzelewski answered questions about Pyston, a new (very early
stage) Python VM based on the new LLVM JIT engine (which is much different
from what defeated Unladen Swallow). Alex Gaynor seemed unconcerned. :-)

- Jukka Lehtosalo gave a talk and answered questions about mypy, his design
and implementation of pragmatic type annotations (no new syntax required,
uses Python 3 function annotations). See mypy-lang.org. In response, Greg P
Smith pointed people to a similar project from Google,
https://github.com/google/pytypedecl, which has annotations in a separate
file (hence amenable to Python 2). Larry Hastings brought up that Argument
Clinic (a new way of specifying signatures for C extensions), released as
part of 3.4) encodes similar information in the docstring of C functions.

- Maybe this is should be the year when we start getting agreement on a
standard use of function annotations to specify argument and return types,
now that we seem to have a somewhat critical mass of experience with
annotations.

- We should make an effort to publicize that we're NOT sunsetting Python
2.7 just yet; support will continue (hopefully with ample support from
distro vendors), and someone should update PEP 373. (Unclear what the new
EOL is but we should definitely rescind the currently published schedule.)

- We (I) still don't want to do a 2.8 release, and I don't want to
accelerate 3.5, but I do think we should make things better for people who
have to straddle Python 2 and 3 in a single codebase, by developing more
tools, and by security and possibly installer updates to 2.7 (PEP 466).

- Some suggestions that were made: PSF financial support for tool
development and/or porting, add more "-3" warnings to a future Python 2.7
release, additional 2to3 fixers to help convert Python-2-only code to
Python-2-and-3-single-source code, a separate linter, a sumo 2.7
distribution that includes all known backported-from-Python-3-stdlib
packages, adding ensure_pip to the 2.7.7 stdlib, and several more I forgot.
IIRC Glyph and Alex Gaynor are going to compile a list of pain points for
people. (I can't honestly say that I convinced Glyph and Alex and a few
others not to pine for 2.8, but I also honestly don't believe it will have
the effect that they expect. Nor do I believe any new feature we add to 3.5
can serve as a big enough carrot.)

- The recommended and least painful way to develop for Python 2 and 3 is
definitely to use a single source that runs under both without translation;
we no longer recommend auto-generating Python 3 compatible source code
using 2to3, for a variety of reasons. Several people attested that
single-source has worked well for them; Mercurial is using the 2to3
approach but they're not too happy with it.

- An argument for releasing something labeled 2.8 was made based on the
unavailability (current or future?) of Visual Studio 2008; the
uncomfortable alternative would be to switch to a newer compiler at some
2.7.x bugfix release, which would break extension modules compiled with for
2.7.0-2.7.6, and that would confuse and upset people.

- Apparently no restaurant in downtown Montreal takes reservations for a
group of 30 people to show up in one hour.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140409/92491ab0/attachment.html>

From benjamin at python.org  Thu Apr 10 03:22:10 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 09 Apr 2014 18:22:10 -0700
Subject: [Python-Dev] death to 2.7; long live 2.7
Message-ID: <1397092930.29268.104798077.13871388@webmail.messagingengine.com>

This email is to share idea that has been bouncing around in my head for
a while about 2.7 releases. Guido's last email containing notes from the
language summit made me think it's time to propose it.

We'll keep doing what we're currently doing for another year, making
normal bug fix releases with installers. After that, we _won't_ close
2.7 to normal bug fixes as is currently implied by the release schedule.
Instead dealing 2.7 will just be completely optional for core
developers. (The much anticipated vendor support arrives at this point.)
Releases will continue at a measured (6-12 months) pace. These releases
will be source only (unless someone steps up to make installers).

Planning-on-making-2.7-releases-'til-the-cows-come-home-ly yours,
Benjamin

From guido at python.org  Thu Apr 10 03:31:13 2014
From: guido at python.org (Guido van Rossum)
Date: Wed, 9 Apr 2014 21:31:13 -0400
Subject: [Python-Dev] death to 2.7; long live 2.7
In-Reply-To: <1397092930.29268.104798077.13871388@webmail.messagingengine.com>
References: <1397092930.29268.104798077.13871388@webmail.messagingengine.com>
Message-ID: <CAP7+vJ+6+=OyjqouApxf4PN0=YAjiTSHC5kg+ji2z+wSG=Qj6A@mail.gmail.com>

I think this is pretty much what Nick Coghlan implied at the summit.


On Wed, Apr 9, 2014 at 9:22 PM, Benjamin Peterson <benjamin at python.org>wrote:

> This email is to share idea that has been bouncing around in my head for
> a while about 2.7 releases. Guido's last email containing notes from the
> language summit made me think it's time to propose it.
>
> We'll keep doing what we're currently doing for another year, making
> normal bug fix releases with installers. After that, we _won't_ close
> 2.7 to normal bug fixes as is currently implied by the release schedule.
> Instead dealing 2.7 will just be completely optional for core
> developers. (The much anticipated vendor support arrives at this point.)
> Releases will continue at a measured (6-12 months) pace. These releases
> will be source only (unless someone steps up to make installers).
>
> Planning-on-making-2.7-releases-'til-the-cows-come-home-ly yours,
> Benjamin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140409/7dc990cd/attachment.html>

From benjamin at python.org  Thu Apr 10 03:39:54 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 09 Apr 2014 18:39:54 -0700
Subject: [Python-Dev] death to 2.7; long live 2.7
In-Reply-To: <CAP7+vJ+6+=OyjqouApxf4PN0=YAjiTSHC5kg+ji2z+wSG=Qj6A@mail.gmail.com>
References: <1397092930.29268.104798077.13871388@webmail.messagingengine.com>
 <CAP7+vJ+6+=OyjqouApxf4PN0=YAjiTSHC5kg+ji2z+wSG=Qj6A@mail.gmail.com>
Message-ID: <1397093994.3287.104803125.7E64D564@webmail.messagingengine.com>



On Wed, Apr 9, 2014, at 18:31, Guido van Rossum wrote:
> I think this is pretty much what Nick Coghlan implied at the summit.

He implied that it's currently the plan or that it should be the plan?

From rosuav at gmail.com  Thu Apr 10 03:43:09 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 10 Apr 2014 11:43:09 +1000
Subject: [Python-Dev] death to 2.7; long live 2.7
In-Reply-To: <1397092930.29268.104798077.13871388@webmail.messagingengine.com>
References: <1397092930.29268.104798077.13871388@webmail.messagingengine.com>
Message-ID: <CAPTjJmpYYcPS1v897ueZU_j=-v5hVfyZLujWjGrrhXw2Euxiww@mail.gmail.com>

On Thu, Apr 10, 2014 at 11:22 AM, Benjamin Peterson <benjamin at python.org> wrote:
> Planning-on-making-2.7-releases-'til-the-cows-come-home-ly yours,

Past 2.7.9, will you make 2.7.10 etc, or does that violate other policies?

What will a lack of provided installers do to Windows support? It's
easy enough on Linux to say "either build it from source, or let your
upstream package provider build it for you", but AIUI, most Windows
users want to get a ready-made binary.

Apologies if these questions were answered at the Summit. Montreal's
treatment of thirty-person-parties at one hour's notice may or may not
be considered a bug to be fixed in 2.7, but its geographic barrier to
Australians is definitely a feature addition. And will need to be
thoroughly bikeshedded on -ideas before implementation (can the time
machine be used to travel in relative dimensions in space?)

ChrisA

From senthil at uthcode.com  Thu Apr 10 03:46:01 2014
From: senthil at uthcode.com (Senthil Kumaran)
Date: Wed, 9 Apr 2014 21:46:01 -0400
Subject: [Python-Dev] death to 2.7; long live 2.7
In-Reply-To: <1397092930.29268.104798077.13871388@webmail.messagingengine.com>
References: <1397092930.29268.104798077.13871388@webmail.messagingengine.com>
Message-ID: <CAPOVWORxLTjbqcQGMg4SWOa=L16mD1JB9VgafnGiSRyrSmCuRA@mail.gmail.com>

On Wed, Apr 9, 2014 at 9:22 PM, Benjamin Peterson <benjamin at python.org>wrote:

> Instead dealing 2.7 will just be completely optional for core
> developers. (The much anticipated vendor support arrives at this point.)
>

Could you clarify your thoughts a bit on the "completely optional" part.
What if vendors take a really long time to come to support the latest minor
release? AFAIK, the discussion centered around keep it "alive", for some
definition of alive. We did not define what do we mean by 'alive'.

Bringing back all the security related enhancement features seem ok.
Guido's last email talks about all the other compatibility goodies /tools
targeting 2.7 that may or may not go with 2.7 code base itself.


-- 
Senthil
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140409/8c05e14c/attachment-0001.html>

From benjamin at python.org  Thu Apr 10 03:58:34 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 09 Apr 2014 18:58:34 -0700
Subject: [Python-Dev] death to 2.7; long live 2.7
In-Reply-To: <CAPTjJmpYYcPS1v897ueZU_j=-v5hVfyZLujWjGrrhXw2Euxiww@mail.gmail.com>
References: <1397092930.29268.104798077.13871388@webmail.messagingengine.com>
 <CAPTjJmpYYcPS1v897ueZU_j=-v5hVfyZLujWjGrrhXw2Euxiww@mail.gmail.com>
Message-ID: <1397095114.11461.104807593.7365B9DD@webmail.messagingengine.com>

On Wed, Apr 9, 2014, at 18:43, Chris Angelico wrote:
> On Thu, Apr 10, 2014 at 11:22 AM, Benjamin Peterson <benjamin at python.org>
> wrote:
> > Planning-on-making-2.7-releases-'til-the-cows-come-home-ly yours,
> 
> Past 2.7.9, will you make 2.7.10 etc, or does that violate other
> policies?

I'm not aware that two digit minor version numbers violate anything but
some people's aesthetic senses.

> 
> What will a lack of provided installers do to Windows support? It's
> easy enough on Linux to say "either build it from source, or let your
> upstream package provider build it for you", but AIUI, most Windows
> users want to get a ready-made binary.

It's not that I don't think Windows installers are important, but rather
that Martin has indicated he is (completely reasonably) not interested
in indefinitely making 2.7 installers.

> 
> Apologies if these questions were answered at the Summit. Montreal's
> treatment of thirty-person-parties at one hour's notice may or may not
> be considered a bug to be fixed in 2.7, but its geographic barrier to
> Australians is definitely a feature addition. And will need to be
> thoroughly bikeshedded on -ideas before implementation (can the time
> machine be used to travel in relative dimensions in space?)

From benjamin at python.org  Thu Apr 10 04:02:01 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 09 Apr 2014 19:02:01 -0700
Subject: [Python-Dev] death to 2.7; long live 2.7
In-Reply-To: <CAPOVWORxLTjbqcQGMg4SWOa=L16mD1JB9VgafnGiSRyrSmCuRA@mail.gmail.com>
References: <1397092930.29268.104798077.13871388@webmail.messagingengine.com>
 <CAPOVWORxLTjbqcQGMg4SWOa=L16mD1JB9VgafnGiSRyrSmCuRA@mail.gmail.com>
Message-ID: <1397095321.12974.104808725.7AB57634@webmail.messagingengine.com>


On Wed, Apr 9, 2014, at 18:46, Senthil Kumaran wrote:
> On Wed, Apr 9, 2014 at 9:22 PM, Benjamin Peterson
> <benjamin at python.org>wrote:
> 
> > Instead dealing 2.7 will just be completely optional for core
> > developers. (The much anticipated vendor support arrives at this point.)
> >
> 
> Could you clarify your thoughts a bit on the "completely optional" part.
> What if vendors take a really long time to come to support the latest
> minor
> release? AFAIK, the discussion centered around keep it "alive", for some
> definition of alive. We did not define what do we mean by 'alive'.

Alive means I will keep making 2.7 releases while there are still
interesting changes to release.

> 
> Bringing back all the security related enhancement features seem ok.
> Guido's last email talks about all the other compatibility goodies /tools
> targeting 2.7 that may or may not go with 2.7 code base itself.

I consider the security enhancement/feature question to be in the domain
of PEP 466. If security stuff lands in the 2.7 branch, it will get
released eventually is all I'm saying.

From guido at python.org  Thu Apr 10 04:09:03 2014
From: guido at python.org (Guido van Rossum)
Date: Wed, 9 Apr 2014 22:09:03 -0400
Subject: [Python-Dev] death to 2.7; long live 2.7
In-Reply-To: <1397093994.3287.104803125.7E64D564@webmail.messagingengine.com>
References: <1397092930.29268.104798077.13871388@webmail.messagingengine.com>
 <CAP7+vJ+6+=OyjqouApxf4PN0=YAjiTSHC5kg+ji2z+wSG=Qj6A@mail.gmail.com>
 <1397093994.3287.104803125.7E64D564@webmail.messagingengine.com>
Message-ID: <CAP7+vJ+FOyCd+AnM-_R6jRSjLZ6u1oK4iK2Ao6gyFgUhnBXk2A@mail.gmail.com>

On Wed, Apr 9, 2014 at 9:39 PM, Benjamin Peterson <benjamin at python.org>wrote:

>
>
> On Wed, Apr 9, 2014, at 18:31, Guido van Rossum wrote:
> > I think this is pretty much what Nick Coghlan implied at the summit.
>
> He implied that it's currently the plan or that it should be the plan?
>

As you might understand, we couldn't actually *decide* anything at the
summit, but that should be the plan. Nick implied that Red Hat has a
relevant announcement it will make in two weeks, but wouldn't say more.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140409/2d542502/attachment.html>

From senthil at uthcode.com  Thu Apr 10 04:09:51 2014
From: senthil at uthcode.com (Senthil Kumaran)
Date: Wed, 9 Apr 2014 22:09:51 -0400
Subject: [Python-Dev] death to 2.7; long live 2.7
In-Reply-To: <1397095321.12974.104808725.7AB57634@webmail.messagingengine.com>
References: <1397092930.29268.104798077.13871388@webmail.messagingengine.com>
 <CAPOVWORxLTjbqcQGMg4SWOa=L16mD1JB9VgafnGiSRyrSmCuRA@mail.gmail.com>
 <1397095321.12974.104808725.7AB57634@webmail.messagingengine.com>
Message-ID: <CAPOVWOSgmEh62jEwA9EtT8iPE2fPtW+xvnEGXGP4yUxYp_ypGg@mail.gmail.com>

On Wed, Apr 9, 2014 at 10:02 PM, Benjamin Peterson <benjamin at python.org>wrote:

> I consider the security enhancement/feature question to be in the domain
> of PEP 466. If security stuff lands in the 2.7 branch, it will get
> released eventually is all I'm saying.
>

Thanks for the response.

>> Instead dealing 2.7 will just be completely optional for core developers

I was worried about this part, that if bug-fixes are
optionally back-ported, then we may end up a inconsistent, undesirable
state.
Instead it could be that bug-fixes fixes will be back-ported as long as it
is alive (and folks seem be excited about keep it alive for a long long
term).

-- 
Senthil
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140409/47bdf83f/attachment.html>

From guido at python.org  Thu Apr 10 04:13:57 2014
From: guido at python.org (Guido van Rossum)
Date: Wed, 9 Apr 2014 22:13:57 -0400
Subject: [Python-Dev] death to 2.7; long live 2.7
In-Reply-To: <1397093994.3287.104803125.7E64D564@webmail.messagingengine.com>
References: <1397092930.29268.104798077.13871388@webmail.messagingengine.com>
 <CAP7+vJ+6+=OyjqouApxf4PN0=YAjiTSHC5kg+ji2z+wSG=Qj6A@mail.gmail.com>
 <1397093994.3287.104803125.7E64D564@webmail.messagingengine.com>
Message-ID: <CAP7+vJ+0+qV238-Hs+qbF1_-FCsr8+=nV3dm_9hY=tR0oq1E6g@mail.gmail.com>

On Wed, Apr 9, 2014 at 9:39 PM, Benjamin Peterson <benjamin at python.org>wrote:

>
>
> On Wed, Apr 9, 2014, at 18:31, Guido van Rossum wrote:
> > I think this is pretty much what Nick Coghlan implied at the summit.
>
> He implied that it's currently the plan or that it should be the plan?
>

As you might understand, we couldn't actually *decide* anything at the
summit, but that should be the plan. Nick implied that Red Hat has a
relevant announcement it will make in two weeks, but wouldn't say more.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140409/e7b90d77/attachment.html>

From benjamin at python.org  Thu Apr 10 04:15:21 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 09 Apr 2014 19:15:21 -0700
Subject: [Python-Dev] death to 2.7; long live 2.7
In-Reply-To: <CAPOVWOSgmEh62jEwA9EtT8iPE2fPtW+xvnEGXGP4yUxYp_ypGg@mail.gmail.com>
References: <1397092930.29268.104798077.13871388@webmail.messagingengine.com>
 <CAPOVWORxLTjbqcQGMg4SWOa=L16mD1JB9VgafnGiSRyrSmCuRA@mail.gmail.com>
 <1397095321.12974.104808725.7AB57634@webmail.messagingengine.com>
 <CAPOVWOSgmEh62jEwA9EtT8iPE2fPtW+xvnEGXGP4yUxYp_ypGg@mail.gmail.com>
Message-ID: <1397096121.17680.104811581.6DC4A0C3@webmail.messagingengine.com>



On Wed, Apr 9, 2014, at 19:09, Senthil Kumaran wrote:
> On Wed, Apr 9, 2014 at 10:02 PM, Benjamin Peterson
> >> Instead dealing 2.7 will just be completely optional for core developers
> 
> I was worried about this part, that if bug-fixes are
> optionally back-ported, then we may end up a inconsistent, undesirable
> state.
> Instead it could be that bug-fixes fixes will be back-ported as long as
> it
> is alive (and folks seem be excited about keep it alive for a long long
> term).

I wouldn't worry about that too much. We should be encouraging a culture
of conservatism for 2.7 anyway.

From guido at python.org  Thu Apr 10 04:16:50 2014
From: guido at python.org (Guido van Rossum)
Date: Wed, 9 Apr 2014 22:16:50 -0400
Subject: [Python-Dev] death to 2.7; long live 2.7
In-Reply-To: <1397095114.11461.104807593.7365B9DD@webmail.messagingengine.com>
References: <1397092930.29268.104798077.13871388@webmail.messagingengine.com>
 <CAPTjJmpYYcPS1v897ueZU_j=-v5hVfyZLujWjGrrhXw2Euxiww@mail.gmail.com>
 <1397095114.11461.104807593.7365B9DD@webmail.messagingengine.com>
Message-ID: <CAP7+vJLYf83jWcv9XW2BkDYwy0sjHJW0fkU48nFz2tcXqSZ96g@mail.gmail.com>

On Wed, Apr 9, 2014 at 9:58 PM, Benjamin Peterson <benjamin at python.org>wrote:

> On Wed, Apr 9, 2014, at 18:43, Chris Angelico wrote:
> > On Thu, Apr 10, 2014 at 11:22 AM, Benjamin Peterson <benjamin at python.org
> >
> > wrote:
> > > Planning-on-making-2.7-releases-'til-the-cows-come-home-ly yours,
> >
> > Past 2.7.9, will you make 2.7.10 etc, or does that violate other
> > policies?
>
> I'm not aware that two digit minor version numbers violate anything but
> some people's aesthetic senses.
>

In particular, mine, but if it's better than stopping at that point, I'll
be fine.


>  > What will a lack of provided installers do to Windows support? It's
> > easy enough on Linux to say "either build it from source, or let your
> > upstream package provider build it for you", but AIUI, most Windows
> > users want to get a ready-made binary.
>
> It's not that I don't think Windows installers are important, but rather
> that Martin has indicated he is (completely reasonably) not interested
> in indefinitely making 2.7 installers.
>

Yeah, this was mentioned a few times. I quipped to Nick that Red Hat's
biggest contribution might be to take over the Windows Installer, but
didn't bite. :-)

But there's always the PSF. We may try to find some folks we trust with
relevant expertise to volunteer their time in return for a stipend from the
PSF.


>  > Apologies if these questions were answered at the Summit. Montreal's
> > treatment of thirty-person-parties at one hour's notice may or may not
> > be considered a bug to be fixed in 2.7, but its geographic barrier to
> > Australians is definitely a feature addition. And will need to be
> > thoroughly bikeshedded on -ideas before implementation (can the time
> > machine be used to travel in relative dimensions in space?)
>

Only across branches of an AST though.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140409/93b03171/attachment.html>

From guido at python.org  Thu Apr 10 04:19:57 2014
From: guido at python.org (Guido van Rossum)
Date: Wed, 9 Apr 2014 22:19:57 -0400
Subject: [Python-Dev] death to 2.7; long live 2.7
In-Reply-To: <1397095114.11461.104807593.7365B9DD@webmail.messagingengine.com>
References: <1397092930.29268.104798077.13871388@webmail.messagingengine.com>
 <CAPTjJmpYYcPS1v897ueZU_j=-v5hVfyZLujWjGrrhXw2Euxiww@mail.gmail.com>
 <1397095114.11461.104807593.7365B9DD@webmail.messagingengine.com>
Message-ID: <CAP7+vJLVrqFHEFsHN+i3mRaHEtvZa_POh9OuLmhYAv6pBNWO4A@mail.gmail.com>

On Wed, Apr 9, 2014 at 9:58 PM, Benjamin Peterson <benjamin at python.org>wrote:

> On Wed, Apr 9, 2014, at 18:43, Chris Angelico wrote:
> > On Thu, Apr 10, 2014 at 11:22 AM, Benjamin Peterson <benjamin at python.org
> >
> > wrote:
> > > Planning-on-making-2.7-releases-'til-the-cows-come-home-ly yours,
> >
> > Past 2.7.9, will you make 2.7.10 etc, or does that violate other
> > policies?
>
> I'm not aware that two digit minor version numbers violate anything but
> some people's aesthetic senses.
>

In particular, mine, but if it's better than stopping at that point, I'll
be fine.


>  > What will a lack of provided installers do to Windows support? It's
> > easy enough on Linux to say "either build it from source, or let your
> > upstream package provider build it for you", but AIUI, most Windows
> > users want to get a ready-made binary.
>
> It's not that I don't think Windows installers are important, but rather
> that Martin has indicated he is (completely reasonably) not interested
> in indefinitely making 2.7 installers.
>

Yeah, this was mentioned a few times. I quipped to Nick that Red Hat's
biggest contribution might be to take over the Windows Installer, but he
didn't bite. :-)

But there's always the PSF. We may try to find some folks we trust with
relevant expertise to volunteer their time in return for a stipend from the
PSF for this and some other unglamorous tasks.


> > Apologies if these questions were answered at the Summit. Montreal's
> > treatment of thirty-person-parties at one hour's notice may or may not
> > be considered a bug to be fixed in 2.7, but its geographic barrier to
> > Australians is definitely a feature addition. And will need to be
> > thoroughly bikeshedded on -ideas before implementation (can the time
> > machine be used to travel in relative dimensions in space?)
>

Only across branches of an AST though.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140409/2af9cc7d/attachment.html>

From senthil at uthcode.com  Thu Apr 10 04:30:56 2014
From: senthil at uthcode.com (Senthil Kumaran)
Date: Wed, 9 Apr 2014 22:30:56 -0400
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
Message-ID: <CAPOVWOTzH5_Yno2a=Zz_k+=N8iaVWyM+HbsutmEHdnZdn-K0pA@mail.gmail.com>

Here are my notes that I jotted down from the back row. Forgive me for any
mistakes. (As I shared in the intro, I am trying to get back and keep up.
:))


Python Release Process:
  * Larry Hastings goes for vote for shortend release process. But Guido
does not seem to be excited about it.
     Would go for go for email based voting.

 PyPy:
   * Alex Gaynor mentioned about the 7th iteration of STM - Software
Transactional Memory that is being worked on.
   * PyPy is targetting go from 2.7.3 to 2.7.6  and Brett teases about STM
enabled CPython interpreter?

 Iron Python:
   * Dino gave an updated on IronPython, 2.7.5 is under development, 3.x is
under development (/will be under development soon)
   * Contributions from community has grown recently. MaL encouraged them
to submit for a funding proposal.

  Jython:
  * Support for Buffer Protocol, For Python 3 support. cffi backend for
Python is coming up.

  Discussion about splitting the standard library:
   * IronPython, Pypy say that it is not a priority request for them.

 Packaging:
   * Nick Coghlan shares his experience on how difficult is get the packing
right. Every agrees and kind of recognize that recent efforts are in the
right direction. Mentioned about https://pypi-preview.a.ssl.fastly.net/ and
wheels packaging format.
   * Well maintained docs at http://packaging.python.org/en/latest/ -
Python packaging user guide.
   * The focus/goal is not get new users easy to understand python
ecosystem and use python packages.

Pyston:
  * Kevin shared his ideas and updates on Pyston. Folks suggested about
using the speed.pypy.org benchmarks to measure the effort.

 MyPy:
  * The optional static typing using functional annotations demonstrated by
MyPy interested a number of developers.  Few felt that it would be a nice
to have feature in Python 3.5 It basically means identify what's lacking in
current function annotations and work on enhancing it.
  * Thomos Wouters suggested to Larry Hastings that Argument clinic could
be enhanced to support such a feature.
  * Interested parties should get together on a python type checking
mailing list.

 Features we care about 3.5:
   ? bytes formatting redux
   ? Binary mode cleanup
   ? Type Annotations.
   ? Improved tooling AI based tooling to convert 2.x code to 3.x and
provide better error messages. (Sounds exciting!)




On Wed, Apr 9, 2014 at 9:08 PM, Guido van Rossum <guido at python.org> wrote:

> To anyone who took notes at the language summit at PyCon today, even if
> you took them just for yourself, would you mind posting them here? It would
> be good to have some kind of (informal!) as soon as possible, before we
> collectively forget. You won't be held responsible for correctness.
>
> Here are some of my own recollections (I didn't take notes but I have a
> decent memory):
>
> - Packaging sucks, but we're improving, and we're actually doing better
> than other dynamic languages.
>
> - Kevin Modzelewski answered questions about Pyston, a new (very early
> stage) Python VM based on the new LLVM JIT engine (which is much different
> from what defeated Unladen Swallow). Alex Gaynor seemed unconcerned. :-)
>
> - Jukka Lehtosalo gave a talk and answered questions about mypy, his
> design and implementation of pragmatic type annotations (no new syntax
> required, uses Python 3 function annotations). See mypy-lang.org. In
> response, Greg P Smith pointed people to a similar project from Google,
> https://github.com/google/pytypedecl, which has annotations in a separate
> file (hence amenable to Python 2). Larry Hastings brought up that Argument
> Clinic (a new way of specifying signatures for C extensions), released as
> part of 3.4) encodes similar information in the docstring of C functions.
>
> - Maybe this is should be the year when we start getting agreement on a
> standard use of function annotations to specify argument and return types,
> now that we seem to have a somewhat critical mass of experience with
> annotations.
>
> - We should make an effort to publicize that we're NOT sunsetting Python
> 2.7 just yet; support will continue (hopefully with ample support from
> distro vendors), and someone should update PEP 373. (Unclear what the new
> EOL is but we should definitely rescind the currently published schedule.)
>
> - We (I) still don't want to do a 2.8 release, and I don't want to
> accelerate 3.5, but I do think we should make things better for people who
> have to straddle Python 2 and 3 in a single codebase, by developing more
> tools, and by security and possibly installer updates to 2.7 (PEP 466).
>
> - Some suggestions that were made: PSF financial support for tool
> development and/or porting, add more "-3" warnings to a future Python 2.7
> release, additional 2to3 fixers to help convert Python-2-only code to
> Python-2-and-3-single-source code, a separate linter, a sumo 2.7
> distribution that includes all known backported-from-Python-3-stdlib
> packages, adding ensure_pip to the 2.7.7 stdlib, and several more I forgot.
> IIRC Glyph and Alex Gaynor are going to compile a list of pain points for
> people. (I can't honestly say that I convinced Glyph and Alex and a few
> others not to pine for 2.8, but I also honestly don't believe it will have
> the effect that they expect. Nor do I believe any new feature we add to 3.5
> can serve as a big enough carrot.)
>
> - The recommended and least painful way to develop for Python 2 and 3 is
> definitely to use a single source that runs under both without translation;
> we no longer recommend auto-generating Python 3 compatible source code
> using 2to3, for a variety of reasons. Several people attested that
> single-source has worked well for them; Mercurial is using the 2to3
> approach but they're not too happy with it.
>
> - An argument for releasing something labeled 2.8 was made based on the
> unavailability (current or future?) of Visual Studio 2008; the
> uncomfortable alternative would be to switch to a newer compiler at some
> 2.7.x bugfix release, which would break extension modules compiled with for
> 2.7.0-2.7.6, and that would confuse and upset people.
>
> - Apparently no restaurant in downtown Montreal takes reservations for a
> group of 30 people to show up in one hour.
>
> --
> --Guido van Rossum (python.org/~guido)
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/senthil%40uthcode.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140409/1f947102/attachment-0001.html>

From nad at acm.org  Thu Apr 10 05:12:42 2014
From: nad at acm.org (Ned Deily)
Date: Wed, 09 Apr 2014 20:12:42 -0700
Subject: [Python-Dev] death to 2.7; long live 2.7
References: <1397092930.29268.104798077.13871388@webmail.messagingengine.com>
 <CAPTjJmpYYcPS1v897ueZU_j=-v5hVfyZLujWjGrrhXw2Euxiww@mail.gmail.com>
 <1397095114.11461.104807593.7365B9DD@webmail.messagingengine.com>
 <CAP7+vJLVrqFHEFsHN+i3mRaHEtvZa_POh9OuLmhYAv6pBNWO4A@mail.gmail.com>
Message-ID: <nad-DBBEA3.20124109042014@news.gmane.org>

In article 
<CAP7+vJLVrqFHEFsHN+i3mRaHEtvZa_POh9OuLmhYAv6pBNWO4A at mail.gmail.com>,
 Guido van Rossum <guido at python.org> wrote:
> On Wed, Apr 9, 2014 at 9:58 PM, Benjamin Peterson <benjamin at python.org>wrote: 
> > It's not that I don't think Windows installers are important, but rather
> > that Martin has indicated he is (completely reasonably) not interested
> > in indefinitely making 2.7 installers. 
> Yeah, this was mentioned a few times. I quipped to Nick that Red Hat's
> biggest contribution might be to take over the Windows Installer, but he
> didn't bite. :-)
> 
> But there's always the PSF. We may try to find some folks we trust with
> relevant expertise to volunteer their time in return for a stipend from the
> PSF for this and some other unglamorous tasks.

WRT the other set of installers we provide, I'm willing to keep 
producing OS X installers for an indeterminate future of 2.7.  I reserve 
the right to get tired of it before 2038.  And I certainly am not 
volunteering to take over the Windows Installer.  I have it easy 
compared to Martin.

If we decide to keep going past 2015, I will likely propose some changes 
in the supported OS X levels of the 2.7 installers, in particular 
dropping at least 10.3 and 10.4 which we already did for Py3 starting 
with 3.3.0.  (By this I am not proposing to do anything to break source 
builds for those older systems.)  That has some potential impact to end 
users, e.g. breaking ABI compatibility in a minor release, but I think 
the impact would be outweighed by the benefits of supporting newer 
os-dependent features and build tools.  Any changes could be mitigated 
by a transitional release with installers for both the old and the new 
configurations.  But that's just a head's up: I'm not prepared to go 
into details at the moment.

-- 
 Ned Deily,
 nad at acm.org


From ncoghlan at gmail.com  Thu Apr 10 05:38:17 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 9 Apr 2014 23:38:17 -0400
Subject: [Python-Dev] death to 2.7; long live 2.7
In-Reply-To: <CAP7+vJ+FOyCd+AnM-_R6jRSjLZ6u1oK4iK2Ao6gyFgUhnBXk2A@mail.gmail.com>
References: <1397092930.29268.104798077.13871388@webmail.messagingengine.com>
 <CAP7+vJ+6+=OyjqouApxf4PN0=YAjiTSHC5kg+ji2z+wSG=Qj6A@mail.gmail.com>
 <1397093994.3287.104803125.7E64D564@webmail.messagingengine.com>
 <CAP7+vJ+FOyCd+AnM-_R6jRSjLZ6u1oK4iK2Ao6gyFgUhnBXk2A@mail.gmail.com>
Message-ID: <CADiSq7fMca_yAzApHuxZArYcHPT_m4bGO5M20dtSue5X6H2Weg@mail.gmail.com>

On 9 Apr 2014 22:11, "Guido van Rossum" <guido at python.org> wrote:
>
> On Wed, Apr 9, 2014 at 9:39 PM, Benjamin Peterson <benjamin at python.org>
wrote:
>>
>>
>>
>> On Wed, Apr 9, 2014, at 18:31, Guido van Rossum wrote:
>> > I think this is pretty much what Nick Coghlan implied at the summit.
>>
>> He implied that it's currently the plan or that it should be the plan?
>
>
> As you might understand, we couldn't actually *decide* anything at the
summit, but that should be the plan.

Yeah, I think it's on us and other commercial redistributors to figure out
how to best meet our long term support commitments to our users, and I
personally think that meeting those obligations cost effectively will
involve more direct *upstream* involvement in the long term maintenance of
the Python 2.7 series. That's largely boring-but-necessary drudgery, and
that's the kind of development gap that open source vendors excel at
addressing.

Still a lot of legwork to get from "I think this is a good way to handle
the problem" to actually making it happen, but as the saying goes: if you
don't ask, you don't get :)

I believe a defined end date for volunteer provided backports actually
helps make that case (and the advance warning also provides lead time to
hopefully do something about it).

> Nick implied that Red Hat has a relevant announcement it will make in two
weeks, but wouldn't say more.

Kinda wishing I hadn't said anything at all, now - I suspect the temporary
vagueness means I'm now building up anticipation that will inevitably lead
to disappointment with any actual announcement :(

I *can* at least say that we're (all too well) aware of the issues
associated with consuming newer dynamic language runtimes on our stable
platforms, and we're definitely interested in making it easier for users to
deploy and use newer versions of these without harming the consistency and
integrity of their system. Assuming we can find an approach (or approaches)
that work well for a wide variety of use cases, this should allow us to
help reduce the pressure on the developers of upstream Python libraries and
frameworks to keep supporting the versions installed system wide in our
long term maintenance releases (OpenShift and software collections are a
couple of existing efforts that handle some aspects of this issue, but I
certainly don't consider the general problem solved at this point).

Regards,
Nick.

>
> --
> --Guido van Rossum (python.org/~guido)
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140409/241f94a0/attachment.html>

From stephen at xemacs.org  Thu Apr 10 06:25:55 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 10 Apr 2014 13:25:55 +0900
Subject: [Python-Dev]  Language Summit notes
In-Reply-To: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
Message-ID: <87r455ize4.fsf@uwakimon.sk.tsukuba.ac.jp>

Guido van Rossum writes:

 > - We should make an effort to publicize that we're NOT sunsetting
 > Python 2.7 just yet;

Maybe just add "Windows XP" to the SEO keywords for that page?  Like
*today* would be perfect timing.<wink/>


From donald at stufft.io  Thu Apr 10 07:13:29 2014
From: donald at stufft.io (Donald Stufft)
Date: Thu, 10 Apr 2014 01:13:29 -0400
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <CAPOVWOTzH5_Yno2a=Zz_k+=N8iaVWyM+HbsutmEHdnZdn-K0pA@mail.gmail.com>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
 <CAPOVWOTzH5_Yno2a=Zz_k+=N8iaVWyM+HbsutmEHdnZdn-K0pA@mail.gmail.com>
Message-ID: <B7CE62CA-22C5-482B-85F8-7E8286CACFC2@stufft.io>


On Apr 9, 2014, at 10:30 PM, Senthil Kumaran <senthil at uthcode.com> wrote:

> Mentioned about https://pypi-preview.a.ssl.fastly.net/ 

For what it?s worth, https://warehouse.python.org/ is a somewhat easier to remember demo url for that :]

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140410/31820825/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140410/31820825/attachment.sig>

From solipsis at pitrou.net  Thu Apr 10 09:43:57 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 10 Apr 2014 09:43:57 +0200
Subject: [Python-Dev] death to 2.7; long live 2.7
In-Reply-To: <CAPOVWOSgmEh62jEwA9EtT8iPE2fPtW+xvnEGXGP4yUxYp_ypGg@mail.gmail.com>
References: <1397092930.29268.104798077.13871388@webmail.messagingengine.com>
 <CAPOVWORxLTjbqcQGMg4SWOa=L16mD1JB9VgafnGiSRyrSmCuRA@mail.gmail.com>
 <1397095321.12974.104808725.7AB57634@webmail.messagingengine.com>
 <CAPOVWOSgmEh62jEwA9EtT8iPE2fPtW+xvnEGXGP4yUxYp_ypGg@mail.gmail.com>
Message-ID: <li5i3t$cs1$1@ger.gmane.org>

Le 10/04/2014 04:09, Senthil Kumaran a ?crit :
>
> On Wed, Apr 9, 2014 at 10:02 PM, Benjamin Peterson <benjamin at python.org
> <mailto:benjamin at python.org>> wrote:
>
>     I consider the security enhancement/feature question to be in the domain
>     of PEP 466. If security stuff lands in the 2.7 branch, it will get
>     released eventually is all I'm saying.
>
>
> Thanks for the response.
>
>>> Instead dealing 2.7 will just be completely optional for coredevelopers
>
> I was worried about this part, that if bug-fixes are
> optionally back-ported, then we may end up a inconsistent, undesirable
> state.

They already are. There are routinely things I (and other developers) 
don't consider for inclusion in 2.x.

Regards

Antoine.



From christian at python.org  Thu Apr 10 10:01:14 2014
From: christian at python.org (Christian Heimes)
Date: Thu, 10 Apr 2014 10:01:14 +0200
Subject: [Python-Dev] death to 2.7; long live 2.7
In-Reply-To: <CAP7+vJLYf83jWcv9XW2BkDYwy0sjHJW0fkU48nFz2tcXqSZ96g@mail.gmail.com>
References: <1397092930.29268.104798077.13871388@webmail.messagingengine.com>
 <CAPTjJmpYYcPS1v897ueZU_j=-v5hVfyZLujWjGrrhXw2Euxiww@mail.gmail.com>
 <1397095114.11461.104807593.7365B9DD@webmail.messagingengine.com>
 <CAP7+vJLYf83jWcv9XW2BkDYwy0sjHJW0fkU48nFz2tcXqSZ96g@mail.gmail.com>
Message-ID: <li5j4a$8uc$1@ger.gmane.org>

On 10.04.2014 04:16, Guido van Rossum wrote:
> Yeah, this was mentioned a few times. I quipped to Nick that Red Hat's
> biggest contribution might be to take over the Windows Installer, but
> didn't bite. :-)
> 
> But there's always the PSF. We may try to find some folks we trust with
> relevant expertise to volunteer their time in return for a stipend from
> the PSF.

The Python core dev team has a couple of people that have the necessary
experience and tools to create Windows installers. Martin has done an
great job in automating the build process, too. AFAIK the new person in
charge for Windows 2.7 builds just need the certificate to sign the
installer.

Christian


From taleinat at gmail.com  Thu Apr 10 10:18:17 2014
From: taleinat at gmail.com (Tal Einat)
Date: Thu, 10 Apr 2014 11:18:17 +0300
Subject: [Python-Dev] A Friendly IDLE
In-Reply-To: <li4erp$6e0$1@ger.gmane.org>
References: <5344cec2.6152c20a.0bb9.ffffce81@mx.google.com>
 <li4erp$6e0$1@ger.gmane.org>
Message-ID: <CALWZvp4eY2bWyS1DDa1TcF_UL4MWCG0+3vPCSjzdjohaDdVjZQ@mail.gmail.com>

On Thu, Apr 10, 2014 at 12:42 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 4/9/2014 12:25 AM, adnanumer95 at gmail.com wrote:

> Python-list, python-ideas, or idle-dev lists might have been better places to put this, but here are my responses.

I'm adding idle-dev, where this belongs. Further discussion should
take place there rather than on python-dev.

>>  3.
>>     In Python Shell Save & Save As menus are enable and using them we
>>     can save shell text as python script (.py) that never executes again
>>     on IDLE. I recommends to either disable this option or save shell
>>     text as plain text. I made a little try to disable this and succeed.
>
>
> We will not disable being able to save the shell window.
> http://bugs.python.org/issue11838 is about saving in runnable form.

If IDLE saves the shell history to a file with a .py extension by
default, that is indeed confusing. It is useful to be able to save the
history to a file, but it shouldn't have a .py extension.

- Tal Einat

From p.f.moore at gmail.com  Thu Apr 10 10:18:49 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 10 Apr 2014 09:18:49 +0100
Subject: [Python-Dev] death to 2.7; long live 2.7
In-Reply-To: <1397095114.11461.104807593.7365B9DD@webmail.messagingengine.com>
References: <1397092930.29268.104798077.13871388@webmail.messagingengine.com>
 <CAPTjJmpYYcPS1v897ueZU_j=-v5hVfyZLujWjGrrhXw2Euxiww@mail.gmail.com>
 <1397095114.11461.104807593.7365B9DD@webmail.messagingengine.com>
Message-ID: <CACac1F_RcDkNRR7ewtRHpuuP0kuByK82J3LyevbQrQ-MB0+CYg@mail.gmail.com>

On 10 April 2014 02:58, Benjamin Peterson <benjamin at python.org> wrote:
>> What will a lack of provided installers do to Windows support? It's
>> easy enough on Linux to say "either build it from source, or let your
>> upstream package provider build it for you", but AIUI, most Windows
>> users want to get a ready-made binary.
>
> It's not that I don't think Windows installers are important, but rather
> that Martin has indicated he is (completely reasonably) not interested
> in indefinitely making 2.7 installers.

I would assume that ActiveState, Enthought and Anaconda are the people
we should be expecting to provide Windows binaries once the core team
stop doing so. In all honesty, if none of those 3 companies can see a
business case for providing 2.7.x binaries for Windows, then that
demonstrates pretty effectively that doing so isn't worth the effort.

Paul.

From kushaldas at gmail.com  Thu Apr 10 13:24:36 2014
From: kushaldas at gmail.com (Kushal Das)
Date: Thu, 10 Apr 2014 16:54:36 +0530
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
Message-ID: <CAAzeMby9RM2K_2xL2UUGQBtbUfHnBt2tMhLgEFETmP0GAHA88A@mail.gmail.com>

On Thu, Apr 10, 2014 at 6:38 AM, Guido van Rossum <guido at python.org> wrote:
> To anyone who took notes at the language summit at PyCon today, even if you
> took them just for yourself, would you mind posting them here? It would be
> good to have some kind of (informal!) as soon as possible, before we
> collectively forget. You won't be held responsible for correctness.
>

The day started with introductions. Guido introduced himself as its
all his fault.

Release management discussion
==============================

Larry Hastings started the day with discussion on 3.5 release. 3.4
release was actually in 16 months. He wanted a
feedback on the next release, if we want it in a smaller release cycle
than the usual 18 months. Guido mentioned to
stay with the 18 month cycle.

Larry also asked about opinions on state of the SCM after release
candidate 1, should we create 3.5 branch and if yes
then should we allow people to commit there or not? Default should
point to 3.5.1 or 3.6 at that time? There can be another
scenario where we do not create the 3.5 branch and keep the default as
3.5 release itself. The discussion will continue
in the mailing list.

Next topic in the agenda was reports from different implementations.

PyPy
=====

Alex Gaynor gave us the current status of `PyPy <http://pypy.org>`_
project. There will be a second fund raiser on STM.
The next release is targeting 2.7.6, there were a million downloads.
While discussing about Python 3 branch he explained
that it it only 3 bugs away from shipping and it is based on 3.2.


There was a small discussion about state of CFFI for standard library
inclusion. Alex and David Beazley are supposed to
work on cleaning PLY for the same. General opinion was that it will be
hidden as a private part of the standard lib and to
be used by the language only.

Ironpython
===========

Dino Viehland talked about the status of `Ironpython
<http://ironpython.net>`_ project. Development is going on both 2.7
and 3.x
series. 2.7.4 was released last year. Many new contributors came into
the project which is a good news.

Jython
=======

The developers sent a detailed report to Micheal Foord and he will
forward it to the python-dev list. The takeaways from the mail are

* Small number of contributors is a big problem.
* 2.7.beta2 is tagged which used Java7.
* Buffer protocol work is done (foundation to Python3 support).
* They are also working on PyPi tooling.
* There is also hope for releasing CFFI backend for Jython during
Europycon sprints.


No standard library as module
==============================

When it was asked that if the other implementations want the standard
library as a separate module to be resused, all agreed as 'No'.

Packaging
===========

It was the longest discussion which made hungry developers really
hungry :) Jokes aside, Nick Coghlan gave a detailed report on the
advancement of the packaging world. Most of the development/design
discussions are now happening on the distutils sig and in pypi mailing
lists.
He managed to put the use cases a very broader audience now, so we can
except better feedbacks. On the development side, Warehouse is now
implementing all old API(s), you may want to try it out at
`https://warehouse.python.org/ <https://warehouse.python.org/>`_.

3.4 has pip included, one usecase was to help people who downloads
binary installers from our site. They can now install Django or other
projects
in wheel format.

Everyone also agreed that having the buildsystem inside the language
is a bad idea. The buildsystem should be able to do cross-version
builds.

Nick also pointed us to `http://packaging.python.org/
<http://packaging.python.org/en/latest/>`_ which is the documentation
for the whole echosystem.  We all agreed that the Python echosystem is
bigger than the core interpreter.

Glyph wants a PSF fund to a usability study on Python. There were a
few other suggestion on PSF support for tooling development.

Pyston
=======

Kevin Modzelewski explained how they are rebuilding a complete vm
which is targeted to Python, this also means too much work but one can
customize. It is targeting Python2.7 as Dropbox runs on it.


At this time of discussion Nick pointed us to
`http://speed.python.org/ <http://speed.python.org/>`_, he asked if
any of the implementations
wants to maintain it. We need more volunteers for that, target is to
have a common set of tests to benchmark different implementations.

Mypy project
=============

Jukka Lehtosalo gave a talk on his `mypy project
<http://mypy-lang.org>`_ which uses Python3 function annotations. Greg
P Smith pointed us to
a similar kind of Google project,
`https://github.com/google/pytypedecl
<https://github.com/google/pytypedecl>`_.

Notes from teaching and outreach
=================================

Selena Deckelmann talked about few pain points from teaching and outreach.

* Website is confusing. (Should I go for Python2 or Python3?)
* Packaging and installer problem
* So many different bug tracking system is also confusing
* OPW program for Cpython, this is the first year we are participating.
* Jessica McKellar will write "brand new coder tutorials".

Mercurial
===========

Matt Mackall talked about Mercurial's painpoints for Python3. It
currently works for 2.4-2.7, though he might drop 2.4 support in near
future.
It will be on 2.7 till RHEL7 is not EOL. He also said startup time is
concern for him. Only big positive point he can see in Python3 is SNI.
That feature allows you to do HTTPS to non ip based virtual hosts.
Porting whole Mercurial to Python 3 is still a very big work. They had
two gsoc students in last two years.

>From here the talks suddenly moved into mythical Python 2.8 which we
will not have, nope, sorry :) Guido wants a feature list from the
people who are asking for 2.8 to understand better. We also want to
help developers to make a single source for Python 2 and Python 3
release less painful.

Python 2.7 is alive and in good health and support will continue on the same.

Few points were talked about from 3.5, like byte formatting, unicode
surrogate, binary mode cleans for bytes etc.

Kushal

From bjourne at gmail.com  Thu Apr 10 13:34:54 2014
From: bjourne at gmail.com (=?ISO-8859-1?Q?Bj=F6rn_Lindqvist?=)
Date: Thu, 10 Apr 2014 13:34:54 +0200
Subject: [Python-Dev] PEP 465: A dedicated infix operator for matrix
	multiplication
In-Reply-To: <CAPJVwBkKLh39G4KV9LeABtBsuT6HSWt-4SN-v7JFjcoRXKLLww@mail.gmail.com>
References: <CAPJVwB=BRt7Z224gJygKfwn-k3vt31L9jDvhx89j4XOrhe+xug@mail.gmail.com>
 <CALG+76cEoo92AecUNAVdK8QOsZvQ+SovJo7rRvFGJKnxj=uEOA@mail.gmail.com>
 <CAPJVwB=JP4Ck01g2zcovUrH3Po-TLPcuZcnqi+M=zCmCp3x+dA@mail.gmail.com>
 <CALG+76e0ZLq1YSHnjAzBveR6F7tvSDzGUZwwMe=ePK4-BW7phg@mail.gmail.com>
 <CAPJVwBkKLh39G4KV9LeABtBsuT6HSWt-4SN-v7JFjcoRXKLLww@mail.gmail.com>
Message-ID: <CALG+76cyVb=+dVA3yjZO1hUW31kY+jQd9uQ81LnQJaLECx15FQ@mail.gmail.com>

2014-04-09 17:37 GMT+02:00 Nathaniel Smith <njs at pobox.com>:
> On Wed, Apr 9, 2014 at 4:25 PM, Bj?rn Lindqvist <bjourne at gmail.com> wrote:
>> 2014-04-08 14:52 GMT+02:00 Nathaniel Smith <njs at pobox.com>:
>>> On Tue, Apr 8, 2014 at 9:58 AM, Bj?rn Lindqvist <bjourne at gmail.com> wrote:
>>>> 2014-04-07 3:41 GMT+02:00 Nathaniel Smith <njs at pobox.com>:
>>>>> So, I guess as far as I'm concerned, this is ready to go. Feedback welcome:
>>>>>   http://legacy.python.org/dev/peps/pep-0465/
>>>>
>>>> Couldn't you please have made your motivation example actually runnable?
>>>>
>>>> import numpy as np
>>>> from numpy.linalg import inv, solve
>>>>
>>>> # Using dot function:
>>>> S = np.dot((np.dot(H, beta) - r).T,
>>>>            np.dot(inv(np.dot(np.dot(H, V), H.T)), np.dot(H, beta) - r))
>>>>
>>>> # Using dot method:
>>>> S = (H.dot(beta) - r).T.dot(inv(H.dot(V).dot(H.T))).dot(H.dot(beta) - r)
>>>>
>>>> Don't keep your reader hanging! Tell us what the magical variables H,
>>>> beta, r and V are. And why import solve when you aren't using it?
>>>> Curious readers that aren't very good at matrix math, like me, should
>>>> still be able to follow your logic. Even if it is just random data,
>>>> it's better than nothing!
>>>
>>> There's a footnote that explains the math in more detail and links to
>>> the real code this was adapted from. And solve is used further down in
>>> the section. But running it is really what you want, just insert:
>>>
>>> beta = np.random.randn(10)
>>> H = np.random.randn(2, 10)
>>> r = np.random.randn(2)
>>> V = np.random.randn(10, 10)
>>>
>>> Does that help? ;-)
>>
>> Thanks! Yes it does help. Then I can see that this expression:
>>
>>   np.dot(H, beta) - r
>>
>> Evaluates to a vector. And a vector transposed is the vector itself.
>> So the .T part in this expression np.dot(H, beta) - r).T is
>> unnecessary, isn't it?
>
> In univariate regressions r and beta are vectors, and the .T is a
> no-op. The formula also works for multivariate regression, in which
> case r and beta become matrices; in this case the .T becomes
> necessary.

Then what is the shape of those variables supposed to be? The earlier
definitions you gave isn't enough for this general case.


-- 
mvh/best regards Bj?rn Lindqvist

From amk at amk.ca  Thu Apr 10 15:05:22 2014
From: amk at amk.ca (A.M. Kuchling)
Date: Thu, 10 Apr 2014 09:05:22 -0400
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
Message-ID: <20140410130522.GA19636@datlandrewk>

On Wed, Apr 09, 2014 at 09:08:04PM -0400, Guido van Rossum wrote:
> To anyone who took notes at the language summit at PyCon today, even if you
> took them just for yourself, would you mind posting them here? It would be
> good to have some kind of (informal!) as soon as possible, before we
> collectively forget. You won't be held responsible for correctness.

>From a 30-second check, the recordings I made on my laptop are
listenable (though the audio levels in the morning were too low).  We
probably don't want to post them publicly, just because they haven't
been listened to.  I provided them to Kushal Das, who was taking
notes; if other attendees want them for reference, please let me know.

--amk

From p.f.moore at gmail.com  Thu Apr 10 15:41:47 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 10 Apr 2014 14:41:47 +0100
Subject: [Python-Dev] Windows installers and OpenSSL
Message-ID: <CACac1F-gGuryDiU9JL4zh0XQ0bpE=ybZ-iBFZ0Q=uvrO8zeGJg@mail.gmail.com>

Given the OpenSSL vulnerability and the fact that we bundle OpenSSL
with the Windows installers (1.0.1e in Python 3.4.0) should we be
releasing updated installers?

Paul

From solipsis at pitrou.net  Thu Apr 10 16:20:04 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 10 Apr 2014 16:20:04 +0200
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <CAAzeMby9RM2K_2xL2UUGQBtbUfHnBt2tMhLgEFETmP0GAHA88A@mail.gmail.com>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
 <CAAzeMby9RM2K_2xL2UUGQBtbUfHnBt2tMhLgEFETmP0GAHA88A@mail.gmail.com>
Message-ID: <li69al$a32$1@ger.gmane.org>

Le 10/04/2014 13:24, Kushal Das a ?crit :
>
> At this time of discussion Nick pointed us to
> `http://speed.python.org/ <http://speed.python.org/>`_, he asked if
> any of the implementations
> wants to maintain it. We need more volunteers for that, target is to
> have a common set of tests to benchmark different implementations.

I feel a bit tired to point out that there *is* a common set of 
cross-implementation benchmarks at http://hg.python.org/benchmarks

It is maintained and there is also a mailing-list to discuss it at 
https://mail.python.org/mailman/listinfo/speed

Regards

Antoine.



From guido at python.org  Thu Apr 10 16:38:51 2014
From: guido at python.org (Guido van Rossum)
Date: Thu, 10 Apr 2014 10:38:51 -0400
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <li69al$a32$1@ger.gmane.org>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
 <CAAzeMby9RM2K_2xL2UUGQBtbUfHnBt2tMhLgEFETmP0GAHA88A@mail.gmail.com>
 <li69al$a32$1@ger.gmane.org>
Message-ID: <CAP7+vJJmR1REo3r9OT68ehDceTa+Xohe3-JLNmGu9hrNi-4+Qw@mail.gmail.com>

Maybe we don't need a volunteer to maintain it, but we sure need a
volunteer to coordinate and spread the knowledge! :-)


On Thu, Apr 10, 2014 at 10:20 AM, Antoine Pitrou <solipsis at pitrou.net>wrote:

> Le 10/04/2014 13:24, Kushal Das a ?crit :
>
>
>> At this time of discussion Nick pointed us to
>> `http://speed.python.org/ <http://speed.python.org/>`_, he asked if
>> any of the implementations
>> wants to maintain it. We need more volunteers for that, target is to
>> have a common set of tests to benchmark different implementations.
>>
>
> I feel a bit tired to point out that there *is* a common set of
> cross-implementation benchmarks at http://hg.python.org/benchmarks
>
> It is maintained and there is also a mailing-list to discuss it at
> https://mail.python.org/mailman/listinfo/speed
>
> Regards
>
> Antoine.
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140410/3d12a6c1/attachment.html>

From python at mrabarnett.plus.com  Thu Apr 10 16:51:39 2014
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 10 Apr 2014 15:51:39 +0100
Subject: [Python-Dev] Windows installers and OpenSSL
In-Reply-To: <CACac1F-gGuryDiU9JL4zh0XQ0bpE=ybZ-iBFZ0Q=uvrO8zeGJg@mail.gmail.com>
References: <CACac1F-gGuryDiU9JL4zh0XQ0bpE=ybZ-iBFZ0Q=uvrO8zeGJg@mail.gmail.com>
Message-ID: <5346AFFB.1060301@mrabarnett.plus.com>

On 2014-04-10 14:41, Paul Moore wrote:
> Given the OpenSSL vulnerability and the fact that we bundle OpenSSL
> with the Windows installers (1.0.1e in Python 3.4.0) should we be
> releasing updated installers?
>
I'd say yes, but, then, I wouldn't be doing any of the work...

From kushaldas at gmail.com  Thu Apr 10 17:11:21 2014
From: kushaldas at gmail.com (Kushal Das)
Date: Thu, 10 Apr 2014 20:41:21 +0530
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <li69al$a32$1@ger.gmane.org>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
 <CAAzeMby9RM2K_2xL2UUGQBtbUfHnBt2tMhLgEFETmP0GAHA88A@mail.gmail.com>
 <li69al$a32$1@ger.gmane.org>
Message-ID: <CAAzeMbxmWDLjj7XrKHZom94Oe0YA_XfKDo9+fj4jqhN5GRcWmg@mail.gmail.com>

On Thu, Apr 10, 2014 at 7:50 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
>
> I feel a bit tired to point out that there *is* a common set of
> cross-implementation benchmarks at http://hg.python.org/benchmarks
>
> It is maintained and there is also a mailing-list to discuss it at
> https://mail.python.org/mailman/listinfo/speed
>
I think I had the comma in the wrong place for that sentence.

Thanks for information, I am adding them in the blog post.

Kushal
-- 
http://fedoraproject.org
http://kushaldas.in

From thomas at python.org  Thu Apr 10 17:41:22 2014
From: thomas at python.org (Thomas Wouters)
Date: Thu, 10 Apr 2014 08:41:22 -0700
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <CAAzeMby9RM2K_2xL2UUGQBtbUfHnBt2tMhLgEFETmP0GAHA88A@mail.gmail.com>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
 <CAAzeMby9RM2K_2xL2UUGQBtbUfHnBt2tMhLgEFETmP0GAHA88A@mail.gmail.com>
Message-ID: <CAPdQG2pwdKkL0MfEJkMcFUoYKPcmaE8_Yzq28D1VnWgfPwm+FA@mail.gmail.com>

On Thu, Apr 10, 2014 at 4:24 AM, Kushal Das <kushaldas at gmail.com> wrote:

> On Thu, Apr 10, 2014 at 6:38 AM, Guido van Rossum <guido at python.org>
> wrote:
> > To anyone who took notes at the language summit at PyCon today, even if
> you
> > took them just for yourself, would you mind posting them here? It would
> be
> > good to have some kind of (informal!) as soon as possible, before we
> > collectively forget. You won't be held responsible for correctness.
> >
>
> The day started with introductions. Guido introduced himself as its
> all his fault.
>
> Release management discussion
> ==============================
>
> Larry Hastings started the day with discussion on 3.5 release. 3.4
> release was actually in 16 months. He wanted a
> feedback on the next release, if we want it in a smaller release cycle
> than the usual 18 months. Guido mentioned to
> stay with the 18 month cycle.
>
> Larry also asked about opinions on state of the SCM after release
> candidate 1, should we create 3.5 branch and if yes
> then should we allow people to commit there or not? Default should
> point to 3.5.1 or 3.6 at that time? There can be another
> scenario where we do not create the 3.5 branch and keep the default as
> 3.5 release itself. The discussion will continue
> in the mailing list.
>
> Next topic in the agenda was reports from different implementations.
>
> PyPy
> =====
>
> Alex Gaynor gave us the current status of `PyPy <http://pypy.org>`_
> project. There will be a second fund raiser on STM.
> The next release is targeting 2.7.6, there were a million downloads.
> While discussing about Python 3 branch he explained
> that it it only 3 bugs away from shipping and it is based on 3.2.
>
>
> There was a small discussion about state of CFFI for standard library
> inclusion. Alex and David Beazley are supposed to
> work on cleaning PLY for the same. General opinion was that it will be
> hidden as a private part of the standard lib and to
> be used by the language only.
>

No, the opinion was that it _shouldn't_ be hidden as a private part of the
standard library :) But some cleanup needs to happen before it can be added
to the stdlib.


>
> Ironpython
> ===========
>
> Dino Viehland talked about the status of `Ironpython
> <http://ironpython.net>`_ project. Development is going on both 2.7
> and 3.x
> series. 2.7.4 was released last year. Many new contributors came into
> the project which is a good news.
>
> Jython
> =======
>
> The developers sent a detailed report to Micheal Foord and he will
> forward it to the python-dev list. The takeaways from the mail are
>
> * Small number of contributors is a big problem.
> * 2.7.beta2 is tagged which used Java7.
> * Buffer protocol work is done (foundation to Python3 support).
> * They are also working on PyPi tooling.
> * There is also hope for releasing CFFI backend for Jython during
> Europycon sprints.
>
>
> No standard library as module
> ==============================
>
> When it was asked that if the other implementations want the standard
> library as a separate module to be resused, all agreed as 'No'.
>

Their answer was mostly "don't care". It has some minor benefits, in
particular when they move to Python 3 and track active development more
closely, but no important ones.


>
> Packaging
> ===========
>
> It was the longest discussion which made hungry developers really
> hungry :) Jokes aside, Nick Coghlan gave a detailed report on the
> advancement of the packaging world. Most of the development/design
> discussions are now happening on the distutils sig and in pypi mailing
> lists.
> He managed to put the use cases a very broader audience now, so we can
> except better feedbacks. On the development side, Warehouse is now
> implementing all old API(s), you may want to try it out at
> `https://warehouse.python.org/ <https://warehouse.python.org/>`_.
>
> 3.4 has pip included, one usecase was to help people who downloads
> binary installers from our site. They can now install Django or other
> projects
> in wheel format.
>
> Everyone also agreed that having the buildsystem inside the language
> is a bad idea. The buildsystem should be able to do cross-version
> builds.
>
> Nick also pointed us to `http://packaging.python.org/
> <http://packaging.python.org/en/latest/>`_ which is the documentation
> for the whole echosystem.  We all agreed that the Python echosystem is
> bigger than the core interpreter.
>
> Glyph wants a PSF fund to a usability study on Python. There were a
> few other suggestion on PSF support for tooling development.
>
> Pyston
> =======
>
> Kevin Modzelewski explained how they are rebuilding a complete vm
> which is targeted to Python, this also means too much work but one can
> customize. It is targeting Python2.7 as Dropbox runs on it.
>
>
> At this time of discussion Nick pointed us to
> `http://speed.python.org/ <http://speed.python.org/>`_, he asked if
> any of the implementations
> wants to maintain it. We need more volunteers for that, target is to
> have a common set of tests to benchmark different implementations.
>
> Mypy project
> =============
>
> Jukka Lehtosalo gave a talk on his `mypy project
> <http://mypy-lang.org>`_ which uses Python3 function annotations. Greg
> P Smith pointed us to
> a similar kind of Google project,
> `https://github.com/google/pytypedecl
> <https://github.com/google/pytypedecl>`_.
>
> Notes from teaching and outreach
> =================================
>
> Selena Deckelmann talked about few pain points from teaching and outreach.
>
> * Website is confusing. (Should I go for Python2 or Python3?)
> * Packaging and installer problem
> * So many different bug tracking system is also confusing
> * OPW program for Cpython, this is the first year we are participating.
> * Jessica McKellar will write "brand new coder tutorials".
>

I believe this was mostly about collecting new coder resources that already
exist, but are hard to find (and to qualitatively judge.)


>
> Mercurial
> ===========
>
> Matt Mackall talked about Mercurial's painpoints for Python3. It
> currently works for 2.4-2.7, though he might drop 2.4 support in near
> future.
> It will be on 2.7 till RHEL7 is not EOL. He also said startup time is
> concern for him. Only big positive point he can see in Python3 is SNI.
> That feature allows you to do HTTPS to non ip based virtual hosts.
> Porting whole Mercurial to Python 3 is still a very big work. They had
> two gsoc students in last two years.
>
> From here the talks suddenly moved into mythical Python 2.8 which we
> will not have, nope, sorry :) Guido wants a feature list from the
> people who are asking for 2.8 to understand better. We also want to
> help developers to make a single source for Python 2 and Python 3
> release less painful.
>
> Python 2.7 is alive and in good health and support will continue on the
> same.
>
> Few points were talked about from 3.5, like byte formatting, unicode
> surrogate, binary mode cleans for bytes etc.
>
> Kushal
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/thomas%40python.org
>



-- 
Thomas Wouters <thomas at python.org>

Hi! I'm an email virus! Think twice before sending your email to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140410/f6cd2541/attachment-0001.html>

From guido at python.org  Thu Apr 10 20:58:25 2014
From: guido at python.org (Guido van Rossum)
Date: Thu, 10 Apr 2014 14:58:25 -0400
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <CAPdQG2pwdKkL0MfEJkMcFUoYKPcmaE8_Yzq28D1VnWgfPwm+FA@mail.gmail.com>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
 <CAAzeMby9RM2K_2xL2UUGQBtbUfHnBt2tMhLgEFETmP0GAHA88A@mail.gmail.com>
 <CAPdQG2pwdKkL0MfEJkMcFUoYKPcmaE8_Yzq28D1VnWgfPwm+FA@mail.gmail.com>
Message-ID: <CAP7+vJLC1i=6t6_fmutYiiizX6Mm-6hY6314+Vwz62g821-6WQ@mail.gmail.com>

On Thu, Apr 10, 2014 at 11:41 AM, Thomas Wouters <thomas at python.org> wrote:


> On Thu, Apr 10, 2014 at 4:24 AM, Kushal Das <kushaldas at gmail.com> wrote:
>


>  There was a small discussion about state of CFFI for standard library
>> inclusion. Alex and David Beazley are supposed to
>> work on cleaning PLY for the same. General opinion was that it will be
>> hidden as a private part of the standard lib and to
>> be used by the language only.
>>
>
> No, the opinion was that it _shouldn't_ be hidden as a private part of the
> standard library :) But some cleanup needs to happen before it can be added
> to the stdlib.
>

Huh, I totally missed this (and I just gave Kushal a confused answer when
he asked me about it in person). Can someone please post here what the plan
is exactly? I don't want to press for a PEP, but I would at least like to
understand the plan for CFFI and PLY before it is executed, since I have
never had to use either one, and it feels like each of these will require
some commitment to maintenance once they are in, in addition to cleanup
before they go in. And no, waving hands and saying "there's already a blog
post about CFFI somewhere" is not good enough. I want the full description
of the plan written up here in python-dev. Blog links might serve to
clarify the motivation though.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140410/bbcf3f49/attachment.html>

From ericsnowcurrently at gmail.com  Thu Apr 10 21:25:15 2014
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Thu, 10 Apr 2014 13:25:15 -0600
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <CAP7+vJLC1i=6t6_fmutYiiizX6Mm-6hY6314+Vwz62g821-6WQ@mail.gmail.com>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
 <CAAzeMby9RM2K_2xL2UUGQBtbUfHnBt2tMhLgEFETmP0GAHA88A@mail.gmail.com>
 <CAPdQG2pwdKkL0MfEJkMcFUoYKPcmaE8_Yzq28D1VnWgfPwm+FA@mail.gmail.com>
 <CAP7+vJLC1i=6t6_fmutYiiizX6Mm-6hY6314+Vwz62g821-6WQ@mail.gmail.com>
Message-ID: <CALFfu7A-Aa9aX85CjsL=BonzbBA=9PiuvftkqgJbmg-_NeWZxg@mail.gmail.com>

On Thu, Apr 10, 2014 at 12:58 PM, Guido van Rossum <guido at python.org> wrote:
> On Thu, Apr 10, 2014 at 11:41 AM, Thomas Wouters <thomas at python.org> wrote:
>
>>
>> On Thu, Apr 10, 2014 at 4:24 AM, Kushal Das <kushaldas at gmail.com> wrote:
>
>
>>>
>>> There was a small discussion about state of CFFI for standard library
>>> inclusion. Alex and David Beazley are supposed to
>>> work on cleaning PLY for the same. General opinion was that it will be
>>> hidden as a private part of the standard lib and to
>>> be used by the language only.
>>
>>
>> No, the opinion was that it _shouldn't_ be hidden as a private part of the
>> standard library :) But some cleanup needs to happen before it can be added
>> to the stdlib.
>
>
> Huh, I totally missed this (and I just gave Kushal a confused answer when he
> asked me about it in person). Can someone please post here what the plan is
> exactly? I don't want to press for a PEP, but I would at least like to
> understand the plan for CFFI and PLY before it is executed, since I have
> never had to use either one, and it feels like each of these will require
> some commitment to maintenance once they are in, in addition to cleanup
> before they go in. And no, waving hands and saying "there's already a blog
> post about CFFI somewhere" is not good enough. I want the full description
> of the plan written up here in python-dev. Blog links might serve to clarify
> the motivation though.

The discussion happened leading up to the language summit in 2013.

ply: https://mail.python.org/pipermail/python-dev/2013-February/124389.html
pycparser: https://mail.python.org/pipermail/python-dev/2013-February/124341.html
(earlier part of same discussion)

My recollection was that we'd add a cleaned-up ply to the stdlib and
leave pycparser as a private implementation detail of cffi.

-eric

From solipsis at pitrou.net  Thu Apr 10 21:30:44 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 10 Apr 2014 21:30:44 +0200
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <CAP7+vJLC1i=6t6_fmutYiiizX6Mm-6hY6314+Vwz62g821-6WQ@mail.gmail.com>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
 <CAAzeMby9RM2K_2xL2UUGQBtbUfHnBt2tMhLgEFETmP0GAHA88A@mail.gmail.com>
 <CAPdQG2pwdKkL0MfEJkMcFUoYKPcmaE8_Yzq28D1VnWgfPwm+FA@mail.gmail.com>
 <CAP7+vJLC1i=6t6_fmutYiiizX6Mm-6hY6314+Vwz62g821-6WQ@mail.gmail.com>
Message-ID: <li6rh4$m7e$1@ger.gmane.org>

Le 10/04/2014 20:58, Guido van Rossum a ?crit :
>
> Huh, I totally missed this (and I just gave Kushal a confused answer
> when he asked me about it in person). Can someone please post here what
> the plan is exactly? I don't want to press for a PEP, but I would at
> least like to understand the plan for CFFI and PLY before it is
> executed, since I have never had to use either one, and it feels like
> each of these will require some commitment to maintenance once they are
> in, in addition to cleanup before they go in.

FWIW, I do hope there would be a PEP before including CFFI... Actually I 
don't understand what would justify an exemption.

Regards

Antoine.



From eliben at gmail.com  Thu Apr 10 21:34:42 2014
From: eliben at gmail.com (Eli Bendersky)
Date: Thu, 10 Apr 2014 12:34:42 -0700
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <li6rh4$m7e$1@ger.gmane.org>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
 <CAAzeMby9RM2K_2xL2UUGQBtbUfHnBt2tMhLgEFETmP0GAHA88A@mail.gmail.com>
 <CAPdQG2pwdKkL0MfEJkMcFUoYKPcmaE8_Yzq28D1VnWgfPwm+FA@mail.gmail.com>
 <CAP7+vJLC1i=6t6_fmutYiiizX6Mm-6hY6314+Vwz62g821-6WQ@mail.gmail.com>
 <li6rh4$m7e$1@ger.gmane.org>
Message-ID: <CAF-Rda-F5U1k0OBmwTg0mE3Gq3SLWG5vJ5Cu1XU=6AtnFxhG1w@mail.gmail.com>

On Thu, Apr 10, 2014 at 12:30 PM, Antoine Pitrou <solipsis at pitrou.net>wrote:

> Le 10/04/2014 20:58, Guido van Rossum a ?crit :
>
>
>> Huh, I totally missed this (and I just gave Kushal a confused answer
>> when he asked me about it in person). Can someone please post here what
>> the plan is exactly? I don't want to press for a PEP, but I would at
>> least like to understand the plan for CFFI and PLY before it is
>> executed, since I have never had to use either one, and it feels like
>> each of these will require some commitment to maintenance once they are
>> in, in addition to cleanup before they go in.
>>
>
> FWIW, I do hope there would be a PEP before including CFFI... Actually I
> don't understand what would justify an exemption


There's absolutely no reason to exempt CFFI, IMHO. On the contrary -- the
dependence on other 3rd party modules (PLY and pycparesr), and the related
dilemma of whether to expose each/both as stdlib modules or hide as
internal implementation details -- makes a PEP even more important here.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140410/bd250d3a/attachment.html>

From guido at python.org  Thu Apr 10 21:55:50 2014
From: guido at python.org (Guido van Rossum)
Date: Thu, 10 Apr 2014 15:55:50 -0400
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <CAF-Rda-F5U1k0OBmwTg0mE3Gq3SLWG5vJ5Cu1XU=6AtnFxhG1w@mail.gmail.com>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
 <CAAzeMby9RM2K_2xL2UUGQBtbUfHnBt2tMhLgEFETmP0GAHA88A@mail.gmail.com>
 <CAPdQG2pwdKkL0MfEJkMcFUoYKPcmaE8_Yzq28D1VnWgfPwm+FA@mail.gmail.com>
 <CAP7+vJLC1i=6t6_fmutYiiizX6Mm-6hY6314+Vwz62g821-6WQ@mail.gmail.com>
 <li6rh4$m7e$1@ger.gmane.org>
 <CAF-Rda-F5U1k0OBmwTg0mE3Gq3SLWG5vJ5Cu1XU=6AtnFxhG1w@mail.gmail.com>
Message-ID: <CAP7+vJJi+uEwV24S9q6LKZehEaA3Gzsh8Eaz+R=KmNXj85yEjA@mail.gmail.com>

Well, I was going to put off requesting a PEP until I'd judged the plan,
but clearly (a) there isn't actually a plan (just some vague description of
an end result that some feel desirable) and (b) it's controversial. So,
yes, it definitely needs a PEP. Also, even though this came up a year ago,
nobody actually cared enough to write a PEP; which makes me think that
there are probably more than a few problems with the whole idea. (One of
which may be that while PLY is stable, it also sounds completely
unmaintainable; another may be that CFFI seems to require too much of a
development environment to be available to count as an alternative to
ctypes.)


On Thu, Apr 10, 2014 at 3:34 PM, Eli Bendersky <eliben at gmail.com> wrote:

> On Thu, Apr 10, 2014 at 12:30 PM, Antoine Pitrou <solipsis at pitrou.net>wrote:
>
>> Le 10/04/2014 20:58, Guido van Rossum a ?crit :
>>
>>
>>> Huh, I totally missed this (and I just gave Kushal a confused answer
>>> when he asked me about it in person). Can someone please post here what
>>> the plan is exactly? I don't want to press for a PEP, but I would at
>>> least like to understand the plan for CFFI and PLY before it is
>>> executed, since I have never had to use either one, and it feels like
>>> each of these will require some commitment to maintenance once they are
>>> in, in addition to cleanup before they go in.
>>>
>>
>> FWIW, I do hope there would be a PEP before including CFFI... Actually I
>> don't understand what would justify an exemption
>
>
> There's absolutely no reason to exempt CFFI, IMHO. On the contrary -- the
> dependence on other 3rd party modules (PLY and pycparesr), and the related
> dilemma of whether to expose each/both as stdlib modules or hide as
> internal implementation details -- makes a PEP even more important here.
>
> Eli
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140410/c4800392/attachment.html>

From p.f.moore at gmail.com  Thu Apr 10 22:12:52 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 10 Apr 2014 21:12:52 +0100
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <li6rh4$m7e$1@ger.gmane.org>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
 <CAAzeMby9RM2K_2xL2UUGQBtbUfHnBt2tMhLgEFETmP0GAHA88A@mail.gmail.com>
 <CAPdQG2pwdKkL0MfEJkMcFUoYKPcmaE8_Yzq28D1VnWgfPwm+FA@mail.gmail.com>
 <CAP7+vJLC1i=6t6_fmutYiiizX6Mm-6hY6314+Vwz62g821-6WQ@mail.gmail.com>
 <li6rh4$m7e$1@ger.gmane.org>
Message-ID: <CACac1F8SEiAgFi4LT7P6FYYYKQ59vFNMp97XRSChGDxwOV9bEg@mail.gmail.com>

On 10 April 2014 20:30, Antoine Pitrou <solipsis at pitrou.net> wrote:
> FWIW, I do hope there would be a PEP before including CFFI... Actually I
> don't understand what would justify an exemption.

I agree. I'd like to see a clear explanation of what advantages (and
disadvantages!) CFFI gives over ctypes, as well as the plan for
inclusion and how the inevitable confusion over whether to use ctypes
or cffi will be handled. The fact that cffi requires bringing in ply
and a vendored-but-not-public copy of pycparser, seems to imply to me
that there's a lot of cost and I'd like to understand the gains.
That's not to say that adding ply to the standard library mightn't be
worth it in its own right, but there are a lot of other parsers out
there, and I'd rather we blessed one as "best of breed" rather than
"because cffi uses it".

In particular, my understanding is that in order to get the key
benefits of cffi (API compatibility rather than ABI) you need to
include some sort of complex "generate and compile C" step in your
project's build. That implies that using cffi requires building
separate wheels for each Python version (as with any C extension) and
having a C compiler to do the build. There are a lot of projects that
I know of (particularly wrappers for Windows APIs like Colorama and
pyreadline) migrating to ctypes precisely because they get a pure
Python solution that doesn't need binary builds or version-dependent
distributions. Those projects won't presumably be migrating to cffi.
Also, a key area where ctypes is used seems to be the Windows API -
and there, ABI compatibility for Windows APIs is the norm so the
advantage of only needing API compatibility is minimal at best.

At the moment, I see no real reason to add cffi to the stdlib - it
certainly isn't an "obvious" decision to do so. This seems like clear
PEP territory.

Paul

From rymg19 at gmail.com  Thu Apr 10 22:47:26 2014
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Thu, 10 Apr 2014 15:47:26 -0500
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <CACac1F8SEiAgFi4LT7P6FYYYKQ59vFNMp97XRSChGDxwOV9bEg@mail.gmail.com>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
 <CAAzeMby9RM2K_2xL2UUGQBtbUfHnBt2tMhLgEFETmP0GAHA88A@mail.gmail.com>
 <CAPdQG2pwdKkL0MfEJkMcFUoYKPcmaE8_Yzq28D1VnWgfPwm+FA@mail.gmail.com>
 <CAP7+vJLC1i=6t6_fmutYiiizX6Mm-6hY6314+Vwz62g821-6WQ@mail.gmail.com>
 <li6rh4$m7e$1@ger.gmane.org>
 <CACac1F8SEiAgFi4LT7P6FYYYKQ59vFNMp97XRSChGDxwOV9bEg@mail.gmail.com>
Message-ID: <CAO41-mMQ27=bkc8HZzHB9G2gkvJjnMYQxNGQL8T-RWxHSgN+3g@mail.gmail.com>

On Thu, Apr 10, 2014 at 3:12 PM, Paul Moore <p.f.moore at gmail.com> wrote:

> On 10 April 2014 20:30, Antoine Pitrou <solipsis at pitrou.net> wrote:
> > FWIW, I do hope there would be a PEP before including CFFI... Actually I
> > don't understand what would justify an exemption.
>
> I agree. I'd like to see a clear explanation of what advantages (and
> disadvantages!) CFFI gives over ctypes, as well as the plan for
> inclusion and how the inevitable confusion over whether to use ctypes
> or cffi will be handled. The fact that cffi requires bringing in ply
> and a vendored-but-not-public copy of pycparser, seems to imply to me
> that there's a lot of cost and I'd like to understand the gains.
> That's not to say that adding ply to the standard library mightn't be
> worth it in its own right, but there are a lot of other parsers out
> there, and I'd rather we blessed one as "best of breed" rather than
> "because cffi uses it".
>
> In particular, my understanding is that in order to get the key
> benefits of cffi (API compatibility rather than ABI) you need to
> include some sort of complex "generate and compile C" step in your
> project's build. That implies that using cffi requires building
> separate wheels for each Python version (as with any C extension) and
> having a C compiler to do the build. There are a lot of projects that
> I know of (particularly wrappers for Windows APIs like Colorama and
> pyreadline) migrating to ctypes precisely because they get a pure
> Python solution that doesn't need binary builds or version-dependent
> distributions. Those projects won't presumably be migrating to cffi.
> Also, a key area where ctypes is used seems to be the Windows API -
> and there, ABI compatibility for Windows APIs is the norm so the
> advantage of only needing API compatibility is minimal at best.
>

Nope. CFFI supports both C-built extensions and ctypes-style shared library
loading. There are pure-Python bindings for modules that use CFFI and don't
require a C compiler(https://github.com/kirbyfan64/clamav-python).


> At the moment, I see no real reason to add cffi to the stdlib - it
> certainly isn't an "obvious" decision to do so. This seems like clear
> PEP territory.


It's somewhat faster but has a longer warm-up time. Useful if the same
module is going to be reused several times.


>
> Paul
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>



-- 
Ryan
If anybody ever asks me why I prefer C++ to C, my answer will be simple:
"It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
nul-terminated."
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140410/10a12b2d/attachment.html>

From barry at python.org  Thu Apr 10 23:20:36 2014
From: barry at python.org (Barry Warsaw)
Date: Thu, 10 Apr 2014 17:20:36 -0400
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <CAF-Rda-F5U1k0OBmwTg0mE3Gq3SLWG5vJ5Cu1XU=6AtnFxhG1w@mail.gmail.com>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
 <CAAzeMby9RM2K_2xL2UUGQBtbUfHnBt2tMhLgEFETmP0GAHA88A@mail.gmail.com>
 <CAPdQG2pwdKkL0MfEJkMcFUoYKPcmaE8_Yzq28D1VnWgfPwm+FA@mail.gmail.com>
 <CAP7+vJLC1i=6t6_fmutYiiizX6Mm-6hY6314+Vwz62g821-6WQ@mail.gmail.com>
 <li6rh4$m7e$1@ger.gmane.org>
 <CAF-Rda-F5U1k0OBmwTg0mE3Gq3SLWG5vJ5Cu1XU=6AtnFxhG1w@mail.gmail.com>
Message-ID: <20140410172036.0c876684@anarchist>

On Apr 10, 2014, at 12:34 PM, Eli Bendersky wrote:

>There's absolutely no reason to exempt CFFI, IMHO. On the contrary -- the
>dependence on other 3rd party modules (PLY and pycparesr), and the related
>dilemma of whether to expose each/both as stdlib modules or hide as
>internal implementation details -- makes a PEP even more important here.

Agreed.  We've already started down the slippery slope of bundling third party
packages (e.g. the ensurepip dependencies) and this is causing headaches for
some of us distro guys.  Either they're in the stdlib or their not.

-Barry

From greg at krypto.org  Thu Apr 10 23:50:34 2014
From: greg at krypto.org (Gregory P. Smith)
Date: Thu, 10 Apr 2014 17:50:34 -0400
Subject: [Python-Dev] Windows installers and OpenSSL
In-Reply-To: <5346AFFB.1060301@mrabarnett.plus.com>
References: <CACac1F-gGuryDiU9JL4zh0XQ0bpE=ybZ-iBFZ0Q=uvrO8zeGJg@mail.gmail.com>
 <5346AFFB.1060301@mrabarnett.plus.com>
Message-ID: <CAGE7PNJ72QTWbD_yoQ-6x+Y89S35aviXC-CPMwHmsyofbnrUeg@mail.gmail.com>

Yep. All binary Python distributions that bundle SSL support need updating.
But... what MRAB said.

We also *likely* have SSL certificates and SSH host keys on
python.orginfrastructure that need to be revoked and new certs
reissued
*after* all of those machines have been patched and their services
restarted. Including hg.python.org.

-gps



On Thu, Apr 10, 2014 at 10:51 AM, MRAB <python at mrabarnett.plus.com> wrote:

> On 2014-04-10 14:41, Paul Moore wrote:
>
>> Given the OpenSSL vulnerability and the fact that we bundle OpenSSL
>> with the Windows installers (1.0.1e in Python 3.4.0) should we be
>> releasing updated installers?
>>
>>  I'd say yes, but, then, I wouldn't be doing any of the work...
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> greg%40krypto.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140410/11011857/attachment-0001.html>

From benjamin at python.org  Fri Apr 11 00:51:09 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 10 Apr 2014 15:51:09 -0700
Subject: [Python-Dev] Windows installers and OpenSSL
In-Reply-To: <CAGE7PNJ72QTWbD_yoQ-6x+Y89S35aviXC-CPMwHmsyofbnrUeg@mail.gmail.com>
References: <CACac1F-gGuryDiU9JL4zh0XQ0bpE=ybZ-iBFZ0Q=uvrO8zeGJg@mail.gmail.com>
 <5346AFFB.1060301@mrabarnett.plus.com>
 <CAGE7PNJ72QTWbD_yoQ-6x+Y89S35aviXC-CPMwHmsyofbnrUeg@mail.gmail.com>
Message-ID: <1397170269.16536.105190241.695A2490@webmail.messagingengine.com>

On Thu, Apr 10, 2014, at 14:50, Gregory P. Smith wrote:
> Yep. All binary Python distributions that bundle SSL support need
> updating.
> But... what MRAB said.
> 
> We also *likely* have SSL certificates and SSH host keys on
> python.orginfrastructure that need to be revoked and new certs
> reissued
> *after* all of those machines have been patched and their services
> restarted. Including hg.python.org.

SSH isn't affected, though, right?

From ncoghlan at gmail.com  Fri Apr 11 03:14:30 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 10 Apr 2014 21:14:30 -0400
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <li69al$a32$1@ger.gmane.org>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
 <CAAzeMby9RM2K_2xL2UUGQBtbUfHnBt2tMhLgEFETmP0GAHA88A@mail.gmail.com>
 <li69al$a32$1@ger.gmane.org>
Message-ID: <CADiSq7eeSkusrjawa_vpou+onyGbTfV5eqodQRLhT4mg6wW8Yg@mail.gmail.com>

On 10 Apr 2014 10:23, "Antoine Pitrou" <solipsis at pitrou.net> wrote:
>
> Le 10/04/2014 13:24, Kushal Das a ?crit :
>
>>
>> At this time of discussion Nick pointed us to
>> `http://speed.python.org/ <http://speed.python.org/>`_, he asked if
>> any of the implementations
>> wants to maintain it. We need more volunteers for that, target is to
>> have a common set of tests to benchmark different implementations.
>
>
> I feel a bit tired to point out that there *is* a common set of
cross-implementation benchmarks at http://hg.python.org/benchmarks

Yes, I pointed that out - the issue is that they're not run regularly and
the results posted as they currently for PyPy.

That is, the discussion was about making speed.python.org a going concern,
not the benchmarks themselves.

Cheers,
Nick.

>p
> It is maintained and there is also a mailing-list to discuss it at
https://mail.python.org/mailman/listinfo/speed

>
> Regards
>
> Antoine.
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140410/885eb5bf/attachment.html>

From ncoghlan at gmail.com  Fri Apr 11 03:15:38 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 10 Apr 2014 21:15:38 -0400
Subject: [Python-Dev] Windows installers and OpenSSL
In-Reply-To: <1397170269.16536.105190241.695A2490@webmail.messagingengine.com>
References: <CACac1F-gGuryDiU9JL4zh0XQ0bpE=ybZ-iBFZ0Q=uvrO8zeGJg@mail.gmail.com>
 <5346AFFB.1060301@mrabarnett.plus.com>
 <CAGE7PNJ72QTWbD_yoQ-6x+Y89S35aviXC-CPMwHmsyofbnrUeg@mail.gmail.com>
 <1397170269.16536.105190241.695A2490@webmail.messagingengine.com>
Message-ID: <CADiSq7eaBYNwatKpnXLF4hyCm5hM7VWFjBgTnS6N61hMO-g1cw@mail.gmail.com>

On 10 Apr 2014 18:55, "Benjamin Peterson" <benjamin at python.org> wrote:
>
> On Thu, Apr 10, 2014, at 14:50, Gregory P. Smith wrote:
> > Yep. All binary Python distributions that bundle SSL support need
> > updating.
> > But... what MRAB said.
> >
> > We also *likely* have SSL certificates and SSH host keys on
> > python.orginfrastructure that need to be revoked and new certs
> > reissued
> > *after* all of those machines have been patched and their services
> > restarted. Including hg.python.org.
>
> SSH isn't affected, though, right?

Just SSL.

Cheers,
Nick.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140410/be7edd02/attachment.html>

From mal at egenix.com  Fri Apr 11 04:27:54 2014
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 11 Apr 2014 04:27:54 +0200
Subject: [Python-Dev] Windows installers and OpenSSL
In-Reply-To: <CADiSq7eaBYNwatKpnXLF4hyCm5hM7VWFjBgTnS6N61hMO-g1cw@mail.gmail.com>
References: <CACac1F-gGuryDiU9JL4zh0XQ0bpE=ybZ-iBFZ0Q=uvrO8zeGJg@mail.gmail.com>	<5346AFFB.1060301@mrabarnett.plus.com>	<CAGE7PNJ72QTWbD_yoQ-6x+Y89S35aviXC-CPMwHmsyofbnrUeg@mail.gmail.com>	<1397170269.16536.105190241.695A2490@webmail.messagingengine.com>
 <CADiSq7eaBYNwatKpnXLF4hyCm5hM7VWFjBgTnS6N61hMO-g1cw@mail.gmail.com>
Message-ID: <5347532A.9090103@egenix.com>

On 11.04.2014 03:15, Nick Coghlan wrote:
> On 10 Apr 2014 18:55, "Benjamin Peterson" <benjamin at python.org> wrote:
>>
>> On Thu, Apr 10, 2014, at 14:50, Gregory P. Smith wrote:
>>> Yep. All binary Python distributions that bundle SSL support need
>>> updating.
>>> But... what MRAB said.
>>>
>>> We also *likely* have SSL certificates and SSH host keys on
>>> python.orginfrastructure that need to be revoked and new certs
>>> reissued
>>> *after* all of those machines have been patched and their services
>>> restarted. Including hg.python.org.
>>
>> SSH isn't affected, though, right?
> 
> Just SSL.

And then only machines using OpenSSL 1.0.1. Not the earlier versions.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From tismer at stackless.com  Fri Apr 11 04:12:50 2014
From: tismer at stackless.com (Christian Tismer)
Date: Fri, 11 Apr 2014 04:12:50 +0200
Subject: [Python-Dev] arguments policy: **kwargs.pop()
Message-ID: <53474FA2.3030106@stackless.com>

Hi guys,

I tried to find advice for hours, but failed so fer, so here is my question:

Whenever I think to adopt a new module that does a good job, I always
can't stand the temptation to look at the coding style and certain
principles.

Then I rather often see things like this:

class someclass(object):
    # note that there is no comment about argument destruction...

    def __init__(self, **kwargs):
        first_arg = kwargs.pop('option_1', somedefault)
        ...
        nth_arg = kwargs.pop('option_n', somedefault')
        ...

That is:
There are arguments "consumed" rigorously by the __init__ of a class,
although it would be equivalent to use kwargs.get(..., ...).

Happened to me again, today and blocked me completely, since I don't
know if I saw bad programming style or if I'm mislead.

I agree there are valid cases when if makes sense to filter out some
arguments
that a subsequently called super() might not be able to handle.
Bu even then, I would probably have a copy and make the filtering more
explicit.

But most of the time, there is no reason for using this pattern, and there
is also no comment stating that the argument dict is modified by the callee
for no good reason.

I always used the policy that arguments are never changed by a function,
unless explicitly stated.
But since I see this pattern quite frequently, I wanted to ask if I am
right by
thinking this way, or if the general policy is more like "arguments may be
destroyed by default".

What do you think? Is this bad style and should be noticed somewhere,
or is the caller supposed to protect the arguments, or are my worries
useless?

Thanks & cheers -- Chris

-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
Software Consulting          :     Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
14482 Potsdam                :     PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776  fax +49 (30) 700143-0023
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From guido at python.org  Fri Apr 11 05:47:49 2014
From: guido at python.org (Guido van Rossum)
Date: Thu, 10 Apr 2014 23:47:49 -0400
Subject: [Python-Dev] arguments policy: **kwargs.pop()
In-Reply-To: <53474FA2.3030106@stackless.com>
References: <53474FA2.3030106@stackless.com>
Message-ID: <CAP7+vJ+5YQ-peV=bMLd2i94=jMGa+wanVdiwdTzOs+s5fHXQ4A@mail.gmail.com>

I'm not sure what you're worried about here. Modifying kwds doesn't
actually modify the dict that was passed in. So are you just talking about
the style issue? Or would you also object to this:

def foo(x=1):
    x += 1

?


On Thu, Apr 10, 2014 at 10:12 PM, Christian Tismer <tismer at stackless.com>wrote:

> Hi guys,
>
> I tried to find advice for hours, but failed so fer, so here is my
> question:
>
> Whenever I think to adopt a new module that does a good job, I always
> can't stand the temptation to look at the coding style and certain
> principles.
>
> Then I rather often see things like this:
>
> class someclass(object):
>     # note that there is no comment about argument destruction...
>
>     def __init__(self, **kwargs):
>         first_arg = kwargs.pop('option_1', somedefault)
>         ...
>         nth_arg = kwargs.pop('option_n', somedefault')
>         ...
>
> That is:
> There are arguments "consumed" rigorously by the __init__ of a class,
> although it would be equivalent to use kwargs.get(..., ...).
>
> Happened to me again, today and blocked me completely, since I don't
> know if I saw bad programming style or if I'm mislead.
>
> I agree there are valid cases when if makes sense to filter out some
> arguments
> that a subsequently called super() might not be able to handle.
> Bu even then, I would probably have a copy and make the filtering more
> explicit.
>
> But most of the time, there is no reason for using this pattern, and there
> is also no comment stating that the argument dict is modified by the callee
> for no good reason.
>
> I always used the policy that arguments are never changed by a function,
> unless explicitly stated.
> But since I see this pattern quite frequently, I wanted to ask if I am
> right by
> thinking this way, or if the general policy is more like "arguments may be
> destroyed by default".
>
> What do you think? Is this bad style and should be noticed somewhere,
> or is the caller supposed to protect the arguments, or are my worries
> useless?
>
> Thanks & cheers -- Chris
>
> --
> Christian Tismer             :^)   <mailto:tismer at stackless.com>
> Software Consulting          :     Have a break! Take a ride on Python's
> Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
> 14482 Potsdam                :     PGP key -> http://pgp.uni-mainz.de
> phone +49 173 24 18 776  fax +49 (30) 700143-0023
> PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
>       whom do you want to sponsor today?   http://www.stackless.com/
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140410/afdb830c/attachment.html>

From tseaver at palladion.com  Fri Apr 11 05:48:44 2014
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 10 Apr 2014 23:48:44 -0400
Subject: [Python-Dev] arguments policy: **kwargs.pop()
In-Reply-To: <53474FA2.3030106@stackless.com>
References: <53474FA2.3030106@stackless.com>
Message-ID: <li7omt$aq$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 04/10/2014 10:12 PM, Christian Tismer wrote:

> I always used the policy that arguments are never changed by a
> function, unless explicitly stated. But since I see this pattern quite
> frequently, I wanted to ask if I am right by thinking this way, or if
> the general policy is more like "arguments may be destroyed by
> default".
> 
> What do you think? Is this bad style and should be noticed somewhere, 
> or is the caller supposed to protect the arguments, or are my worries 
> useless?

The caller can't know or care that the function / method pops arguments::

    $ python
    Python 2.7.3 (default, Feb 27 2014, 19:58:35)
    [GCC 4.6.3] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> def foo(**kw):
    ...     bar = kw.pop('bar', 'BAR')
    ...     print 'bar: %s' % bar
    ...     print 'kw: %s' % kw
    ...
    >>> foo()
    bar: BAR
    kw: {}
    >>> foo(bar='baz')
    bar: baz
    kw: {}
    >>> foo(bar='baz', bam='qux')
    bar: baz
    kw: {'bam': 'qux'}
    >>> mykw = {'bar': 'baz', 'bam': 'qux'}
    >>> foo(**mykw)
    bar: baz
    kw: {'bam': 'qux'}
    >>> mykw
    {'bam': 'qux', 'bar': 'baz'}

because the caller gets its own copy of 'kw', even when called with an
existing dict.


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlNHZhwACgkQ+gerLs4ltQ5RLQCeMaFvMDNexmCw9ggbg34w+AjP
DKMAn1U1WRGW9PV8R/xqJs1HPWUBVEse
=A8nP
-----END PGP SIGNATURE-----


From arigo at tunes.org  Fri Apr 11 11:36:53 2014
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 11 Apr 2014 11:36:53 +0200
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <CACac1F8SEiAgFi4LT7P6FYYYKQ59vFNMp97XRSChGDxwOV9bEg@mail.gmail.com>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
 <CAAzeMby9RM2K_2xL2UUGQBtbUfHnBt2tMhLgEFETmP0GAHA88A@mail.gmail.com>
 <CAPdQG2pwdKkL0MfEJkMcFUoYKPcmaE8_Yzq28D1VnWgfPwm+FA@mail.gmail.com>
 <CAP7+vJLC1i=6t6_fmutYiiizX6Mm-6hY6314+Vwz62g821-6WQ@mail.gmail.com>
 <li6rh4$m7e$1@ger.gmane.org>
 <CACac1F8SEiAgFi4LT7P6FYYYKQ59vFNMp97XRSChGDxwOV9bEg@mail.gmail.com>
Message-ID: <CAMSv6X3AP4jk4DxcA3DNOVLq8KTry04ZOi=3gPRKkneg-2SHNg@mail.gmail.com>

Hi,

On 10 April 2014 22:12, Paul Moore <p.f.moore at gmail.com> wrote:
> I agree. I'd like to see a clear explanation of what advantages (and
> disadvantages!) CFFI gives over ctypes, as well as the plan for
> inclusion and how the inevitable confusion over whether to use ctypes
> or cffi will be handled.  (...)

I can't judge exactly what was told in the Language Summit, but here
is my own position about CFFI. Code-wise, we're in precisely the same
spot as last year.  The usage of CFFI seems to be growing a lot.
However, it's not in any stdlib-ready state right now.

Why not?  Because we have a plan to go forward and fix the main issues
people seem to be having: when used in "API mode" there is some
building-C-sources-and-compiling-them going on under your feet;
however, "explicit is better than implicit" seems to apply here too.
Thus, it seems that the basic model might be changed toward a variant
in which you put your C declarations into some separate file that you
need to execute once, in order to build and compile a regular C
extension module.

This would be superficial, but change the perception of CFFI to be "a
preprocessor that produces C extension modules".

This affects the "API mode" but not the "ABI mode" (which is basically
the same as ctypes, modulo the syntax).


A bient?t,

Armin.

From ncoghlan at gmail.com  Fri Apr 11 13:41:12 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 11 Apr 2014 07:41:12 -0400
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <CAMSv6X3AP4jk4DxcA3DNOVLq8KTry04ZOi=3gPRKkneg-2SHNg@mail.gmail.com>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
 <CAAzeMby9RM2K_2xL2UUGQBtbUfHnBt2tMhLgEFETmP0GAHA88A@mail.gmail.com>
 <CAPdQG2pwdKkL0MfEJkMcFUoYKPcmaE8_Yzq28D1VnWgfPwm+FA@mail.gmail.com>
 <CAP7+vJLC1i=6t6_fmutYiiizX6Mm-6hY6314+Vwz62g821-6WQ@mail.gmail.com>
 <li6rh4$m7e$1@ger.gmane.org>
 <CACac1F8SEiAgFi4LT7P6FYYYKQ59vFNMp97XRSChGDxwOV9bEg@mail.gmail.com>
 <CAMSv6X3AP4jk4DxcA3DNOVLq8KTry04ZOi=3gPRKkneg-2SHNg@mail.gmail.com>
Message-ID: <CADiSq7cJ7_yf0LXwkQQwp1XLNTk0LQhCxsg9vDccjufc1e8kzQ@mail.gmail.com>

On 11 Apr 2014 05:39, "Armin Rigo" <arigo at tunes.org> wrote:
>
> Hi,
>
> On 10 April 2014 22:12, Paul Moore <p.f.moore at gmail.com> wrote:
> > I agree. I'd like to see a clear explanation of what advantages (and
> > disadvantages!) CFFI gives over ctypes, as well as the plan for
> > inclusion and how the inevitable confusion over whether to use ctypes
> > or cffi will be handled.  (...)
>
> I can't judge exactly what was told in the Language Summit, but here
> is my own position about CFFI. Code-wise, we're in precisely the same
> spot as last year.  The usage of CFFI seems to be growing a lot.
> However, it's not in any stdlib-ready state right now.
>
> Why not?  Because we have a plan to go forward and fix the main issues
> people seem to be having: when used in "API mode" there is some
> building-C-sources-and-compiling-them going on under your feet;
> however, "explicit is better than implicit" seems to apply here too.
> Thus, it seems that the basic model might be changed toward a variant
> in which you put your C declarations into some separate file that you
> need to execute once, in order to build and compile a regular C
> extension module.
>
> This would be superficial, but change the perception of CFFI to be "a
> preprocessor that produces C extension modules".
>
> This affects the "API mode" but not the "ABI mode" (which is basically
> the same as ctypes, modulo the syntax).

This is along the lines of what Alex said at the summit (i.e. it wasn't
proposed for 3.4 because you didn't consider it stdlib ready yet). The
clearer separation of the build and import steps definitely sounds
promising, as the current implicit approach poses challenges getting cffi
extensions to play nice with wheel based distribution.

Cheers,
Nick.

>
>
> A bient?t,
>
> Armin.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140411/fa1a7f3b/attachment.html>

From p.f.moore at gmail.com  Fri Apr 11 14:22:31 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 11 Apr 2014 13:22:31 +0100
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <CAMSv6X3AP4jk4DxcA3DNOVLq8KTry04ZOi=3gPRKkneg-2SHNg@mail.gmail.com>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
 <CAAzeMby9RM2K_2xL2UUGQBtbUfHnBt2tMhLgEFETmP0GAHA88A@mail.gmail.com>
 <CAPdQG2pwdKkL0MfEJkMcFUoYKPcmaE8_Yzq28D1VnWgfPwm+FA@mail.gmail.com>
 <CAP7+vJLC1i=6t6_fmutYiiizX6Mm-6hY6314+Vwz62g821-6WQ@mail.gmail.com>
 <li6rh4$m7e$1@ger.gmane.org>
 <CACac1F8SEiAgFi4LT7P6FYYYKQ59vFNMp97XRSChGDxwOV9bEg@mail.gmail.com>
 <CAMSv6X3AP4jk4DxcA3DNOVLq8KTry04ZOi=3gPRKkneg-2SHNg@mail.gmail.com>
Message-ID: <CACac1F_7RCbSbA2UpXmVL6-nsSxVgAFyaO-607O3uhzp7D+9pQ@mail.gmail.com>

On 11 April 2014 10:36, Armin Rigo <arigo at tunes.org> wrote:
> This would be superficial, but change the perception of CFFI to be "a
> preprocessor that produces C extension modules".

Thanks, that clarification helps a lot. Does this mean that "API-mode"
CFFI is competing with things like swig (which is not used much these
days, as far as I know) and Cython (which is used a lot in the numeric
community)? ("ABI-mode" CFFI is obviously directly competing with
ctypes).

Paul

From jdornak at redhat.com  Fri Apr 11 14:58:16 2014
From: jdornak at redhat.com (Jakub QB =?utf-8?B?RG9yxYjDoWs=?=)
Date: Fri, 11 Apr 2014 08:58:16 -0400 (EDT)
Subject: [Python-Dev] flock in Python 3
In-Reply-To: <796149019.3768397.1397219470829.JavaMail.zimbra@redhat.com>
Message-ID: <1867801184.3782831.1397221096189.JavaMail.zimbra@redhat.com>

Hi all,

writing a threaded application I've been surprised that there is no object api for fcntl.flock implementing __enter__() and __exit__() methods to be used with 'with' statement.
I have written one (https://pypi.python.org/pypi), but I wonder whether this could get into the Python Standard Library.

Regards,
QB

From status at bugs.python.org  Fri Apr 11 18:07:53 2014
From: status at bugs.python.org (Python tracker)
Date: Fri, 11 Apr 2014 18:07:53 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20140411160753.8ACA856A2E@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2014-04-04 - 2014-04-11)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    4567 (+17)
  closed 28405 (+29)
  total  32972 (+46)

Open issues with patches: 2096 


Issues opened (32)
==================

#21015: support SSL_CTX_set_ecdh_auto on newer OpenSSLs
http://bugs.python.org/issue21015  reopened by ned.deily

#21156: Consider moving importlib.abc.InspectLoader.source_to_code() t
http://bugs.python.org/issue21156  opened by brett.cannon

#21157: Update imp docs for a PEP 451 world
http://bugs.python.org/issue21157  opened by brett.cannon

#21158: Windows installer service could not be accessed (Python bug!)
http://bugs.python.org/issue21158  opened by thecheater887

#21159: configparser.InterpolationMissingOptionError is not very intui
http://bugs.python.org/issue21159  opened by Claudiu.Popa

#21160: incorrect comments in nturl2path.py
http://bugs.python.org/issue21160  opened by Jurko.Gospodneti??

#21162: code in multiprocessing.pool freeze if inside some code from s
http://bugs.python.org/issue21162  opened by Ivan.K

#21163: asyncio task possibly incorrectly garbage collected
http://bugs.python.org/issue21163  opened by richard.kiss

#21164: print unicode in Windows console
http://bugs.python.org/issue21164  opened by LeslieK

#21165: Optimize str.translate() for replacement with substrings and n
http://bugs.python.org/issue21165  opened by haypo

#21167: float('nan') returns 0.0 on Python compiled with icc
http://bugs.python.org/issue21167  opened by hniksic

#21169: getpass.getpass() fails with non-ASCII characters in prompt
http://bugs.python.org/issue21169  opened by Arfrever

#21170: Incorrect signature for unittest.TestResult.startTestRun(), .s
http://bugs.python.org/issue21170  opened by Volodymyr.Sapsai

#21171: Outdated usage str.encode('rot-13') in rot13 codec
http://bugs.python.org/issue21171  opened by Pix

#21173: WeakKeyDictionary.__len__ fragile w/ _IterationGuards
http://bugs.python.org/issue21173  opened by pjenvey

#21177: ValueError: byte must be in range(0, 256)
http://bugs.python.org/issue21177  opened by zart

#21178: doctest cause warnings in tests using generators
http://bugs.python.org/issue21178  opened by oluensdorf

#21180: Cannot efficiently create empty array.array of given size, inc
http://bugs.python.org/issue21180  opened by pfalcon

#21183: Doctest capture only AssertionError but not printed text
http://bugs.python.org/issue21183  opened by osantana

#21184: statistics.pvariance with known mean does not work as expected
http://bugs.python.org/issue21184  opened by steven.daprano

#21186: RawConfigParser __name__ option handling inconsistent
http://bugs.python.org/issue21186  opened by Adam.Groszer

#21188: Broken link
http://bugs.python.org/issue21188  opened by tiposchi

#21189: Broken link to patch
http://bugs.python.org/issue21189  opened by tiposchi

#21192: Idle: Print filename when running a file from editor
http://bugs.python.org/issue21192  opened by terry.reedy

#21193: pow(a, b, c) should not raise TypeError when b is negative and
http://bugs.python.org/issue21193  opened by josh.rosenberg

#21194: json.dumps with ensure_ascii=False doesn't escape control char
http://bugs.python.org/issue21194  opened by weeble

#21195: None float format: incomplete documentation
http://bugs.python.org/issue21195  opened by lebigot

#21196: Name mangling example in Python tutorial
http://bugs.python.org/issue21196  opened by chandan

#21197: venv does not create lib64 directory and appropriate symlinks
http://bugs.python.org/issue21197  opened by BinaryKhaos

#21198: Minor tarfile documentation bug
http://bugs.python.org/issue21198  opened by lurchman

#21199: Python on 64-bit Windows uses signed 32-bit type for read leng
http://bugs.python.org/issue21199  opened by Matt.Mackall

#21200: pkgutil.get_loader() fails on "__main__"
http://bugs.python.org/issue21200  opened by eric.snow



Most recent 15 issues with no replies (15)
==========================================

#21200: pkgutil.get_loader() fails on "__main__"
http://bugs.python.org/issue21200

#21198: Minor tarfile documentation bug
http://bugs.python.org/issue21198

#21188: Broken link
http://bugs.python.org/issue21188

#21173: WeakKeyDictionary.__len__ fragile w/ _IterationGuards
http://bugs.python.org/issue21173

#21160: incorrect comments in nturl2path.py
http://bugs.python.org/issue21160

#21152: Idle: timed autosave for shell (and maybe editor) window
http://bugs.python.org/issue21152

#21132: Failure to import win32api
http://bugs.python.org/issue21132

#21126: Integrate doctest.run_docstring_examples() with unittest
http://bugs.python.org/issue21126

#21120: PyArena type is used in headers from the limited API
http://bugs.python.org/issue21120

#21114: wsgiref.simple_server doesn't handle multi-line headers correc
http://bugs.python.org/issue21114

#21112: 3.4 regression: unittest.expectedFailure no longer works on Te
http://bugs.python.org/issue21112

#21110: Slowdown and high memory usage when adding a new module in emb
http://bugs.python.org/issue21110

#21107: Add pgen.vcxproj to allow regenerating grammar files on Window
http://bugs.python.org/issue21107

#21095: EmailMessage should support Header objects
http://bugs.python.org/issue21095

#21091: EmailMessage.is_attachment should be a method
http://bugs.python.org/issue21091



Most recent 15 issues waiting for review (15)
=============================================

#21193: pow(a, b, c) should not raise TypeError when b is negative and
http://bugs.python.org/issue21193

#21173: WeakKeyDictionary.__len__ fragile w/ _IterationGuards
http://bugs.python.org/issue21173

#21171: Outdated usage str.encode('rot-13') in rot13 codec
http://bugs.python.org/issue21171

#21170: Incorrect signature for unittest.TestResult.startTestRun(), .s
http://bugs.python.org/issue21170

#21169: getpass.getpass() fails with non-ASCII characters in prompt
http://bugs.python.org/issue21169

#21167: float('nan') returns 0.0 on Python compiled with icc
http://bugs.python.org/issue21167

#21148: avoid memset in small tuple creation
http://bugs.python.org/issue21148

#21147: sqlite3 doesn't complain if the request contains a null charac
http://bugs.python.org/issue21147

#21146: update gzip usage examples in docs
http://bugs.python.org/issue21146

#21141: Don't mention Perl in Windows build output
http://bugs.python.org/issue21141

#21140: Idle: saving an OutputWindow should default to .txt
http://bugs.python.org/issue21140

#21139: Idle: change default reformat width from 70 to 72
http://bugs.python.org/issue21139

#21137: Better repr for threading.Lock()
http://bugs.python.org/issue21137

#21127: Path objects cannot be constructed from str subclasses
http://bugs.python.org/issue21127

#21126: Integrate doctest.run_docstring_examples() with unittest
http://bugs.python.org/issue21126



Top 10 most discussed issues (10)
=================================

#21177: ValueError: byte must be in range(0, 256)
http://bugs.python.org/issue21177  13 msgs

#21111: PyLong_AsUnsignedLongAndOverflow does not exist
http://bugs.python.org/issue21111   9 msgs

#21167: float('nan') returns 0.0 on Python compiled with icc
http://bugs.python.org/issue21167   9 msgs

#16927: Separate built-in types from functions and group similar funct
http://bugs.python.org/issue16927   8 msgs

#21162: code in multiprocessing.pool freeze if inside some code from s
http://bugs.python.org/issue21162   8 msgs

#21101: Extend the PyDict C API to handle cases where the hash value i
http://bugs.python.org/issue21101   7 msgs

#1191964: asynchronous Subprocess
http://bugs.python.org/issue1191964   7 msgs

#20383: Add a keyword-only spec argument to types.ModuleType
http://bugs.python.org/issue20383   6 msgs

#21163: asyncio task possibly incorrectly garbage collected
http://bugs.python.org/issue21163   6 msgs

#21193: pow(a, b, c) should not raise TypeError when b is negative and
http://bugs.python.org/issue21193   6 msgs



Issues closed (30)
==================

#12014: str.format parses replacement field incorrectly
http://bugs.python.org/issue12014  closed by benjamin.peterson

#16395: Documentation claims that PySequence_Fast returns a tuple, whe
http://bugs.python.org/issue16395  closed by python-dev

#17621: Create a lazy import loader mixin
http://bugs.python.org/issue17621  closed by brett.cannon

#19281: add __objclass__ to the docs
http://bugs.python.org/issue19281  closed by python-dev

#20334: make inspect Signature hashable
http://bugs.python.org/issue20334  closed by yselivanov

#20539: math.factorial may throw OverflowError
http://bugs.python.org/issue20539  closed by mark.dickinson

#20644: OS X installer build broken by changes to documentation build
http://bugs.python.org/issue20644  closed by ned.deily

#21076: Turn signal.SIG* constants into enums
http://bugs.python.org/issue21076  closed by giampaolo.rodola

#21117: inspect.signature: inaccuracies for partial functions
http://bugs.python.org/issue21117  closed by yselivanov

#21118: str.translate is absurdly slow in majority of use cases (takes
http://bugs.python.org/issue21118  closed by haypo

#21128: testing stdlib and compatibility with pypy
http://bugs.python.org/issue21128  closed by benjamin.peterson

#21133: unittest discover should allow option to run each package sepa
http://bugs.python.org/issue21133  closed by michael.foord

#21136: fractions.Fraction.__pow__ does unneeded renormalization
http://bugs.python.org/issue21136  closed by mark.dickinson

#21144: ensurepip "AssertionError: Multiple .dist-info directories"
http://bugs.python.org/issue21144  closed by ned.deily

#21153: bdist_rpm fails if project contains files with spaces in the n
http://bugs.python.org/issue21153  closed by jason.coombs

#21155: asyncio: calling _UnixSelectorEventLoop.create_unix_server(soc
http://bugs.python.org/issue21155  closed by haypo

#21161: list comprehensions don't see local variables in pdb in python
http://bugs.python.org/issue21161  closed by georg.brandl

#21166: Bus error on "AMD64 FreeBSD 9.x 3.4" buildbot
http://bugs.python.org/issue21166  closed by haypo

#21168: unicodeobject.c: likely type-punning may break strict-aliasing
http://bugs.python.org/issue21168  closed by haypo

#21172: Unexpected behaviour with logging.LogRecord "first arg is dict
http://bugs.python.org/issue21172  closed by vinay.sajip

#21174: A typo in the docs for "exception GeneratorExit"
http://bugs.python.org/issue21174  closed by python-dev

#21175: Refcounting error in str.translate fastpath
http://bugs.python.org/issue21175  closed by python-dev

#21176: Implement matrix multiplication operator (PEP 465)
http://bugs.python.org/issue21176  closed by python-dev

#21179: Rounding half to even
http://bugs.python.org/issue21179  closed by mark.dickinson

#21181: Inconsistent Definition of collections.namedtuple.__dict__ in 
http://bugs.python.org/issue21181  closed by rhettinger

#21182: json.loads errors out on valid JSON
http://bugs.python.org/issue21182  closed by ned.deily

#21185: heapq fails to print in sorted order for certain inputs
http://bugs.python.org/issue21185  closed by wchlm

#21187: build-installer.py fails using Xcode 5.1 on OS X 10.9
http://bugs.python.org/issue21187  closed by ned.deily

#21190: Broken download link on README for CPython docs
http://bugs.python.org/issue21190  closed by orsenthil

#21191: os.fdopen() may eat file descriptor and still raise exception
http://bugs.python.org/issue21191  closed by benjamin.peterson

From tjreedy at udel.edu  Fri Apr 11 18:35:36 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 11 Apr 2014 12:35:36 -0400
Subject: [Python-Dev] flock in Python 3
In-Reply-To: <1867801184.3782831.1397221096189.JavaMail.zimbra@redhat.com>
References: <796149019.3768397.1397219470829.JavaMail.zimbra@redhat.com>
 <1867801184.3782831.1397221096189.JavaMail.zimbra@redhat.com>
Message-ID: <li95l8$m9v$1@ger.gmane.org>

On 4/11/2014 8:58 AM, Jakub QB Dor??k wrote:

> writing a threaded application I've been surprised that there is no
> object api for fcntl.flock implementing __enter__() and __exit__()
> methods to be used with 'with' statement.

Several things have been turned into context managers because someone 
filed a request on the tracker. I do not see one for flock.

> I have written one
> (https://pypi.python.org/pypi), but I wonder whether this could get
> into the Python Standard Library.

It might if you open an issue and submitted a patch.

-- 
Terry Jan Reedy



From benjamin at python.org  Fri Apr 11 18:38:43 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 11 Apr 2014 09:38:43 -0700
Subject: [Python-Dev] flock in Python 3
In-Reply-To: <1867801184.3782831.1397221096189.JavaMail.zimbra@redhat.com>
References: <1867801184.3782831.1397221096189.JavaMail.zimbra@redhat.com>
Message-ID: <1397234323.20322.105474653.7638E691@webmail.messagingengine.com>

On Fri, Apr 11, 2014, at 5:58, Jakub QB Dor??k wrote:
> Hi all,
> 
> writing a threaded application I've been surprised that there is no
> object api for fcntl.flock implementing __enter__() and __exit__()
> methods to be used with 'with' statement.
> I have written one (https://pypi.python.org/pypi), but I wonder whether
> this could get into the Python Standard Library.

That link PyPi link is broken.

I think it would be nicer to have a high-level API for cross-platform
file locking. IIRC, flock can be rather non-portable.

From tismer at stackless.com  Fri Apr 11 19:26:07 2014
From: tismer at stackless.com (Christian Tismer)
Date: Fri, 11 Apr 2014 19:26:07 +0200
Subject: [Python-Dev] arguments policy: **kwargs.pop()
In-Reply-To: <CAP7+vJ+5YQ-peV=bMLd2i94=jMGa+wanVdiwdTzOs+s5fHXQ4A@mail.gmail.com>
References: <53474FA2.3030106@stackless.com>
 <CAP7+vJ+5YQ-peV=bMLd2i94=jMGa+wanVdiwdTzOs+s5fHXQ4A@mail.gmail.com>
Message-ID: <534825AF.1070500@stackless.com>

Ah, now I see it.

For some reason, I forgot that the dict is always newly created.
That was really wrong thinking. Sorry!

On 11/04/14 05:47, Guido van Rossum wrote:
> I'm not sure what you're worried about here. Modifying kwds doesn't
> actually modify the dict that was passed in. So are you just talking about
> the style issue? Or would you also object to this:
> 
> def foo(x=1):
>     x += 1
> 

No, this is ok, although I personally tend not to modify args.

No, it was just my temporary forgetting that the star
args/kwds are always in a new container, and the whole reasoning chain
was pointless, therefore.

Thanks and cheers - Chris


> On Thu, Apr 10, 2014 at 10:12 PM, Christian Tismer <tismer at stackless.com>wrote:
> 
>> Hi guys,
>>
>> I tried to find advice for hours, but failed so fer, so here is my
>> question:
>>
>> Whenever I think to adopt a new module that does a good job, I always
>> can't stand the temptation to look at the coding style and certain
>> principles.
>>
>> Then I rather often see things like this:
>>
>> class someclass(object):
>>     # note that there is no comment about argument destruction...
>>
>>     def __init__(self, **kwargs):
>>         first_arg = kwargs.pop('option_1', somedefault)
>>         ...
>>         nth_arg = kwargs.pop('option_n', somedefault')
>>         ...
>>
>> That is:
>> There are arguments "consumed" rigorously by the __init__ of a class,
>> although it would be equivalent to use kwargs.get(..., ...).
>>
>> Happened to me again, today and blocked me completely, since I don't
>> know if I saw bad programming style or if I'm mislead.
>>
>> I agree there are valid cases when if makes sense to filter out some
>> arguments
>> that a subsequently called super() might not be able to handle.
>> Bu even then, I would probably have a copy and make the filtering more
>> explicit.
>>
>> But most of the time, there is no reason for using this pattern, and there
>> is also no comment stating that the argument dict is modified by the callee
>> for no good reason.
>>
>> I always used the policy that arguments are never changed by a function,
>> unless explicitly stated.
>> But since I see this pattern quite frequently, I wanted to ask if I am
>> right by
>> thinking this way, or if the general policy is more like "arguments may be
>> destroyed by default".
>>
>> What do you think? Is this bad style and should be noticed somewhere,
>> or is the caller supposed to protect the arguments, or are my worries
>> useless?
>>
>> Thanks & cheers -- Chris
>>


-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
Software Consulting          :     Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
14482 Potsdam                :     PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776  fax +49 (30) 700143-0023
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From tismer at stackless.com  Fri Apr 11 19:30:52 2014
From: tismer at stackless.com (Christian Tismer)
Date: Fri, 11 Apr 2014 19:30:52 +0200
Subject: [Python-Dev] arguments policy: **kwargs.pop()
In-Reply-To: <li7omt$aq$1@ger.gmane.org>
References: <53474FA2.3030106@stackless.com> <li7omt$aq$1@ger.gmane.org>
Message-ID: <534826CC.9080503@stackless.com>

Thank you too, Tres.
Somehow I had a brain shortcut and forgot that
the dict is locally generated, *because* of the stars.

Good to become adjusted and restarted, sorry about the noise.

ciao - Chris


On 11/04/14 05:48, Tres Seaver wrote:
> On 04/10/2014 10:12 PM, Christian Tismer wrote:
> 
>> I always used the policy that arguments are never changed by a 
>> function, unless explicitly stated. But since I see this pattern 
>> quite frequently, I wanted to ask if I am right by thinking this 
>> way, or if the general policy is more like "arguments may be 
>> destroyed by default".
> 
>> What do you think? Is this bad style and should be noticed 
>> somewhere, or is the caller supposed to protect the arguments,
>> or are my worries useless?
> 
> The caller can't know or care that the function / method pops 
> arguments::
> 
> $ python Python 2.7.3 (default, Feb 27 2014, 19:58:35) [GCC 4.6.3] 
> on linux2 Type "help", "copyright", "credits" or "license" for
> more information.
>>>> def foo(**kw):
> ...     bar = kw.pop('bar', 'BAR') ...     print 'bar: %s' % bar 
> ...     print 'kw: %s' % kw ...
>>>> foo()
> bar: BAR kw: {}
>>>> foo(bar='baz')
> bar: baz kw: {}
>>>> foo(bar='baz', bam='qux')
> bar: baz kw: {'bam': 'qux'}
>>>> mykw = {'bar': 'baz', 'bam': 'qux'} foo(**mykw)
> bar: baz kw: {'bam': 'qux'}
>>>> mykw
> {'bam': 'qux', 'bar': 'baz'}
> 
> because the caller gets its own copy of 'kw', even when called
> with an existing dict.
> 
> 
> Tres.
> 
> _______________________________________________ Python-Dev mailing 
> list Python-Dev at python.org 
> https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/tismer%40stackless.com
>
>
> 
-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
Software Consulting          :     Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
14482 Potsdam                :     PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776  fax +49 (30) 700143-0023
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From antony.lee at berkeley.edu  Fri Apr 11 19:47:55 2014
From: antony.lee at berkeley.edu (Antony Lee)
Date: Fri, 11 Apr 2014 10:47:55 -0700
Subject: [Python-Dev] Receiving email updates for the bug tracker
Message-ID: <CAGRr6BEgbH=oPYU6S5T1do3J0c6AvEGitc6-9wJdhp-R=otvsg@mail.gmail.com>

Hi,
Sorry for the slightly off-topic(?) question but I would like to know how
to receive email notifications for activity on bugs I've opened on the
bugs.python.org -- so far I don't receive anything even though I'm on the
nosy list.
Thanks,
Antony
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140411/1c39cc54/attachment.html>

From fijall at gmail.com  Fri Apr 11 19:55:57 2014
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 11 Apr 2014 19:55:57 +0200
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <CACac1F_7RCbSbA2UpXmVL6-nsSxVgAFyaO-607O3uhzp7D+9pQ@mail.gmail.com>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
 <CAAzeMby9RM2K_2xL2UUGQBtbUfHnBt2tMhLgEFETmP0GAHA88A@mail.gmail.com>
 <CAPdQG2pwdKkL0MfEJkMcFUoYKPcmaE8_Yzq28D1VnWgfPwm+FA@mail.gmail.com>
 <CAP7+vJLC1i=6t6_fmutYiiizX6Mm-6hY6314+Vwz62g821-6WQ@mail.gmail.com>
 <li6rh4$m7e$1@ger.gmane.org>
 <CACac1F8SEiAgFi4LT7P6FYYYKQ59vFNMp97XRSChGDxwOV9bEg@mail.gmail.com>
 <CAMSv6X3AP4jk4DxcA3DNOVLq8KTry04ZOi=3gPRKkneg-2SHNg@mail.gmail.com>
 <CACac1F_7RCbSbA2UpXmVL6-nsSxVgAFyaO-607O3uhzp7D+9pQ@mail.gmail.com>
Message-ID: <CAK5idxRjgY8Wg5yVZd5sDCSyrkpLjj0gs+eBuu2CaiB-QK7nfw@mail.gmail.com>

On Fri, Apr 11, 2014 at 2:22 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> On 11 April 2014 10:36, Armin Rigo <arigo at tunes.org> wrote:
>> This would be superficial, but change the perception of CFFI to be "a
>> preprocessor that produces C extension modules".
>
> Thanks, that clarification helps a lot. Does this mean that "API-mode"
> CFFI is competing with things like swig (which is not used much these
> days, as far as I know) and Cython (which is used a lot in the numeric
> community)? ("ABI-mode" CFFI is obviously directly competing with
> ctypes).

Yes.

From brett at python.org  Fri Apr 11 21:12:34 2014
From: brett at python.org (Brett Cannon)
Date: Fri, 11 Apr 2014 15:12:34 -0400
Subject: [Python-Dev] Receiving email updates for the bug tracker
In-Reply-To: <CAGRr6BEgbH=oPYU6S5T1do3J0c6AvEGitc6-9wJdhp-R=otvsg@mail.gmail.com>
References: <CAGRr6BEgbH=oPYU6S5T1do3J0c6AvEGitc6-9wJdhp-R=otvsg@mail.gmail.com>
Message-ID: <CAP1=2W596Con8RWSnYuDcp=N1KZBXz9cMUhuPBjqrnybtsnoNg@mail.gmail.com>

On Fri, Apr 11, 2014 at 1:47 PM, Antony Lee <antony.lee at berkeley.edu> wrote:

> Hi,
> Sorry for the slightly off-topic(?) question but I would like to know how
> to receive email notifications for activity on bugs I've opened on the
> bugs.python.org -- so far I don't receive anything even though I'm on the
> nosy list.
>

Should be automatic. I would check your email address in your account
settings.

-Brett


>  Thanks,
> Antony
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140411/eaeb7655/attachment.html>

From brett at python.org  Fri Apr 11 21:27:49 2014
From: brett at python.org (Brett Cannon)
Date: Fri, 11 Apr 2014 15:27:49 -0400
Subject: [Python-Dev] Receiving email updates for the bug tracker
In-Reply-To: <CAGRr6BFvbaBePQM3NoSh5wkKnEYSBUKdmm__squmyf2pSA-XBQ@mail.gmail.com>
References: <CAGRr6BFvbaBePQM3NoSh5wkKnEYSBUKdmm__squmyf2pSA-XBQ@mail.gmail.com>
Message-ID: <CAP1=2W4bbbQJVHwjvS4abT_uKwsvzW8xPKigkgwPi7ctWi2pHg@mail.gmail.com>

On Fri, Apr 11, 2014 at 3:25 PM, Antony Lee <antony.lee at berkeley.edu> wrote:

> Nope, the email is correct...
>

Then you can try reporting a bug at
http://psf.upfronthosting.co.za/roundup/meta/


>
> 2014-04-11 12:12 GMT-07:00 Brett Cannon <brett at python.org>:
>
>
>>
>>
>> On Fri, Apr 11, 2014 at 1:47 PM, Antony Lee <antony.lee at berkeley.edu>wrote:
>>
>>> Hi,
>>> Sorry for the slightly off-topic(?) question but I would like to know
>>> how to receive email notifications for activity on bugs I've opened on the
>>> bugs.python.org -- so far I don't receive anything even though I'm on
>>> the nosy list.
>>>
>>
>> Should be automatic. I would check your email address in your account
>> settings.
>>
>> -Brett
>>
>>
>>>  Thanks,
>>> Antony
>>>
>>> _______________________________________________
>>> Python-Dev mailing list
>>> Python-Dev at python.org
>>> https://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe:
>>> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140411/c61ccd43/attachment.html>

From chris.barker at noaa.gov  Fri Apr 11 21:50:10 2014
From: chris.barker at noaa.gov (Chris Barker)
Date: Fri, 11 Apr 2014 12:50:10 -0700
Subject: [Python-Dev] arguments policy: **kwargs.pop()
In-Reply-To: <53474FA2.3030106@stackless.com>
References: <53474FA2.3030106@stackless.com>
Message-ID: <CALGmxEKza==gz=pEySZu8q33Hi0VhfXFw0zohxb9=tdyoLFzFw@mail.gmail.com>

On Thu, Apr 10, 2014 at 7:12 PM, Christian Tismer <tismer at stackless.com>wrote:

> Then I rather often see things like this:
>
> class someclass(object):
>     # note that there is no comment about argument destruction...
>
>     def __init__(self, **kwargs):
>         first_arg = kwargs.pop('option_1', somedefault)
>         ...
>         nth_arg = kwargs.pop('option_n', somedefault')
>         ...
>

While it's been clarified that this isn't dangerous, I find it to be a
really annoying style, as you've lost the opurtuniyt to docuemnt somethign
in the signature. Is:

    def __init__(self, option_1=some_default, option_n=some_default,
**kwargs):
        first_arg = kwargs.pop('option_1')
        nth_arg = kwargs.pop('option_n')

*that* much harder to write?

And many of these come with virtually no docstring, either.....

oh well,

  -Chris


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140411/b8138009/attachment.html>

From guido at python.org  Fri Apr 11 22:20:50 2014
From: guido at python.org (Guido van Rossum)
Date: Fri, 11 Apr 2014 16:20:50 -0400
Subject: [Python-Dev] Receiving email updates for the bug tracker
In-Reply-To: <CAGRr6BEgbH=oPYU6S5T1do3J0c6AvEGitc6-9wJdhp-R=otvsg@mail.gmail.com>
References: <CAGRr6BEgbH=oPYU6S5T1do3J0c6AvEGitc6-9wJdhp-R=otvsg@mail.gmail.com>
Message-ID: <CAP7+vJKTyMmGD3dcdeGoJh44-fzGX6yhOw9Hq7uBFGe0XMmz8Q@mail.gmail.com>

Did you check your spam folder?


On Fri, Apr 11, 2014 at 1:47 PM, Antony Lee <antony.lee at berkeley.edu> wrote:

> Hi,
> Sorry for the slightly off-topic(?) question but I would like to know how
> to receive email notifications for activity on bugs I've opened on the
> bugs.python.org -- so far I don't receive anything even though I'm on the
> nosy list.
> Thanks,
> Antony
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140411/1273a494/attachment.html>

From antony.lee at berkeley.edu  Fri Apr 11 21:25:54 2014
From: antony.lee at berkeley.edu (Antony Lee)
Date: Fri, 11 Apr 2014 12:25:54 -0700
Subject: [Python-Dev] Receiving email updates for the bug tracker
Message-ID: <CAGRr6BFvbaBePQM3NoSh5wkKnEYSBUKdmm__squmyf2pSA-XBQ@mail.gmail.com>

Nope, the email is correct...

2014-04-11 12:12 GMT-07:00 Brett Cannon <brett at python.org>:

>
>
>
> On Fri, Apr 11, 2014 at 1:47 PM, Antony Lee <antony.lee at berkeley.edu>wrote:
>
>> Hi,
>> Sorry for the slightly off-topic(?) question but I would like to know how
>> to receive email notifications for activity on bugs I've opened on the
>> bugs.python.org -- so far I don't receive anything even though I'm on
>> the nosy list.
>>
>
> Should be automatic. I would check your email address in your account
> settings.
>
> -Brett
>
>
>>  Thanks,
>> Antony
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140411/4580f41b/attachment.html>

From tismer at stackless.com  Fri Apr 11 23:01:50 2014
From: tismer at stackless.com (Christian Tismer)
Date: Fri, 11 Apr 2014 23:01:50 +0200
Subject: [Python-Dev] arguments policy: **kwargs.pop()
In-Reply-To: <CALGmxEKza==gz=pEySZu8q33Hi0VhfXFw0zohxb9=tdyoLFzFw@mail.gmail.com>
References: <53474FA2.3030106@stackless.com>
 <CALGmxEKza==gz=pEySZu8q33Hi0VhfXFw0zohxb9=tdyoLFzFw@mail.gmail.com>
Message-ID: <5348583E.1070806@stackless.com>

Hi Chris,

On 11/04/14 21:50, Chris Barker wrote:
> On Thu, Apr 10, 2014 at 7:12 PM, Christian Tismer <tismer at stackless.com>wrote:
> 
>> Then I rather often see things like this:
>>
>> class someclass(object):
>>     # note that there is no comment about argument destruction...
>>
>>     def __init__(self, **kwargs):
>>         first_arg = kwargs.pop('option_1', somedefault)
>>         ...
>>         nth_arg = kwargs.pop('option_n', somedefault')
>>         ...
>>
> 
> While it's been clarified that this isn't dangerous, I find it to be a
> really annoying style, as you've lost the opurtuniyt to docuemnt somethign
> in the signature. Is:
> 
>     def __init__(self, option_1=some_default, option_n=some_default,
> **kwargs):
>         first_arg = kwargs.pop('option_1')
>         nth_arg = kwargs.pop('option_n')
> 
> *that* much harder to write?
> 
> And many of these come with virtually no docstring, either.....

Thank you for re-validating my rant, after my wrong start was clarified.
Yes, the style is of course annoying, still.

This is the style of
"""hey look how cool I am, just taking an interface, picking args if
they happen to be there and otherwise not treating them""".

So while I'm still ashamed of my mis-interpretion, I am happy to still
not like that very much.

At least for myself, I like to be way more explicit and tell actively
what I expect as arguments, what I do care about and what not, just to
make sure that people see right by looking at the interface what they
may ignore and what they should probably put in as an argument.

Actually, putting so many defaults in without documenting that in the
interface is this new-fangled sloppiness that is perceived as cool-ness.

May be I am getting old, but I dislike this and tend to tell much more
in the interface. And not in the 35th iteration, but when writing the
first public version.

This is because I don't want to throw an interface at somebody, but to
discuss and improve it, and for that I put comments in that invite to
agree or create a better version.

I have these style problems with several modules that I am reluctant to
use, therefore. I know that I'm pretty alone with that. But my idea
of a published module is such that it should try to motivate why it is
doing things in which way, and why it thinks this is good to do.

Doing that not and nothing instead is my definition of "sloppy".

(interested people may get the actual module from me why this came up)

cheers -- Chris


-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
Software Consulting          :     Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
14482 Potsdam                :     PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776  fax +49 (30) 700143-0023
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From antony.lee at berkeley.edu  Fri Apr 11 23:15:36 2014
From: antony.lee at berkeley.edu (Antony Lee)
Date: Fri, 11 Apr 2014 14:15:36 -0700
Subject: [Python-Dev] Receiving email updates for the bug tracker
In-Reply-To: <CAP7+vJKTyMmGD3dcdeGoJh44-fzGX6yhOw9Hq7uBFGe0XMmz8Q@mail.gmail.com>
References: <CAGRr6BEgbH=oPYU6S5T1do3J0c6AvEGitc6-9wJdhp-R=otvsg@mail.gmail.com>
 <CAP7+vJKTyMmGD3dcdeGoJh44-fzGX6yhOw9Hq7uBFGe0XMmz8Q@mail.gmail.com>
Message-ID: <CAGRr6BF6x46JRpwGh+3wDj+wBo0c+s5rpdGXcpeoAcwVzjmpGQ@mail.gmail.com>

Actually that was a better idea...
Antony


2014-04-11 13:20 GMT-07:00 Guido van Rossum <guido at python.org>:

> Did you check your spam folder?
>
>
> On Fri, Apr 11, 2014 at 1:47 PM, Antony Lee <antony.lee at berkeley.edu>wrote:
>
>> Hi,
>> Sorry for the slightly off-topic(?) question but I would like to know how
>> to receive email notifications for activity on bugs I've opened on the
>> bugs.python.org -- so far I don't receive anything even though I'm on
>> the nosy list.
>> Thanks,
>> Antony
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140411/f8da05cb/attachment.html>

From greg.ewing at canterbury.ac.nz  Sat Apr 12 02:25:08 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 12 Apr 2014 12:25:08 +1200
Subject: [Python-Dev] arguments policy: **kwargs.pop()
In-Reply-To: <5348583E.1070806@stackless.com>
References: <53474FA2.3030106@stackless.com>
 <CALGmxEKza==gz=pEySZu8q33Hi0VhfXFw0zohxb9=tdyoLFzFw@mail.gmail.com>
 <5348583E.1070806@stackless.com>
Message-ID: <534887E4.4020103@canterbury.ac.nz>

> On 11/04/14 21:50, Chris Barker wrote:
> 
>>On Thu, Apr 10, 2014 at 7:12 PM, Christian Tismer <tismer at stackless.com>wrote:
>>
>>>    def __init__(self, **kwargs):
>>>        first_arg = kwargs.pop('option_1', somedefault)
>>>        ...
>>>        nth_arg = kwargs.pop('option_n', somedefault')
>>>        ...
>>
>> Is:
>>
>>    def __init__(self, option_1=some_default, option_n=some_default,
>>**kwargs):
>>
>>*that* much harder to write?

I've done this kind of thing (extracting arguments out of **kwds)
in various places in PyGUI, but the situation is somewhat special.
Most classes in PyGUI have an api that accepts *any* of the object's
properties as keyword arguments to the constructor, implemented by
code in the base class's __init__ method. It would be impractical
to explicitly document all of them as constructor args, so I don't.
I just say in one place in the docs that this is general behaviour
to be expected of all PyGUI classes.

Sometimes a class needs to treat the initial values of some of its
properties in a special way, instead of just passing them on to the
base __init__. But this is an implementation detail -- having those
particular args appear explicitly in the signature, but not any of
the others, would serve no purpose, and would just clutter up the
function header. In that situation, I find the extract-from-kwds
style is often easier to read.

Also, often I just want to *read* the value of some arguments,
without popping them. Putting all of those in as explicit keyword
args would mean explicitly passing them on to the base __init__,
further cluttering up the code.

-- 
Greg

From ethan at stoneleaf.us  Sat Apr 12 01:55:55 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 11 Apr 2014 16:55:55 -0700
Subject: [Python-Dev] arguments policy: **kwargs.pop()
In-Reply-To: <5348583E.1070806@stackless.com>
References: <53474FA2.3030106@stackless.com>
 <CALGmxEKza==gz=pEySZu8q33Hi0VhfXFw0zohxb9=tdyoLFzFw@mail.gmail.com>
 <5348583E.1070806@stackless.com>
Message-ID: <5348810B.3050205@stoneleaf.us>

On 04/11/2014 02:01 PM, Christian Tismer wrote:
>
> I have these style problems with several modules that I am reluctant to
> use, therefore. I know that I'm pretty alone with that.

You are not alone in that.

--
~Ethan~

From tismer at stackless.com  Sat Apr 12 04:53:53 2014
From: tismer at stackless.com (Christian Tismer)
Date: Sat, 12 Apr 2014 04:53:53 +0200
Subject: [Python-Dev] arguments policy: **kwargs.pop()
In-Reply-To: <5348810B.3050205@stoneleaf.us>
References: <53474FA2.3030106@stackless.com>
 <CALGmxEKza==gz=pEySZu8q33Hi0VhfXFw0zohxb9=tdyoLFzFw@mail.gmail.com>
 <5348583E.1070806@stackless.com> <5348810B.3050205@stoneleaf.us>
Message-ID: <5348AAC1.1060500@stackless.com>

On 12.04.14 01:55, Ethan Furman wrote:
> On 04/11/2014 02:01 PM, Christian Tismer wrote:
>>
>> I have these style problems with several modules that I am reluctant to
>> use, therefore. I know that I'm pretty alone with that.
>
> You are not alone in that.

Funny not to be alone in being alone in that :-)

hugs -- Chris

-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
Software Consulting          :     Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
14482 Potsdam                :     PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776  fax +49 (30) 700143-0023
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From arigo at tunes.org  Sat Apr 12 09:45:58 2014
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 12 Apr 2014 09:45:58 +0200
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <CAK5idxRjgY8Wg5yVZd5sDCSyrkpLjj0gs+eBuu2CaiB-QK7nfw@mail.gmail.com>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
 <CAAzeMby9RM2K_2xL2UUGQBtbUfHnBt2tMhLgEFETmP0GAHA88A@mail.gmail.com>
 <CAPdQG2pwdKkL0MfEJkMcFUoYKPcmaE8_Yzq28D1VnWgfPwm+FA@mail.gmail.com>
 <CAP7+vJLC1i=6t6_fmutYiiizX6Mm-6hY6314+Vwz62g821-6WQ@mail.gmail.com>
 <li6rh4$m7e$1@ger.gmane.org>
 <CACac1F8SEiAgFi4LT7P6FYYYKQ59vFNMp97XRSChGDxwOV9bEg@mail.gmail.com>
 <CAMSv6X3AP4jk4DxcA3DNOVLq8KTry04ZOi=3gPRKkneg-2SHNg@mail.gmail.com>
 <CACac1F_7RCbSbA2UpXmVL6-nsSxVgAFyaO-607O3uhzp7D+9pQ@mail.gmail.com>
 <CAK5idxRjgY8Wg5yVZd5sDCSyrkpLjj0gs+eBuu2CaiB-QK7nfw@mail.gmail.com>
Message-ID: <CAMSv6X1LEznL840EMYq575ns4eLMWrjBMkK1wBE=QJ9KcHow8Q@mail.gmail.com>

Hi,

On 11 April 2014 19:55, Maciej Fijalkowski <fijall at gmail.com> wrote:
>> Thanks, that clarification helps a lot. Does this mean that "API-mode"
>> CFFI is competing with things like swig (which is not used much these
>> days, as far as I know) and Cython (which is used a lot in the numeric
>> community)? ("ABI-mode" CFFI is obviously directly competing with
>> ctypes).
>
> Yes.

Funny how a high-level shift --- not really changing anything under
the hood --- changes how we look at a project :-)


A bient?t,

Armin.

From raf at durin42.com  Sat Apr 12 17:08:13 2014
From: raf at durin42.com (Augie Fackler)
Date: Sat, 12 Apr 2014 11:08:13 -0400
Subject: [Python-Dev] PEP 461: Adding % formatting to bytes and
	bytearray -- Final, Take 3
In-Reply-To: <CAGE7PNJE0imdTV9zpv89ZUbfxNbo2xc3sSSVrywdsHPau0_vQQ@mail.gmail.com>
References: <53320517.3010101@stoneleaf.us> <20140327122449.71a50ea2@fsol>
 <CADiSq7d0khUMsyTiLSWhTPmaGmxc1Cu5o+ZomAG8XFCO=Yub7A@mail.gmail.com>
 <53344AB4.50904@stoneleaf.us>
 <CAP7+vJKu6gs0SbcoN-msCnP8NRNBZxH0DOO1QUfr5yLRdRX67w@mail.gmail.com>
 <CAP7+vJKMQnGwBDT8KzcZtHsq+NQ8++GzRGUfe0z6igf60iBmQA@mail.gmail.com>
 <53346603.1000402@stoneleaf.us> <5334681C.40904@stoneleaf.us>
 <CAP7+vJ+qUfGc=9gcPMX6Cmbz6jNxwgeZUAAz01A24w6S+HpT7Q@mail.gmail.com>
 <CAP1=2W48Zw8SJa2RwLtWxJYjCjHN2WaxbU+cQF9kKA4LM_n4Xg@mail.gmail.com>
 <20140327230555.571227d2@fsol>
 <CAGE7PNJE0imdTV9zpv89ZUbfxNbo2xc3sSSVrywdsHPau0_vQQ@mail.gmail.com>
Message-ID: <75AA7BD9-21AD-45D7-9F14-716C0070890C@durin42.com>


On Mar 29, 2014, at 2:53 PM, Gregory P. Smith <greg at krypto.org> wrote:

> 
> On Thu, Mar 27, 2014 at 3:05 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Thu, 27 Mar 2014 18:47:59 +0000
> Brett Cannon <bcannon at gmail.com> wrote:
> > On Thu Mar 27 2014 at 2:42:40 PM, Guido van Rossum <guido at python.org> wrote:
> >
> > > Much better, but I'm still not happy with including %s at all. Otherwise
> > > it's accept-worthy. (How's that for pressure. :-)
> > >
> >
> > But if we only add %b and leave out %s then how is this going to lead to
> > Python 2/3 compatible code since %b is not in Python 2? Or am I
> > misunderstanding you?
> 
> I think we have reached a point where adding porting-related facilities
> in 3.5 may actually slow down the pace of porting, rather than
> accelerate it (because people will then wait for 3.5 to start porting
> stuff).
> 
> I understand that sentiment but that is an unjustified fear. It is not a good reason not to do it. Projects are already trying to port stuff today and running into roadblocks when it comes to ascii-compatible bytes formatting for real world data formats in code needing to be 2.x compatible. I'm pulling out my practicality beats purity card here.
> 
> Mercurial is one of the large Python 2.4-2.7 code bases that needs this feature in order to support Python 3 in a sane manner. (+Augie Fackler to look at the latest http://legacy.python.org/dev/peps/pep-0461/ to confirm usefulness)

That looks sufficient to me - the biggest thing is being able to do 

"abort: %s is broken" % some_filename_that_is_bytes

and have that work sanely, as well as the numerics. This looks like exactly what we need, but I'd love to test it soon (I'm happy to build a 3.5 from tip for testing) so that if it's not Right[0] changes can be made before it's permanent. Feel encouraged to CC me on patches or something for testing (or mail me directly when it lands).

Thanks!

AF

> 
> -gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140412/edf0d20f/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140412/edf0d20f/attachment.sig>

From michael at voidspace.org.uk  Sat Apr 12 17:56:46 2014
From: michael at voidspace.org.uk (Michael Foord)
Date: Sat, 12 Apr 2014 16:56:46 +0100
Subject: [Python-Dev] Fwd: Jython Report
References: <CAL9RhrVxaoe7f18F+xnsvj=9WW-gKtZsdGnnUB8OW5S6RDqpjg@mail.gmail.com>
Message-ID: <EB728A51-F961-4B58-A0DC-09A6A77F9579@voidspace.org.uk>

Below is the Jython "status update" report on Jython I received from Jim Baker and summarised in the Language Summit. It comes with one addendum from Frank:

Jim's list is fantastic - the one bit I'd like to add to the list:

Jython now supports a buffer protocol that parallels CPython's C API buffer protocol. This provided the basis for support of buffer() and memoryview(). The work was done with Jython3 in mind and will be a huge boost to that eventual effort.

Begin forwarded message:

> From: Jim Baker <jbaker at zyasoft.com>
> Subject: Re: Jython Report
> Date: 7 April 2014 06:42:51 BST
> To: Michael Foord <michael at voidspace.org.uk>
> Cc: Frank Wierzbicki <fwierzbicki at gmail.com>
> 
> Recent changes to trunk (last 6 months)
> 
> * Recently tagged a soft beta 2!
> * Java 7 JVM is now the minimum version, which gives a larger base of functionality to work with (such as using Java 7's AutoCloseable to imply corresponding context manager support in using Python code)
> * Enable mixing Python and Java types in the bases of a class when using a metaclass
> * Added support for buffer, memoryview, although not complete yet with respect to Java integration
> * Console and encoding support, such as unicodedata/idna updates
> * Many, many small fixes
> 
> About to be in trunk, to support beta 3
> 
> * socket-reboot reimplements socket/select/ssl on top of Netty 4, a popular event loop networking framework for the JVM (used by a large number of performant projects in Java space and originally part of JBoss). There was no ssl support before, but now socket and especially select semantics are much closer to CPython as well (basically close to the Windows socket model). 
> * socket-reboot in turn enables requests and thereby pip. A branch of pip currently works, actually modifying an upstream vendor lib (html5lib) so that it doesn't use isolated UTF-16 surrogates in literals, since this is not actually legal unicode, nor does it work in Jython's UTF-16 based representation. Ironically this usage is to detect such illegal use in input streams.
> * Relative star imports, which seems to impact a number of interesting projects.
> * Performance tuning of sre. Jython has a port of CPython's sre, however our use of UTF-16 requires expansion into an array of codepoints. Currently this is done on demand, which can potentially add another O(n) factor in evaluating regexes. A pull request we will apply memoizes. In the future, we will rewrite the logic in sre so that it does next/prev, much like JRuby currently does for similar encoding issues.
> 
> Related work
> 
> * Other PyPA tooling including virtualenv and wheel needs more diagnosis to see why they currently fail on Jython, but our hope is that this is minor. 
> * New project jythontools by a number of Jython developers (including Frank and Jim). This includes a number of projects that will help evolve Jython, but outside the usual release schedule and the usual problem of being in core (such as eventual deprecation):
>       - Clamp - precise integration with Java, enabling such capabilities as Java directly importing Python modules without explicitly initializing the Jython runtime or using object factories. Future work will enable Java annotation integration, as decorators. Integrates with setuptools; future integration as well with Maven via Aether.
>       - Jiffy - provide a CFFI backend for Jython. Right now it is pure vaporware, but cursory examination of cffi.backend_ctypes suggests that it should be straightforward and of modest effort to provide a similar backend by using JFFI, which Jython and JRuby both use to access native runtime services (such as Posix API) as part of the Java native runtime project.
> * The Patois project has been started to collect examples for cross-implementation support, as seen in surrogate support, but it will be a good question to get that really going, vs just talking about it.
> * JyNI - simply adding this jar to the classpath enables C extension API support. Note that this project has been licensed by its developer (not a Jython committer) under an LGPL license.
> 
> Release schedule
> 
> * Complete beta 2
> * Beta 3 is forthcoming, likely in 2 weeks
> * For beta 4, need to perform a comprehensive bug triage - what will be in, not in for 2.7.0
> * EuroPython sprint to finalize a release candidate for 2.7.0?
> 
> Future
> 
> * Mostly around performance, Java integration, and of course the usual bug fixes
> * Python bytecode compiler remains important, including for support targeting Android and removing restriction on getting too large a method for the JVM
> * More hooks for Java integration, including managing generated bytecode
> * Integrating Zippy could provide for PyPy-like performance, but requires Graal JVM
> * Supporting invokedynamic is a more realistic solution, but without the use of annotations (eg turn off Python frames) is going to be limited (maybe 2x?) based on earlier analysis
> 
> Jython 3.x?
> 
> * This comes up periodically, and it would be super nice for us to complete this support. At the very least it would make unicode strings and bytestrings correspond directly to how they are represented in Java, so that will be a nice cleanup.
> * Release schedule: we will get there at some point!
> 
> 
> On Sun, Apr 6, 2014 at 5:20 PM, Jim Baker <jbaker at zyasoft.com> wrote:
> Michael,
> 
> I was thinking about this very topic this morning! Will send you the latest status in the next 24h, specifically our work to support pypa (setuptools, pip, virtualenv, wheel) and related tooling.
> 
> - Jim
> 
> 
> On Sun, Apr 6, 2014 at 11:30 AM, Michael Foord <michael at voidspace.org.uk> wrote:
> Hey guys,
> 
> Would you be able to write up a brief report on the current and future status of Jython, for me to read out at the Python language summit on Wednesday? (Unless someone who works on Jython will be there - but as far as I know that isn't the case.)
> 
> All the best,
> 
> Michael Foord
> 
> --
> http://www.voidspace.org.uk/
> 
> 
> May you do good and not evil
> May you find forgiveness for yourself and forgive others
> May you share freely, never taking more than you give.
> -- the sqlite blessing
> http://www.sqlite.org/different.html
> 
> 
> 
> 
> 
> 
> 
> 


--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140412/1fcab5b3/attachment.html>

From stefan_ml at behnel.de  Sat Apr 12 19:11:36 2014
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 12 Apr 2014 19:11:36 +0200
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
Message-ID: <libs4b$qqg$1@ger.gmane.org>

Guido van Rossum, 10.04.2014 03:08:
> - Jukka Lehtosalo gave a talk and answered questions about mypy, his design
> and implementation of pragmatic type annotations (no new syntax required,
> uses Python 3 function annotations).

FWIW, signature type annotations aren't enough for a static compiler like
Cython, which also benefits from local and global variable declarations,
static functions, etc. However, we initially discussed this feature in the
project some five years ago or so and never actually implemented it, so I
finally decided to add support for it to Cython. There already was a way to
provide static Cython/C type declarations in pure Python code, also for
function arguments, but it's nice to have a way that is also naturally
runtime inspectable in the signature.

It essentially looks like this now:

    def func(plain_python_type: dict,
             named_python_type: 'list',
             explicit_python_type: {'type': dict},
             explicit_named_python_type: {'type': 'tuple'},
             explicit_c_type: {'ctype': 'int'}):
        ...

Pretty straight forward, I think.

Stefan



From tjreedy at udel.edu  Sat Apr 12 19:19:42 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 12 Apr 2014 13:19:42 -0400
Subject: [Python-Dev] PEP 461: Adding % formatting to bytes and
 bytearray -- Final, Take 3
In-Reply-To: <75AA7BD9-21AD-45D7-9F14-716C0070890C@durin42.com>
References: <53320517.3010101@stoneleaf.us> <20140327122449.71a50ea2@fsol>
 <CADiSq7d0khUMsyTiLSWhTPmaGmxc1Cu5o+ZomAG8XFCO=Yub7A@mail.gmail.com>
 <53344AB4.50904@stoneleaf.us>
 <CAP7+vJKu6gs0SbcoN-msCnP8NRNBZxH0DOO1QUfr5yLRdRX67w@mail.gmail.com>
 <CAP7+vJKMQnGwBDT8KzcZtHsq+NQ8++GzRGUfe0z6igf60iBmQA@mail.gmail.com>
 <53346603.1000402@stoneleaf.us> <5334681C.40904@stoneleaf.us>
 <CAP7+vJ+qUfGc=9gcPMX6Cmbz6jNxwgeZUAAz01A24w6S+HpT7Q@mail.gmail.com>
 <CAP1=2W48Zw8SJa2RwLtWxJYjCjHN2WaxbU+cQF9kKA4LM_n4Xg@mail.gmail.com>
 <20140327230555.571227d2@fsol>
 <CAGE7PNJE0imdTV9zpv89ZUbfxNbo2xc3sSSVrywdsHPau0_vQQ@mail.gmail.com>
 <75AA7BD9-21AD-45D7-9F14-716C0070890C@durin42.com>
Message-ID: <libsjv$1i2$1@ger.gmane.org>

On 4/12/2014 11:08 AM, Augie Fackler wrote:
>
> On Mar 29, 2014, at 2:53 PM, Gregory P. Smith <greg at krypto.org
> <mailto:greg at krypto.org>> wrote:
>
>>
>> On Thu, Mar 27, 2014 at 3:05 PM, Antoine Pitrou <solipsis at pitrou.net
>> <mailto:solipsis at pitrou.net>> wrote:
>>
>>     On Thu, 27 Mar 2014 18:47:59 +0000
>>     Brett Cannon <bcannon at gmail.com <mailto:bcannon at gmail.com>> wrote:
>>     > On Thu Mar 27 2014 at 2:42:40 PM, Guido van Rossum
>>     <guido at python.org <mailto:guido at python.org>> wrote:
>>     >
>>     > > Much better, but I'm still not happy with including %s at all.
>>     Otherwise
>>     > > it's accept-worthy. (How's that for pressure. :-)
>>     > >
>>     >
>>     > But if we only add %b and leave out %s then how is this going to
>>     lead to
>>     > Python 2/3 compatible code since %b is not in Python 2? Or am I
>>     > misunderstanding you?
>>
>>     I think we have reached a point where adding porting-related
>>     facilities
>>     in 3.5 may actually slow down the pace of porting, rather than
>>     accelerate it (because people will then wait for 3.5 to start porting
>>     stuff).
>>
>>
>> I understand that sentiment but that is an unjustified fear. It is not
>> a good reason not to do it. Projects are already trying to port stuff
>> today and running into roadblocks when it comes to ascii-compatible
>> bytes formatting for real world data formats in code needing to be 2.x
>> compatible. I'm pulling out my practicality beats purity card here.
>>
>> Mercurial is one of the large Python 2.4-2.7 code bases that needs
>> this feature in order to support Python 3 in a sane manner. (+Augie
>> Fackler to look at the latest
>> http://legacy.python.org/dev/peps/pep-0461/ to confirm usefulness)
>
> That looks sufficient to me - the biggest thing is being able to do
>
> "abort: %s is broken" % some_filename_that_is_bytes
>
> and have that work sanely, as well as the numerics. This looks like
> exactly what we need, but I'd love to test it soon (I'm happy to build a
> 3.5 from tip for testing) so that if it's not Right[0] changes can be
> made before it's permanent. Feel encouraged to CC me on patches or
> something for testing (or mail me directly when it lands).

Add yourself as nosy to http://bugs.python.org/issue20284
"patch to implement PEP 461 (%-interpolation for bytes)"

Indeed, you could help test it the latest version, and others as posted.

-- 
Terry Jan Reedy


From Nikolaus at rath.org  Sat Apr 12 20:58:19 2014
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Sat, 12 Apr 2014 11:58:19 -0700
Subject: [Python-Dev] Appeal for reviews
Message-ID: <87sipiwf1w.fsf@vostro.rath.org>

Hello,

I've accumulated a number of patches in the issue tracker that are
waiting for someone to review/commit/reject them. I'm eager to make
corrections as necessary, I just need someone to look the work that I've
done so far:

* http://bugs.python.org/issue20951 (SSLSocket.send() returns 0 for
  non-blocking socket)

* http://bugs.python.org/issue1738 (filecmp.dircmp does exact match
  only)

* http://bugs.python.org/issue7776 (http.client.HTTPConnection
  tunneling is broken)

* http://bugs.python.org/issue20177 (Derby #8: Convert 28 sites to
  Argument Clinic across 2 files)

* http://bugs.python.org/issue19414 (iter(ordered_dict) yields keys
  not in dict in some circumstances)

* http://bugs.python.org/issue20578 (BufferedIOBase.readinto1 is
  missing)

* http://bugs.python.org/issue21057 (TextIOWrapper does not support
  reading bytearrays or memoryviews)

  
I realize that core developer time is scarce, so I have started to only
work on patches after I've confirmed that someone is available and
interested to review them. However, it would be great if some people
could find time to look at the issues above as well. Having your
contributions just languish in the bugtracker is really dispiriting... I
*want* to contribute, and I can't believe that Python is the one
open-source project that is not in need of more manpower. But for some
reason it seems really hard to actually find something to do..


Best,
Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             ?Time flies like an arrow, fruit flies like a Banana.?

From tjreedy at udel.edu  Sun Apr 13 00:06:52 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 12 Apr 2014 18:06:52 -0400
Subject: [Python-Dev] Appeal for reviews
In-Reply-To: <87sipiwf1w.fsf@vostro.rath.org>
References: <87sipiwf1w.fsf@vostro.rath.org>
Message-ID: <licded$juc$1@ger.gmane.org>

On 4/12/2014 2:58 PM, Nikolaus Rath wrote:

> I've accumulated a number of patches in the issue tracker that are
> waiting for someone to review/commit/reject them. I'm eager to make
> corrections as necessary, I just need someone to look the work that I've
> done so far:

If I did not have several Idle patches to review from other new people 
like you, I would. Many core developers are at the PyCon conference, so 
it may be a few days before they can respond.

> * http://bugs.python.org/issue20951 (SSLSocket.send() returns 0 for
>    non-blocking socket)
>
> * http://bugs.python.org/issue1738 (filecmp.dircmp does exact match
>    only)
>
> * http://bugs.python.org/issue7776 (http.client.HTTPConnection
>    tunneling is broken)
>
> * http://bugs.python.org/issue20177 (Derby #8: Convert 28 sites to
>    Argument Clinic across 2 files)
>
> * http://bugs.python.org/issue19414 (iter(ordered_dict) yields keys
>    not in dict in some circumstances)
>
> * http://bugs.python.org/issue20578 (BufferedIOBase.readinto1 is
>    missing)
>
> * http://bugs.python.org/issue21057 (TextIOWrapper does not support
>    reading bytearrays or memoryviews)

Do you consider any of these 'ready-to-commit'?
* before-fail, after-pass test?
* required doc changes?
* tested patch for all versions that should be patched?
* any new Misc/ACKS entry needed (for new person other than you)?
* checked for stray end-of-line whitespace?

I intentionally omitted news entry. There is a list of commit checks in 
in devguide, and a script to do some.

> I realize that core developer time is scarce, so I have started to only
> work on patches after I've confirmed that someone is available and
> interested to review them. However, it would be great if some people
> could find time to look at the issues above as well. Having your
> contributions just languish in the bugtracker is really dispiriting... I
> *want* to contribute, and I can't believe that Python is the one
> open-source project that is not in need of more manpower.

We need either more core developer personpower or more efficient use of 
the effort we do have -- or both. We seem to lose people as fast as we 
promote them. More efficient use of time might also reduce attrition.

Right now we seem to be in an awkward phase of the core mentorship 
program. We have gotten some new people, like you, submitting multiple 
patches, but have not yet gotten enough new people to review and commit.

 > But for some
> reason it seems really hard to actually find something to do.

Review and test other people's patches, if you are not already.  If a 
3.4 bugfix patch works and is ready to commit, and plausibly should be 
applied to 2.7 (maybe ask), but does not apply cleanly (common), see if 
the core developer on the issue would like you to tweak, test, and 
upload a 2.7 version.

-- 
Terry Jan Reedy


From ncoghlan at gmail.com  Sun Apr 13 02:05:55 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 12 Apr 2014 20:05:55 -0400
Subject: [Python-Dev] Appeal for reviews
In-Reply-To: <licded$juc$1@ger.gmane.org>
References: <87sipiwf1w.fsf@vostro.rath.org>
	<licded$juc$1@ger.gmane.org>
Message-ID: <CADiSq7eK9u_TB0eQvddUMqe5Vra0Dx0v2AESsJHCgmyE0f73eA@mail.gmail.com>

On 12 Apr 2014 18:08, "Terry Reedy" <tjreedy at udel.edu> wrote:
> On 4/12/2014 2:58 PM, Nikolaus Rath wrote:
>
>> I realize that core developer time is scarce, so I have started to only
>> work on patches after I've confirmed that someone is available and
>> interested to review them. However, it would be great if some people
>> could find time to look at the issues above as well. Having your
>> contributions just languish in the bugtracker is really dispiriting... I
>> *want* to contribute, and I can't believe that Python is the one
>> open-source project that is not in need of more manpower.
>
>
> We need either more core developer personpower or more efficient use of
the effort we do have -- or both. We seem to lose people as fast as we
promote them. More efficient use of time might also reduce attrition.
>
> Right now we seem to be in an awkward phase of the core mentorship
program. We have gotten some new people, like you, submitting multiple
patches, but have not yet gotten enough new people to review and commit.

PEP 462 covers the workflow problems that currently make it difficult not
only to get patches merged, but also makes the existing core developers
reluctant to add new core developers too hastily. We'll eventually get
those resolved, but it's going to take time, since infrastructure support
is incredibly short of contributors (even more so than core development in
general), and there's already a non-trivial backlog of unresolved issues.

Regards,
Nick.

>
>
> > But for some
>>
>> reason it seems really hard to actually find something to do.
>
>
> Review and test other people's patches, if you are not already.  If a 3.4
bugfix patch works and is ready to commit, and plausibly should be applied
to 2.7 (maybe ask), but does not apply cleanly (common), see if the core
developer on the issue would like you to tweak, test, and upload a 2.7
version.
>
> --
> Terry Jan Reedy
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140412/b38a1460/attachment-0001.html>

From stephen at xemacs.org  Sun Apr 13 02:30:02 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 13 Apr 2014 09:30:02 +0900
Subject: [Python-Dev]  Appeal for reviews
In-Reply-To: <87sipiwf1w.fsf@vostro.rath.org>
References: <87sipiwf1w.fsf@vostro.rath.org>
Message-ID: <87ppkmgjg5.fsf@uwakimon.sk.tsukuba.ac.jp>

I apologize for the tone.  I need to go *right* now, and can't fix
that.  Really, I'm sympathetic and my goal is not just to defend
python-dev, but to help you get the reviews your work deserves.
Please read with that in mind.

Steve

Nikolaus Rath writes:

 > I've accumulated a number of patches in the issue tracker that are
 > waiting for someone to review/commit/reject them.

This is the wrong place to post a personal request like this, now that
the core-mentorship channel is available.  The problem itself is well-
known, but you don't offer a solution that makes it easier to review,
so it's not really appropriate for Python-Dev.

 > I'm eager to make corrections as necessary, I just need someone to
 > look the work that I've done so far:
 > 
 > * http://bugs.python.org/issue20951 (SSLSocket.send() returns 0 for
 >   non-blocking socket)
 > 
 > * http://bugs.python.org/issue1738 (filecmp.dircmp does exact match
 >   only)
 > 
 > * http://bugs.python.org/issue7776 (http.client.HTTPConnection
 >   tunneling is broken)
 > 
 > * http://bugs.python.org/issue20177 (Derby #8: Convert 28 sites to
 >   Argument Clinic across 2 files)
 > 
 > * http://bugs.python.org/issue19414 (iter(ordered_dict) yields keys
 >   not in dict in some circumstances)
 > 
 > * http://bugs.python.org/issue20578 (BufferedIOBase.readinto1 is
 >   missing)
 > 
 > * http://bugs.python.org/issue21057 (TextIOWrapper does not support
 >   reading bytearrays or memoryviews)

That's an impressive list, but it doesn't tell us what work you've
done.  There's a checklist in the devguide which should be more
accurate (sorry about lack of URL), but I imagine it includes the
following:

 * Has the issue been classified as bug or feature?
 * If bug, is it confirmed?
 * If complex, does it need a PEP?  (probably irrelevant to your patches)
 * Is there agreement about requirements in the issue comments?
 * Is there a patch?
 * Does the design need review?
 * Has relevant documentation been added/updated (including
   docstrings, manuals, Whats New at least)?
 * Are there tests that the requirements are satisfied?
 * Are there tests for any regressions that arose in the process of
   developing the patch?
 * Have you signed a CA?  (irrelevant to you IIRC)
 * Is the issue status updated to reflect the work done so far?

It would help in getting reviewer attention to your work if in
addition to a list of patches you provided an indication of (1) how
complete the patch is and (2) what review is requested.  That in
itself is a strong indicator of quality.

 > I realize that core developer time is scarce, so I have started to only
 > work on patches after I've confirmed that someone is available and
 > interested to review them. However, it would be great if some people
 > could find time to look at the issues above as well. Having your
 > contributions just languish in the bugtracker is really
 > dispiriting...

 > I *want* to contribute,

Not really.  You *want* attention.  You have *already* contributed,
it's a matter of time before the contribution is integrated.  There's
nothing wrong with wanting to see your contribution integrated, but as
you write yourself, core developer time is very scarce.  Please be
patient and keep pinging core-mentorship at intervals.

 > and I can't believe that Python is the one open-source project that
 > is not in need of more manpower.

It's not.  What is unusual (though hardly unique) about Python is the
high quality demanded of both contributions and reviews.  It is well-
known that demand for quality screens out contributors who can't
provide the quality, and it is equally well-known that it frustrates
high-quality contributors who have not achieved "trust" yet.
Motivating reviewers is not an easy problem to solve.

 > But for some reason it seems really hard to actually find something
 > to do..

Your list above makes it clear that you had little trouble finding
things to do.  I'm sure you can find more.  Eventually people will
start getting embarrassed by the length of your list of patches
awaiting review.<wink/>

If you want to *do* something about this, maybe you could add a
feature to the tracker to note that patches are awaiting review (nb --
it needs to identify who requested review or the feature will
undoubtedly be abused).  Ping the tracker-discuss list.

You also might want to pay attention what's happening in the Python
world in various seasons.  Right now it's PyCon US (actually NA,
emphasized by its venue in Montreal).  Your post was poorly timed in
that sense.  Several of the obvious candidates for reviewer for your
patches have been busy preparing for the language summit, and many
more preparing talks or tutorials, as well as arranging for travel.

There must be stuff of interest to you happening on twitter, etc.,
during PyCon.


From tseaver at palladion.com  Sun Apr 13 03:04:26 2014
From: tseaver at palladion.com (Tres Seaver)
Date: Sat, 12 Apr 2014 21:04:26 -0400
Subject: [Python-Dev] Appeal for reviews
In-Reply-To: <87ppkmgjg5.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <87sipiwf1w.fsf@vostro.rath.org>
 <87ppkmgjg5.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <licnq1$3ba$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 04/12/2014 08:30 PM, Stephen J. Turnbull wrote:
> it's a matter of time before the contribution is integrated.

Our current backlog is bad enough that many contributions are effectively
wasted:  they rot on the vine before they can be merged.


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlNJ4pUACgkQ+gerLs4ltQ4WIACfeoWID19lDf1wpFF2vtl1ZHRk
q6gAnRrwLETLirZ6ulS0NivLmYcOOkzF
=VHtW
-----END PGP SIGNATURE-----


From benjamin at python.org  Sun Apr 13 03:07:34 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 12 Apr 2014 18:07:34 -0700
Subject: [Python-Dev] Appeal for reviews
In-Reply-To: <87ppkmgjg5.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <87sipiwf1w.fsf@vostro.rath.org>
 <87ppkmgjg5.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <1397351254.611.105839425.39EA328C@webmail.messagingengine.com>

On Sat, Apr 12, 2014, at 17:30, Stephen J. Turnbull wrote:
> I apologize for the tone.  I need to go *right* now, and can't fix
> that.  Really, I'm sympathetic and my goal is not just to defend
> python-dev, but to help you get the reviews your work deserves.
> Please read with that in mind.

I don't think Nikolaus is wrong to post here. I often tell people that
sometimes the only way to get your patches in is to constantly poke us
about it.

From stefan_ml at behnel.de  Sun Apr 13 06:48:16 2014
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 13 Apr 2014 06:48:16 +0200
Subject: [Python-Dev] static typing of input arguments in signatures (was:
 Language Summit notes)
In-Reply-To: <libs4b$qqg$1@ger.gmane.org>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
 <libs4b$qqg$1@ger.gmane.org>
Message-ID: <lid4uj$qj1$1@ger.gmane.org>

Stefan Behnel, 12.04.2014 19:11:
> Guido van Rossum, 10.04.2014 03:08:
>> - Jukka Lehtosalo gave a talk and answered questions about mypy, his design
>> and implementation of pragmatic type annotations (no new syntax required,
>> uses Python 3 function annotations).
> 
> FWIW, signature type annotations aren't enough for a static compiler like
> Cython, which also benefits from local and global variable declarations,
> static functions, etc. However, we initially discussed this feature in the
> project some five years ago or so and never actually implemented it, so I
> finally decided to add support for it to Cython. There already was a way to
> provide static Cython/C type declarations in pure Python code, also for
> function arguments, but it's nice to have a way that is also naturally
> runtime inspectable in the signature.
> 
> It essentially looks like this now:
> 
>     def func(plain_python_type: dict,
>              named_python_type: 'list',
>              explicit_python_type: {'type': dict},
>              explicit_named_python_type: {'type': 'tuple'},
>              explicit_c_type: {'ctype': 'int'}):
>         ...

It may not be obvious to everyone, so I guess I should add a comment on why
this wasn't considered important enough during the last five years to
implement it before (and that didn't change much now). One thing I learned
in the Cython project is that it's often a bad idea to statically type
input arguments at all. The reason is that it violates the principle of
being liberal with input (but strict with output).

There are cases where this is ok, e.g. using the C type "double" is
perfectly ok most of the time, because it's not (really) range restricting
and all numeric-ish Python types can happily coerce to it. Using integer
types is ok only if their restricted range really is enough, but can lead
to the Py2.5 problem of having too small integer types if things grow
larger. Similarly, using self-defined (extension) types to type input
arguments is ok because at the native level, the code will almost certainly
depend on their internal implementation details, so accepting anything else
here would be useless and wrong (CPython does the same for "self" type
checking, for example).

However, what I've often seen people do is to write something like the
above example, i.e. they use explicit Python types for input arguments.
This leads to an overly narrow API that rejects lots of otherwise
acceptable input types. What people usually want as input type is something
like "iterable", or "mapping". What many users end up writing instead is
"list" or "dict". The reason is that in Python code, input really *is* a
list or a dict most of the time, and at the C level (to which Cython
translates), list and dict have (limited) performance advantages, whereas
"iterable" and "mapping" do not. So using something like the ABC types
Iterable or Mapping for static typing is almost completely useless, except
for documentation purposes (and maybe a bit earlier type checking, in some
cases).

The point where this is extremely visible is string/bytes input. I've seen
many people actually use "str" or "bytes" to type their input. In Python 2,
"str" is a very bad idea, because it excludes "unicode". In Py3, that's
less of a problem, because all reasonable string input really is "str" (or,
more rarely, a subtype). In both Py2 and Py3, however, statically typing
input as "bytes" is a *very* bad idea, because it excludes everything
bytes-ish: bytearray, memoryview, buffers. Many users don't see this need
at first. While using buffers instead is easy enough in Cython, getting it
right for a specific use case and properly using it to interface with
external native libraries is less straight forward and requires more
thought than just writing "I want bytes".

So, what I've learned from seven years of Cython is that static typing in
signatures is actually less interesting than you might think at first
sight. It might be ok for documentation purposes, although its verboseness
makes that also a bit questionable. But for actual input checking, there's
substantially more than just the (generic) type, even at the C level, so
users who properly understand the problem don't use static argument typing
in many cases and instead write their own input validation, conversion,
normalisation code. Or just let the code raise an exception if the input
doesn't work the way it's being used. Errors don't always have to be raised
at signature evaluation time.

Stefan



From stephen at xemacs.org  Sun Apr 13 08:36:19 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 13 Apr 2014 15:36:19 +0900
Subject: [Python-Dev] Appeal for reviews
In-Reply-To: <1397351254.611.105839425.39EA328C@webmail.messagingengine.com>
References: <87sipiwf1w.fsf@vostro.rath.org>
 <87ppkmgjg5.fsf@uwakimon.sk.tsukuba.ac.jp>
 <1397351254.611.105839425.39EA328C@webmail.messagingengine.com>
Message-ID: <87k3athh24.fsf@uwakimon.sk.tsukuba.ac.jp>

Benjamin Peterson writes:

 > I don't think Nikolaus is wrong to post here. I often tell people that
 > sometimes the only way to get your patches in is to constantly poke us
 > about it.

I admit the tone was biased toward nagging or "blaming the victim",
and again I apologize for causing misunderstanding.  Nikolaus isn't
"wrong" for posting here.  My claim is that in current circumstances,
core-mentorship would be a more *effective* channel because

 - core-mentorship is *explicitly* for poking Those Who Can Help
   (among other requests for help);

 - a surprisingly large (to me, anyway :-) fraction of core committers
   and people who may not be "core" but do a lot of mentoring for
   central projects like IDLE do hang out there; and

 - when reading core-mentorship their "mentor modes" will be engaged,
   whereas on python-dev they will often be mostly interested in a
   particular thread.

I also suggested that some tweaks to the weekly issue report might
help to catch the attention of those who can commit patches, but my
ideas about that are still basically vaporware.

From lukasz at langa.pl  Sun Apr 13 10:11:50 2014
From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=)
Date: Sun, 13 Apr 2014 04:11:50 -0400
Subject: [Python-Dev] static typing of input arguments in signatures
	(was: Language Summit notes)
In-Reply-To: <lid4uj$qj1$1@ger.gmane.org>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
 <libs4b$qqg$1@ger.gmane.org> <lid4uj$qj1$1@ger.gmane.org>
Message-ID: <21C31DB4-6F80-43CE-B811-FB61938E9423@langa.pl>

On Apr 13, 2014, at 12:48 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:

> Stefan Behnel, 12.04.2014 19:11:
> 
> So, what I've learned from seven years of Cython is that static typing in
> signatures is actually less interesting than you might think at first
> sight. It might be ok for documentation purposes, although its verboseness
> makes that also a bit questionable.

You raise a valid point that type hinting a dict instead of a Mapping is
likely to create an overly limiting API. This sort of error is however
quite easy to fix forward.

To counter, we?ve had multiple data points during the summit suggesting
that duck typing is not really that often used in production code. In
other words, after initial prototyping and testing, an API of a callable
gets settled and is later used with a very limited number of types.

Of course, fuzzy terms like ?often?, ?seldom?, and ?limited? suggest
that we don?t have hard data here. That being said, there is a clear
need to improve our ability to lint code, refactor code easier, and
document the contracts better. This has been voiced by various
conference attendees and the consensus is tremendous. Moreover, this
need has been proven by owners of large codebases like Google, Facebook,
and Microsoft by creating optionally typed variants of known dynamic
languages (namely Dart, Hack and TypeScript).

For Python specifically the existence of prototypes like Mypy, obiwan,
or pytypedecl shows that there is growing interest in standardizing this
functionality. Therefore, I'm drafting a PEP for opt-in type hints with
optional checks at runtime and by static analysis. Will post the initial
form in the upcoming days, and then we can go paint that bike shed.

I'm aware that great care here is needed for the outcome to feel
Pythonic and not overwhelming. Don't worry before you get your hands on
the draft, though.

-- 
Best regards,
?ukasz Langa

WWW: http://lukasz.langa.pl/
Twitter: @llanga
IRC: ambv on #python-dev

From janzert at janzert.com  Sun Apr 13 14:12:50 2014
From: janzert at janzert.com (Janzert)
Date: Sun, 13 Apr 2014 08:12:50 -0400
Subject: [Python-Dev] Appeal for reviews
In-Reply-To: <87k3athh24.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <87sipiwf1w.fsf@vostro.rath.org>
 <87ppkmgjg5.fsf@uwakimon.sk.tsukuba.ac.jp>
 <1397351254.611.105839425.39EA328C@webmail.messagingengine.com>
 <87k3athh24.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <lidv02$3c2$1@ger.gmane.org>

On 4/13/2014 2:36 AM, Stephen J. Turnbull wrote:
[snip]
> My claim is that in current circumstances,
> core-mentorship would be a more *effective* channel because
>
>   - core-mentorship is *explicitly* for poking Those Who Can Help
>     (among other requests for help);
>
>   - a surprisingly large (to me, anyway :-) fraction of core committers
>     and people who may not be "core" but do a lot of mentoring for
>     central projects like IDLE do hang out there; and
>
>   - when reading core-mentorship their "mentor modes" will be engaged,
>     whereas on python-dev they will often be mostly interested in a
>     particular thread.
>

It saddens me that the core-mentorship list has grown into the primary 
acceptable place to ask these getting into core development questions. 
Any question answered there can never benefit the wider community since 
it is kept a closed list for the specific purpose of keeping its 
archives out of public view.

Janzert


From jdornak at redhat.com  Sun Apr 13 10:38:57 2014
From: jdornak at redhat.com (Jakub QB =?utf-8?B?RG9yxYjDoWs=?=)
Date: Sun, 13 Apr 2014 04:38:57 -0400 (EDT)
Subject: [Python-Dev] flock in Python 3
In-Reply-To: <1397234323.20322.105474653.7638E691@webmail.messagingengine.com>
References: <1867801184.3782831.1397221096189.JavaMail.zimbra@redhat.com>
 <1397234323.20322.105474653.7638E691@webmail.messagingengine.com>
Message-ID: <2114138273.4233234.1397378337094.JavaMail.zimbra@redhat.com>

You are right, it should be https://pypi.python.org/pypi/flock

----- Original Message -----
From: "Benjamin Peterson" <benjamin at python.org>
To: "Jakub QB Dor??k" <jdornak at redhat.com>, "Python Dev" <python-dev at python.org>
Sent: Friday, April 11, 2014 6:38:43 PM
Subject: Re: [Python-Dev] flock in Python 3

On Fri, Apr 11, 2014, at 5:58, Jakub QB Dor??k wrote:
> Hi all,
> 
> writing a threaded application I've been surprised that there is no
> object api for fcntl.flock implementing __enter__() and __exit__()
> methods to be used with 'with' statement.
> I have written one (https://pypi.python.org/pypi), but I wonder whether
> this could get into the Python Standard Library.

That link PyPi link is broken.

I think it would be nicer to have a high-level API for cross-platform
file locking. IIRC, flock can be rather non-portable.

From stephen at xemacs.org  Sun Apr 13 17:39:58 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 14 Apr 2014 00:39:58 +0900
Subject: [Python-Dev] Appeal for reviews
In-Reply-To: <lidv02$3c2$1@ger.gmane.org>
References: <87sipiwf1w.fsf@vostro.rath.org>
 <87ppkmgjg5.fsf@uwakimon.sk.tsukuba.ac.jp>
 <1397351254.611.105839425.39EA328C@webmail.messagingengine.com>
 <87k3athh24.fsf@uwakimon.sk.tsukuba.ac.jp>
 <lidv02$3c2$1@ger.gmane.org>
Message-ID: <87ha5xgrw1.fsf@uwakimon.sk.tsukuba.ac.jp>

Janzert writes:
 > On 4/13/2014 2:36 AM, Stephen J. Turnbull wrote:
 > [snip]
 > > My claim is that in current circumstances,
 > > core-mentorship would be a more *effective* channel because
 > >
 > >   - core-mentorship is *explicitly* for poking Those Who Can Help
 > >     (among other requests for help);
 > >
 > >   - a surprisingly large (to me, anyway :-) fraction of core committers
 > >     and people who may not be "core" but do a lot of mentoring for
 > >     central projects like IDLE do hang out there; and
 > >
 > >   - when reading core-mentorship their "mentor modes" will be engaged,
 > >     whereas on python-dev they will often be mostly interested in a
 > >     particular thread.
 > >
 > 
 > It saddens me that the core-mentorship list has grown into the primary 
 > acceptable place to ask these getting into core development questions. 
 > Any question answered there can never benefit the wider community since 
 > it is kept a closed list

That is false.  The list is open, anybody interested can join the list
and read the archives.

 > for the specific purpose of keeping its archives out of public
 > view.

Nobody said core-mentorship is the primary place for getting into core
development questions.  I said that for requesting review of your
patches, it's probably a better venue than here.

However, I take issue with your implied opinion that a lot of the
discussion on core-mentorship is of general interest.  AFAICS most
questions that are being asked are FAQs and requests for review.  Most
are handled well by the dev guide, and the answers don't belong on
this list because they contain little new content *of general
interest*.

Of course there's some loss of information to the "wider community"
due to the (tiny) cost of subscribing and lack of indexing by search
engines, but the benefit of directing new developers to
core-mentorship is that S/N increases for *both* sets of users (not
just the subscribers to -dev, but the new developers, too).

What bothers me is a *much* larger leak: the issue tracker.  Although
I suppose that technically speaking it's a little easier to access and
indexed by Google, there is surely a lot of implementation and style
advice that just isn't going to be accessible to anybody who doesn't
already know it -- because without that knowledge they won't have the
search terms needed to find it in Google.  I don't, and I imagine you
don't, think it's a good idea to abandon the tracker and discuss those
issues here instead.  Eventually we'll have technology to aggregate
that, I hope.


From greg.mildenstein at hotmail.co.uk  Sun Apr 13 17:31:24 2014
From: greg.mildenstein at hotmail.co.uk (Greg Mildenstein)
Date: Sun, 13 Apr 2014 16:31:24 +0100
Subject: [Python-Dev] python-3.4.0
Message-ID: <DUB126-W82DFB1A64EAEC6E4C6E36DC9560@phx.gbl>

Hi,
 
I'm running windows 8.1 64 bit. I was using 'python-3.3.1.amd64' but have uninstalled it and installed the above version. However, when I try running 'PyScripter', I get an error stating, 'ERROR - unable to open python, it will now exit'. Is there something I'm doing wrong?
 
Thanks
 
Greg Mildenstein
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140413/a551d884/attachment.html>

From zachary.ware+pydev at gmail.com  Sun Apr 13 18:06:18 2014
From: zachary.ware+pydev at gmail.com (Zachary Ware)
Date: Sun, 13 Apr 2014 11:06:18 -0500
Subject: [Python-Dev] python-3.4.0
In-Reply-To: <DUB126-W82DFB1A64EAEC6E4C6E36DC9560@phx.gbl>
References: <DUB126-W82DFB1A64EAEC6E4C6E36DC9560@phx.gbl>
Message-ID: <2a6bcd43-396b-4d50-902e-fa4055f42ecc@email.android.com>



On April 13, 2014 10:31:24 AM CDT, Greg Mildenstein <greg.mildenstein at hotmail.co.uk> wrote:
>Hi,
> 
>I'm running windows 8.1 64 bit. I was using 'python-3.3.1.amd64' but
>have uninstalled it and installed the above version. However, when I
>try running 'PyScripter', I get an error stating, 'ERROR - unable to
>open python, it will now exit'. Is there something I'm doing wrong?

Hi Greg,

All you've done wrong is use the wrong list for your question ;-).  Python-dev is intended for discussing the development *of* Python, not development *with* Python. Python-list would have been a better choice of venue, or a PyScripter-specific discussion forum.

That said, the problem you're having is because PyScripter has not yet been updated to run with Python 3.4. If you want to stick with PyScripter, you'll have to reinstall Python 3.3 (though I would suggest updating to 3.3.5 instead of 3.3.1). You can install both versions side-by-side without any problems. If you want an IDE that can run Python 3.4, you'll need to use something else until PyScripter updates.

Hope this helps,
--
Zach (on a phone)

From solipsis at pitrou.net  Sun Apr 13 18:47:33 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 13 Apr 2014 18:47:33 +0200
Subject: [Python-Dev] Appeal for reviews
In-Reply-To: <1397351254.611.105839425.39EA328C@webmail.messagingengine.com>
References: <87sipiwf1w.fsf@vostro.rath.org>
 <87ppkmgjg5.fsf@uwakimon.sk.tsukuba.ac.jp>
 <1397351254.611.105839425.39EA328C@webmail.messagingengine.com>
Message-ID: <lief35$vl5$1@ger.gmane.org>

Le 13/04/2014 03:07, Benjamin Peterson a ?crit :
> On Sat, Apr 12, 2014, at 17:30, Stephen J. Turnbull wrote:
>> I apologize for the tone.  I need to go *right* now, and can't fix
>> that.  Really, I'm sympathetic and my goal is not just to defend
>> python-dev, but to help you get the reviews your work deserves.
>> Please read with that in mind.
>
> I don't think Nikolaus is wrong to post here. I often tell people that
> sometimes the only way to get your patches in is to constantly poke us
> about it.
>

FWIW - I agree with Benjamin, there's nothing wrong in posting here.

Regards

Antoine.


From martin at v.loewis.de  Sun Apr 13 20:28:21 2014
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 13 Apr 2014 20:28:21 +0200
Subject: [Python-Dev] 2.7 Windows installers (Was: death to 2.7;
 long live 2.7)
In-Reply-To: <1397092930.29268.104798077.13871388@webmail.messagingengine.com>
References: <1397092930.29268.104798077.13871388@webmail.messagingengine.com>
Message-ID: <534AD745.9000908@v.loewis.de>

Am 10.04.14 03:22, schrieb Benjamin Peterson:
> We'll keep doing what we're currently doing for another year, making
> normal bug fix releases with installers. After that, we _won't_ close
> 2.7 to normal bug fixes as is currently implied by the release schedule.

After thinking about this plan, I believe it is best if I step down as
the Windows installers engineer right away. To my knowledge, no specific
date for the next 2.7 bugfix release has been set, so this should allow
to find a replacement before the next release.

I would have volunteered to do two more releases if these had been the
definite last ones, as it would have been impractical to replace me for
just two releases. If this is going until 2020, anybody up to the task
should start working on it now.

I'll be willing to help anybody taking over with advise, but I'll
consider the 2.7 branch otherwise dead (i.e. not worthwhile
contributing to).

I'll be happy to continue making installers for the 3.x branches,
but if anybody considering to volunteer would do so on the condition
that he/she gets it "all", I'd be happy to hand these other branches
over as well.

Of course, anybody taking over that role would be free to either keep
or replace the current technology (e.g. switching to InstallShield
or WiX or whatever).

The one option I'd rather not support is an "open" committee of
Windows installer engineers. Anybody taking this role will have a
copy of the PSF code signing key, so I'd like to keep this list
rather small, and the people attached to the task.

Regards,
Martin


From martin at v.loewis.de  Sun Apr 13 20:46:48 2014
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 13 Apr 2014 20:46:48 +0200
Subject: [Python-Dev] Appeal for reviews
In-Reply-To: <1397351254.611.105839425.39EA328C@webmail.messagingengine.com>
References: <87sipiwf1w.fsf@vostro.rath.org>
 <87ppkmgjg5.fsf@uwakimon.sk.tsukuba.ac.jp>
 <1397351254.611.105839425.39EA328C@webmail.messagingengine.com>
Message-ID: <534ADB98.2070600@v.loewis.de>

Am 13.04.14 03:07, schrieb Benjamin Peterson:
> On Sat, Apr 12, 2014, at 17:30, Stephen J. Turnbull wrote:
>> I apologize for the tone.  I need to go *right* now, and can't fix
>> that.  Really, I'm sympathetic and my goal is not just to defend
>> python-dev, but to help you get the reviews your work deserves.
>> Please read with that in mind.
> 
> I don't think Nikolaus is wrong to post here. I often tell people that
> sometimes the only way to get your patches in is to constantly poke us
> about it.

I agree. If I had more time than I actually do, I would have responded
with my 5-for-1 deal. Instead, I just looked at some of the patches,
and commented on further improvements.

As for the request "Are you sure that the patch is ready": this is
*very* difficult to answer for the author. We all have experienced
that patches that we considered good were critized out of nowhere,
and I just did the same to Nikolaus. There is just no way to tell
in advance whether people will approve a certain change or not, because
people will apply other criteria than oneself.

Regards,
Martin



From martin at v.loewis.de  Sun Apr 13 20:38:47 2014
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 13 Apr 2014 20:38:47 +0200
Subject: [Python-Dev] Windows installers and OpenSSL
In-Reply-To: <CACac1F-gGuryDiU9JL4zh0XQ0bpE=ybZ-iBFZ0Q=uvrO8zeGJg@mail.gmail.com>
References: <CACac1F-gGuryDiU9JL4zh0XQ0bpE=ybZ-iBFZ0Q=uvrO8zeGJg@mail.gmail.com>
Message-ID: <534AD9B7.4020209@v.loewis.de>

Am 10.04.14 15:41, schrieb Paul Moore:
> Given the OpenSSL vulnerability and the fact that we bundle OpenSSL
> with the Windows installers (1.0.1e in Python 3.4.0) should we be
> releasing updated installers?

As others have said: certainly, and only for 3.4.0 (i.e. 2.7 in
particular is not affected - I'm glad I didn't update OpenSSL
there past 0.9.8).

My feeling with these things is that it is often better to wait
until the dust settles - people in a hurry of fixing security
bugs tend to introduce new ones in the process.

I'm tempted to experiment with installer patch files for this
(.msp); it's technically just a single DLL that would need to be
replaced. Contributions are welcome.

Regards,
Martin



From martin at v.loewis.de  Sun Apr 13 20:49:13 2014
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 13 Apr 2014 20:49:13 +0200
Subject: [Python-Dev] Appeal for reviews
In-Reply-To: <87k3athh24.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <87sipiwf1w.fsf@vostro.rath.org>
 <87ppkmgjg5.fsf@uwakimon.sk.tsukuba.ac.jp>
 <1397351254.611.105839425.39EA328C@webmail.messagingengine.com>
 <87k3athh24.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <534ADC29.3070401@v.loewis.de>

Am 13.04.14 08:36, schrieb Stephen J. Turnbull:

> I admit the tone was biased toward nagging or "blaming the victim",
> and again I apologize for causing misunderstanding.  Nikolaus isn't
> "wrong" for posting here.  My claim is that in current circumstances,
> core-mentorship would be a more *effective* channel because
> 
>  - core-mentorship is *explicitly* for poking Those Who Can Help
>    (among other requests for help);

It would be worth an experiment. I know that I wouldn't have reviewed
Nikolaus' patches if he had posted to core-mentorship - I'm not a core
mentor.

> I also suggested that some tweaks to the weekly issue report might
> help to catch the attention of those who can commit patches, but my
> ideas about that are still basically vaporware.

Well, the code of the weekly reports is free software, contributions
are welcome :-)

Regards,
Martin



From tjreedy at udel.edu  Sun Apr 13 21:36:39 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 13 Apr 2014 15:36:39 -0400
Subject: [Python-Dev] Appeal for reviews
In-Reply-To: <534ADB98.2070600@v.loewis.de>
References: <87sipiwf1w.fsf@vostro.rath.org>
 <87ppkmgjg5.fsf@uwakimon.sk.tsukuba.ac.jp>
 <1397351254.611.105839425.39EA328C@webmail.messagingengine.com>
 <534ADB98.2070600@v.loewis.de>
Message-ID: <liep0q$qc6$1@ger.gmane.org>

On 4/13/2014 2:46 PM, "Martin v. L?wis" wrote:

> As for the request "Are you sure that the patch is ready": this is
> *very* difficult to answer for the author. We all have experienced
> that patches that we considered good were critized out of nowhere,
> and I just did the same to Nikolaus. There is just no way to tell
> in advance whether people will approve a certain change or not, because
> people will apply other criteria than oneself.

What I suggested is that he make sure that the patch *appears* to be 
ready, as far as he can tell, according to our *minimum* criteria -- and 
that he so indicate. There is a big difference between a bare code patch 
and a whitespace-tested code + test + doc + ack patch, with a news entry 
suggested only in the text, if at all.

-- 
Terry Jan Reedy



From Steve.Dower at microsoft.com  Sun Apr 13 21:41:27 2014
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Sun, 13 Apr 2014 19:41:27 +0000
Subject: [Python-Dev] Windows installers and OpenSSL
In-Reply-To: <534AD9B7.4020209@v.loewis.de>
References: <CACac1F-gGuryDiU9JL4zh0XQ0bpE=ybZ-iBFZ0Q=uvrO8zeGJg@mail.gmail.com>,
 <534AD9B7.4020209@v.loewis.de>
Message-ID: <c16f4128037c4e088a3aa5dc1a3278a1@BLUPR03MB389.namprd03.prod.outlook.com>

I'm willing to embark on migrating the entire installer to WiX, which doesn't directly fix any particular issue, but could significantly reduce the overhead of building and maintaining the Windows installers.

Martin - are you at PyCon today? Can we chat to figure out what how much of the work you do can be automated? (If not, email on or off list is good too.)

Alternatively, if you don't want me mucking about with this, tell me and I'll stop (being willing does not mean being keen ;) )

Cheers,
Steve

Top-posted from my Windows Phone
________________________________
From: Martin v. L?wis<mailto:martin at v.loewis.de>
Sent: ?4/?13/?2014 14:51
To: Paul Moore<mailto:p.f.moore at gmail.com>; Python Dev<mailto:python-dev at python.org>
Subject: Re: [Python-Dev] Windows installers and OpenSSL

Am 10.04.14 15:41, schrieb Paul Moore:
> Given the OpenSSL vulnerability and the fact that we bundle OpenSSL
> with the Windows installers (1.0.1e in Python 3.4.0) should we be
> releasing updated installers?

As others have said: certainly, and only for 3.4.0 (i.e. 2.7 in
particular is not affected - I'm glad I didn't update OpenSSL
there past 0.9.8).

My feeling with these things is that it is often better to wait
until the dust settles - people in a hurry of fixing security
bugs tend to introduce new ones in the process.

I'm tempted to experiment with installer patch files for this
(.msp); it's technically just a single DLL that would need to be
replaced. Contributions are welcome.

Regards,
Martin


_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40microsoft.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140413/4a444f50/attachment-0001.html>

From tjreedy at udel.edu  Sun Apr 13 21:59:36 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 13 Apr 2014 15:59:36 -0400
Subject: [Python-Dev] static typing of input arguments in signatures
In-Reply-To: <21C31DB4-6F80-43CE-B811-FB61938E9423@langa.pl>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
 <libs4b$qqg$1@ger.gmane.org> <lid4uj$qj1$1@ger.gmane.org>
 <21C31DB4-6F80-43CE-B811-FB61938E9423@langa.pl>
Message-ID: <lieqbs$d6s$1@ger.gmane.org>

On 4/13/2014 4:11 AM, ?ukasz Langa wrote:
> On Apr 13, 2014, at 12:48 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
>
>> Stefan Behnel, 12.04.2014 19:11:
>>
>> So, what I've learned from seven years of Cython is that static typing in
>> signatures is actually less interesting than you might think at first
>> sight. It might be ok for documentation purposes, although its verboseness
>> makes that also a bit questionable.
>
> You raise a valid point that type hinting a dict instead of a Mapping is
> likely to create an overly limiting API. This sort of error is however
> quite easy to fix forward.
>
> To counter, we?ve had multiple data points during the summit suggesting
> that duck typing is not really that often used in production code. In
> other words, after initial prototyping and testing, an API of a callable
> gets settled and is later used with a very limited number of types.

For public library code, where the use case is not known, apis should 
usually be as generic as sensible. For private library code, I can 
imagine that apis are and possibly even should be limited to classes 
actually used. There naturally is a bit of a bias here for public code.

> Of course, fuzzy terms like ?often?, ?seldom?, and ?limited? suggest
> that we don?t have hard data here. That being said, there is a clear
> need to improve our ability to lint code, refactor code easier, and
> document the contracts better. This has been voiced by various
> conference attendees and the consensus is tremendous.

This need applies to public as well as private code. Private code is a 
good place for experiments.

> For Python specifically the existence of prototypes like Mypy, obiwan,
> or pytypedecl shows that there is growing interest in standardizing this
> functionality. Therefore, I'm drafting a PEP for opt-in type hints with
> optional checks at runtime and by static analysis. Will post the initial
> form in the upcoming days, and then we can go paint that bike shed.

I look forward to it.

> I'm aware that great care here is needed for the outcome to feel
> Pythonic and not overwhelming. Don't worry before you get your hands on
> the draft, though.

I'm not. PEPs routinely get revised, and ofter improved, no matter the 
author.

-- 
Terry Jan Reedy



From Steve.Dower at microsoft.com  Sun Apr 13 22:02:02 2014
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Sun, 13 Apr 2014 20:02:02 +0000
Subject: [Python-Dev] 2.7 Windows installers (Was: death to 2.7;
 long live 2.7)
In-Reply-To: <534AD745.9000908@v.loewis.de>
References: <1397092930.29268.104798077.13871388@webmail.messagingengine.com>,
 <534AD745.9000908@v.loewis.de>
Message-ID: <7f27a783ee474bd7806300151bfadb50@BLUPR03MB389.namprd03.prod.outlook.com>

I replied to the other email before I saw this one.

Consider this my self-nomination to take over, pending a quick email to Microsoft's lawyers to make sure it's okay (it should be, but IANAL and they wrote the policy).

My plan would be to rewrite the installer using WiX but retaining compatibility with the existing MSIs. I'd also automate as much as possible to make it very easy for someone else with the certificates to build/verify the release.

Happy to answer any other questions or provide more background about myself if people ask.

Cheers,
Steve

Sent from my Windows Phone
________________________________
From: Martin v. L?wis<mailto:martin at v.loewis.de>
Sent: ?4/?13/?2014 14:29
To: Benjamin Peterson<mailto:benjamin at python.org>; python-dev at python.org<mailto:python-dev at python.org>
Subject: Re: [Python-Dev] 2.7 Windows installers (Was: death to 2.7; long live 2.7)

Am 10.04.14 03:22, schrieb Benjamin Peterson:
> We'll keep doing what we're currently doing for another year, making
> normal bug fix releases with installers. After that, we _won't_ close
> 2.7 to normal bug fixes as is currently implied by the release schedule.

After thinking about this plan, I believe it is best if I step down as
the Windows installers engineer right away. To my knowledge, no specific
date for the next 2.7 bugfix release has been set, so this should allow
to find a replacement before the next release.

I would have volunteered to do two more releases if these had been the
definite last ones, as it would have been impractical to replace me for
just two releases. If this is going until 2020, anybody up to the task
should start working on it now.

I'll be willing to help anybody taking over with advise, but I'll
consider the 2.7 branch otherwise dead (i.e. not worthwhile
contributing to).

I'll be happy to continue making installers for the 3.x branches,
but if anybody considering to volunteer would do so on the condition
that he/she gets it "all", I'd be happy to hand these other branches
over as well.

Of course, anybody taking over that role would be free to either keep
or replace the current technology (e.g. switching to InstallShield
or WiX or whatever).

The one option I'd rather not support is an "open" committee of
Windows installer engineers. Anybody taking this role will have a
copy of the PSF code signing key, so I'd like to keep this list
rather small, and the people attached to the task.

Regards,
Martin

_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40microsoft.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140413/cf5d73df/attachment.html>

From Nikolaus at rath.org  Sun Apr 13 21:21:09 2014
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Sun, 13 Apr 2014 12:21:09 -0700
Subject: [Python-Dev] Appeal for reviews
In-Reply-To: <licded$juc$1@ger.gmane.org> (Terry Reedy's message of "Sat, 12
 Apr 2014 18:06:52 -0400")
References: <87sipiwf1w.fsf@vostro.rath.org> <licded$juc$1@ger.gmane.org>
Message-ID: <87ioqd11ei.fsf@vostro.rath.org>

Terry Reedy <tjreedy at udel.edu> writes:
[Quote conveniently rearranged]
>> I've accumulated a number of patches in the issue tracker that are
>> waiting for someone to review/commit/reject them. I'm eager to make
>> corrections as necessary, I just need someone to look the work that I've
>> done so far:
>
> Do you consider any of these 'ready-to-commit'?
> * before-fail, after-pass test?
> * required doc changes?
> * tested patch for all versions that should be patched?
> * any new Misc/ACKS entry needed (for new person other than you)?
> * checked for stray end-of-line whitespace?

Actually, they should all be in this stage (assuming make patchcheck
would complain about eol whitespace). I wouldn't call it ready-to-commit
though, I'm sure a second pair of eyes would find issues that need to be
addressed. But I'm at a point where I don't see any possible issues.

In more detail:

>> * http://bugs.python.org/issue20951 (SSLSocket.send() returns 0 for
>>    non-blocking socket)

This contains a docpatch for 3.4 that (I believe) should be applied in
any case. In addition, there does not seem to be a consensus whether the
same behavior should be kept for 3.5 (in that case the docpatch should
be applied for 3.5 as well), deprecated in 3.5 (patch available as
well), or changed right away (no patch attached yet, I'm waiting for
consensus). Maybe it would be enough if some more developers could chime
in? I'm not sure how issues like this are typically decided.

>> * http://bugs.python.org/issue1738 (filecmp.dircmp does exact match
>>    only)

Contains testcase, docpatch and code-patch reviewed and updated by
me. New feature, so should go only into 3.5 and applies cleanly.

I just updated the patch to include the original author in Misc/ACKS.

>> * http://bugs.python.org/issue7776 (http.client.HTTPConnection
>>    tunneling is broken)

Contains testcase, docpatch and code-patch. Applies cleanly to 3.4, and
I just refreshed the patch for 3.5. Not sure if this should be fixed in
2.7?

>> * http://bugs.python.org/issue20177 (Derby #8: Convert 28 sites to
>>    Argument Clinic across 2 files)

No test case and no doc patch necessary (I believe). I have some doubts
about the patch though, see
http://bugs.python.org/issue20177#msg209153. 

>> * http://bugs.python.org/issue19414 (iter(ordered_dict) yields keys
>>    not in dict in some circumstances)

There are several different ways to fix this problem. I have created a
patch implementing one of them. It applies cleanly to 3.4 and 3.5 - not
sure why the review link does not show up. I've just attached the same
patch again, maybe that helps.

>> * http://bugs.python.org/issue20578 (BufferedIOBase.readinto1 is
>>    missing)

Contains doc update, test case, and code patch. Applies cleanly to 3.5,
and should not go in older versions (new feature).

Martin v. L?wis has kindly started reviewing this.

>> * http://bugs.python.org/issue21057 (TextIOWrapper does not support
>>    reading bytearrays or memoryviews)

Contains test case, and code patch. I don't think the documentation
needs to be updated. Patch applies cleanly to 3.5, and should not go in
older versions (new feature).

Martin v. L?wis has kindly started reviewing this as well :-).


Best,
-Nikolaus


-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             ?Time flies like an arrow, fruit flies like a Banana.?

From Nikolaus at rath.org  Sun Apr 13 21:31:46 2014
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Sun, 13 Apr 2014 12:31:46 -0700
Subject: [Python-Dev] Appeal for reviews
In-Reply-To: <87ppkmgjg5.fsf@uwakimon.sk.tsukuba.ac.jp> (Stephen J. Turnbull's
 message of "Sun, 13 Apr 2014 09:30:02 +0900")
References: <87sipiwf1w.fsf@vostro.rath.org>
 <87ppkmgjg5.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <87fvlh10wt.fsf@vostro.rath.org>

"Stephen J. Turnbull" <stephen at xemacs.org> writes:
> I apologize for the tone.  I need to go *right* now, and can't fix
> that.  Really, I'm sympathetic and my goal is not just to defend
> python-dev, but to help you get the reviews your work deserves.
> Please read with that in mind.

Will do - but why the rush? Be assured that I would have read the mail
even if it came a day later :-).

> That's an impressive list, but it doesn't tell us what work you've
> done.  There's a checklist in the devguide which should be more
> accurate (sorry about lack of URL), but I imagine it includes the
> following:

>  * Has the issue been classified as bug or feature?
>  * If bug, is it confirmed?
>  * If complex, does it need a PEP?  (probably irrelevant to your patches)
>  * Is there agreement about requirements in the issue comments?
>  * Is there a patch?
>  * Does the design need review?
>  * Has relevant documentation been added/updated (including
>    docstrings, manuals, Whats New at least)?
>  * Are there tests that the requirements are satisfied?
>  * Are there tests for any regressions that arose in the process of
>    developing the patch?
>  * Have you signed a CA?  (irrelevant to you IIRC)
>  * Is the issue status updated to reflect the work done so far?

I've described the status of each bug in more detail in my reply 
Terry, but generally all the issues contain a testcase (i.e., so I
consider them confirmed), do not require a PEP, contain a patch that
needs review and include documentation updates. I have signed the CA,
and I have updated the issue status as much as I can (I don't seem to
have privileges to add/change keywords). 

I have a second list with issues that I worked on that are more nasty,
but I deliberately did not include them in the list.

> It would help in getting reviewer attention to your work if in
> addition to a list of patches you provided an indication of (1) how
> complete the patch is and (2) what review is requested.  That in
> itself is a strong indicator of quality.

All the patches are complete in the sense that I think they can be
committed and would improve Python. Of course, further improvements are
always possible. I'm not sure about (2) -- what different kind of
reviews are there?


> There must be stuff of interest to you happening on twitter, etc.,
> during PyCon.

I'm not quite sure what you're getting at (maybe because I don't use
twitter).

Best,
-Nikolaus
-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             ?Time flies like an arrow, fruit flies like a Banana.?

From Nikolaus at rath.org  Sun Apr 13 21:36:29 2014
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Sun, 13 Apr 2014 12:36:29 -0700
Subject: [Python-Dev] Appeal for reviews
In-Reply-To: <534ADC29.3070401@v.loewis.de> ("Martin v. =?utf-8?Q?L=C3=B6w?=
 =?utf-8?Q?is=22's?= message of "Sun, 13 Apr 2014 20:49:13 +0200")
References: <87sipiwf1w.fsf@vostro.rath.org>
 <87ppkmgjg5.fsf@uwakimon.sk.tsukuba.ac.jp>
 <1397351254.611.105839425.39EA328C@webmail.messagingengine.com>
 <87k3athh24.fsf@uwakimon.sk.tsukuba.ac.jp>
 <534ADC29.3070401@v.loewis.de>
Message-ID: <87d2gl10oy.fsf@vostro.rath.org>

"Martin v. L?wis" <martin at v.loewis.de> writes:
> Am 13.04.14 08:36, schrieb Stephen J. Turnbull:
>
>> I admit the tone was biased toward nagging or "blaming the victim",
>> and again I apologize for causing misunderstanding.  Nikolaus isn't
>> "wrong" for posting here.  My claim is that in current circumstances,
>> core-mentorship would be a more *effective* channel because
>> 
>>  - core-mentorship is *explicitly* for poking Those Who Can Help
>>    (among other requests for help);
>
> It would be worth an experiment. I know that I wouldn't have reviewed
> Nikolaus' patches if he had posted to core-mentorship - I'm not a core
> mentor.

This is actually not the first time I sent this (though luckily, the
number of issues did change somewhat since the last attempt). My last
two mails went to core-mentorship, so I figured that maybe this time I
ought to try a different list to get other people's attention as well.


Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             ?Time flies like an arrow, fruit flies like a Banana.?

From martin at v.loewis.de  Sun Apr 13 22:46:36 2014
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 13 Apr 2014 22:46:36 +0200
Subject: [Python-Dev] Windows installers and OpenSSL
In-Reply-To: <c16f4128037c4e088a3aa5dc1a3278a1@BLUPR03MB389.namprd03.prod.outlook.com>
References: <CACac1F-gGuryDiU9JL4zh0XQ0bpE=ybZ-iBFZ0Q=uvrO8zeGJg@mail.gmail.com>,
 <534AD9B7.4020209@v.loewis.de>
 <c16f4128037c4e088a3aa5dc1a3278a1@BLUPR03MB389.namprd03.prod.outlook.com>
Message-ID: <534AF7AC.9090307@v.loewis.de>

Am 13.04.14 21:41, schrieb Steve Dower:
> I'm willing to embark on migrating the entire installer to WiX, which
> doesn't directly fix any particular issue, but could significantly
> reduce the overhead of building and maintaining the Windows installers.

I somewhat doubt that it could reduce the overhead - the actual
overhead for the MSI generation is fairly small. It's more that the
current Python-based approach is not too familiar to people, in
particular to those that are familiar with WiX (which I, in turn, am
not).

> Martin - are you at PyCon today? Can we chat to figure out what how much
> of the work you do can be automated? (If not, email on or off list is
> good too.)

Unfortunately, I'm not, and I can't really do chats ATM due to timezone
differences. It's all fairly automated; what takes time is that there
is manual steps in-between (e.g. getting the sources from Mercurial,
launching Visual Studio, running the CHM generator, running the MSI
generator, signging with PGP, uploading the files, running the
installer for testing, uninstalling the previous installation, and so
on). It could be done in a single batch file mostly, except that I
never got to write this batch file.

If it's automated, it would still take an hour or so, but it would be
possible to do other things while waiting for the build to complete.

> Alternatively, if you don't want me mucking about with this, tell me and
> I'll stop (being willing does not mean being keen ;) )

If it's that, I'd say stop. It's not that I want you to, but that I'm
skeptical that it solves anything. If you do want to start using WiX,
please do run the current build process at least once before.

The basic instruction for that is "run msi.py". It will tell you
if it's missing files, and anybody familiar with the build process
should be able to figure out how to get the missing files (if not,
just ask).

Regards,
Martin




From martin at v.loewis.de  Sun Apr 13 22:51:28 2014
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 13 Apr 2014 22:51:28 +0200
Subject: [Python-Dev] 2.7 Windows installers (Was: death to 2.7;
 long live 2.7)
In-Reply-To: <7f27a783ee474bd7806300151bfadb50@BLUPR03MB389.namprd03.prod.outlook.com>
References: <1397092930.29268.104798077.13871388@webmail.messagingengine.com>,
 <534AD745.9000908@v.loewis.de>
 <7f27a783ee474bd7806300151bfadb50@BLUPR03MB389.namprd03.prod.outlook.com>
Message-ID: <534AF8D0.8050703@v.loewis.de>

Am 13.04.14 22:02, schrieb Steve Dower:
> I replied to the other email before I saw this one.

Same here :-)

> Consider this my self-nomination to take over, pending a quick email to
> Microsoft's lawyers to make sure it's okay (it should be, but IANAL and
> they wrote the policy).
> 
> My plan would be to rewrite the installer using WiX but retaining
> compatibility with the existing MSIs. I'd also automate as much as
> possible to make it very easy for someone else with the certificates to
> build/verify the release.
> 
> Happy to answer any other questions or provide more background about
> myself if people ask.

As I just said: to get started, run the current build process. Without
knowing WiX in detail, I'd still claim that msi.py is superiour in
terms of expressiveness (i.e. it can better compute what files go into
the MSI). I'm almost certain that you need a full programming language
to actually perform the entire build; one alternative to msi.py would
be to have a Python script generating the WiX configuration files.

Regards,
Martin


From rdmurray at bitdance.com  Mon Apr 14 01:58:25 2014
From: rdmurray at bitdance.com (R. David Murray)
Date: Sun, 13 Apr 2014 19:58:25 -0400
Subject: [Python-Dev] static typing of input arguments in signatures
In-Reply-To: <lieqbs$d6s$1@ger.gmane.org>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
 <libs4b$qqg$1@ger.gmane.org> <lid4uj$qj1$1@ger.gmane.org>
 <21C31DB4-6F80-43CE-B811-FB61938E9423@langa.pl> <lieqbs$d6s$1@ger.gmane.org>
Message-ID: <20140413235826.142BF250030@webabinitio.net>

On Sun, 13 Apr 2014 15:59:36 -0400, Terry Reedy <tjreedy at udel.edu> wrote:
> On 4/13/2014 4:11 AM, ??ukasz Langa wrote:
> > On Apr 13, 2014, at 12:48 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> >
> >> Stefan Behnel, 12.04.2014 19:11:
> >>
> >> So, what I've learned from seven years of Cython is that static typing in
> >> signatures is actually less interesting than you might think at first
> >> sight. It might be ok for documentation purposes, although its verboseness
> >> makes that also a bit questionable.
> >
> > You raise a valid point that type hinting a dict instead of a Mapping is
> > likely to create an overly limiting API. This sort of error is however
> > quite easy to fix forward.
> >
> > To counter, we???ve had multiple data points during the summit suggesting
> > that duck typing is not really that often used in production code. In
> > other words, after initial prototyping and testing, an API of a callable
> > gets settled and is later used with a very limited number of types.
> 
> For public library code, where the use case is not known, apis should 
> usually be as generic as sensible. For private library code, I can 
> imagine that apis are and possibly even should be limited to classes 
> actually used. There naturally is a bit of a bias here for public code.

The way *I* heard it was that types of the input arguments did not, in
general, change *during the running of an application*.  That doesn't
mean that the types wouldn't be different in a different application,
which means that in a library that duck types, duck typing is indeed used
in production, it's just that the types don't *change* for a production
application.  So, yeah, pretty much what Terry said about library code
versus application code.  That is, after all, what duck typing is about,
and there is a *reason* we use it.

--David

From stephen at xemacs.org  Mon Apr 14 03:43:14 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 14 Apr 2014 10:43:14 +0900
Subject: [Python-Dev] Appeal for reviews
In-Reply-To: <87fvlh10wt.fsf@vostro.rath.org>
References: <87sipiwf1w.fsf@vostro.rath.org>
 <87ppkmgjg5.fsf@uwakimon.sk.tsukuba.ac.jp>
 <87fvlh10wt.fsf@vostro.rath.org>
Message-ID: <87fvlghej1.fsf@uwakimon.sk.tsukuba.ac.jp>

Nikolaus Rath writes:

 > I've described the status of each bug in more detail in my reply 
 > Terry, but generally all the issues contain a testcase (i.e., so I
 > consider them confirmed), do not require a PEP, contain a patch that
 > needs review and include documentation updates. I have signed the CA,
 > and I have updated the issue status as much as I can (I don't seem to
 > have privileges to add/change keywords). 

AFAIK that can be described as:

 > All the patches are complete in the sense that I think they can be
 > committed and would improve Python.

I don't suggest you need to say more than that.  Perhaps I overlooked
such a statement in the OP?  If so, I'm sorry.

For somebody who is submitting their first round of patches I would
recommend some statement indicating that they know about the
"checklist" you described above, but by now you're well-known to the
Right People.

 > Of course, further improvements are always possible.

Your modesty is not in question. :-)  An explicit statement that "These
are ready" allows an experienced developer to give you feedback not
only about whether the patches are in fact ready, but whether your
judgment about patches is ready for commit privileges -- in Python
it's desirable that code be "Pythonic" as well as correct and clean.

 > I'm not sure about (2) -- what different kind of reviews are there?

>From the contributor point of view, I'd broadly say

 - I'm stuck (this is harder than I thought), somebody please look at
   what I've done and tell me I haven't screwed up so far, and then
   I'll go back to work.

 - I'm a little stuck, maybe I'm done, but I'm not sure.  Please look.

 - I'm done, please commit it if you agree.

Evidently you specifically listed patches for which you want an "I'm
done" review.  Part of why I wrote what I did is that I didn't
recognize that.

 > > There must be stuff of interest to you happening on twitter, etc.,
 > > during PyCon.
 > 
 > I'm not quite sure what you're getting at (maybe because I don't use
 > twitter).

Just that on the one hand, it was very obvious to me that over the
last few weeks people were very busy preparing for PyCon (and so even
less time than usual for reviewing), and on the other, if you can't be
at PyCon, there's much to hear about Python design and implementation,
and about Pythonic thinking by following the language summit and what
people are talking about during PyCon.


From guido at python.org  Mon Apr 14 03:51:57 2014
From: guido at python.org (Guido van Rossum)
Date: Sun, 13 Apr 2014 21:51:57 -0400
Subject: [Python-Dev] Fwd: Jython Report
In-Reply-To: <EB728A51-F961-4B58-A0DC-09A6A77F9579@voidspace.org.uk>
References: <CAL9RhrVxaoe7f18F+xnsvj=9WW-gKtZsdGnnUB8OW5S6RDqpjg@mail.gmail.com>
 <EB728A51-F961-4B58-A0DC-09A6A77F9579@voidspace.org.uk>
Message-ID: <CAP7+vJ+6YgqeH-nbJ6gXWeJj-8yKg4KWN9iBEg1Ohp79pDeFEQ@mail.gmail.com>

Jim and Frank,

I just wanted to thank you both (and all other Jython developers) for
sending this update, and for your great and continued work on Jython. I'm
sorry neither of you could come to PyCon but I am very happy that the torch
is being carried!! It's a fantastic list indeed.

--Guido


On Sat, Apr 12, 2014 at 11:56 AM, Michael Foord <michael at voidspace.org.uk>wrote:

> Below is the Jython "status update" report on Jython I received from Jim
> Baker and summarised in the Language Summit. It comes with one addendum
> from Frank:
>
> Jim's list is fantastic - the one bit I'd like to add to the list:
>
> Jython now supports a buffer protocol that parallels CPython's C API
> buffer protocol. This provided the basis for support of buffer() and
> memoryview(). The work was done with Jython3 in mind and will be a huge
> boost to that eventual effort.
>
> Begin forwarded message:
>
> *From: *Jim Baker <jbaker at zyasoft.com>
> *Subject: **Re: Jython Report*
> *Date: *7 April 2014 06:42:51 BST
> *To: *Michael Foord <michael at voidspace.org.uk>
> *Cc: *Frank Wierzbicki <fwierzbicki at gmail.com>
>
> Recent changes to trunk (last 6 months)
>
> * Recently tagged a soft beta 2!
> * Java 7 JVM is now the minimum version, which gives a larger base of
> functionality to work with (such as using Java 7's AutoCloseable to imply
> corresponding context manager support in using Python code)
> * Enable mixing Python and Java types in the bases of a class when using a
> metaclass
> * Added support for buffer, memoryview, although not complete yet with
> respect to Java integration
> * Console and encoding support, such as unicodedata/idna updates
> * Many, many small fixes
>
> About to be in trunk, to support beta 3
>
> * socket-reboot reimplements socket/select/ssl on top of Netty 4, a
> popular event loop networking framework for the JVM (used by a large number
> of performant projects in Java space and originally part of JBoss). There
> was no ssl support before, but now socket and especially select semantics
> are much closer to CPython as well (basically close to the Windows socket
> model).
> * socket-reboot in turn enables requests and thereby pip. A branch of pip
> currently works, actually modifying an upstream vendor lib (html5lib) so
> that it doesn't use isolated UTF-16 surrogates in literals, since this is
> not actually legal unicode, nor does it work in Jython's UTF-16 based
> representation. Ironically this usage is to detect such illegal use in
> input streams.
> * Relative star imports, which seems to impact a number of interesting
> projects.
> * Performance tuning of sre. Jython has a port of CPython's sre, however
> our use of UTF-16 requires expansion into an array of codepoints. Currently
> this is done on demand, which can potentially add another O(n) factor in
> evaluating regexes. A pull request we will apply memoizes. In the future,
> we will rewrite the logic in sre so that it does next/prev, much like JRuby
> currently does for similar encoding issues.
>
> Related work
>
> * Other PyPA tooling including virtualenv and wheel needs more diagnosis
> to see why they currently fail on Jython, but our hope is that this is
> minor.
> * New project jythontools by a number of Jython developers (including
> Frank and Jim). This includes a number of projects that will help evolve
> Jython, but outside the usual release schedule and the usual problem of
> being in core (such as eventual deprecation):
>       - Clamp - precise integration with Java, enabling such capabilities
> as Java directly importing Python modules without explicitly initializing
> the Jython runtime or using object factories. Future work will enable Java
> annotation integration, as decorators. Integrates with setuptools; future
> integration as well with Maven via Aether.
>       - Jiffy - provide a CFFI backend for Jython. Right now it is pure
> vaporware, but cursory examination of cffi.backend_ctypes suggests that it
> should be straightforward and of modest effort to provide a similar backend
> by using JFFI, which Jython and JRuby both use to access native runtime
> services (such as Posix API) as part of the Java native runtime project.
> * The Patois project has been started to collect examples for
> cross-implementation support, as seen in surrogate support, but it will be
> a good question to get that really going, vs just talking about it.
> * JyNI - simply adding this jar to the classpath enables C extension API
> support. Note that this project has been licensed by its developer (not a
> Jython committer) under an LGPL license.
>
> Release schedule
>
> * Complete beta 2
> * Beta 3 is forthcoming, likely in 2 weeks
> * For beta 4, need to perform a comprehensive bug triage - what will be
> in, not in for 2.7.0
> * EuroPython sprint to finalize a release candidate for 2.7.0?
>
> Future
>
> * Mostly around performance, Java integration, and of course the usual bug
> fixes
> * Python bytecode compiler remains important, including for support
> targeting Android and removing restriction on getting too large a method
> for the JVM
> * More hooks for Java integration, including managing generated bytecode
> * Integrating Zippy could provide for PyPy-like performance, but requires
> Graal JVM
> * Supporting invokedynamic is a more realistic solution, but without the
> use of annotations (eg turn off Python frames) is going to be limited
> (maybe 2x?) based on earlier analysis
>
> Jython 3.x?
>
> * This comes up periodically, and it would be super nice for us to
> complete this support. At the very least it would make unicode strings and
> bytestrings correspond directly to how they are represented in Java, so
> that will be a nice cleanup.
> * Release schedule: we will get there at some point!
>
>
> On Sun, Apr 6, 2014 at 5:20 PM, Jim Baker <jbaker at zyasoft.com> wrote:
>
>> Michael,
>>
>> I was thinking about this very topic this morning! Will send you the
>> latest status in the next 24h, specifically our work to support pypa
>> (setuptools, pip, virtualenv, wheel) and related tooling.
>>
>> - Jim
>>
>>
>> On Sun, Apr 6, 2014 at 11:30 AM, Michael Foord <michael at voidspace.org.uk>wrote:
>>
>>> Hey guys,
>>>
>>> Would you be able to write up a brief report on the current and future
>>> status of Jython, for me to read out at the Python language summit on
>>> Wednesday? (Unless someone who works on Jython will be there - but as far
>>> as I know that isn't the case.)
>>>
>>> All the best,
>>>
>>> Michael Foord
>>>
>>> --
>>> http://www.voidspace.org.uk/
>>>
>>>
>>> May you do good and not evil
>>> May you find forgiveness for yourself and forgive others
>>> May you share freely, never taking more than you give.
>>> -- the sqlite blessing
>>> http://www.sqlite.org/different.html
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>
>
> --
>
> http://www.voidspace.org.uk/
>
> May you do good and not evil
> May you find forgiveness for yourself and forgive others
> May you share freely, never taking more than you give.
> -- the sqlite blessing http://www.sqlite.org/different.html
>
>
>
>
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140413/61110f6b/attachment.html>

From stephen at xemacs.org  Mon Apr 14 03:54:05 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 14 Apr 2014 10:54:05 +0900
Subject: [Python-Dev] Appeal for reviews
In-Reply-To: <534ADC29.3070401@v.loewis.de>
References: <87sipiwf1w.fsf@vostro.rath.org>
 <87ppkmgjg5.fsf@uwakimon.sk.tsukuba.ac.jp>
 <1397351254.611.105839425.39EA328C@webmail.messagingengine.com>
 <87k3athh24.fsf@uwakimon.sk.tsukuba.ac.jp>
 <534ADC29.3070401@v.loewis.de>
Message-ID: <87eh10he0y.fsf@uwakimon.sk.tsukuba.ac.jp>

"Martin v. L?wis" writes:
 > Am 13.04.14 08:36, schrieb Stephen J. Turnbull:

 > >  - core-mentorship is *explicitly* for poking Those Who Can Help
 > >    (among other requests for help);
 > 
 > It would be worth an experiment. I know that I wouldn't have reviewed
 > Nikolaus' patches if he had posted to core-mentorship - I'm not a core
 > mentor.

I imagine there's useful data on this already available.  I'll go
look, in a week or two.

 > > I also suggested that some tweaks to the weekly issue report might
 > > help to catch the attention of those who can commit patches, but my
 > > ideas about that are still basically vaporware.
 > 
 > Well, the code of the weekly reports is free software, contributions
 > are welcome :-)

XEmacs uses roundup, too, including (a now-obsolete version of) the
Python patches.  Time horizon is longer, but I may look into this too.

Note that what I have in mind also may require changes to the schema,
I believe (there needs to be a user control to mark the issue as
"commit requested", and I don't think a keyword would be appropriate).


From martin at v.loewis.de  Mon Apr 14 04:05:57 2014
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Mon, 14 Apr 2014 04:05:57 +0200
Subject: [Python-Dev] Appeal for reviews
In-Reply-To: <87fvlghej1.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <87sipiwf1w.fsf@vostro.rath.org>
 <87ppkmgjg5.fsf@uwakimon.sk.tsukuba.ac.jp> <87fvlh10wt.fsf@vostro.rath.org>
 <87fvlghej1.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20140414040557.Horde.FRrDDDnMkJvhmQHf-0NFHg1@webmail.df.eu>


Quoting "Stephen J. Turnbull" <stephen at xemacs.org>:

> Your modesty is not in question. :-)  An explicit statement that "These
> are ready" allows an experienced developer to give you feedback not
> only about whether the patches are in fact ready, but whether your
> judgment about patches is ready for commit privileges -- in Python
> it's desirable that code be "Pythonic" as well as correct and clean.

For a regular patch, I think this is asking to much. The default
assumption should be that the patch is finished, and not work-in-progress.
If it was, people typically indicate so. But I then regularly ask that
the tracker is not abused as a version control system to develop a
change; it's easy enough to use a Mercurial clone for that.

For gaining commit access, it's really more important that the patch
is factually finished, than that it's author believes it to. If people
get it right the first time often enough, they get commit access.

Regards,
Martin


From benjamin at python.org  Mon Apr 14 04:46:56 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 13 Apr 2014 19:46:56 -0700
Subject: [Python-Dev] your next regular installment of python 2.7
Message-ID: <1397443616.11961.106105065.43688254@webmail.messagingengine.com>

Yep, it's almost that time of the year again. My proposed schedule for
2.7.7 is

May 12th, 2.7.7rc1
May 26th, 2.7.7 final

That means if you want your 2.7 improvement to see the light of day
before November, the next few weeks are the time to do it. (PEP 446, I'm
looking at you.)

Also, I doubt anyone cares, but I'm going to do the last security
release of python 3.1 concurrently. That will, of course, be a
source-only release.

Regards,
Benjamin

From stephen at xemacs.org  Mon Apr 14 07:54:33 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 14 Apr 2014 14:54:33 +0900
Subject: [Python-Dev] Appeal for reviews
In-Reply-To: <20140414040557.Horde.FRrDDDnMkJvhmQHf-0NFHg1@webmail.df.eu>
References: <87sipiwf1w.fsf@vostro.rath.org>
 <87ppkmgjg5.fsf@uwakimon.sk.tsukuba.ac.jp>
 <87fvlh10wt.fsf@vostro.rath.org>
 <87fvlghej1.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140414040557.Horde.FRrDDDnMkJvhmQHf-0NFHg1@webmail.df.eu>
Message-ID: <877g6sh2w6.fsf@uwakimon.sk.tsukuba.ac.jp>

martin at v.loewis.de writes:

 > For gaining commit access, it's really more important that the patch
 > is factually finished, than that it's author believes it to. If people
 > get it right the first time often enough, they get commit access.

Yes, that's what I had in mind, but I guess I explained it poorly.



From jdhardy at gmail.com  Mon Apr 14 14:12:14 2014
From: jdhardy at gmail.com (Jeff Hardy)
Date: Mon, 14 Apr 2014 13:12:14 +0100
Subject: [Python-Dev] 2.7 Windows installers (Was: death to 2.7;
	long live 2.7)
In-Reply-To: <534AF8D0.8050703@v.loewis.de>
References: <1397092930.29268.104798077.13871388@webmail.messagingengine.com>
 <534AD745.9000908@v.loewis.de>
 <7f27a783ee474bd7806300151bfadb50@BLUPR03MB389.namprd03.prod.outlook.com>
 <534AF8D0.8050703@v.loewis.de>
Message-ID: <CAF7AXFGaHtZvBwB2fAZU7AC1rdcbhuMoWrePT7uzh03ghNoxUQ@mail.gmail.com>

On Sun, Apr 13, 2014 at 9:51 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> As I just said: to get started, run the current build process. Without
> knowing WiX in detail, I'd still claim that msi.py is superiour in
> terms of expressiveness (i.e. it can better compute what files go into
> the MSI). I'm almost certain that you need a full programming language
> to actually perform the entire build; one alternative to msi.py would
> be to have a Python script generating the WiX configuration files.

FWIW, that's exactly what IronPython does - most of the files are
fixed, but there are scripts to generate inclusions for the stdlib.
WiX and MSBuild are pretty powerful combo, even if the excess of
stabby brackets makes *me* stabby after a while.

- Jeff

From ncoghlan at gmail.com  Mon Apr 14 14:18:13 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 14 Apr 2014 08:18:13 -0400
Subject: [Python-Dev] Appeal for reviews
In-Reply-To: <877g6sh2w6.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <87sipiwf1w.fsf@vostro.rath.org>
 <87ppkmgjg5.fsf@uwakimon.sk.tsukuba.ac.jp>
 <87fvlh10wt.fsf@vostro.rath.org>
 <87fvlghej1.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140414040557.Horde.FRrDDDnMkJvhmQHf-0NFHg1@webmail.df.eu>
 <877g6sh2w6.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CADiSq7fdJSuhsW0cuRVT-ZzqGjzQKMd7Vi8+vNbF0LOmNofxWA@mail.gmail.com>

On 14 Apr 2014 01:56, "Stephen J. Turnbull" <stephen at xemacs.org> wrote:
>
> martin at v.loewis.de writes:
>
>  > For gaining commit access, it's really more important that the patch
>  > is factually finished, than that it's author believes it to. If people
>  > get it right the first time often enough, they get commit access.
>
> Yes, that's what I had in mind, but I guess I explained it poorly.

We should capture this discussion clearly in the dev guide. Even if we
switch to a core reviewer model at some point (as I propose in PEP 462),
the criteria for core reviewer status will match those for core commiter
status.

There are actually a few things I'm personally looking for:

* good judgement on when a patch is "finished enough" to merge
* good judgement on whether a change is a new feature or a bug fix
* good judgement whether a new feature is worth the additional cognitive
burden
* good ability to assess backwards compatibility risks
* sufficient humility to answer "I don't know" to the above questions when
appropriate and ask the relevant domain experts, their sponsoring mentor,
the core-mentorship list or python-dev at large for advice on what to do

It's that last one which is really most critical - even Guido asks for
additional input when he's uncertain about something, and that's a key part
of why I trust his decisions on those rare occasions when he finds it
necessary to exercise BDFL fiat (although his long history of demonstrating
excellent language design instincts certainly helps!)

Cheers,
Nick.

>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140414/1903c332/attachment.html>

From rdmurray at bitdance.com  Mon Apr 14 14:40:53 2014
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 14 Apr 2014 08:40:53 -0400
Subject: [Python-Dev] Appeal for reviews
In-Reply-To: <CADiSq7fdJSuhsW0cuRVT-ZzqGjzQKMd7Vi8+vNbF0LOmNofxWA@mail.gmail.com>
References: <87sipiwf1w.fsf@vostro.rath.org>
 <87ppkmgjg5.fsf@uwakimon.sk.tsukuba.ac.jp> <87fvlh10wt.fsf@vostro.rath.org>
 <87fvlghej1.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140414040557.Horde.FRrDDDnMkJvhmQHf-0NFHg1@webmail.df.eu>
 <877g6sh2w6.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CADiSq7fdJSuhsW0cuRVT-ZzqGjzQKMd7Vi8+vNbF0LOmNofxWA@mail.gmail.com>
Message-ID: <20140414124054.133AA250CAE@webabinitio.net>

On Mon, 14 Apr 2014 08:18:13 -0400, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 14 Apr 2014 01:56, "Stephen J. Turnbull" <stephen at xemacs.org> wrote:
> >
> > martin at v.loewis.de writes:
> >
> >  > For gaining commit access, it's really more important that the patch
> >  > is factually finished, than that it's author believes it to. If people
> >  > get it right the first time often enough, they get commit access.
> >
> > Yes, that's what I had in mind, but I guess I explained it poorly.
> 
> We should capture this discussion clearly in the dev guide. Even if we
> switch to a core reviewer model at some point (as I propose in PEP 462),
> the criteria for core reviewer status will match those for core commiter
> status.
> 
> There are actually a few things I'm personally looking for:
> 
> * good judgement on when a patch is "finished enough" to merge
> * good judgement on whether a change is a new feature or a bug fix
> * good judgement whether a new feature is worth the additional cognitive
> burden
> * good ability to assess backwards compatibility risks
> * sufficient humility to answer "I don't know" to the above questions when
> appropriate and ask the relevant domain experts, their sponsoring mentor,
> the core-mentorship list or python-dev at large for advice on what to do

When considering who we give commit access to, I think we would be
well served to start giving more weight to the quality of the code
reviews that someone does.  Producing good patches is important,
but even without moving the infrastructure to Nick's "core reviewer"
model, doing those reviews is an important part of what committers
do, and it is a different (although related) skill to that of
writing good patches.

Or to put it another way, I'd like to encourage contributors who
want to get commit access to focus just as much on doing good reviews as
they do on writing new patches.  Currently the focus is all on
getting patches accepted.

--David

From njs at pobox.com  Mon Apr 14 07:39:01 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Mon, 14 Apr 2014 07:39:01 +0200
Subject: [Python-Dev] [numpy wishlist] PyMem_*Calloc
Message-ID: <CAPJVwBnFu9_KzEZz9R-cwhK2=K_gx_prfu-t+VEAb=hMGXU0WQ@mail.gmail.com>

Hi all,

The new tracemalloc infrastructure in python 3.4 is super-interesting
to numerical folks, because we really like memory profiling. Numerical
programs allocate a lot of memory, and sometimes it's not clear which
operations allocate memory (some numpy operations return views of the
original array without allocating anything; others return copies). So
people actually use memory tracking tools[1], even though
traditionally these have been pretty hacky (i.e., just checking RSS
before and after each line is executed), and numpy has even grown its
own little tracemalloc-like infrastructure [2], but it only works for
numpy data.

BUT, we also really like calloc(). One of the basic array creation
routines in numpy is numpy.zeros(), which returns an array full of --
you guessed it -- zeros. For pretty much all the data types numpy
supports, the value zero is represented by the bytestring consisting
of all zeros. So numpy.zeros() usually uses calloc() to allocate its
memory.

calloc() is more awesome than malloc()+memset() for two reasons.
First, calloc() for larger allocations is usually implemented using
clever VM tricks, so that it doesn't actually allocate any memory up
front, it just creates a COW mapping of the system zero page and then
does the actual allocation one page at a time as different entries are
written to. This means that in the somewhat common case where you
allocate a large array full of zeros, and then only set a few
scattered entries to non-zero values, you can end up using much much
less memory than otherwise. It's entirely possible for this to make
the difference between being able to run an analysis versus not.
memset() forces the whole amount of RAM to be committed immediately.

Secondly, even if you *are* going to touch all the memory, then
calloc() is still faster than malloc()+memset(). The reason is that
for large allocations, malloc() usually does a calloc() no matter what
-- when you get a new page from the kernel, the kernel has to make
sure you can't see random bits of other processes's memory, so it
unconditionally zeros out the page before you get to see it. calloc()
knows this, so it doesn't bother zeroing it again. malloc()+memset(),
by contrast, zeros the page twice, producing twice as much memory
traffic, which is huge.

SO, we'd like to route our allocations through PyMem_* in order to let
tracemalloc "see" them, but because there is no PyMem_*Calloc, doing
this would force us to give up on the calloc() optimizations.

The obvious solution is to add a PyMem_*Calloc to the API. Would this
be possible? Unfortunately it would require adding a new field to the
PyMemAllocator struct, which would be an ABI/API break; PyMemAllocator
is exposed directly in the C API and passed by value:
  https://docs.python.org/3.5/c-api/memory.html#c.PyMemAllocator
(Too bad we didn't notice this a few months ago before 3.4 was
released :-(.) I guess we could just rename the struct in 3.5, to
force people to update their code. (I guess there aren't too many
people who would have to update their code.)

Thoughts?

-n

[1] http://scikit-learn.org/stable/developers/performance.html#memory-usage-profiling
[2] https://github.com/numpy/numpy/pull/309

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

From Steve.Dower at microsoft.com  Mon Apr 14 17:32:25 2014
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Mon, 14 Apr 2014 15:32:25 +0000
Subject: [Python-Dev] Python "2migr8"
Message-ID: <9f950fc6989849fbba2d604cdb7db0c2@BLUPR03MB389.namprd03.prod.outlook.com>


Just in case there's anyone out there who isn't yet sick of discussing how to proceed with Python 2.7, I have some more inputs to contribute.

To put it up front, I'm totally against "CPython 2.8" ever becoming a real thing. Anything that comes out should be seen as a migration path, not an upgrade path. I'll also admit I'm not heavily invested in working on it myself, but I had a number of conversations during PyCon (as well as being at the language summit) that puts me in a position to share the ideas and concerns that have been raised.

The main trigger was a conversation I had with two employees of a very large bank that has about 3000 Python users (not developers - mostly financial analysts) and 16 million lines of code running on 2.7. They are keen to migrate to 3.x but cannot afford to stop work entirely while their code is updated. (There was much more to the conversation than I'm relating here - I'm keeping to the directly relevant bits.)

In describing the approach they'd like to take, they made me realise that there is definitely a place for a Python that is different but mostly compatible with 2.7, in a way that 2.7.x could not be. For the sake of having a name, I'll refer to this as "Python 2migr8" (pronounced "to migrate" :) ).

The two important components of Python 2migr8 would be the ability to disable 2.7-only features, and to do so on a module-by-module basis.

My best idea so far would be to have a magic comment (to ensure 2.7 compatibility better than a "from __future__ ...") near the top of the file that marks that file as "must straddle 2.7 and 3.3". Adding this comment causes (for example) the parser to treat "except x, y" as a syntax error in this file, forces "from __future__ import ...", hides "dict.iterkeys", undefines "basestring", etc., but only for this file. (I haven't thought through all the possibilities or implications - Eric Snow said he was going to sprint on this today/tomorrow, so he'll soon have a better idea just what can be done.)

In effect, 2migr8 would be the version that *only* supports "single-source" files. This allows large code bases to progressively migrate modules from 2.x to single-source while continuing to run against Python 2.7. As files are updated, they are marked as being single-source. When all files have this marker, it should be possible to flip the switch and run with Python 3.3 or later.

You could also think of this as enabling "-3 --warnings-as-errors" for individual modules, though since the user has already opted in to 2migr8, it isn't unreasonable to make more significant changes, like having dict.keys returning a list that warns if it is mutated. This sort of warning can only really be done by changing the interpreter - static analysis just can't catch everything - and only when users accept a potential performance hit and low probability of breakage when they move to 2migr8 (followed by a not-quite-as-low probability of breaking when they eventually move from 2migr8 to 3.x, but it's still better than guaranteed breakage).

As a fork, it would also be possible to bundle the modules that have been backported, and possibly also to disallow importing deprecated stdlib modules when 2.7 functionality is disabled. As I said, I haven't thought through all the possibilities, but the general idea is to take 2.7 and *remove* features so it becomes easier to migrate.

Where does python-dev come in? Obviously this is where a fork like this would have to start - there has been such strong and public opposition to any significant changes like this that you'd be hard pressed to find someone willing to start and promote it from outside. There is also a good opportunity to make a start and directly invite those using it to contribute the rules or warnings that they need - the 3000 Python "users" I mentioned earlier are backed by a team of true developers who are more than capable of contributing, and this would be a great opportunity to directly invite them. However unfair and incorrect it may be, there is a perception in some businesses that open-source projects do not want contributions from them. I invited more than one business to have someone join python-dev and get involved during PyCon, and I heard that others did the same - it may not be at the level of employing a core developer full time, but it's the starting point that some companies will need to be able to become comfortable with employing a core dev.

I'm not pretending to have a full plan on how this will work. I was privileged to have some private conversations during PyCon that are directly relevant, so I'm bringing it here to promote the discussion. Thanks to everyone I had a chance to chat to, and to everyone generally for a great PyCon.

Cheers,
Steve

 

From benjamin at python.org  Mon Apr 14 17:36:55 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 14 Apr 2014 08:36:55 -0700
Subject: [Python-Dev] [numpy wishlist] PyMem_*Calloc
In-Reply-To: <CAPJVwBnFu9_KzEZz9R-cwhK2=K_gx_prfu-t+VEAb=hMGXU0WQ@mail.gmail.com>
References: <CAPJVwBnFu9_KzEZz9R-cwhK2=K_gx_prfu-t+VEAb=hMGXU0WQ@mail.gmail.com>
Message-ID: <1397489815.23597.106333761.0614A127@webmail.messagingengine.com>

On Sun, Apr 13, 2014, at 22:39, Nathaniel Smith wrote:
> Hi all,
> 
> The new tracemalloc infrastructure in python 3.4 is super-interesting
> to numerical folks, because we really like memory profiling. Numerical
> programs allocate a lot of memory, and sometimes it's not clear which
> operations allocate memory (some numpy operations return views of the
> original array without allocating anything; others return copies). So
> people actually use memory tracking tools[1], even though
> traditionally these have been pretty hacky (i.e., just checking RSS
> before and after each line is executed), and numpy has even grown its
> own little tracemalloc-like infrastructure [2], but it only works for
> numpy data.
> 
> BUT, we also really like calloc(). One of the basic array creation
> routines in numpy is numpy.zeros(), which returns an array full of --
> you guessed it -- zeros. For pretty much all the data types numpy
> supports, the value zero is represented by the bytestring consisting
> of all zeros. So numpy.zeros() usually uses calloc() to allocate its
> memory.
> 
> calloc() is more awesome than malloc()+memset() for two reasons.
> First, calloc() for larger allocations is usually implemented using
> clever VM tricks, so that it doesn't actually allocate any memory up
> front, it just creates a COW mapping of the system zero page and then
> does the actual allocation one page at a time as different entries are
> written to. This means that in the somewhat common case where you
> allocate a large array full of zeros, and then only set a few
> scattered entries to non-zero values, you can end up using much much
> less memory than otherwise. It's entirely possible for this to make
> the difference between being able to run an analysis versus not.
> memset() forces the whole amount of RAM to be committed immediately.
> 
> Secondly, even if you *are* going to touch all the memory, then
> calloc() is still faster than malloc()+memset(). The reason is that
> for large allocations, malloc() usually does a calloc() no matter what
> -- when you get a new page from the kernel, the kernel has to make
> sure you can't see random bits of other processes's memory, so it
> unconditionally zeros out the page before you get to see it. calloc()
> knows this, so it doesn't bother zeroing it again. malloc()+memset(),
> by contrast, zeros the page twice, producing twice as much memory
> traffic, which is huge.
> 
> SO, we'd like to route our allocations through PyMem_* in order to let
> tracemalloc "see" them, but because there is no PyMem_*Calloc, doing
> this would force us to give up on the calloc() optimizations.
> 
> The obvious solution is to add a PyMem_*Calloc to the API. Would this
> be possible? Unfortunately it would require adding a new field to the
> PyMemAllocator struct, which would be an ABI/API break; PyMemAllocator
> is exposed directly in the C API and passed by value:
>   https://docs.python.org/3.5/c-api/memory.html#c.PyMemAllocator
> (Too bad we didn't notice this a few months ago before 3.4 was
> released :-(.) I guess we could just rename the struct in 3.5, to
> force people to update their code. (I guess there aren't too many
> people who would have to update their code.)

Well, the allocator API is not part of the stable ABI, so we can change
it if we want.

> 
> Thoughts?

I think the request is completely reasonable.

From rosuav at gmail.com  Mon Apr 14 18:04:22 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 15 Apr 2014 02:04:22 +1000
Subject: [Python-Dev] Python "2migr8"
In-Reply-To: <9f950fc6989849fbba2d604cdb7db0c2@BLUPR03MB389.namprd03.prod.outlook.com>
References: <9f950fc6989849fbba2d604cdb7db0c2@BLUPR03MB389.namprd03.prod.outlook.com>
Message-ID: <CAPTjJmoHi53rQB7xexJEWyp6U6nCHXWmdjBiHnNJKWfJk9MRDw@mail.gmail.com>

On Tue, Apr 15, 2014 at 1:32 AM, Steve Dower <Steve.Dower at microsoft.com> wrote:
> My best idea so far would be to have a magic comment (to ensure 2.7 compatibility better than a "from __future__ ...") near the top of the file that marks that file as "must straddle 2.7 and 3.3". Adding this comment causes (for example) the parser to treat "except x, y" as a syntax error in this file, forces "from __future__ import ...", hides "dict.iterkeys", undefines "basestring", etc., but only for this file.
>

So if I understand you, this isn't really a "new version of Python" so
much as "Python 2.7 with checks for anything that would break on 3.3".
(BTW, any particular reason for targeting 3.3 specifically? PEP 461,
currently slated for 3.5, may make your job rather easier.) And the
file should run perfectly on both 2.7 and 3.3, and have the exact same
semantics. Is that correct?

If that's the case, how critical are false positives and false
negatives? You could make something that heuristically tries to figure
out when something could be a problem, and in dubious cases issues a
warning. That could afford to be a bit more aggressive in its warnings
than something that actually throws stuff out. You could even do a
whole lot of messy monkey-patching that checks stuff at run-time; if
performance becomes a problem, just switch back to 2.7 and it'll still
all run fine.

Ultimately, the best test of whether or not a piece of code runs in
2.7 and 3.3 is to simply run it in 2.7 and 3.3, but obviously that's
hard to do on a partial codebase. The trouble with a directive like
this is that it's hard to pin-point the source of a problem. For
instance, you say:

> ... it isn't unreasonable to make more significant changes, like having dict.keys returning a list that warns if it is mutated.
>

Suppose function foo calls function bar, passing it the .keys() of a
dict; function bar treats its argument as a list. Which one is at
fault? If they're in different modules and only one of them is marked
for migration mode, do you get a warning/error or not? Going for
warnings is the safe option here. You could warn for both half-cases
and error for the double. That is, dict.keys() returns a subclass of
list that, on mutation, will warn if the calling module is marked
migration-safe xor the module that originally created it is so marked,
and throws an error if both were marked. But for that to work, you
have to accept that there'll be both false positives (warnings issued
when the problem isn't in the migration-safe code) and false negatives
(warnings, rather than errors, when your code has a problem). Does
that kill the proposal's usefulness, or is that an acceptable cost?

ChrisA

From zachary.ware+pydev at gmail.com  Mon Apr 14 18:14:23 2014
From: zachary.ware+pydev at gmail.com (Zachary Ware)
Date: Mon, 14 Apr 2014 11:14:23 -0500
Subject: [Python-Dev] [Python-checkins] cpython (3.1): disallow a
	negative idx parameter
In-Reply-To: <3g6vTr1fhBz7LkJ@mail.python.org>
References: <3g6vTr1fhBz7LkJ@mail.python.org>
Message-ID: <CAKJDb-ONGDebUsdh3yM_LWwJJOPeK_w_tpp7V6QOWKV8Xh2=-w@mail.gmail.com>

On Mon, Apr 14, 2014 at 10:52 AM, benjamin.peterson
<python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/4bd1fb0f4f44
> changeset:   90256:4bd1fb0f4f44
> branch:      3.1
> parent:      90235:a8facac493ef
> user:        Benjamin Peterson <benjamin at python.org>
> date:        Mon Apr 14 11:45:21 2014 -0400
> summary:
>   disallow a negative idx parameter
>
> files:
>   Modules/_json.c |  9 +++++----
>   1 files changed, 5 insertions(+), 4 deletions(-)
>
>
> diff --git a/Modules/_json.c b/Modules/_json.c
> --- a/Modules/_json.c
> +++ b/Modules/_json.c
> @@ -902,10 +902,11 @@
>      PyObject *res;
>      Py_UNICODE *str = PyUnicode_AS_UNICODE(pystr);
>      Py_ssize_t length = PyUnicode_GET_SIZE(pystr);
> -    if (idx < 0)
> -        /* Compatibility with Python version. */
> -        idx += length;
> -    if (idx < 0 || idx >= length) {
> +    if (idx < 0) {
> +        PyErr_SetString(PyExc_ValueError, "idx canont be negative");

s/canont/cannot/

Also in the comparable 3.2 commit, but not the 3.3+ merge.

Regards,
-- 
Zach

From benjamin at python.org  Mon Apr 14 18:16:33 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 14 Apr 2014 09:16:33 -0700
Subject: [Python-Dev] [Python-checkins] cpython (3.1): disallow a
 negative idx parameter
In-Reply-To: <CAKJDb-ONGDebUsdh3yM_LWwJJOPeK_w_tpp7V6QOWKV8Xh2=-w@mail.gmail.com>
References: <3g6vTr1fhBz7LkJ@mail.python.org>
 <CAKJDb-ONGDebUsdh3yM_LWwJJOPeK_w_tpp7V6QOWKV8Xh2=-w@mail.gmail.com>
Message-ID: <1397492193.2024.106352309.33F18E5B@webmail.messagingengine.com>



On Mon, Apr 14, 2014, at 9:14, Zachary Ware wrote:
> On Mon, Apr 14, 2014 at 10:52 AM, benjamin.peterson
> <python-checkins at python.org> wrote:
> > http://hg.python.org/cpython/rev/4bd1fb0f4f44
> > changeset:   90256:4bd1fb0f4f44
> > branch:      3.1
> > parent:      90235:a8facac493ef
> > user:        Benjamin Peterson <benjamin at python.org>
> > date:        Mon Apr 14 11:45:21 2014 -0400
> > summary:
> >   disallow a negative idx parameter
> >
> > files:
> >   Modules/_json.c |  9 +++++----
> >   1 files changed, 5 insertions(+), 4 deletions(-)
> >
> >
> > diff --git a/Modules/_json.c b/Modules/_json.c
> > --- a/Modules/_json.c
> > +++ b/Modules/_json.c
> > @@ -902,10 +902,11 @@
> >      PyObject *res;
> >      Py_UNICODE *str = PyUnicode_AS_UNICODE(pystr);
> >      Py_ssize_t length = PyUnicode_GET_SIZE(pystr);
> > -    if (idx < 0)
> > -        /* Compatibility with Python version. */
> > -        idx += length;
> > -    if (idx < 0 || idx >= length) {
> > +    if (idx < 0) {
> > +        PyErr_SetString(PyExc_ValueError, "idx canont be negative");
> 
> s/canont/cannot/
> 
> Also in the comparable 3.2 commit, but not the 3.3+ merge.

Thanks

From guido at python.org  Mon Apr 14 19:19:10 2014
From: guido at python.org (Guido van Rossum)
Date: Mon, 14 Apr 2014 13:19:10 -0400
Subject: [Python-Dev] Python "2migr8"
In-Reply-To: <9f950fc6989849fbba2d604cdb7db0c2@BLUPR03MB389.namprd03.prod.outlook.com>
References: <9f950fc6989849fbba2d604cdb7db0c2@BLUPR03MB389.namprd03.prod.outlook.com>
Message-ID: <CAP7+vJKRpZ1KDxaVAFbJye++1CkZkbwp_pV43Rpje34uwuFf=w@mail.gmail.com>

Some quick thoughts:

- I'd prefer a name that plays on 2 and 3, not 2 and 8. :-)

- Are you sure this isn't better directed to python-ideas first? Most ideas
have to prove their worth in that list before python-dev will give them the
light of day.

- When it comes to purely syntactic issues (e.g. "except x, y:") a linter
or some other separate tool can handle this well (heck, you can build it
into an import hook probably :-).

- When it's about backported modules, a sumo distribution is probably the
way to go; when it's about renamed stdlib modules, six (perhaps an extended
version) should cover you.

- Regarding warning about the changed dict API, I wonder how you plan to
implement that if you allow passing dict object back and forth between code
that has opted in to single-source and code that hasn't yet. Please think
through some specific examples before responding.

- But the biggest issue is of course bytes vs. text. You would have to
first do a careful analysis of the *whole* problem before you can even
think about proposing a solution. Too many people think their is an easy
solution for this; but almost everybody is focused on only part of the
problem (the part that causes them immediate pain) without realizing that
other people's pain may be different.

- As far as your assertion that it would have to come from python-dev
because nobody outside is going to tackle it, I think it's the opposite:
the core developers would prefer not to have to deal with this, while some
folks outside the inner circles will not be discouraged by our opinions
(e.g. Stackless is working on "Stackless 2.8").

- Regarding open source projects having a reputation of "not taking
contributions", I would guess that this is usually about those
"contributions" violating the most basic rules of the project (and I don't
mean the coding style). I do want to discourage discussions with users like
the company you referred to, but I think it would be much more useful if
they laid out their problems for us instead of expecting they can buy
acceptance for a "solution" they develop in-house. We could then hopefully
have a productive dialog over many months where we iterate over possible
approaches that could be acceptable both to Python and to the customer. But
it will take a significant investment of time on both sides -- there is no
shortcut. And it's not a particularly interesting problem (for most people)
to work on -- things like designing a notation for optional type
declarations are always much more fun. :-)




On Mon, Apr 14, 2014 at 11:32 AM, Steve Dower <Steve.Dower at microsoft.com>wrote:

>
> Just in case there's anyone out there who isn't yet sick of discussing how
> to proceed with Python 2.7, I have some more inputs to contribute.
>
> To put it up front, I'm totally against "CPython 2.8" ever becoming a real
> thing. Anything that comes out should be seen as a migration path, not an
> upgrade path. I'll also admit I'm not heavily invested in working on it
> myself, but I had a number of conversations during PyCon (as well as being
> at the language summit) that puts me in a position to share the ideas and
> concerns that have been raised.
>
> The main trigger was a conversation I had with two employees of a very
> large bank that has about 3000 Python users (not developers - mostly
> financial analysts) and 16 million lines of code running on 2.7. They are
> keen to migrate to 3.x but cannot afford to stop work entirely while their
> code is updated. (There was much more to the conversation than I'm relating
> here - I'm keeping to the directly relevant bits.)
>
> In describing the approach they'd like to take, they made me realise that
> there is definitely a place for a Python that is different but mostly
> compatible with 2.7, in a way that 2.7.x could not be. For the sake of
> having a name, I'll refer to this as "Python 2migr8" (pronounced "to
> migrate" :) ).
>
> The two important components of Python 2migr8 would be the ability to
> disable 2.7-only features, and to do so on a module-by-module basis.
>
> My best idea so far would be to have a magic comment (to ensure 2.7
> compatibility better than a "from __future__ ...") near the top of the file
> that marks that file as "must straddle 2.7 and 3.3". Adding this comment
> causes (for example) the parser to treat "except x, y" as a syntax error in
> this file, forces "from __future__ import ...", hides "dict.iterkeys",
> undefines "basestring", etc., but only for this file. (I haven't thought
> through all the possibilities or implications - Eric Snow said he was going
> to sprint on this today/tomorrow, so he'll soon have a better idea just
> what can be done.)
>
> In effect, 2migr8 would be the version that *only* supports
> "single-source" files. This allows large code bases to progressively
> migrate modules from 2.x to single-source while continuing to run against
> Python 2.7. As files are updated, they are marked as being single-source.
> When all files have this marker, it should be possible to flip the switch
> and run with Python 3.3 or later.
>
> You could also think of this as enabling "-3 --warnings-as-errors" for
> individual modules, though since the user has already opted in to 2migr8,
> it isn't unreasonable to make more significant changes, like having
> dict.keys returning a list that warns if it is mutated. This sort of
> warning can only really be done by changing the interpreter - static
> analysis just can't catch everything - and only when users accept a
> potential performance hit and low probability of breakage when they move to
> 2migr8 (followed by a not-quite-as-low probability of breaking when they
> eventually move from 2migr8 to 3.x, but it's still better than guaranteed
> breakage).
>
> As a fork, it would also be possible to bundle the modules that have been
> backported, and possibly also to disallow importing deprecated stdlib
> modules when 2.7 functionality is disabled. As I said, I haven't thought
> through all the possibilities, but the general idea is to take 2.7 and
> *remove* features so it becomes easier to migrate.
>
> Where does python-dev come in? Obviously this is where a fork like this
> would have to start - there has been such strong and public opposition to
> any significant changes like this that you'd be hard pressed to find
> someone willing to start and promote it from outside. There is also a good
> opportunity to make a start and directly invite those using it to
> contribute the rules or warnings that they need - the 3000 Python "users" I
> mentioned earlier are backed by a team of true developers who are more than
> capable of contributing, and this would be a great opportunity to directly
> invite them. However unfair and incorrect it may be, there is a perception
> in some businesses that open-source projects do not want contributions from
> them. I invited more than one business to have someone join python-dev and
> get involved during PyCon, and I heard that others did the same - it may
> not be at the level of employing a core developer full time, but it's the
> starting point that some companies will ne
>  ed to be able to become comfortable with employing a core dev.
>
> I'm not pretending to have a full plan on how this will work. I was
> privileged to have some private conversations during PyCon that are
> directly relevant, so I'm bringing it here to promote the discussion.
> Thanks to everyone I had a chance to chat to, and to everyone generally for
> a great PyCon.
>
> Cheers,
> Steve
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140414/745de0df/attachment-0001.html>

From tjreedy at udel.edu  Mon Apr 14 20:41:09 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 14 Apr 2014 14:41:09 -0400
Subject: [Python-Dev] Python "2migr8"
In-Reply-To: <CAP7+vJKRpZ1KDxaVAFbJye++1CkZkbwp_pV43Rpje34uwuFf=w@mail.gmail.com>
References: <9f950fc6989849fbba2d604cdb7db0c2@BLUPR03MB389.namprd03.prod.outlook.com>
 <CAP7+vJKRpZ1KDxaVAFbJye++1CkZkbwp_pV43Rpje34uwuFf=w@mail.gmail.com>
Message-ID: <liha4p$ev0$1@ger.gmane.org>

On 4/14/2014 1:19 PM, Guido van Rossum wrote:
> Some quick thoughts:
>
> - I'd prefer a name that plays on 2 and 3, not 2 and 8. :-)
>
> - Are you sure this isn't better directed to python-ideas first? Most
> ideas have to prove their worth in that list before python-dev will give
> them the light of day.
>
> - When it comes to purely syntactic issues (e.g. "except x, y:") a
> linter or some other separate tool can handle this well (heck, you can
> build it into an import hook probably :-).
>
> - When it's about backported modules, a sumo distribution is probably
> the way to go; when it's about renamed stdlib modules, six (perhaps an
> extended version) should cover you.
>
> - Regarding warning about the changed dict API, I wonder how you plan to
> implement that if you allow passing dict object back and forth between
> code that has opted in to single-source and code that hasn't yet. Please
> think through some specific examples before responding.
>
> - But the biggest issue is of course bytes vs. text. You would have to
> first do a careful analysis of the *whole* problem before you can even
> think about proposing a solution. Too many people think their is an easy
> solution for this; but almost everybody is focused on only part of the
> problem (the part that causes them immediate pain) without realizing
> that other people's pain may be different.
>
> - As far as your assertion that it would have to come from python-dev
> because nobody outside is going to tackle it, I think it's the opposite:
> the core developers would prefer not to have to deal with this, while
> some folks outside the inner circles will not be discouraged by our
> opinions (e.g. Stackless is working on "Stackless 2.8").
>
> - Regarding open source projects having a reputation of "not taking
> contributions", I would guess that this is usually about those
> "contributions" violating the most basic rules of the project (and I
> don't mean the coding style). I do want to discourage discussions with

Did you mean 'don't want to discourage'?

> users like the company you referred to, but I think it would be much
> more useful if they laid out their problems for us instead of expecting
> they can buy acceptance for a "solution" they develop in-house. We could
> then hopefully have a productive dialog over many months where we
> iterate over possible approaches that could be acceptable both to Python
> and to the customer. But it will take a significant investment of time
> on both sides -- there is no shortcut. And it's not a particularly
> interesting problem (for most people) to work on -- things like
> designing a notation for optional type declarations are always much more
> fun. :-)
>
>
>
>
> On Mon, Apr 14, 2014 at 11:32 AM, Steve Dower <Steve.Dower at microsoft.com
> <mailto:Steve.Dower at microsoft.com>> wrote:
>
>
>     Just in case there's anyone out there who isn't yet sick of
>     discussing how to proceed with Python 2.7, I have some more inputs
>     to contribute.
>
>     To put it up front, I'm totally against "CPython 2.8" ever becoming
>     a real thing. Anything that comes out should be seen as a migration
>     path, not an upgrade path. I'll also admit I'm not heavily invested
>     in working on it myself, but I had a number of conversations during
>     PyCon (as well as being at the language summit) that puts me in a
>     position to share the ideas and concerns that have been raised.
>
>     The main trigger was a conversation I had with two employees of a
>     very large bank that has about 3000 Python users (not developers -
>     mostly financial analysts) and 16 million lines of code running on
>     2.7. They are keen to migrate to 3.x but cannot afford to stop work
>     entirely while their code is updated. (There was much more to the
>     conversation than I'm relating here - I'm keeping to the directly
>     relevant bits.)
>
>     In describing the approach they'd like to take, they made me realise
>     that there is definitely a place for a Python that is different but
>     mostly compatible with 2.7, in a way that 2.7.x could not be. For
>     the sake of having a name, I'll refer to this as "Python 2migr8"
>     (pronounced "to migrate" :) ).
>
>     The two important components of Python 2migr8 would be the ability
>     to disable 2.7-only features, and to do so on a module-by-module basis.
>
>     My best idea so far would be to have a magic comment (to ensure 2.7
>     compatibility better than a "from __future__ ...") near the top of
>     the file that marks that file as "must straddle 2.7 and 3.3". Adding
>     this comment causes (for example) the parser to treat "except x, y"
>     as a syntax error in this file, forces "from __future__ import ...",
>     hides "dict.iterkeys", undefines "basestring", etc., but only for
>     this file. (I haven't thought through all the possibilities or
>     implications - Eric Snow said he was going to sprint on this
>     today/tomorrow, so he'll soon have a better idea just what can be done.)
>
>     In effect, 2migr8 would be the version that *only* supports
>     "single-source" files. This allows large code bases to progressively
>     migrate modules from 2.x to single-source while continuing to run
>     against Python 2.7. As files are updated, they are marked as being
>     single-source. When all files have this marker, it should be
>     possible to flip the switch and run with Python 3.3 or later.
>
>     You could also think of this as enabling "-3 --warnings-as-errors"
>     for individual modules, though since the user has already opted in
>     to 2migr8, it isn't unreasonable to make more significant changes,
>     like having dict.keys returning a list that warns if it is mutated.
>     This sort of warning can only really be done by changing the
>     interpreter - static analysis just can't catch everything - and only
>     when users accept a potential performance hit and low probability of
>     breakage when they move to 2migr8 (followed by a not-quite-as-low
>     probability of breaking when they eventually move from 2migr8 to
>     3.x, but it's still better than guaranteed breakage).
>
>     As a fork, it would also be possible to bundle the modules that have
>     been backported, and possibly also to disallow importing deprecated
>     stdlib modules when 2.7 functionality is disabled. As I said, I
>     haven't thought through all the possibilities, but the general idea
>     is to take 2.7 and *remove* features so it becomes easier to migrate.
>
>     Where does python-dev come in? Obviously this is where a fork like
>     this would have to start - there has been such strong and public
>     opposition to any significant changes like this that you'd be hard
>     pressed to find someone willing to start and promote it from
>     outside. There is also a good opportunity to make a start and
>     directly invite those using it to contribute the rules or warnings
>     that they need - the 3000 Python "users" I mentioned earlier are
>     backed by a team of true developers who are more than capable of
>     contributing, and this would be a great opportunity to directly
>     invite them. However unfair and incorrect it may be, there is a
>     perception in some businesses that open-source projects do not want
>     contributions from them. I invited more than one business to have
>     someone join python-dev and get involved during PyCon, and I heard
>     that others did the same - it may not be at the level of employing a
>     core developer full time, but it's the starting point that some
>     companies will ne
>       ed to be able to become comfortable with employing a core dev.
>
>     I'm not pretending to have a full plan on how this will work. I was
>     privileged to have some private conversations during PyCon that are
>     directly relevant, so I'm bringing it here to promote the
>     discussion. Thanks to everyone I had a chance to chat to, and to
>     everyone generally for a great PyCon.
>
>     Cheers,
>     Steve
>
>
>     _______________________________________________
>     Python-Dev mailing list
>     Python-Dev at python.org <mailto:Python-Dev at python.org>
>     https://mail.python.org/mailman/listinfo/python-dev
>     Unsubscribe:
>     https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>
>
>
> --
> --Guido van Rossum (python.org/~guido <http://python.org/~guido>)
>
>


-- 
Terry Jan Reedy


From tjreedy at udel.edu  Mon Apr 14 21:53:12 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 14 Apr 2014 15:53:12 -0400
Subject: [Python-Dev] Python "2migr8"
In-Reply-To: <9f950fc6989849fbba2d604cdb7db0c2@BLUPR03MB389.namprd03.prod.outlook.com>
References: <9f950fc6989849fbba2d604cdb7db0c2@BLUPR03MB389.namprd03.prod.outlook.com>
Message-ID: <lihebs$m8l$1@ger.gmane.org>

On 4/14/2014 11:32 AM, Steve Dower wrote:

> To put it up front, I'm totally against "CPython 2.8" ever becoming a
> real thing. Anything that comes out should be seen as a migration
> path, not an upgrade path. I'll also admit I'm not heavily invested
> in working on it myself, but I had a number of conversations during
> PyCon (as well as being at the language summit) that puts me in a
> position to share the ideas and concerns that have been raised.

I think it great that you 'volunteered' to be a neutral, hopefully 
trusted go-between.

> The main trigger was a conversation I had with two employees of a
> very large bank that has about 3000 Python users (not developers -
> mostly financial analysts) and 16 million lines of code running on
> 2.7.

Sounds like a billion-dollar company. Are they a PSF sponsor, and a 
top-tier one at that? If the company is profitable, it could afford to 
fund a half- to full-time developer.

 > They are keen to migrate to 3.x but cannot afford to stop work
> entirely while their code is updated.

Sounds like they are looking ahead several years and anxious to avoid 
the 'comforable with XP' trap.

> In describing the approach they'd like to take, they made me realise
> that there is definitely a place for a Python that is different but
> mostly compatible with 2.7, in a way that 2.7.x could not be. For the
> sake of having a name, I'll refer to this as "Python 2migr8"
> (pronounced "to migrate" :) ).

This should be a separate project from pydev, even if under the PSF 
umbrella.

> The two important components of Python 2migr8 would be the ability to
> disable 2.7-only features, and to do so on a module-by-module basis.

A reasonable request of pydev would be for python-coded stdlib modules 
to be updated as much as possible, if that has not already been done. No 
'apply', no 'except SomeException, e'.

> However unfair
> and incorrect it may be, there is a perception in some businesses
> that open-source projects do not want contributions from them.

For PSF/CPython, this is so untrue that it looks to me like an excuse to 
take without giving back. This might be 'unfair and incorrect', but it 
is my perception.

> I invited more than one business to have someone join python -dev and
> get involved during PyCon, and I heard that others did the same - it
> may not be at the level of employing a core developer full time, but
> it's the starting point that some companies will need to be able to
> become comfortable with employing a core dev.

Let's hope some act on your invitation.

-- 
Terry Jan Reedy


From donald at stufft.io  Mon Apr 14 22:02:05 2014
From: donald at stufft.io (Donald Stufft)
Date: Mon, 14 Apr 2014 16:02:05 -0400
Subject: [Python-Dev] Python "2migr8"
In-Reply-To: <lihebs$m8l$1@ger.gmane.org>
References: <9f950fc6989849fbba2d604cdb7db0c2@BLUPR03MB389.namprd03.prod.outlook.com>
 <lihebs$m8l$1@ger.gmane.org>
Message-ID: <B4E5EB00-93AD-4867-B055-4CD7D9231F18@stufft.io>


On Apr 14, 2014, at 3:53 PM, Terry Reedy <tjreedy at udel.edu> wrote:

> On 4/14/2014 11:32 AM, Steve Dower wrote:
> 
>> To put it up front, I'm totally against "CPython 2.8" ever becoming a
>> real thing. Anything that comes out should be seen as a migration
>> path, not an upgrade path. I'll also admit I'm not heavily invested
>> in working on it myself, but I had a number of conversations during
>> PyCon (as well as being at the language summit) that puts me in a
>> position to share the ideas and concerns that have been raised.
> 
> I think it great that you 'volunteered' to be a neutral, hopefully trusted go-between.
> 
>> The main trigger was a conversation I had with two employees of a
>> very large bank that has about 3000 Python users (not developers -
>> mostly financial analysts) and 16 million lines of code running on
>> 2.7.
> 
> Sounds like a billion-dollar company. Are they a PSF sponsor, and a top-tier one at that? If the company is profitable, it could afford to fund a half- to full-time developer.
> 
> > They are keen to migrate to 3.x but cannot afford to stop work
>> entirely while their code is updated.
> 
> Sounds like they are looking ahead several years and anxious to avoid the 'comforable with XP' trap.
> 
>> In describing the approach they'd like to take, they made me realise
>> that there is definitely a place for a Python that is different but
>> mostly compatible with 2.7, in a way that 2.7.x could not be. For the
>> sake of having a name, I'll refer to this as "Python 2migr8"
>> (pronounced "to migrate" :) ).
> 
> This should be a separate project from pydev, even if under the PSF umbrella.
> 
>> The two important components of Python 2migr8 would be the ability to
>> disable 2.7-only features, and to do so on a module-by-module basis.
> 
> A reasonable request of pydev would be for python-coded stdlib modules to be updated as much as possible, if that has not already been done. No 'apply', no 'except SomeException, e'.
> 
>> However unfair
>> and incorrect it may be, there is a perception in some businesses
>> that open-source projects do not want contributions from them.
> 
> For PSF/CPython, this is so untrue that it looks to me like an excuse to take without giving back. This might be 'unfair and incorrect', but it is my perception.

As someone who *has* given back, I can certainly understand why someone would feel that way. It often times *does* feel like CPython doesn?t want contributions.

> 
>> I invited more than one business to have someone join python -dev and
>> get involved during PyCon, and I heard that others did the same - it
>> may not be at the level of employing a core developer full time, but
>> it's the starting point that some companies will need to be able to
>> become comfortable with employing a core dev.
> 
> Let's hope some act on your invitation.
> 
> -- 
> Terry Jan Reedy
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/donald%40stufft.io


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140414/0aee72e5/attachment.sig>

From guido at python.org  Mon Apr 14 22:26:45 2014
From: guido at python.org (Guido van Rossum)
Date: Mon, 14 Apr 2014 16:26:45 -0400
Subject: [Python-Dev] Python "2migr8"
In-Reply-To: <liha4p$ev0$1@ger.gmane.org>
References: <9f950fc6989849fbba2d604cdb7db0c2@BLUPR03MB389.namprd03.prod.outlook.com>
 <CAP7+vJKRpZ1KDxaVAFbJye++1CkZkbwp_pV43Rpje34uwuFf=w@mail.gmail.com>
 <liha4p$ev0$1@ger.gmane.org>
Message-ID: <CAP7+vJLOEeHc8X741OfFsEJBymYrT9RU1O4Yo0Jz81BXadaxqQ@mail.gmail.com>

On Apr 14, 2014 2:42 PM, "Terry Reedy" <tjreedy at udel.edu> wrote:
>
> On 4/14/2014 1:19 PM, Guido van Rossum wrote:
>>
>> Some quick thoughts:
>>
>> - I'd prefer a name that plays on 2 and 3, not 2 and 8. :-)
>>
>> - Are you sure this isn't better directed to python-ideas first? Most
>> ideas have to prove their worth in that list before python-dev will give
>> them the light of day.
>>
>> - When it comes to purely syntactic issues (e.g. "except x, y:") a
>> linter or some other separate tool can handle this well (heck, you can
>> build it into an import hook probably :-).
>>
>> - When it's about backported modules, a sumo distribution is probably
>> the way to go; when it's about renamed stdlib modules, six (perhaps an
>> extended version) should cover you.
>>
>> - Regarding warning about the changed dict API, I wonder how you plan to
>> implement that if you allow passing dict object back and forth between
>> code that has opted in to single-source and code that hasn't yet. Please
>> think through some specific examples before responding.
>>
>> - But the biggest issue is of course bytes vs. text. You would have to
>> first do a careful analysis of the *whole* problem before you can even
>> think about proposing a solution. Too many people think their is an easy
>> solution for this; but almost everybody is focused on only part of the
>> problem (the part that causes them immediate pain) without realizing
>> that other people's pain may be different.
>>
>> - As far as your assertion that it would have to come from python-dev
>> because nobody outside is going to tackle it, I think it's the opposite:
>> the core developers would prefer not to have to deal with this, while
>> some folks outside the inner circles will not be discouraged by our
>> opinions (e.g. Stackless is working on "Stackless 2.8").
>>
>> - Regarding open source projects having a reputation of "not taking
>> contributions", I would guess that this is usually about those
>> "contributions" violating the most basic rules of the project (and I
>> don't mean the coding style). I do want to discourage discussions with
>
>
> Did you mean 'don't want to discourage'?

Yes. Sorry. Pylon brain fry...

>> users like the company you referred to, but I think it would be much
>> more useful if they laid out their problems for us instead of expecting
>> they can buy acceptance for a "solution" they develop in-house. We could
>> then hopefully have a productive dialog over many months where we
>> iterate over possible approaches that could be acceptable both to Python
>> and to the customer. But it will take a significant investment of time
>> on both sides -- there is no shortcut. And it's not a particularly
>> interesting problem (for most people) to work on -- things like
>> designing a notation for optional type declarations are always much more
>> fun. :-)
>>
>>
>>
>>
>> On Mon, Apr 14, 2014 at 11:32 AM, Steve Dower <Steve.Dower at microsoft.com
>> <mailto:Steve.Dower at microsoft.com>> wrote:
>>
>>
>>     Just in case there's anyone out there who isn't yet sick of
>>     discussing how to proceed with Python 2.7, I have some more inputs
>>     to contribute.
>>
>>     To put it up front, I'm totally against "CPython 2.8" ever becoming
>>     a real thing. Anything that comes out should be seen as a migration
>>     path, not an upgrade path. I'll also admit I'm not heavily invested
>>     in working on it myself, but I had a number of conversations during
>>     PyCon (as well as being at the language summit) that puts me in a
>>     position to share the ideas and concerns that have been raised.
>>
>>     The main trigger was a conversation I had with two employees of a
>>     very large bank that has about 3000 Python users (not developers -
>>     mostly financial analysts) and 16 million lines of code running on
>>     2.7. They are keen to migrate to 3.x but cannot afford to stop work
>>     entirely while their code is updated. (There was much more to the
>>     conversation than I'm relating here - I'm keeping to the directly
>>     relevant bits.)
>>
>>     In describing the approach they'd like to take, they made me realise
>>     that there is definitely a place for a Python that is different but
>>     mostly compatible with 2.7, in a way that 2.7.x could not be. For
>>     the sake of having a name, I'll refer to this as "Python 2migr8"
>>     (pronounced "to migrate" :) ).
>>
>>     The two important components of Python 2migr8 would be the ability
>>     to disable 2.7-only features, and to do so on a module-by-module
basis.
>>
>>     My best idea so far would be to have a magic comment (to ensure 2.7
>>     compatibility better than a "from __future__ ...") near the top of
>>     the file that marks that file as "must straddle 2.7 and 3.3". Adding
>>     this comment causes (for example) the parser to treat "except x, y"
>>     as a syntax error in this file, forces "from __future__ import ...",
>>     hides "dict.iterkeys", undefines "basestring", etc., but only for
>>     this file. (I haven't thought through all the possibilities or
>>     implications - Eric Snow said he was going to sprint on this
>>     today/tomorrow, so he'll soon have a better idea just what can be
done.)
>>
>>     In effect, 2migr8 would be the version that *only* supports
>>     "single-source" files. This allows large code bases to progressively
>>     migrate modules from 2.x to single-source while continuing to run
>>     against Python 2.7. As files are updated, they are marked as being
>>     single-source. When all files have this marker, it should be
>>     possible to flip the switch and run with Python 3.3 or later.
>>
>>     You could also think of this as enabling "-3 --warnings-as-errors"
>>     for individual modules, though since the user has already opted in
>>     to 2migr8, it isn't unreasonable to make more significant changes,
>>     like having dict.keys returning a list that warns if it is mutated.
>>     This sort of warning can only really be done by changing the
>>     interpreter - static analysis just can't catch everything - and only
>>     when users accept a potential performance hit and low probability of
>>     breakage when they move to 2migr8 (followed by a not-quite-as-low
>>     probability of breaking when they eventually move from 2migr8 to
>>     3.x, but it's still better than guaranteed breakage).
>>
>>     As a fork, it would also be possible to bundle the modules that have
>>     been backported, and possibly also to disallow importing deprecated
>>     stdlib modules when 2.7 functionality is disabled. As I said, I
>>     haven't thought through all the possibilities, but the general idea
>>     is to take 2.7 and *remove* features so it becomes easier to migrate.
>>
>>     Where does python-dev come in? Obviously this is where a fork like
>>     this would have to start - there has been such strong and public
>>     opposition to any significant changes like this that you'd be hard
>>     pressed to find someone willing to start and promote it from
>>     outside. There is also a good opportunity to make a start and
>>     directly invite those using it to contribute the rules or warnings
>>     that they need - the 3000 Python "users" I mentioned earlier are
>>     backed by a team of true developers who are more than capable of
>>     contributing, and this would be a great opportunity to directly
>>     invite them. However unfair and incorrect it may be, there is a
>>     perception in some businesses that open-source projects do not want
>>     contributions from them. I invited more than one business to have
>>     someone join python-dev and get involved during PyCon, and I heard
>>     that others did the same - it may not be at the level of employing a
>>     core developer full time, but it's the starting point that some
>>     companies will ne
>>       ed to be able to become comfortable with employing a core dev.
>>
>>     I'm not pretending to have a full plan on how this will work. I was
>>     privileged to have some private conversations during PyCon that are
>>     directly relevant, so I'm bringing it here to promote the
>>     discussion. Thanks to everyone I had a chance to chat to, and to
>>     everyone generally for a great PyCon.
>>
>>     Cheers,
>>     Steve
>>
>>
>>     _______________________________________________
>>     Python-Dev mailing list
>>     Python-Dev at python.org <mailto:Python-Dev at python.org>
>>
>>     https://mail.python.org/mailman/listinfo/python-dev
>>     Unsubscribe:
>>     https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>>
>>
>>
>> --
>> --Guido van Rossum (python.org/~guido <http://python.org/~guido>)
>>
>>
>
>
> --
> Terry Jan Reedy
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/guido%40python.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140414/7cc30db0/attachment.html>

From ethan at stoneleaf.us  Mon Apr 14 22:10:33 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 14 Apr 2014 13:10:33 -0700
Subject: [Python-Dev] [numpy wishlist] PyMem_*Calloc
In-Reply-To: <1397489815.23597.106333761.0614A127@webmail.messagingengine.com>
References: <CAPJVwBnFu9_KzEZz9R-cwhK2=K_gx_prfu-t+VEAb=hMGXU0WQ@mail.gmail.com>
 <1397489815.23597.106333761.0614A127@webmail.messagingengine.com>
Message-ID: <534C40B9.2090303@stoneleaf.us>

On 04/14/2014 08:36 AM, Benjamin Peterson wrote:
> On Sun, Apr 13, 2014, at 22:39, Nathaniel Smith wrote:
>>
>> SO, we'd like to route our allocations through PyMem_* in order to let
>> tracemalloc "see" them, but because there is no PyMem_*Calloc, doing
>> this would force us to give up on the calloc() optimizations.
>
> Well, the allocator API is not part of the stable ABI, so we can change
> it if we want.


>> Thoughts?
>
> I think the request is completely reasonable.

+1

--
~Ethan~

From guido at python.org  Mon Apr 14 22:39:33 2014
From: guido at python.org (Guido van Rossum)
Date: Mon, 14 Apr 2014 16:39:33 -0400
Subject: [Python-Dev] Python "2migr8"
In-Reply-To: <B4E5EB00-93AD-4867-B055-4CD7D9231F18@stufft.io>
References: <9f950fc6989849fbba2d604cdb7db0c2@BLUPR03MB389.namprd03.prod.outlook.com>
 <lihebs$m8l$1@ger.gmane.org> <B4E5EB00-93AD-4867-B055-4CD7D9231F18@stufft.io>
Message-ID: <CAP7+vJ+ZjNMQ9C+TTxbqnrznnR+UmXaVQ8EdjgtoftAcL_Kkng@mail.gmail.com>

On Mon, Apr 14, 2014 at 4:02 PM, Donald Stufft <donald at stufft.io> wrote:

>
> On Apr 14, 2014, at 3:53 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>
> > On 4/14/2014 11:32 AM, Steve Dower wrote:
> [...]
> >> However unfair
> >> and incorrect it may be, there is a perception in some businesses
> >> that open-source projects do not want contributions from them.
>
> > For PSF/CPython, this is so untrue that it looks to me like an excuse to
> take without giving back. This might be 'unfair and incorrect', but it is
> my perception.
>
> As someone who *has* given back, I can certainly understand why someone
> would feel that way. It often times *does* feel like CPython doesn?t want
> contributions.
>

Donald, your remark in itself sounds unnecessarily (and unproductively!)
passive-aggressive. What have we done wrong to you, and what can we do to
avoid making the same mistake in the future (to you, and to others)?

[PS. When I appeared to write "Pylon brain fry" earlier in this thread,
that was an unfortunate auto-correct for "PyCon brain fry". We need to get
"PyCon" into the dictionary...]

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140414/411a22c2/attachment.html>

From guido at python.org  Mon Apr 14 23:00:22 2014
From: guido at python.org (Guido van Rossum)
Date: Mon, 14 Apr 2014 17:00:22 -0400
Subject: [Python-Dev] Python "2migr8"
In-Reply-To: <lihebs$m8l$1@ger.gmane.org>
References: <9f950fc6989849fbba2d604cdb7db0c2@BLUPR03MB389.namprd03.prod.outlook.com>
 <lihebs$m8l$1@ger.gmane.org>
Message-ID: <CAP7+vJKNcmS70RsR8CpJmYGLUyCU01A6WziSJkGYA9YdQGyxFg@mail.gmail.com>

On Mon, Apr 14, 2014 at 3:53 PM, Terry Reedy <tjreedy at udel.edu> wrote:

> On 4/14/2014 11:32 AM, Steve Dower wrote:
>
[...]
>
 The main trigger was a conversation I had with two employees of a
>> very large bank that has about 3000 Python users (not developers -
>> mostly financial analysts) and 16 million lines of code running on
>> 2.7.
>>
>
> Sounds like a billion-dollar company. Are they a PSF sponsor, and a
> top-tier one at that? If the company is profitable, it could afford to fund
> a half- to full-time developer.


A few people have made similar suggestions to me at the conference, but I
personally believe that there is a better way.

I don't think we ought to make companies feel bad about not donating to the
PSF. The PSF is doing fine, but IMO it shouldn't be in the business of
employing core developers. Being an employer is fraught with difficulties,
and there are serious risks both for the PSF (due to the rigidity of
employment laws, for example) and for the employee (e.g. benefits, worry
about continuity, oversight and direction).

IMO a much better approach would be to convince companies to free up some
of their current employees or new hires to invest e.g. 50% of their time
into core Python development. Such "liaison" folks can become a valuable
resource both for the other Python users in the company and for the core
Python developers (by telling us about pain points in their company). In
the ideal situation such people will help search for solutions that benefit
their employers while also conforming to (and sometimes influencing! :-)
the general direction of Python core development. The (roughly) 50/50
division ensures that such employees are rooted in the practices of their
company and feel the pain of other Python users directly. It also makes
sure that they won't be considered as having too much of a "special status"
by their colleagues and managers, can share freely in the companies
benefits and incentives, and so on.

(I should say that this is my own situation at Dropbox and previously at
Google, and I personally wouldn't want it any other way. I know not
everybody wants to be a "regular employee", and some folks think of
themselves as "free spirits" who don't want to be "working for the man".
There's plenty of space for such folks in the Python core dev community,
but I think that the kind of company that Steve Dower is speaking of would
not be helped as much by hiring such a "consultant".)


> > They are keen to migrate to 3.x but cannot afford to stop work
>
>> entirely while their code is updated.
>>
>
> Sounds like they are looking ahead several years and anxious to avoid the
> 'comforable with XP' trap.


Ohhh, nice analogy!


>  In describing the approach they'd like to take, they made me realise
>> that there is definitely a place for a Python that is different but
>> mostly compatible with 2.7, in a way that 2.7.x could not be. For the
>> sake of having a name, I'll refer to this as "Python 2migr8"
>> (pronounced "to migrate" :) ).
>>
>
> This should be a separate project from pydev, even if under the PSF
> umbrella.
>
>
>  The two important components of Python 2migr8 would be the ability to
>> disable 2.7-only features, and to do so on a module-by-module basis.
>>
>
> A reasonable request of pydev would be for python-coded stdlib modules to
> be updated as much as possible, if that has not already been done. No
> 'apply', no 'except SomeException, e'.


I'm not sure what you're proposing here, but I don't think it would be wise
to go on a code modernizing spree of the 2.7 stdlib. Contrary to what some
believe, the stdlib often isn't a great example, because it lives under the
pressure of quite unusual constraints.


>
>
>  However unfair
>> and incorrect it may be, there is a perception in some businesses
>> that open-source projects do not want contributions from them.
>>
>
> For PSF/CPython, this is so untrue that it looks to me like an excuse to
> take without giving back. This might be 'unfair and incorrect', but it is
> my perception.
>
>  I invited more than one business to have someone join python -dev and
>>
>> get involved during PyCon, and I heard that others did the same - it
>> may not be at the level of employing a core developer full time, but
>> it's the starting point that some companies will need to be able to
>>
>> become comfortable with employing a core dev.
>>
>
> Let's hope some act on your invitation.


I really hope to have a direct conversation with some companies in this
situation, but unfortunately they didn't approach me at PyCon -- they only
approached Steve (perhaps because he works for a brand they recognize and
trust :-).

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140414/144aedeb/attachment.html>

From donald at stufft.io  Mon Apr 14 23:16:10 2014
From: donald at stufft.io (Donald Stufft)
Date: Mon, 14 Apr 2014 17:16:10 -0400
Subject: [Python-Dev] Python "2migr8"
In-Reply-To: <CAP7+vJ+ZjNMQ9C+TTxbqnrznnR+UmXaVQ8EdjgtoftAcL_Kkng@mail.gmail.com>
References: <9f950fc6989849fbba2d604cdb7db0c2@BLUPR03MB389.namprd03.prod.outlook.com>
 <lihebs$m8l$1@ger.gmane.org> <B4E5EB00-93AD-4867-B055-4CD7D9231F18@stufft.io>
 <CAP7+vJ+ZjNMQ9C+TTxbqnrznnR+UmXaVQ8EdjgtoftAcL_Kkng@mail.gmail.com>
Message-ID: <E53C833C-8FC8-43FA-B881-A0828BF43FDC@stufft.io>


On Apr 14, 2014, at 4:39 PM, Guido van Rossum <guido at python.org> wrote:

> On Mon, Apr 14, 2014 at 4:02 PM, Donald Stufft <donald at stufft.io> wrote:
> 
> On Apr 14, 2014, at 3:53 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> 
> > On 4/14/2014 11:32 AM, Steve Dower wrote:
> [...]
> >> However unfair
> >> and incorrect it may be, there is a perception in some businesses
> >> that open-source projects do not want contributions from them.
> 
> > For PSF/CPython, this is so untrue that it looks to me like an excuse to take without giving back. This might be 'unfair and incorrect', but it is my perception.
> 
> As someone who *has* given back, I can certainly understand why someone would feel that way. It often times *does* feel like CPython doesn?t want contributions.
>  
> Donald, your remark in itself sounds unnecessarily (and unproductively!) passive-aggressive. What have we done wrong to you, and what can we do to avoid making the same mistake in the future (to you, and to others)?
> 
> [PS. When I appeared to write "Pylon brain fry" earlier in this thread, that was an unfortunate auto-correct for "PyCon brain fry". We need to get "PyCon" into the dictionary...]
> 
> -- 
> --Guido van Rossum (python.org/~guido)


Hmm, I?m sorry if I came across that way. I didn?t mean to. I do think contributing directly to CPython is often times off-putting to people (and given I know others who have privately expressed the same sentiment to me I don?t think I?m alone in that [1]). I only brought it up because I don?t think ignoring a problem (which maybe some disagree that there is a problem! but in my view there is so thus my comment) is a useful thing to do. Generally when I say something it?s because I?m trying to be helpful :) I?m sorry that my wording didn?t convey that appropriately.

Now I will admit I personally have probably had a harder time than some others because of the nature of the things I was trying to work on, and lately it?s gotten better (although I think that?s partially because I?m more known now, and I think in general the experience of contributing to CPython changes depending on who you are, so the more integrated into the culture you are, the less likely you are to see the issues and unfortunately those people are also the ones with the most power to change it). I do however think that just in general it might be getting better too? 

Specific details are hard because it?s nothing major and obvious like having Linus go off on rants and tearing things apart, it?s death by a thousand cuts so it?s hard to point a finger at one behavior (or a few behaviors) and look at them in isolation and ?see? it. That being said I?m more than happy to *try* and explain it, but right this moment I don?t have a lot of time as I?m getting ready to step out the door, but I didn?t want to leave this email hanging without a reply.

[1] See Also http://www.curiousefficiency.org/posts/2011/04/musings-on-culture-of-python-dev.html

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140414/7e43444a/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140414/7e43444a/attachment.sig>

From chris.barker at noaa.gov  Mon Apr 14 23:17:54 2014
From: chris.barker at noaa.gov (Chris Barker)
Date: Mon, 14 Apr 2014 14:17:54 -0700
Subject: [Python-Dev] Python "2migr8"
In-Reply-To: <CAP7+vJLOEeHc8X741OfFsEJBymYrT9RU1O4Yo0Jz81BXadaxqQ@mail.gmail.com>
References: <9f950fc6989849fbba2d604cdb7db0c2@BLUPR03MB389.namprd03.prod.outlook.com>
 <CAP7+vJKRpZ1KDxaVAFbJye++1CkZkbwp_pV43Rpje34uwuFf=w@mail.gmail.com>
 <liha4p$ev0$1@ger.gmane.org>
 <CAP7+vJLOEeHc8X741OfFsEJBymYrT9RU1O4Yo0Jz81BXadaxqQ@mail.gmail.com>
Message-ID: <CALGmxEJL3qA=Km+=d5uW7XTPKr_82BG9cw-3RbDpKEf-psJWJw@mail.gmail.com>

On Mon, Apr 14, 2014 at 1:26 PM, Guido van Rossum <guido at python.org> wrote:


> >> - I'd prefer a name that plays on 2 and 3, not 2 and 8. :-)
>
How about mirg2**3 (pronounced "migrate") ?

;-)

-Chris



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140414/5581852f/attachment.html>

From breamoreboy at yahoo.co.uk  Mon Apr 14 23:41:14 2014
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 14 Apr 2014 22:41:14 +0100
Subject: [Python-Dev] Python "2migr8"
In-Reply-To: <CALGmxEJL3qA=Km+=d5uW7XTPKr_82BG9cw-3RbDpKEf-psJWJw@mail.gmail.com>
References: <9f950fc6989849fbba2d604cdb7db0c2@BLUPR03MB389.namprd03.prod.outlook.com>
 <CAP7+vJKRpZ1KDxaVAFbJye++1CkZkbwp_pV43Rpje34uwuFf=w@mail.gmail.com>
 <liha4p$ev0$1@ger.gmane.org>
 <CAP7+vJLOEeHc8X741OfFsEJBymYrT9RU1O4Yo0Jz81BXadaxqQ@mail.gmail.com>
 <CALGmxEJL3qA=Km+=d5uW7XTPKr_82BG9cw-3RbDpKEf-psJWJw@mail.gmail.com>
Message-ID: <lihkl5$pqq$1@ger.gmane.org>

On 14/04/2014 22:17, Chris Barker wrote:
> On Mon, Apr 14, 2014 at 1:26 PM, Guido van Rossum <guido at python.org
> <mailto:guido at python.org>> wrote:
>
>      >> - I'd prefer a name that plays on 2 and 3, not 2 and 8. :-)
>
> How about mirg2**3 (pronounced "migrate") ?
>
> ;-)
>
> -Chris
>
>

I agree with the grate part.  Failing that, how about a play on 2 + 8 = 
ache?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.

Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com



From bcannon at gmail.com  Mon Apr 14 23:51:23 2014
From: bcannon at gmail.com (Brett Cannon)
Date: Mon, 14 Apr 2014 17:51:23 -0400
Subject: [Python-Dev] this is what happens if you freeze all the modules
	required for startup
Message-ID: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>

It was realized during PyCon that since we are freezing importlib we could
now consider freezing all the modules to cut out having to stat or read
them from disk. So for day 1 of the sprints I I decided to hack up a
proof-of-concept to see what kind of performance gain it would get.

Freezing everything except encodings.__init__, os, and _sysconfigdata, it
speeds up startup by 11% compared to default. Compared to 2.7 it shaves 14%
from the slowdown (27% slower vs. 41% slower). The full results are at the
end of the email.

Now the question is whether the maintenance cost of having to rebuild
Python for a select number of stdlib modules is enough to warrant putting
in the effort to make this work. My guess is the best approach would be
adding a Lib/_frozen directory where any modules that we treat like this
would be kept to act as a reminder that you need to rebuild for them (I
would probably move importlib/_boostrap.py as well to make this consistent).

Thoughts?


--------------------------------------

default vs the freezing:

### normal_startup ###

Min: 0.524812 -> 0.473339: 1.11x faster

Avg: 0.534403 -> 0.481245: 1.11x faster

Significant (t=61.80)

Stddev: 0.00466 -> 0.00391: 1.1909x smaller


### startup_nosite ###

Min: 0.307359 -> 0.291939: 1.05x faster

Avg: 0.317667 -> 0.300156: 1.06x faster

Significant (t=26.29)

Stddev: 0.00543 -> 0.00385: 1.4099x smaller

---------

2.7 vs the freezing:

### normal_startup ###

Min: 0.367571 -> 0.465264: 1.27x slower

Avg: 0.374404 -> 0.476662: 1.27x slower

Significant (t=-90.26)

Stddev: 0.00313 -> 0.00738: 2.3603x larger


### startup_nosite ###

Min: 0.164510 -> 0.290544: 1.77x slower

Avg: 0.169833 -> 0.301109: 1.77x slower

Significant (t=-286.30)

Stddev: 0.00211 -> 0.00407: 1.9310x larger

---------

As a baseline, 2.7 vs default:

### normal_startup ###

Min: 0.368916 -> 0.521758: 1.41x slower

Avg: 0.376784 -> 0.531883: 1.41x slower

Significant (t=-172.82)

Stddev: 0.00423 -> 0.00474: 1.1207x larger


### startup_nosite ###

Min: 0.165156 -> 0.309090: 1.87x slower

Avg: 0.171516 -> 0.319004: 1.86x slower

Significant (t=-283.45)

Stddev: 0.00334 -> 0.00399: 1.1948x larger
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140414/c9250d70/attachment.html>

From v+python at g.nevcal.com  Tue Apr 15 00:07:28 2014
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Mon, 14 Apr 2014 15:07:28 -0700
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
Message-ID: <534C5C20.5010805@g.nevcal.com>

On 4/14/2014 2:51 PM, Brett Cannon wrote:
> consider freezing all the modules
...
> Now the question is whether the maintenance cost of having to rebuild 
> Python for a select number of stdlib modules

"all" versus "select number".

So I'm guessing the proposal is to freeze all the modules that Python 
imports just to get itself running, which would consume no additional 
memory when frozen, and saves time per your performance numbers, rather 
than the whole stdlib, which is what is sort of implied by "all".
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140414/ef36662c/attachment-0001.html>

From skip at pobox.com  Tue Apr 15 00:15:37 2014
From: skip at pobox.com (Skip Montanaro)
Date: Mon, 14 Apr 2014 17:15:37 -0500
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
Message-ID: <CANc-5Uwanud0_CJ4ieeORyPjRz3z8=-UAzP2bVRXOXJjrDrbCg@mail.gmail.com>

On Mon, Apr 14, 2014 at 4:51 PM, Brett Cannon <bcannon at gmail.com> wrote:
> Thoughts?

Interesting idea, but YAGNI?

In my work environment (Python 2.7.2, all the heavy lifting done in
C++), startup costs are dominated by dynamic linking of all our C++
libraries and their Boost wrappers:

% time python -c 'import tradelink.snake.v11_2 ; raise SystemExit'

real 0m0.671s
user 0m0.405s
sys 0m0.044s

% time python -c 'raise SystemExit'

real 0m0.022s
user 0m0.011s
sys 0m0.009s

Skip

From bcannon at gmail.com  Tue Apr 15 00:21:19 2014
From: bcannon at gmail.com (Brett Cannon)
Date: Mon, 14 Apr 2014 18:21:19 -0400
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <CANc-5Uwanud0_CJ4ieeORyPjRz3z8=-UAzP2bVRXOXJjrDrbCg@mail.gmail.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <CANc-5Uwanud0_CJ4ieeORyPjRz3z8=-UAzP2bVRXOXJjrDrbCg@mail.gmail.com>
Message-ID: <CAP1=2W7pRNcHMasszuSPa0KU4GLHNh4F2r27xjQZxq8tsVZeqQ@mail.gmail.com>

On Mon, Apr 14, 2014 at 6:15 PM, Skip Montanaro <skip at pobox.com> wrote:

> On Mon, Apr 14, 2014 at 4:51 PM, Brett Cannon <bcannon at gmail.com> wrote:
> > Thoughts?
>
> Interesting idea, but YAGNI?
>

Not at all. Think of every script you execute that's written in Python. One
of the things the Mercurial folks say is hindering any motivation to switch
to Python 3 is the startup performance.


>
> In my work environment (Python 2.7.2, all the heavy lifting done in
> C++), startup costs are dominated by dynamic linking of all our C++
> libraries and their Boost wrappers:
>

Sure, but not everyone uses Boost or has long running processes where
startup time is minuscule compared to the total execution time.

-Brett


>
> % time python -c 'import tradelink.snake.v11_2 ; raise SystemExit'
>
> real 0m0.671s
> user 0m0.405s
> sys 0m0.044s
>
> % time python -c 'raise SystemExit'
>
> real 0m0.022s
> user 0m0.011s
> sys 0m0.009s
>
> Skip
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140414/7c3b38a3/attachment.html>

From ericsnowcurrently at gmail.com  Tue Apr 15 00:24:23 2014
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Mon, 14 Apr 2014 16:24:23 -0600
Subject: [Python-Dev] Python "2migr8"
In-Reply-To: <9f950fc6989849fbba2d604cdb7db0c2@BLUPR03MB389.namprd03.prod.outlook.com>
References: <9f950fc6989849fbba2d604cdb7db0c2@BLUPR03MB389.namprd03.prod.outlook.com>
Message-ID: <CALFfu7D+3nd8_n+qu2F66yEefT38ZiOyfR+JyPdpJcG2ovLejw@mail.gmail.com>

On Mon, Apr 14, 2014 at 9:32 AM, Steve Dower <Steve.Dower at microsoft.com> wrote:
[snip]
> The two important components of Python 2migr8 would be the ability to disable 2.7-only features, and to do so on a module-by-module basis.

This should be doable with an import hook.  For that matter it would
be pretty straight-forward to provide a script around the Python
binary that installs the import hook.  All this could be bundled up
and distributed through the cheeseshop.

-eric

>
> My best idea so far would be to have a magic comment (to ensure 2.7 compatibility better than a "from __future__ ...") near the top of the file that marks that file as "must straddle 2.7 and 3.3". Adding this comment causes (for example) the parser to treat "except x, y" as a syntax error in this file, forces "from __future__ import ...", hides "dict.iterkeys", undefines "basestring", etc., but only for this file. (I haven't thought through all the possibilities or implications - Eric Snow said he was going to sprint on this today/tomorrow, so he'll soon have a better idea just what can be done.)
>
> In effect, 2migr8 would be the version that *only* supports "single-source" files. This allows large code bases to progressively migrate modules from 2.x to single-source while continuing to run against Python 2.7. As files are updated, they are marked as being single-source. When all files have this marker, it should be possible to flip the switch and run with Python 3.3 or later.
>
> You could also think of this as enabling "-3 --warnings-as-errors" for individual modules, though since the user has already opted in to 2migr8, it isn't unreasonable to make more significant changes, like having dict.keys returning a list that warns if it is mutated. This sort of warning can only really be done by changing the interpreter - static analysis just can't catch everything - and only when users accept a potential performance hit and low probability of breakage when they move to 2migr8 (followed by a not-quite-as-low probability of breaking when they eventually move from 2migr8 to 3.x, but it's still better than guaranteed breakage).
>
> As a fork, it would also be possible to bundle the modules that have been backported, and possibly also to disallow importing deprecated stdlib modules when 2.7 functionality is disabled. As I said, I haven't thought through all the possibilities, but the general idea is to take 2.7 and *remove* features so it becomes easier to migrate.
[snip]
> I'm not pretending to have a full plan on how this will work. I was privileged to have some private conversations during PyCon that are directly relevant, so I'm bringing it here to promote the discussion. Thanks to everyone I had a chance to chat to, and to everyone generally for a great PyCon.

From brett at python.org  Tue Apr 15 00:18:40 2014
From: brett at python.org (Brett Cannon)
Date: Mon, 14 Apr 2014 18:18:40 -0400
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <534C5C20.5010805@g.nevcal.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <534C5C20.5010805@g.nevcal.com>
Message-ID: <CAP1=2W7fD35TcWBCMuDtEwnotEupybJ97QzUXLR95+EBCLP9+Q@mail.gmail.com>

On Mon, Apr 14, 2014 at 6:07 PM, Glenn Linderman <v+python at g.nevcal.com>wrote:

>  On 4/14/2014 2:51 PM, Brett Cannon wrote:
>
>  consider freezing all the modules
>
> ...
>
>  Now the question is whether the maintenance cost of having to rebuild
> Python for a select number of stdlib modules
>
>
> "all" versus "select number".
>
> So I'm guessing the proposal is to freeze all the modules that Python
> imports just to get itself running, which would consume no additional
> memory when frozen, and saves time per your performance numbers, rather
> than the whole stdlib, which is what is sort of implied by "all".
>

Yes, exactly.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140414/bda57acd/attachment.html>

From v+python at g.nevcal.com  Tue Apr 15 00:34:42 2014
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Mon, 14 Apr 2014 15:34:42 -0700
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
Message-ID: <534C6282.2060308@g.nevcal.com>

On 4/14/2014 2:51 PM, Brett Cannon wrote:
> Freezing everything except encodings.__init__, os, and _sysconfigdata,

I suppose these are omitted because they can vary in different environments?

But isn't Python built for a particular environment... seems like os 
could be included?

Seems like it would be helpful to have the utf8 encoding preloaded both 
to encourage people to use it rather than something else for the 
load-time performance gain (although likely minuscule for one encoding), 
and because they might as well, since they are spending the memory on it 
anyway!  :)


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140414/327f4bd0/attachment.html>

From greg.ewing at canterbury.ac.nz  Tue Apr 15 01:08:18 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 15 Apr 2014 11:08:18 +1200
Subject: [Python-Dev] Python "2migr8"
In-Reply-To: <CAP7+vJKRpZ1KDxaVAFbJye++1CkZkbwp_pV43Rpje34uwuFf=w@mail.gmail.com>
References: <9f950fc6989849fbba2d604cdb7db0c2@BLUPR03MB389.namprd03.prod.outlook.com>
 <CAP7+vJKRpZ1KDxaVAFbJye++1CkZkbwp_pV43Rpje34uwuFf=w@mail.gmail.com>
Message-ID: <534C6A62.3010408@canterbury.ac.nz>

Guido van Rossum wrote:
> Some quick thoughts:
> 
> - I'd prefer a name that plays on 2 and 3, not 2 and 8. :-)

Python Twee?

Or maybe Python Tween, as in "between 2 and 3".

-- 
Greg

From tjreedy at udel.edu  Tue Apr 15 01:27:13 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 14 Apr 2014 19:27:13 -0400
Subject: [Python-Dev] Python "2migr8"
In-Reply-To: <E53C833C-8FC8-43FA-B881-A0828BF43FDC@stufft.io>
References: <9f950fc6989849fbba2d604cdb7db0c2@BLUPR03MB389.namprd03.prod.outlook.com>
 <lihebs$m8l$1@ger.gmane.org> <B4E5EB00-93AD-4867-B055-4CD7D9231F18@stufft.io>
 <CAP7+vJ+ZjNMQ9C+TTxbqnrznnR+UmXaVQ8EdjgtoftAcL_Kkng@mail.gmail.com>
 <E53C833C-8FC8-43FA-B881-A0828BF43FDC@stufft.io>
Message-ID: <lihqt5$knn$1@ger.gmane.org>

On 4/14/2014 5:16 PM, Donald Stufft wrote:

> On Apr 14, 2014, at 4:39 PM, Guido van Rossum <guido at python.org
> <mailto:guido at python.org>> wrote:

>> On Mon, Apr 14, 2014 at 4:02 PM, Donald Stufft <donald at stufft.io
>> <mailto:donald at stufft.io>> wrote:

>>     On Apr 14, 2014, at 3:53 PM, Terry Reedy <tjreedy at udel.edu
>>     <mailto:tjreedy at udel.edu>> wrote:

>>     > On 4/14/2014 11:32 AM, Steve Dower wrote:
>>     [...]
>>     >> However unfair
>>     >> and incorrect it may be, there is a perception in some businesses
>>     >> that open-source projects do not want contributions from them.

I took this to mean that they think any 'do not want' is peculiar to 
them as from a business. Perhaps I was wrong to project that on them.

>>     > For PSF/CPython, this is so untrue that it looks to me like an
>>     excuse to take without giving back.

What I meant would be untrue would be a claim that this project does not 
want contributions in particular from people in business organizations. 
A generic claim that pydev is unwelcoming or sometimes off-putting to 
newcomers, for instance, would be a different issue.

>>     As someone who *has* given back, I can certainly understand why
>>     someone would feel that way. It often times *does* feel like
>>     CPython doesn?t want contributions.

>> Donald, your remark in itself sounds unnecessarily (and
>> unproductively!) passive-aggressive. What have we done wrong to you,
>> and what can we do to avoid making the same mistake in the future (to
>> you, and to others)?

> Hmm, I?m sorry if I came across that way. I didn?t mean to. I do think
> contributing directly to CPython is often times off-putting to people

Change 'often' to 'sometimes' and I would agree.

-- 
Terry Jan Reedy



From tjreedy at udel.edu  Tue Apr 15 02:15:17 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 14 Apr 2014 20:15:17 -0400
Subject: [Python-Dev] Python "2migr8"
In-Reply-To: <CAP7+vJKNcmS70RsR8CpJmYGLUyCU01A6WziSJkGYA9YdQGyxFg@mail.gmail.com>
References: <9f950fc6989849fbba2d604cdb7db0c2@BLUPR03MB389.namprd03.prod.outlook.com>
 <lihebs$m8l$1@ger.gmane.org>
 <CAP7+vJKNcmS70RsR8CpJmYGLUyCU01A6WziSJkGYA9YdQGyxFg@mail.gmail.com>
Message-ID: <lihtna$mfm$1@ger.gmane.org>

On 4/14/2014 5:00 PM, Guido van Rossum wrote:
> On Mon, Apr 14, 2014 at 3:53 PM, Terry Reedy <tjreedy at udel.edu
> <mailto:tjreedy at udel.edu>> wrote:

>>     If the company is profitable, it could afford
>>     to fund a half- to full-time developer.

By using the vague 'fund' I meant either hire themselves or donate to 
PSF to somehow 'fund' work. I think we need both.

> A few people have made similar suggestions to me at the conference, but
> I personally believe that there is a better way.
>
> I don't think we ought to make companies feel bad about not donating to
> the PSF.

My only beef is with people who use Python *and* complain about unpaid 
volunteers not doing the un-fun work they want done that they could do 
or fund themselves. But that is part of life.

  The PSF is doing fine, but IMO it shouldn't be in the business
> of employing core developers. Being an employer is fraught with
> difficulties, and there are serious risks both for the PSF (due to the
> rigidity of employment laws, for example) and for the employee (e.g.
> benefits, worry about continuity, oversight and direction).

I was thinking in terms of contracting rather than employing -- perhaps 
for working on the backlog of hundreds of doc issues. But even that 
requires selection, training, and supervision. Perhaps I should write a 
grant application to pick and supervise one or more college students to 
work on neglected issues. Then the grants committee volunteers would 
just have to say yes or no and later continue or stop.

> IMO a much better approach would be to convince companies to free up
> some of their current employees or new hires to invest e.g. 50% of their
> time into core Python development.

This would be great, but it is not something I can be very convincing 
about ;-). I hope you succeed. But I suspect that some of the things I 
think need to be done will not be done by corporate employees. Hence the 
'both' above.
...
> (I should say that this is my own situation at Dropbox and previously at
> Google, and I personally wouldn't want it any other way.)

I think your continued practical experience is good for Python.

>> Sounds like they are looking ahead several years and anxious to
>> avoid the 'comfortable with XP' trap.
>
> Ohhh, nice analogy!

>>> The two important components of Python 2migr8 would be the ability to
>>> disable 2.7-only features, and to do so on a module-by-module basis.

>> A reasonable request of pydev would be for python-coded stdlib
>> modules to be updated as much as possible, if that has not already
>> been done. No 'apply', no 'except SomeException, e'.

'Reasonable request': a request plausible enough that we should discuss 
it and maybe say yes.

I looked and apply is already not hardly used except in lib2to3 and its 
test in test_builtins. There are hundreds of the old exception form. The 
re for an re.sub call would have to not match tuple commas, such as in 
'except (KeyError, IndexError):

> I'm not sure what you're proposing here,

Focused, carefully considered, behavior neutral changes that help 
migration, if indeed there are such, by letting an stdlib module work 
with an altered 2.7 interpreter. The existence of decent test coverage 
for a module would be a consideration.

-- 
Terry Jan Reedy


From rosuav at gmail.com  Tue Apr 15 04:03:38 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 15 Apr 2014 12:03:38 +1000
Subject: [Python-Dev] Python "2migr8"
In-Reply-To: <CALGmxEJL3qA=Km+=d5uW7XTPKr_82BG9cw-3RbDpKEf-psJWJw@mail.gmail.com>
References: <9f950fc6989849fbba2d604cdb7db0c2@BLUPR03MB389.namprd03.prod.outlook.com>
 <CAP7+vJKRpZ1KDxaVAFbJye++1CkZkbwp_pV43Rpje34uwuFf=w@mail.gmail.com>
 <liha4p$ev0$1@ger.gmane.org>
 <CAP7+vJLOEeHc8X741OfFsEJBymYrT9RU1O4Yo0Jz81BXadaxqQ@mail.gmail.com>
 <CALGmxEJL3qA=Km+=d5uW7XTPKr_82BG9cw-3RbDpKEf-psJWJw@mail.gmail.com>
Message-ID: <CAPTjJmq159_qhJ2s_O-MtGzx8VWPq=aNFGGMiKyfxs0DXwf73w@mail.gmail.com>

On Tue, Apr 15, 2014 at 7:17 AM, Chris Barker <chris.barker at noaa.gov> wrote:
> On Mon, Apr 14, 2014 at 1:26 PM, Guido van Rossum <guido at python.org> wrote:
>
>>
>> >> - I'd prefer a name that plays on 2 and 3, not 2 and 8. :-)
>
> How about mirg2**3 (pronounced "migrate") ?
>
> ;-)
>

Just read this, and laughed so loudly and suddenly that my brother's
cat jumped in fright.

Spelled 2**3 and pronounced 8 makes perfect sense, same as spelling
"Mercurial" as "hg".

ChrisA

From rymg19 at gmail.com  Tue Apr 15 04:15:32 2014
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Mon, 14 Apr 2014 21:15:32 -0500
Subject: [Python-Dev] Python "2migr8"
In-Reply-To: <CAPTjJmq159_qhJ2s_O-MtGzx8VWPq=aNFGGMiKyfxs0DXwf73w@mail.gmail.com>
References: <9f950fc6989849fbba2d604cdb7db0c2@BLUPR03MB389.namprd03.prod.outlook.com>
 <CAP7+vJKRpZ1KDxaVAFbJye++1CkZkbwp_pV43Rpje34uwuFf=w@mail.gmail.com>
 <liha4p$ev0$1@ger.gmane.org>
 <CAP7+vJLOEeHc8X741OfFsEJBymYrT9RU1O4Yo0Jz81BXadaxqQ@mail.gmail.com>
 <CALGmxEJL3qA=Km+=d5uW7XTPKr_82BG9cw-3RbDpKEf-psJWJw@mail.gmail.com>
 <CAPTjJmq159_qhJ2s_O-MtGzx8VWPq=aNFGGMiKyfxs0DXwf73w@mail.gmail.com>
Message-ID: <CAO41-mN25CiupKT=T_bjsARM65Q1SARr1iC9aTeRZNc8412o1w@mail.gmail.com>

On Mon, Apr 14, 2014 at 9:03 PM, Chris Angelico <rosuav at gmail.com> wrote:

> On Tue, Apr 15, 2014 at 7:17 AM, Chris Barker <chris.barker at noaa.gov>
> wrote:
> > On Mon, Apr 14, 2014 at 1:26 PM, Guido van Rossum <guido at python.org>
> wrote:
> >
> >>
> >> >> - I'd prefer a name that plays on 2 and 3, not 2 and 8. :-)
> >
> > How about mirg2**3 (pronounced "migrate") ?
> >
> > ;-)
> >
>
> Just read this, and laughed so loudly and suddenly that my brother's
> cat jumped in fright.
>
> Spelled 2**3 and pronounced 8 makes perfect sense, same as spelling
> "Mercurial" as "hg".
>

...unless the reader doesn't know math. That would be
funny...*migrtoostarstarthree*


>
> ChrisA
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>



-- 
Ryan
If anybody ever asks me why I prefer C++ to C, my answer will be simple:
"It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
nul-terminated."
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140414/1261d41d/attachment-0001.html>

From alexander.belopolsky at gmail.com  Tue Apr 15 04:22:26 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Mon, 14 Apr 2014 22:22:26 -0400
Subject: [Python-Dev] Python "2migr8"
In-Reply-To: <CAPTjJmq159_qhJ2s_O-MtGzx8VWPq=aNFGGMiKyfxs0DXwf73w@mail.gmail.com>
References: <9f950fc6989849fbba2d604cdb7db0c2@BLUPR03MB389.namprd03.prod.outlook.com>
 <CAP7+vJKRpZ1KDxaVAFbJye++1CkZkbwp_pV43Rpje34uwuFf=w@mail.gmail.com>
 <liha4p$ev0$1@ger.gmane.org>
 <CAP7+vJLOEeHc8X741OfFsEJBymYrT9RU1O4Yo0Jz81BXadaxqQ@mail.gmail.com>
 <CALGmxEJL3qA=Km+=d5uW7XTPKr_82BG9cw-3RbDpKEf-psJWJw@mail.gmail.com>
 <CAPTjJmq159_qhJ2s_O-MtGzx8VWPq=aNFGGMiKyfxs0DXwf73w@mail.gmail.com>
Message-ID: <CAP7h-xZKW1dWWNLdeswG3JCWrTiL5_a0z9cA1xrs7=__NAQNbw@mail.gmail.com>

On Mon, Apr 14, 2014 at 10:03 PM, Chris Angelico <rosuav at gmail.com> wrote:

> > How about mirg2**3 (pronounced "migrate") ?
> >
> > ;-)
> >
>
> Just read this, and laughed so loudly and suddenly that my brother's
> cat jumped in fright.
>
> Spelled 2**3 and pronounced 8 makes perfect sense, same as spelling
> "Mercurial" as "hg".


Why is six afraid of seven? :-)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140414/11baac20/attachment.html>

From stephen at xemacs.org  Tue Apr 15 04:48:40 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 15 Apr 2014 11:48:40 +0900
Subject: [Python-Dev] Python "2migr8"
In-Reply-To: <CAP7+vJ+ZjNMQ9C+TTxbqnrznnR+UmXaVQ8EdjgtoftAcL_Kkng@mail.gmail.com>
References: <9f950fc6989849fbba2d604cdb7db0c2@BLUPR03MB389.namprd03.prod.outlook.com>
 <lihebs$m8l$1@ger.gmane.org>
 <B4E5EB00-93AD-4867-B055-4CD7D9231F18@stufft.io>
 <CAP7+vJ+ZjNMQ9C+TTxbqnrznnR+UmXaVQ8EdjgtoftAcL_Kkng@mail.gmail.com>
Message-ID: <87tx9vfgtz.fsf@uwakimon.sk.tsukuba.ac.jp>

Guido van Rossum writes:

 > On Mon, Apr 14, 2014 at 4:02 PM, Donald Stufft <donald at stufft.io> wrote:

 >> As someone who *has* given back, I can certainly understand why
 >> someone would feel that way. It often times *does* feel like
 >> CPython doesn?t want contributions.

Sure, but most of the time that feeling is based on a completely
contributor-centric viewpoint.  (Not to mention that I think this is a
little ironic given the position you're taking in the PEP 440
vs. semantic versioning thread.  I agree 100%, but still: ironic.)

As a user and wannabe developer of Python, one of the things I love
about Python (compared to say, Emacsen *shudder*) is that it has been
very good about avoiding the

    1. I have a problem.
    2. This patch fixes my problem.
    3. Now Python has a problem.

syndrome.  Contributors typically feel like they're unwanted, though,
at least at some points in the process of resolving (3) *before* the
patch gets into the mainline.  Of course mostly they're grownups and
get over it, but I suppose that feeling is what Donald is pointing to.

But there's a problem beyond "feeling" for developers paid to work *in
but not on* Python.  I suppose that often their employers *are*
sufficiently enlightened to be willing to allow their people to work
up a full patch (including tests and documentation), but *not* willing
to allow their people to spend twice as much time (or more) getting to
a "fully Pythonic" solution.

And it's not just the employer; the developer herself probably feels
that it's not a profitable use of her time.  She's solved her problem,
she's met the formal requirements for a Python contribution, it's
probably code she's proud of, and she's got 10 more tasks of the same
ilk listed on a whiteboard in her cubicle.  As a professional, of
course she wants to get to work on those.

I can see a few possible ways to address this.

1.  Commit core resources to fast-tracking (through review, concrete
    suggestions for improvement, and additional code) contributions
    from people who are "on deadline".  (Yeah, I know, that's
    problematic, and not just because of the scarcity of core
    reviewers.  But what else can we do?)

2.  Educate the employers about the benefits, not only from higher-
    quality code, but access to resources like core developer
    attention and (perhaps future hiring :-), and happier developers,
    that comes from encouraging their people to participate fully in
    the process.

    The point is to create a "you're *supposed* to spend the extra
    time" atmosphere *in the employment context*.

    This probably would work best if Guido's boss talks to the
    developer's boss at $BILLION_DOLLAR_BUSINESS. :-)

    Do Google and DropBox have blogs where they explain their "tithe
    to open source" policies?

3.  Related to 2.  I think there's an impression out there that "open
    source tithes" are for the top 1% of programmers.  I know that at
    LUG meetings and the like I've often heard comments like "nice
    work if you can get it, but my employer never will offer it to me".

    This impression must be debunked.  And it can be debunked.  Sure,
    there are many "we created this job for a particular genius
    programmer" positions out there, but many excellent companies have
    a generic tithe for all their developers.  We need examples other
    than Google.

4.  Get those developers to PyCon.  Get their bosses to PyCon. :-)
    Or maybe this should be point #1. :-)

 >?Donald, your remark in itself sounds unnecessarily (and
 > unproductively!) passive-aggressive.

FWIW, I didn't read it that way.  I took it as a genuinely "more in
sorrow than in anger" statement.


From cf.natali at gmail.com  Tue Apr 15 09:31:49 2014
From: cf.natali at gmail.com (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=)
Date: Tue, 15 Apr 2014 08:31:49 +0100
Subject: [Python-Dev] [numpy wishlist] PyMem_*Calloc
In-Reply-To: <534C40B9.2090303@stoneleaf.us>
References: <CAPJVwBnFu9_KzEZz9R-cwhK2=K_gx_prfu-t+VEAb=hMGXU0WQ@mail.gmail.com>
 <1397489815.23597.106333761.0614A127@webmail.messagingengine.com>
 <534C40B9.2090303@stoneleaf.us>
Message-ID: <CAH_1eM1YiNVfUbW+3aH0zcG8v8ttaqcRaAMSPmUhnMoznZ3UCA@mail.gmail.com>

Indeed, that's very reasonable.

Please open an issue on the tracker!

From rosuav at gmail.com  Tue Apr 15 09:45:02 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 15 Apr 2014 17:45:02 +1000
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <CAP1=2W7pRNcHMasszuSPa0KU4GLHNh4F2r27xjQZxq8tsVZeqQ@mail.gmail.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <CANc-5Uwanud0_CJ4ieeORyPjRz3z8=-UAzP2bVRXOXJjrDrbCg@mail.gmail.com>
 <CAP1=2W7pRNcHMasszuSPa0KU4GLHNh4F2r27xjQZxq8tsVZeqQ@mail.gmail.com>
Message-ID: <CAPTjJmoJXr-HLz7FvXndc7ix9n-MwEbZ2-8=WiLNKOcMRKWFPw@mail.gmail.com>

On Tue, Apr 15, 2014 at 8:21 AM, Brett Cannon <bcannon at gmail.com> wrote:
>> In my work environment (Python 2.7.2, all the heavy lifting done in
>> C++), startup costs are dominated by dynamic linking of all our C++
>> libraries and their Boost wrappers:
>
>
> Sure, but not everyone uses Boost or has long running processes where
> startup time is minuscule compared to the total execution time.
>

Specific use-case that I can see: Mercurial. In a git vs hg shoot-out,
git will usually win on performance, and hg is using Py2; migrating hg
to Py3 will (if I understand the above figures correctly) widen that
gap, so any improvement done to startup performance will give a very
real advantage.

ChrisA

From ncoghlan at gmail.com  Tue Apr 15 15:21:02 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 15 Apr 2014 09:21:02 -0400
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <534C6282.2060308@g.nevcal.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <534C6282.2060308@g.nevcal.com>
Message-ID: <CADiSq7eHhhSxU-pFmrFNGiaKh5NhBtP4enrCnyttnW2qEUCK2g@mail.gmail.com>

On 14 Apr 2014 18:37, "Glenn Linderman" <v+python at g.nevcal.com> wrote:
>
> On 4/14/2014 2:51 PM, Brett Cannon wrote:
>>
>> Freezing everything except encodings.__init__, os, and _sysconfigdata,
>
>
> I suppose these are omitted because they can vary in different
environments?
>
> But isn't Python built for a particular environment... seems like os
could be included?
>
> Seems like it would be helpful to have the utf8 encoding preloaded both
to encourage people to use it rather than something else for the load-time
performance gain (although likely minuscule for one encoding), and because
they might as well, since they are spending the memory on it anyway!  :)

Via some moderately arcane hackery, UTF-8 support is already built in to
the Py3 interpreter :)

Cheers,
Nick.

>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140415/d523fca0/attachment.html>

From mal at egenix.com  Tue Apr 15 16:11:16 2014
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 15 Apr 2014 16:11:16 +0200
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <CAPTjJmoJXr-HLz7FvXndc7ix9n-MwEbZ2-8=WiLNKOcMRKWFPw@mail.gmail.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>	<CANc-5Uwanud0_CJ4ieeORyPjRz3z8=-UAzP2bVRXOXJjrDrbCg@mail.gmail.com>	<CAP1=2W7pRNcHMasszuSPa0KU4GLHNh4F2r27xjQZxq8tsVZeqQ@mail.gmail.com>
 <CAPTjJmoJXr-HLz7FvXndc7ix9n-MwEbZ2-8=WiLNKOcMRKWFPw@mail.gmail.com>
Message-ID: <534D3E04.7000903@egenix.com>

On 15.04.2014 09:45, Chris Angelico wrote:
> On Tue, Apr 15, 2014 at 8:21 AM, Brett Cannon <bcannon at gmail.com> wrote:
>>> In my work environment (Python 2.7.2, all the heavy lifting done in
>>> C++), startup costs are dominated by dynamic linking of all our C++
>>> libraries and their Boost wrappers:
>>
>>
>> Sure, but not everyone uses Boost or has long running processes where
>> startup time is minuscule compared to the total execution time.
>>
> 
> Specific use-case that I can see: Mercurial. In a git vs hg shoot-out,
> git will usually win on performance, and hg is using Py2; migrating hg
> to Py3 will (if I understand the above figures correctly) widen that
> gap, so any improvement done to startup performance will give a very
> real advantage.

You might want to have a look at this project:

http://pyrun.org/

It's currently Python 2 only, but we'll try to get it to work
with Python 3.4 as well, now that freeze.py and some other bits
have been fixed to make it work again.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From ericsnowcurrently at gmail.com  Tue Apr 15 16:30:12 2014
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Tue, 15 Apr 2014 08:30:12 -0600
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
Message-ID: <CALFfu7B46zu9VceRr=B-tcJEQLQ7KeDMnuJXCpz0f1d=vZOtDQ@mail.gmail.com>

On Mon, Apr 14, 2014 at 3:51 PM, Brett Cannon <bcannon at gmail.com> wrote:
> It was realized during PyCon that since we are freezing importlib we could
> now consider freezing all the modules to cut out having to stat or read them
> from disk. So for day 1 of the sprints I I decided to hack up a
> proof-of-concept to see what kind of performance gain it would get.
>
> Freezing everything except encodings.__init__, os, and _sysconfigdata, it
> speeds up startup by 11% compared to default. Compared to 2.7 it shaves 14%
> from the slowdown (27% slower vs. 41% slower). The full results are at the
> end of the email.

Nice.  I was hoping it would be even bigger (given the hyper-focus
people put on the impact of FS-access on startup time imports), but
this is definitely a significant improvement.  I wonder then where the
remaining slowdown lies; are there any remaining low hanging fruit
elsewhere?

>
> Now the question is whether the maintenance cost of having to rebuild Python
> for a select number of stdlib modules is enough to warrant putting in the
> effort to make this work.

Yeah.  Definitely the big question.  Who cares the most about startup
time?  Would this improvement please them?  Does that alone make it
worth the increased maintenance burden?  Is that group big enough or
important enough to justify it?  At the very least it may be good for
the PR value alone, but the maintenance cost will long outlive the PR
benefit. :)

> My guess is the best approach would be adding a
> Lib/_frozen directory where any modules that we treat like this would be
> kept to act as a reminder that you need to rebuild for them (I would
> probably move importlib/_boostrap.py as well to make this consistent).

That makes sense.  I also wonder if we could accomplish the same thing
with a marker (e.g. a comment) in each related module (and leave them
where they are).  A marker would allow for easily finding the
freezable modules.

Personally, I think the speedup would be worth it if it doesn't add
significant to the maintenance burden.

-eric

From ericsnowcurrently at gmail.com  Tue Apr 15 16:34:00 2014
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Tue, 15 Apr 2014 08:34:00 -0600
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <CAPTjJmoJXr-HLz7FvXndc7ix9n-MwEbZ2-8=WiLNKOcMRKWFPw@mail.gmail.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <CANc-5Uwanud0_CJ4ieeORyPjRz3z8=-UAzP2bVRXOXJjrDrbCg@mail.gmail.com>
 <CAP1=2W7pRNcHMasszuSPa0KU4GLHNh4F2r27xjQZxq8tsVZeqQ@mail.gmail.com>
 <CAPTjJmoJXr-HLz7FvXndc7ix9n-MwEbZ2-8=WiLNKOcMRKWFPw@mail.gmail.com>
Message-ID: <CALFfu7DVaEge-_f3_nORrozR419ENrA+OGkqdL9e0m88xDXt5A@mail.gmail.com>

On Tue, Apr 15, 2014 at 1:45 AM, Chris Angelico <rosuav at gmail.com> wrote:
> Specific use-case that I can see: Mercurial. In a git vs hg shoot-out,
> git will usually win on performance, and hg is using Py2; migrating hg
> to Py3 will (if I understand the above figures correctly) widen that
> gap, so any improvement done to startup performance will give a very
> real advantage.

Perhaps not so much "a very real advantage" as "less of a
distraction". It's still significantly slower than 2.7.  :)

-eric

From ncoghlan at gmail.com  Tue Apr 15 16:34:30 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 15 Apr 2014 10:34:30 -0400
Subject: [Python-Dev] Python "2migr8"
In-Reply-To: <E53C833C-8FC8-43FA-B881-A0828BF43FDC@stufft.io>
References: <9f950fc6989849fbba2d604cdb7db0c2@BLUPR03MB389.namprd03.prod.outlook.com>
 <lihebs$m8l$1@ger.gmane.org>
 <B4E5EB00-93AD-4867-B055-4CD7D9231F18@stufft.io>
 <CAP7+vJ+ZjNMQ9C+TTxbqnrznnR+UmXaVQ8EdjgtoftAcL_Kkng@mail.gmail.com>
 <E53C833C-8FC8-43FA-B881-A0828BF43FDC@stufft.io>
Message-ID: <CADiSq7cSQ0mJK+QTPVaz4gv_U=3GZV5pVSWeTSvBPibwzP6Rdw@mail.gmail.com>

On 14 Apr 2014 17:17, "Donald Stufft" <donald at stufft.io> wrote:
>
> Now I will admit I personally have probably had a harder time than some
others because of the nature of the things I was trying to work on, and
lately it?s gotten better (although I think that?s partially because I?m
more known now, and I think in general the experience of contributing to
CPython changes depending on who you are, so the more integrated into the
culture you are, the less likely you are to see the issues and
unfortunately those people are also the ones with the most power to change
it). I do however think that just in general it might be getting better too?

>From my perspective, the main issue is that new contributors typically
don't acknowledge the large number of competing constraints we're balancing
in core development, and hence grow frustrated when we reject their
"obvious" idea on the basis of a concern that seems completely irrelevant
to them. (Due to my background, I'll occasionally phrase this along the
lines of "Supporting submarines and secure enclaves matters to me, but most
open source devs don't care").

Core development for a programming language is a genuinely hard problem,
and one that often involves thinking on timescales of years and decades
rather than the weeks and months involved in more typical development
environments. There's a huge impedance mismatch in expectations that can
lead to major communication problems if people don't take the time to lurk
for a while and get a feel for the kinds of novel concerns that arise when
sitting at the centre of a massive ecosystem like Python's.

> Specific details are hard because it?s nothing major and obvious like
having Linus go off on rants and tearing things apart, it?s death by a
thousand cuts so it?s hard to point a finger at one behavior (or a few
behaviors) and look at them in isolation and ?see? it. That being said I?m
more than happy to *try* and explain it, but right this moment I don?t have
a lot of time as I?m getting ready to step out the door, but I didn?t want
to leave this email hanging without a reply.

>From my perspective, the heart of the issue is personal time management on
the part of the core development team. We're highly cognizant of the limits
of what can be sustained through volunteer development, and one of the big
issues is that loading volunteers up with too many activities that they
feel obliged to do but don't find inherently enjoyable is a recipe for
burnout. So we "default to no" not just because the number of ways we can
make Python worse is unbounded, but also because the time we have available
to do anything at all is incredibly limited.

Now, consider that we're operating in an environment where multi-billion
dollar companies are relying on our software while making only relatively
small contributions to its ongoing support and evolution, and where we have
multiple prominent community members wishing vocally (and encouraging
others to advocate) for the core development team to devote our volunteer
efforts to improving a legacy language rather than the new one we shifted
en masse to working on instead. (Note that the latter actually makes about
as much sense to me as telling the Rust and Go developers they should spend
their free time working on C compilers instead because the latter would be
more immediately useful to commercial users)

On top of this, various outreach efforts to encourage new contributors are
working, and working well enough that they are actually *exceeding* the
existing team's ability to effectively *absorb* those new contributors.

So, tremendously high stress levels for the core development team, on top
of whatever stress we have to deal with in our personal and professional
lives.

This stress then manifests as irritability, impatience and outright anger -
my own stress levels reached sufficiently high levels earlier this year
that I almost decided to walk away from my job. Red Hat management
intervened to keep that from actually happening, but I think it's important
to make that incident more public to help people understand how utterly
unsustainable the status quo is when it comes to CPython - we can't keep
relying on almost entirely volunteer effort to maintain 2.7 LTS, 3.x, all
the python.org infrastructure *and* the PSF without also anticipating
complete and total burnout of some highly invested contributors.

PEP 462 describes some ideas to make more effective use of core developer
time, and to potentially distribute particular tasks to better suited
groups of people (such as the tutorial and HOWTO guides), but in itself
involves a substantial amount of up front work.

That's where Guido's suggestion of corporations offering more "50%" jobs
for core developers comes in. That 50% time wouldn't be about working on
things we would have done anyway - it would be about working on things that
are difficult to get jumpstarted with purely volunteer effort. More work on
2.7 maintenance, more work on CPython infrastructure, more work on
encouraging and supporting new contributors, more work on improving
communications.

As things stand, the centre cannot hold (not indefinitely, anyway).
However, I think Guido's proposed approach to resolving the situation is a
good one, and similar in many respects to the way the Linux and OpenStack
communities already work. I think we also have the necessary contacts at
various organisations to make it a reality if we make it clear not only
that we're open to the idea, but also emphasise the strategic risk
corporate users are currently taking by treating CPython as magic
programming pixie dust spontaneously generated on the internet - a few
relatively flexible roles for core developers at various organisations
would drastically reduce that risk without ceding any one organisation
undue levels of influence.

Regards,
Nick.

>
> [1] See Also
http://www.curiousefficiency.org/posts/2011/04/musings-on-culture-of-python-dev.html
>
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372
DCFA
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140415/0bbf518c/attachment.html>

From ncoghlan at gmail.com  Tue Apr 15 16:41:17 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 15 Apr 2014 10:41:17 -0400
Subject: [Python-Dev] Appeal for reviews
In-Reply-To: <20140414124054.133AA250CAE@webabinitio.net>
References: <87sipiwf1w.fsf@vostro.rath.org>
 <87ppkmgjg5.fsf@uwakimon.sk.tsukuba.ac.jp>
 <87fvlh10wt.fsf@vostro.rath.org>
 <87fvlghej1.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140414040557.Horde.FRrDDDnMkJvhmQHf-0NFHg1@webmail.df.eu>
 <877g6sh2w6.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CADiSq7fdJSuhsW0cuRVT-ZzqGjzQKMd7Vi8+vNbF0LOmNofxWA@mail.gmail.com>
 <20140414124054.133AA250CAE@webabinitio.net>
Message-ID: <CADiSq7eucW-LgKYGe5H4faJqov19YaS3AU4M=yKmLhOHpOzMXQ@mail.gmail.com>

On 14 Apr 2014 08:42, "R. David Murray" <rdmurray at bitdance.com> wrote:
>
> On Mon, 14 Apr 2014 08:18:13 -0400, Nick Coghlan <ncoghlan at gmail.com>
wrote:
> > On 14 Apr 2014 01:56, "Stephen J. Turnbull" <stephen at xemacs.org> wrote:
> > >
> > > martin at v.loewis.de writes:
> > >
> > >  > For gaining commit access, it's really more important that the
patch
> > >  > is factually finished, than that it's author believes it to. If
people
> > >  > get it right the first time often enough, they get commit access.
> > >
> > > Yes, that's what I had in mind, but I guess I explained it poorly.
> >
> > We should capture this discussion clearly in the dev guide. Even if we
> > switch to a core reviewer model at some point (as I propose in PEP 462),
> > the criteria for core reviewer status will match those for core commiter
> > status.
> >
> > There are actually a few things I'm personally looking for:
> >
> > * good judgement on when a patch is "finished enough" to merge
> > * good judgement on whether a change is a new feature or a bug fix
> > * good judgement whether a new feature is worth the additional cognitive
> > burden
> > * good ability to assess backwards compatibility risks
> > * sufficient humility to answer "I don't know" to the above questions
when
> > appropriate and ask the relevant domain experts, their sponsoring
mentor,
> > the core-mentorship list or python-dev at large for advice on what to do
>
> When considering who we give commit access to, I think we would be
> well served to start giving more weight to the quality of the code
> reviews that someone does.  Producing good patches is important,
> but even without moving the infrastructure to Nick's "core reviewer"
> model, doing those reviews is an important part of what committers
> do, and it is a different (although related) skill to that of
> writing good patches.
>
> Or to put it another way, I'd like to encourage contributors who
> want to get commit access to focus just as much on doing good reviews as
> they do on writing new patches.  Currently the focus is all on
> getting patches accepted.

Huh, I hadn't thought of it that way before, but it's a very good point.

Cheers,
Nick.

>
> --David
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140415/20940f1b/attachment.html>

From victor.stinner at gmail.com  Tue Apr 15 16:51:37 2014
From: victor.stinner at gmail.com (Victor Stinner)
Date: Tue, 15 Apr 2014 10:51:37 -0400
Subject: [Python-Dev] [numpy wishlist] PyMem_*Calloc
In-Reply-To: <CAPJVwBnFu9_KzEZz9R-cwhK2=K_gx_prfu-t+VEAb=hMGXU0WQ@mail.gmail.com>
References: <CAPJVwBnFu9_KzEZz9R-cwhK2=K_gx_prfu-t+VEAb=hMGXU0WQ@mail.gmail.com>
Message-ID: <CAMpsgwa3-12iRA+9qkutU-dQ83WiJRfgsx_Mo6g4fz3Z=hYOxA@mail.gmail.com>

  Hi,

2014-04-14 1:39 GMT-04:00 Nathaniel Smith <njs at pobox.com>:
> The new tracemalloc infrastructure in python 3.4 is super-interesting
> to numerical folks, because we really like memory profiling.

Cool, thanks :-)

> calloc() is more awesome than malloc()+memset() (...)

I had a discussion with someone about tracemalloc and numpy at Pycon,
was it you? After this discussion, I realized that calloc() exists
because the operating system can have a very efficient implementation
for calloc() (as you described).

> SO, we'd like to route our allocations through PyMem_* in order to let
> tracemalloc "see" them, but because there is no PyMem_*Calloc, doing
> this would force us to give up on the calloc() optimizations.

It would also be useful in Python because in many places, Python uses
memset() to fill memory with zeros.

> The obvious solution is to add a PyMem_*Calloc to the API. Would this
> be possible? Unfortunately it would require adding a new field to the
> PyMemAllocator struct, which would be an ABI/API break; PyMemAllocator
> is exposed directly in the C API and passed by value:
>   https://docs.python.org/3.5/c-api/memory.html#c.PyMemAllocator

I don't want to change the structure in Python 3.4, but I'm interested
to implement the change in Python 3.5.

Please open an issue and add me to the nosy list.

For Python 3.4, you can maybe add a compilation flag to use Python
allocators but reimplementing calloc() which will be slower as you
explained.

Victor

From guido at python.org  Tue Apr 15 16:56:45 2014
From: guido at python.org (Guido van Rossum)
Date: Tue, 15 Apr 2014 10:56:45 -0400
Subject: [Python-Dev] Appeal for reviews
In-Reply-To: <CADiSq7eucW-LgKYGe5H4faJqov19YaS3AU4M=yKmLhOHpOzMXQ@mail.gmail.com>
References: <87sipiwf1w.fsf@vostro.rath.org>
 <87ppkmgjg5.fsf@uwakimon.sk.tsukuba.ac.jp>
 <87fvlh10wt.fsf@vostro.rath.org> <87fvlghej1.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140414040557.Horde.FRrDDDnMkJvhmQHf-0NFHg1@webmail.df.eu>
 <877g6sh2w6.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CADiSq7fdJSuhsW0cuRVT-ZzqGjzQKMd7Vi8+vNbF0LOmNofxWA@mail.gmail.com>
 <20140414124054.133AA250CAE@webabinitio.net>
 <CADiSq7eucW-LgKYGe5H4faJqov19YaS3AU4M=yKmLhOHpOzMXQ@mail.gmail.com>
Message-ID: <CAP7+vJLVtkik1NWyQ-Eoq4FdhEo8hEYT4CwDpabFa635xKKchw@mail.gmail.com>

On Tue, Apr 15, 2014 at 10:41 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 14 Apr 2014 08:42, "R. David Murray" <rdmurray at bitdance.com> wrote:
> > When considering who we give commit access to, I think we would be
> > well served to start giving more weight to the quality of the code
> > reviews that someone does.  Producing good patches is important,
> > but even without moving the infrastructure to Nick's "core reviewer"
> > model, doing those reviews is an important part of what committers
> > do, and it is a different (although related) skill to that of
> > writing good patches.
> >
> > Or to put it another way, I'd like to encourage contributors who
> > want to get commit access to focus just as much on doing good reviews as
> > they do on writing new patches.  Currently the focus is all on
> > getting patches accepted.
>
> Huh, I hadn't thought of it that way before, but it's a very good point.
>
Indeed. This is also a reflection of a good hiring policy for software
developers (wannabe startup founders pay attention): Don't just try to hire
brilliant coders -- look for people who work really well with a team.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140415/4d8da57d/attachment-0001.html>

From njs at pobox.com  Tue Apr 15 10:56:56 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Tue, 15 Apr 2014 10:56:56 +0200
Subject: [Python-Dev] [numpy wishlist] PyMem_*Calloc
In-Reply-To: <CAH_1eM1YiNVfUbW+3aH0zcG8v8ttaqcRaAMSPmUhnMoznZ3UCA@mail.gmail.com>
References: <CAPJVwBnFu9_KzEZz9R-cwhK2=K_gx_prfu-t+VEAb=hMGXU0WQ@mail.gmail.com>
 <1397489815.23597.106333761.0614A127@webmail.messagingengine.com>
 <534C40B9.2090303@stoneleaf.us>
 <CAH_1eM1YiNVfUbW+3aH0zcG8v8ttaqcRaAMSPmUhnMoznZ3UCA@mail.gmail.com>
Message-ID: <CAPJVwBmKAaa5wVKkBiWuE4J2AQLKY2w1KoJhTwVA6q4NzsfM9Q@mail.gmail.com>

On Tue, Apr 15, 2014 at 9:31 AM, Charles-Fran?ois Natali
<cf.natali at gmail.com> wrote:
> Indeed, that's very reasonable.
>
> Please open an issue on the tracker!

Done!

http://bugs.python.org/issue21233

I'll ping numpy-discussion and see if I can convince someone to do the work ;-).

-n

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

From dholth at gmail.com  Tue Apr 15 17:19:32 2014
From: dholth at gmail.com (Daniel Holth)
Date: Tue, 15 Apr 2014 11:19:32 -0400
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <CALFfu7DVaEge-_f3_nORrozR419ENrA+OGkqdL9e0m88xDXt5A@mail.gmail.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <CANc-5Uwanud0_CJ4ieeORyPjRz3z8=-UAzP2bVRXOXJjrDrbCg@mail.gmail.com>
 <CAP1=2W7pRNcHMasszuSPa0KU4GLHNh4F2r27xjQZxq8tsVZeqQ@mail.gmail.com>
 <CAPTjJmoJXr-HLz7FvXndc7ix9n-MwEbZ2-8=WiLNKOcMRKWFPw@mail.gmail.com>
 <CALFfu7DVaEge-_f3_nORrozR419ENrA+OGkqdL9e0m88xDXt5A@mail.gmail.com>
Message-ID: <CAG8k2+7ktuEr0Q=em6CtisGgd_YPK41pLiDnUoO1ti+Ntf75Mg@mail.gmail.com>

IIRC it is no longer the case that ZIP imports (involving only one
file for a lot of modules) are much faster than regular FS imports?

On Tue, Apr 15, 2014 at 10:34 AM, Eric Snow <ericsnowcurrently at gmail.com> wrote:
> On Tue, Apr 15, 2014 at 1:45 AM, Chris Angelico <rosuav at gmail.com> wrote:
>> Specific use-case that I can see: Mercurial. In a git vs hg shoot-out,
>> git will usually win on performance, and hg is using Py2; migrating hg
>> to Py3 will (if I understand the above figures correctly) widen that
>> gap, so any improvement done to startup performance will give a very
>> real advantage.
>
> Perhaps not so much "a very real advantage" as "less of a
> distraction". It's still significantly slower than 2.7.  :)
>
> -eric
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/dholth%40gmail.com

From skip at pobox.com  Tue Apr 15 17:34:14 2014
From: skip at pobox.com (Skip Montanaro)
Date: Tue, 15 Apr 2014 10:34:14 -0500
Subject: [Python-Dev] Mercurial sluggishness (was: this is what happens if
 you freeze all the modules required for startup)
Message-ID: <CANc-5UwL++W0_d8KXVF2+aG+_mcRC6FgvR+heTNYrw1-XmvpOg@mail.gmail.com>

Eric wrote:

> Perhaps not so much "a very real advantage" as "less of a
> distraction".  It's still significantly slower than 2.7.  :)

I'm still confused. I peeked in /usr/bin/hg. The only "system" modules
it imports directly are os and system (maybe I'm using an ancient
version?). After that, it imports its own lazy import module. This
suggests to me that Mercurial's import slowness is mostly in its own
code (I counted 104 Python modules and six shared objects in its
mercurial package, which isn't going to be affected (much?) by
freezing the Python standard modules.

I'm not trying to be difficult here. I thought that way back in the
day a huge amount of work was done to remove needless filesystem
activity, and zip packaging has been around for quite awhile.

As an experiment, I ran hg pull as

    /usr/bin/python -v /usr/bin/hg pull

in my cpython repo then looked at the -v output. Summarizing the
output I saw the following:

    30 imports (0 dlopens)

    Python banner printed

    86 imports (18 dlopens)

    adding changesets message

    5 imports (2 dlopens)

    adding manifests message

    1 import (0 dlopens)

    adding file changes message

    7 imports (3 dlopens)

    added ... changesets message

    4 imports (0 dlopens)

    run 'hg update' message

(I missed a "searching" message in there somewhere.)

That's a total of 133 imports, 23 of which were satisfied by loading
an extension module. The imports break down as follows:

51 imports (4 dlopens) from the mercurial package
5 imports from the hgext package
7 imports from the encodings package

Everything else is imported from the top level, and at a glance
appears to be all Python stdlib stuff.  The key period of execution
looks to be between the printing of the Python banner and the printing
of the adding changesets message. I see 46 imports (2 dlopens) from
the mercurial or hgext packages. That leaves 40 stdlib imports, of
which 16 were satisfied by dlopen.

As a final check, I saved all the stdlib import statements from the -v
output (77 statements) to a file and timed its execution:

    % time /usr/bin/python ~/tmp/pyimp.py

    real    0m0.162s
    user    0m0.034s
    sys     0m0.010s

It doesn't take much time to import every stdlib module that Mercurial
needs.  I don't know how much slower all this import machinery is in
3.x (and can't test at work, as we don't have a copy laying about). It
would probably have to be 3x or more slower for it to have much
visible impact on even simple hg commands.  I find it hard to believe
that freezing the stdlib is going to lower the barrier enough for the
Mercurial folks, if, in fact, import slowness is their main reason for
not moving to 3.x.

Skip

From brett at python.org  Tue Apr 15 17:29:43 2014
From: brett at python.org (Brett Cannon)
Date: Tue, 15 Apr 2014 11:29:43 -0400
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <CAG8k2+7ktuEr0Q=em6CtisGgd_YPK41pLiDnUoO1ti+Ntf75Mg@mail.gmail.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <CANc-5Uwanud0_CJ4ieeORyPjRz3z8=-UAzP2bVRXOXJjrDrbCg@mail.gmail.com>
 <CAP1=2W7pRNcHMasszuSPa0KU4GLHNh4F2r27xjQZxq8tsVZeqQ@mail.gmail.com>
 <CAPTjJmoJXr-HLz7FvXndc7ix9n-MwEbZ2-8=WiLNKOcMRKWFPw@mail.gmail.com>
 <CALFfu7DVaEge-_f3_nORrozR419ENrA+OGkqdL9e0m88xDXt5A@mail.gmail.com>
 <CAG8k2+7ktuEr0Q=em6CtisGgd_YPK41pLiDnUoO1ti+Ntf75Mg@mail.gmail.com>
Message-ID: <CAP1=2W7W8qt5i5C0d22Nm14g2DJzFB6D15rk4LYTw0QoMMLxqw@mail.gmail.com>

On Tue, Apr 15, 2014 at 11:19 AM, Daniel Holth <dholth at gmail.com> wrote:

> IIRC it is no longer the case that ZIP imports (involving only one
> file for a lot of modules) are much faster than regular FS imports?
>

It's definitely minimized since Python 3.3 and the caching of stat results
at the directory level for a small amount of time.

-Brett


>
> On Tue, Apr 15, 2014 at 10:34 AM, Eric Snow <ericsnowcurrently at gmail.com>
> wrote:
> > On Tue, Apr 15, 2014 at 1:45 AM, Chris Angelico <rosuav at gmail.com>
> wrote:
> >> Specific use-case that I can see: Mercurial. In a git vs hg shoot-out,
> >> git will usually win on performance, and hg is using Py2; migrating hg
> >> to Py3 will (if I understand the above figures correctly) widen that
> >> gap, so any improvement done to startup performance will give a very
> >> real advantage.
> >
> > Perhaps not so much "a very real advantage" as "less of a
> > distraction". It's still significantly slower than 2.7.  :)
> >
> > -eric
> > _______________________________________________
> > Python-Dev mailing list
> > Python-Dev at python.org
> > https://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/dholth%40gmail.com
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140415/58e1d000/attachment.html>

From rosuav at gmail.com  Tue Apr 15 17:40:21 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 16 Apr 2014 01:40:21 +1000
Subject: [Python-Dev] Mercurial sluggishness (was: this is what happens
 if you freeze all the modules required for startup)
In-Reply-To: <CANc-5UwL++W0_d8KXVF2+aG+_mcRC6FgvR+heTNYrw1-XmvpOg@mail.gmail.com>
References: <CANc-5UwL++W0_d8KXVF2+aG+_mcRC6FgvR+heTNYrw1-XmvpOg@mail.gmail.com>
Message-ID: <CAPTjJmqOGsTswpphxR_o9vTFPxOsPSg_iz_tu3cZcZKT_MPuyQ@mail.gmail.com>

On Wed, Apr 16, 2014 at 1:34 AM, Skip Montanaro <skip at pobox.com> wrote:
> I find it hard to believe
> that freezing the stdlib is going to lower the barrier enough for the
> Mercurial folks, if, in fact, import slowness is their main reason for
> not moving to 3.x.

I've no idea whether that's the case or not. All I know is, every time
I need to work with a Mercurial repo it feels a lot slower than doing
similar work on a similar size git repo [1], so any improvement (or
reduction of penalty) will be noticeable. Every time you type 'hg
something', it has to do those imports, so startup time is
significant.

ChrisA
[1] One time I actually did a conversion of CPython from hg into git,
just so I could do some analysis and digging. Worked out considerably
faster, although that's only because I left the import/conversion
process to run by itself while I made a hot chocolate, which meant I
was waiting less time.

From dholth at gmail.com  Tue Apr 15 17:42:52 2014
From: dholth at gmail.com (Daniel Holth)
Date: Tue, 15 Apr 2014 11:42:52 -0400
Subject: [Python-Dev] Mercurial sluggishness (was: this is what happens
 if you freeze all the modules required for startup)
In-Reply-To: <CANc-5UwL++W0_d8KXVF2+aG+_mcRC6FgvR+heTNYrw1-XmvpOg@mail.gmail.com>
References: <CANc-5UwL++W0_d8KXVF2+aG+_mcRC6FgvR+heTNYrw1-XmvpOg@mail.gmail.com>
Message-ID: <CAG8k2+6m5N8TaV01wkUHyuVXExCzupVFqAUC9oxqz605-Z1sgQ@mail.gmail.com>

I find Python's startup time to be very sluggish. I wish it was less
than 50 milliseconds (0.05 seconds) including running hg, which is the
common threshold for "instant". On my machine 'python -c ""' takes
about 0.05 seconds but 'python3 -c ""' takes 0.125 seconds. I will be
very happy to see any speedup.

On Tue, Apr 15, 2014 at 11:34 AM, Skip Montanaro <skip at pobox.com> wrote:
> Eric wrote:
>
>> Perhaps not so much "a very real advantage" as "less of a
>> distraction".  It's still significantly slower than 2.7.  :)
>
> I'm still confused. I peeked in /usr/bin/hg. The only "system" modules
> it imports directly are os and system (maybe I'm using an ancient
> version?). After that, it imports its own lazy import module. This
> suggests to me that Mercurial's import slowness is mostly in its own
> code (I counted 104 Python modules and six shared objects in its
> mercurial package, which isn't going to be affected (much?) by
> freezing the Python standard modules.
>
> I'm not trying to be difficult here. I thought that way back in the
> day a huge amount of work was done to remove needless filesystem
> activity, and zip packaging has been around for quite awhile.
>
> As an experiment, I ran hg pull as
>
>     /usr/bin/python -v /usr/bin/hg pull
>
> in my cpython repo then looked at the -v output. Summarizing the
> output I saw the following:
>
>     30 imports (0 dlopens)
>
>     Python banner printed
>
>     86 imports (18 dlopens)
>
>     adding changesets message
>
>     5 imports (2 dlopens)
>
>     adding manifests message
>
>     1 import (0 dlopens)
>
>     adding file changes message
>
>     7 imports (3 dlopens)
>
>     added ... changesets message
>
>     4 imports (0 dlopens)
>
>     run 'hg update' message
>
> (I missed a "searching" message in there somewhere.)
>
> That's a total of 133 imports, 23 of which were satisfied by loading
> an extension module. The imports break down as follows:
>
> 51 imports (4 dlopens) from the mercurial package
> 5 imports from the hgext package
> 7 imports from the encodings package
>
> Everything else is imported from the top level, and at a glance
> appears to be all Python stdlib stuff.  The key period of execution
> looks to be between the printing of the Python banner and the printing
> of the adding changesets message. I see 46 imports (2 dlopens) from
> the mercurial or hgext packages. That leaves 40 stdlib imports, of
> which 16 were satisfied by dlopen.
>
> As a final check, I saved all the stdlib import statements from the -v
> output (77 statements) to a file and timed its execution:
>
>     % time /usr/bin/python ~/tmp/pyimp.py
>
>     real    0m0.162s
>     user    0m0.034s
>     sys     0m0.010s
>
> It doesn't take much time to import every stdlib module that Mercurial
> needs.  I don't know how much slower all this import machinery is in
> 3.x (and can't test at work, as we don't have a copy laying about). It
> would probably have to be 3x or more slower for it to have much
> visible impact on even simple hg commands.  I find it hard to believe
> that freezing the stdlib is going to lower the barrier enough for the
> Mercurial folks, if, in fact, import slowness is their main reason for
> not moving to 3.x.
>
> Skip
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/dholth%40gmail.com

From tseaver at palladion.com  Tue Apr 15 17:46:56 2014
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 15 Apr 2014 11:46:56 -0400
Subject: [Python-Dev] Mercurial sluggishness (was: this is what happens
 if you freeze all the modules required for startup)
In-Reply-To: <CANc-5UwL++W0_d8KXVF2+aG+_mcRC6FgvR+heTNYrw1-XmvpOg@mail.gmail.com>
References: <CANc-5UwL++W0_d8KXVF2+aG+_mcRC6FgvR+heTNYrw1-XmvpOg@mail.gmail.com>
Message-ID: <lijk8i$cf0$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 04/15/2014 11:34 AM, Skip Montanaro wrote:
> I find it hard to believe that freezing the stdlib is going to lower
> the barrier enough for the Mercurial folks, if, in fact, import
> slowness is their main reason for not moving to 3.x.

My understanding of what Matt said at the language summit is that the
need to support really old versions of Python 2.x (back to 2.4) is a big
part of the holdup ("straddling" is *much* more painful without
constraining to
Python2 >= 2.6).

As I heard it, the real reason for the inertia is that the Python3 port
is a lot of effort / pain for zero perceived gain outside of "because it
is the Right Thing(TM)."  After my porting experience, I can sympathize
with that sensibility, and my stuff gets an advantage (frameworks /
libraries marketing to Python3 devs) that Hg doesn't (most users don't
really care which Python is used to drive the standalone tool).


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlNNVHAACgkQ+gerLs4ltQ4lpwCeJTYvfBBlE3cS+eq+kA4/zEi3
R+8AnRy4HYLRZ4DHhHDop/8A86MJt5Ei
=fORL
-----END PGP SIGNATURE-----


From skip at pobox.com  Tue Apr 15 17:53:26 2014
From: skip at pobox.com (Skip Montanaro)
Date: Tue, 15 Apr 2014 10:53:26 -0500
Subject: [Python-Dev] Mercurial sluggishness (was: this is what happens
 if you freeze all the modules required for startup)
In-Reply-To: <CAPTjJmqOGsTswpphxR_o9vTFPxOsPSg_iz_tu3cZcZKT_MPuyQ@mail.gmail.com>
References: <CANc-5UwL++W0_d8KXVF2+aG+_mcRC6FgvR+heTNYrw1-XmvpOg@mail.gmail.com>
 <CAPTjJmqOGsTswpphxR_o9vTFPxOsPSg_iz_tu3cZcZKT_MPuyQ@mail.gmail.com>
Message-ID: <CANc-5UwaK=8ZVRarVZfi_EExRzpy+EBHEBKHHe2oZQYCVLDLuw@mail.gmail.com>

On Tue, Apr 15, 2014 at 10:40 AM, Chris Angelico <rosuav at gmail.com> wrote:
> I've no idea whether that's the case or not. All I know is, every time
> I need to work with a Mercurial repo it feels a lot slower than doing
> similar work on a similar size git repo [1], so any improvement (or
> reduction of penalty) will be noticeable.

Based on what I saw, I really don't think that startup slowness is in
imports of Python stdlib modules, which is all Brett was aiming at. I
*can* believe overall import slowness might be a problem, but in that
case, Brett's work isn't going to help the Mercurial folks much.

Skip

From skip at pobox.com  Tue Apr 15 17:56:09 2014
From: skip at pobox.com (Skip Montanaro)
Date: Tue, 15 Apr 2014 10:56:09 -0500
Subject: [Python-Dev] Mercurial sluggishness (was: this is what happens
 if you freeze all the modules required for startup)
In-Reply-To: <CAG8k2+6m5N8TaV01wkUHyuVXExCzupVFqAUC9oxqz605-Z1sgQ@mail.gmail.com>
References: <CANc-5UwL++W0_d8KXVF2+aG+_mcRC6FgvR+heTNYrw1-XmvpOg@mail.gmail.com>
 <CAG8k2+6m5N8TaV01wkUHyuVXExCzupVFqAUC9oxqz605-Z1sgQ@mail.gmail.com>
Message-ID: <CANc-5Uyp0UxZWZguC-ttkts0EYmFioLR4AFZSqwRo1vbcEmYgw@mail.gmail.com>

On Tue, Apr 15, 2014 at 10:42 AM, Daniel Holth <dholth at gmail.com> wrote:
> I wish it was less
> than 50 milliseconds (0.05 seconds) including running hg, which is the
> common threshold for "instant".

"Instant" for me is "the blink of an eye," which Wikipedia reports as
typically between 100ms and 400ms.
<http://en.wikipedia.org/wiki/Blink> If you blink, you've missed
Python 2.7 startup on a relatively modern machine.

<wink>

Skip

From rosuav at gmail.com  Tue Apr 15 17:58:51 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 16 Apr 2014 01:58:51 +1000
Subject: [Python-Dev] Mercurial sluggishness (was: this is what happens
 if you freeze all the modules required for startup)
In-Reply-To: <CANc-5Uyp0UxZWZguC-ttkts0EYmFioLR4AFZSqwRo1vbcEmYgw@mail.gmail.com>
References: <CANc-5UwL++W0_d8KXVF2+aG+_mcRC6FgvR+heTNYrw1-XmvpOg@mail.gmail.com>
 <CAG8k2+6m5N8TaV01wkUHyuVXExCzupVFqAUC9oxqz605-Z1sgQ@mail.gmail.com>
 <CANc-5Uyp0UxZWZguC-ttkts0EYmFioLR4AFZSqwRo1vbcEmYgw@mail.gmail.com>
Message-ID: <CAPTjJmrPYjJLnZ+VhdQHXPYmhJw0qLS8JrirSSYLw4sozhWGmQ@mail.gmail.com>

On Wed, Apr 16, 2014 at 1:56 AM, Skip Montanaro <skip at pobox.com> wrote:
> On Tue, Apr 15, 2014 at 10:42 AM, Daniel Holth <dholth at gmail.com> wrote:
>> I wish it was less
>> than 50 milliseconds (0.05 seconds) including running hg, which is the
>> common threshold for "instant".
>
> "Instant" for me is "the blink of an eye," which Wikipedia reports as
> typically between 100ms and 400ms.
> <http://en.wikipedia.org/wiki/Blink> If you blink, you've missed
> Python 2.7 startup on a relatively modern machine.
>

400ms feels glitchy. 100ms is the absolute maximum for immediate
interaction. If I can sense a musical beat between doing something and
seeing its result, it's not instant.

And if there's a comedic beat between them, it's... laughably slow.

*dives for cover*

ChrisA

From stephen at xemacs.org  Tue Apr 15 18:15:57 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 16 Apr 2014 01:15:57 +0900
Subject: [Python-Dev] Appeal for reviews
In-Reply-To: <CADiSq7eucW-LgKYGe5H4faJqov19YaS3AU4M=yKmLhOHpOzMXQ@mail.gmail.com>
References: <87sipiwf1w.fsf@vostro.rath.org>
 <87ppkmgjg5.fsf@uwakimon.sk.tsukuba.ac.jp>
 <87fvlh10wt.fsf@vostro.rath.org>
 <87fvlghej1.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140414040557.Horde.FRrDDDnMkJvhmQHf-0NFHg1@webmail.df.eu>
 <877g6sh2w6.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CADiSq7fdJSuhsW0cuRVT-ZzqGjzQKMd7Vi8+vNbF0LOmNofxWA@mail.gmail.com>
 <20140414124054.133AA250CAE@webabinitio.net>
 <CADiSq7eucW-LgKYGe5H4faJqov19YaS3AU4M=yKmLhOHpOzMXQ@mail.gmail.com>
Message-ID: <87lhv6fu0y.fsf@uwakimon.sk.tsukuba.ac.jp>

On 14 Apr 2014 08:42, "R. David Murray" <rdmurray at bitdance.com> wrote:

 >> Or to put it another way, I'd like to encourage contributors who
 >> want to get commit access to focus just as much on doing good
 >> reviews as they do on writing new patches.  Currently the focus is
 >> all on getting patches accepted.

 >Huh, I hadn't thought of it that way before, but it's a very good
 >point.

AFAICS Python already does very well at getting people to do reviewing
by comparison to most projects, though.  And it's *not* all about
getting patches accepted -- newer people seem to do a lot of work on
PEPs and testing compared to most projects I've seen, and not just
because Python-Dev insists on them before getting code approved.

I've always really liked MvL's 5-reviews-to-get-1 approach.  In the
context of this thread it has a number of nice properties.  First, it
makes it explicit that cooperative work (even if it's expressed as
out-and-out horse-trading, it's still working together) is central to
python-dev.  Second, it makes that work visible if people post their
requests to either python-dev or core-mentorship.[1][2]  Third, it
emphasizes reviewing as a good thing and an important contribution.
People tend to think of reviews as "criticism", or even us-against-
them.  Again, the activity and the idea that it is a Good Thing is (or
can be) visible to the contributors in general.

The only thing I don't like about it[3] is that it puts an explicit
price on core developer time ("my time is worth 5x as much as
yours").  I fear that it may be a little off-putting.

In this vein, I wonder if a slot for "new comments on old issues" in
the tracker report might not be useful.  (Yeah, I know, the tracker
reporting function is free software. :)  Maybe a "most active issues"
report, too.

This isn't to deprecate the "50% core developer" idea at all -- it's
great!  I just don't know enough bosses in the field to know how to
sell that one.


Footnotes: 
[1]  Bazaar had a "patch pilot" program where an experienced developer
was detailed to clean the patch queue by shepherding newer developers
through their rather detailed process.  But it had two disadvantages:
first, it really was all about getting patches accepted, and second,
the actual piloting tended to be done off-list.

[2]  I know some people don't like core-mentorship because it's
somewhat less visible than python-dev.  Let's not discuss that now,
it's just an example.

[3]  Well, actually, when wearing my economist hat I like it a lot. :-)


From solipsis at pitrou.net  Tue Apr 15 18:40:50 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Apr 2014 18:40:50 +0200
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <CAPTjJmoJXr-HLz7FvXndc7ix9n-MwEbZ2-8=WiLNKOcMRKWFPw@mail.gmail.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <CANc-5Uwanud0_CJ4ieeORyPjRz3z8=-UAzP2bVRXOXJjrDrbCg@mail.gmail.com>
 <CAP1=2W7pRNcHMasszuSPa0KU4GLHNh4F2r27xjQZxq8tsVZeqQ@mail.gmail.com>
 <CAPTjJmoJXr-HLz7FvXndc7ix9n-MwEbZ2-8=WiLNKOcMRKWFPw@mail.gmail.com>
Message-ID: <lijnei$t6f$1@ger.gmane.org>

Le 15/04/2014 09:45, Chris Angelico a ?crit :
>
> Specific use-case that I can see: Mercurial. In a git vs hg shoot-out,
> git will usually win on performance, and hg is using Py2;

Keep in mind those shoot-outs usually rely on large repositories and/or 
non-trivial operations, so startup time is not necessarily a significant 
contributor in Mercurial "being slower" (when it actually is slower than 
git, which may not be all the time).

Regards

Antoine.



From solipsis at pitrou.net  Tue Apr 15 18:43:03 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Apr 2014 18:43:03 +0200
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
Message-ID: <lijnin$t6f$2@ger.gmane.org>

Le 14/04/2014 23:51, Brett Cannon a ?crit :
> It was realized during PyCon that since we are freezing importlib we
> could now consider freezing all the modules to cut out having to stat or
> read them from disk. So for day 1 of the sprints I I decided to hack up
> a proof-of-concept to see what kind of performance gain it would get.
>
> Freezing everything except encodings.__init__, os, and _sysconfigdata,
> it speeds up startup by 11% compared to default.

That sounds like a rather small number for the amount of complication 
and opacity it adds into the build and startup process.

Regards

Antoine.



From solipsis at pitrou.net  Tue Apr 15 18:47:39 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Apr 2014 18:47:39 +0200
Subject: [Python-Dev] Mercurial sluggishness (was: this is what happens
 if you freeze all the modules required for startup)
In-Reply-To: <CAG8k2+6m5N8TaV01wkUHyuVXExCzupVFqAUC9oxqz605-Z1sgQ@mail.gmail.com>
References: <CANc-5UwL++W0_d8KXVF2+aG+_mcRC6FgvR+heTNYrw1-XmvpOg@mail.gmail.com>
 <CAG8k2+6m5N8TaV01wkUHyuVXExCzupVFqAUC9oxqz605-Z1sgQ@mail.gmail.com>
Message-ID: <lijnrb$t6f$4@ger.gmane.org>

Le 15/04/2014 17:42, Daniel Holth a ?crit :
> I find Python's startup time to be very sluggish. I wish it was less
> than 50 milliseconds (0.05 seconds) including running hg, which is the
> common threshold for "instant". On my machine 'python -c ""' takes
> about 0.05 seconds but 'python3 -c ""' takes 0.125 seconds.

Please quote exact versions. Different versions of Python 3 will have 
different startup characteristics.

Regards

Antoine.



From rosuav at gmail.com  Tue Apr 15 18:54:05 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 16 Apr 2014 02:54:05 +1000
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <lijnei$t6f$1@ger.gmane.org>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <CANc-5Uwanud0_CJ4ieeORyPjRz3z8=-UAzP2bVRXOXJjrDrbCg@mail.gmail.com>
 <CAP1=2W7pRNcHMasszuSPa0KU4GLHNh4F2r27xjQZxq8tsVZeqQ@mail.gmail.com>
 <CAPTjJmoJXr-HLz7FvXndc7ix9n-MwEbZ2-8=WiLNKOcMRKWFPw@mail.gmail.com>
 <lijnei$t6f$1@ger.gmane.org>
Message-ID: <CAPTjJmpPbm+Fu-iENN35DMKYQSpO88qQ1fGNp3_XWm2CczUQog@mail.gmail.com>

On Wed, Apr 16, 2014 at 2:40 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le 15/04/2014 09:45, Chris Angelico a ?crit :
>
>>
>> Specific use-case that I can see: Mercurial. In a git vs hg shoot-out,
>> git will usually win on performance, and hg is using Py2;
>
>
> Keep in mind those shoot-outs usually rely on large repositories and/or
> non-trivial operations, so startup time is not necessarily a significant
> contributor in Mercurial "being slower" (when it actually is slower than
> git, which may not be all the time).

I'm talking also about the feel of actual daily use, partly on big
repos like git (git), CPython (hg), and Pike (git), and partly on some
smaller ones. Whether it's startup cost or operational cost I don't
know, but if I want it consistently fast, I generally go for git.

ChrisA

From solipsis at pitrou.net  Tue Apr 15 18:46:16 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Apr 2014 18:46:16 +0200
Subject: [Python-Dev] Mercurial sluggishness (was: this is what happens
 if you freeze all the modules required for startup)
In-Reply-To: <CANc-5UwL++W0_d8KXVF2+aG+_mcRC6FgvR+heTNYrw1-XmvpOg@mail.gmail.com>
References: <CANc-5UwL++W0_d8KXVF2+aG+_mcRC6FgvR+heTNYrw1-XmvpOg@mail.gmail.com>
Message-ID: <lijnoo$t6f$3@ger.gmane.org>

Le 15/04/2014 17:34, Skip Montanaro a ?crit :
> This
> suggests to me that Mercurial's import slowness is mostly in its own
> code (I counted 104 Python modules and six shared objects in its
> mercurial package, which isn't going to be affected (much?) by
> freezing the Python standard modules.

Skip is right. When trying to find out why the hgprompt extension (which 
is a rather nifty extension adding color-coded repository information 
into your bash prompt) made the shell so much slower, it turned out that 
most of the execution time comes from importing *Mercurial* modules, not 
stdlib modules.

Regards

Antoine.



From stephen at xemacs.org  Tue Apr 15 18:59:59 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 16 Apr 2014 01:59:59 +0900
Subject: [Python-Dev] Mercurial sluggishness (was: this is what happens
 if you freeze all the modules required for startup)
In-Reply-To: <CANc-5Uyp0UxZWZguC-ttkts0EYmFioLR4AFZSqwRo1vbcEmYgw@mail.gmail.com>
References: <CANc-5UwL++W0_d8KXVF2+aG+_mcRC6FgvR+heTNYrw1-XmvpOg@mail.gmail.com>
 <CAG8k2+6m5N8TaV01wkUHyuVXExCzupVFqAUC9oxqz605-Z1sgQ@mail.gmail.com>
 <CANc-5Uyp0UxZWZguC-ttkts0EYmFioLR4AFZSqwRo1vbcEmYgw@mail.gmail.com>
Message-ID: <87ioqafrzk.fsf@uwakimon.sk.tsukuba.ac.jp>

Skip Montanaro writes:

 > "Instant" for me is "the blink of an eye," which Wikipedia reports as
 > typically between 100ms and 400ms.
 > <http://en.wikipedia.org/wiki/Blink> If you blink, you've missed
 > Python 2.7 startup on a relatively modern machine.

This is what I see for Mac OS X "Mavericks" on a 2.7GHz Core i7:

# Apple's /usr/bin/python, Python 2.7.5, built with clang 5.0.0
$ time python -c ""  #     cold      warm      warm
               real      0.967s    0.020s    0.022s
               user      0.025s    0.011s    0.012s
               sys       0.061s    0.009s    0.007s
# MacPorts /opt/local/bin/python3.3, Python 3.3.5, same compiler
$ time python3.3 -c ""
               real      1.034s    0.041s    0.037s
               user      0.065s    0.030s    0.028s
               sys       0.036s    0.008s    0.007s

Further iterations of warm cache starts remain (coincidentally, I
would guess, but it's indicative) in the ranges above.  I don't feel
like rebooting or otherwise figuring out how to evict python from the
cache to get a sense of variation for cold cache startup, but
obviously it's more than an order of magnitude slower than warm start
for both Pythons.

Warm start numbers are well within Daniel's "50ms" spec.  Granted user
time for Python 3.3 is 2.5-3X that of Python 2.7 warm or cold, it's
still below human JND (if you can see it, it's probably a slow display
;-).  So it's all about waiting on the OS, it seems to me.

By comparison, git:

$ time git --version  #    cold      warm      warm
               real      0.430s    0.017s    0.021s
               user      0.007s    0.006s    0.006s
               sys       0.009s    0.006s    0.006s

OK, Python 2.7 is slower than git and Python 3.3 much slower.  But I
don't think this explains anybody's feeling that hg is sluggish
compared to git -- git also shows perceptible delay on a cold start, I
didn't notice it being faster (I wasn't thinking about it, though, and
I wasn't in a hurry to see the version string).  I suspect Linus has
spiked everybody's Kool-Aid and it's a mass hallucination.  More
seriously, I wouldn't be surprised if git is just better optimized for
certain operations that people expect to be fast.

From dholth at gmail.com  Tue Apr 15 19:09:16 2014
From: dholth at gmail.com (Daniel Holth)
Date: Tue, 15 Apr 2014 13:09:16 -0400
Subject: [Python-Dev] Mercurial sluggishness (was: this is what happens
 if you freeze all the modules required for startup)
In-Reply-To: <lijnrb$t6f$4@ger.gmane.org>
References: <CANc-5UwL++W0_d8KXVF2+aG+_mcRC6FgvR+heTNYrw1-XmvpOg@mail.gmail.com>
 <CAG8k2+6m5N8TaV01wkUHyuVXExCzupVFqAUC9oxqz605-Z1sgQ@mail.gmail.com>
 <lijnrb$t6f$4@ger.gmane.org>
Message-ID: <CAG8k2+7jw=3Z0FwfVKcvpn=T+-moWq=zOXKY3uScbOPe3pt51Q@mail.gmail.com>

In case you were wondering, I'm using Ubuntu's "2.7.5+" and "3.3.2+".

My feeling has long been that the speed of getting at the "--help"
option or any initial user feedback from Mercurial or git is a big
driver in perceived speed as opposed to how long the entire operation
might take. But for me any initial speed improvements from git are
fully offset by the feeling of irritation afterwards. /troll

For me Python's startup time (warm) takes about 1/4 of the hg startup
time in the worst case. I expect to both notice and appreciate any
speedups and encourage all optimizers to optimize.

On Tue, Apr 15, 2014 at 12:47 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le 15/04/2014 17:42, Daniel Holth a ?crit :
>
>> I find Python's startup time to be very sluggish. I wish it was less
>> than 50 milliseconds (0.05 seconds) including running hg, which is the
>> common threshold for "instant". On my machine 'python -c ""' takes
>> about 0.05 seconds but 'python3 -c ""' takes 0.125 seconds.
>
>
> Please quote exact versions. Different versions of Python 3 will have
> different startup characteristics.
>
> Regards
>
> Antoine.
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/dholth%40gmail.com

From solipsis at pitrou.net  Tue Apr 15 19:29:48 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Apr 2014 19:29:48 +0200
Subject: [Python-Dev] Mercurial sluggishness (was: this is what happens
 if you freeze all the modules required for startup)
In-Reply-To: <CAG8k2+7jw=3Z0FwfVKcvpn=T+-moWq=zOXKY3uScbOPe3pt51Q@mail.gmail.com>
References: <CANc-5UwL++W0_d8KXVF2+aG+_mcRC6FgvR+heTNYrw1-XmvpOg@mail.gmail.com>
 <CAG8k2+6m5N8TaV01wkUHyuVXExCzupVFqAUC9oxqz605-Z1sgQ@mail.gmail.com>
 <lijnrb$t6f$4@ger.gmane.org>
 <CAG8k2+7jw=3Z0FwfVKcvpn=T+-moWq=zOXKY3uScbOPe3pt51Q@mail.gmail.com>
Message-ID: <lijqac$jgk$1@ger.gmane.org>

Le 15/04/2014 19:09, Daniel Holth a ?crit :
> In case you were wondering, I'm using Ubuntu's "2.7.5+" and "3.3.2+".
>
> My feeling has long been that the speed of getting at the "--help"
> option or any initial user feedback from Mercurial or git is a big
> driver in perceived speed as opposed to how long the entire operation
> might take. But for me any initial speed improvements from git are
> fully offset by the feeling of irritation afterwards. /troll
>
> For me Python's startup time (warm) takes about 1/4 of the hg startup
> time in the worst case. I expect to both notice and appreciate any
> speedups and encourage all optimizers to optimize.

Well, if we optimize 11% out of that 1/4, I don't expect you to notice 
the speedup at all ;-)

Regards

Antoine.



From p.f.moore at gmail.com  Tue Apr 15 19:36:58 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 15 Apr 2014 18:36:58 +0100
Subject: [Python-Dev] Mercurial sluggishness (was: this is what happens
 if you freeze all the modules required for startup)
In-Reply-To: <CAG8k2+7jw=3Z0FwfVKcvpn=T+-moWq=zOXKY3uScbOPe3pt51Q@mail.gmail.com>
References: <CANc-5UwL++W0_d8KXVF2+aG+_mcRC6FgvR+heTNYrw1-XmvpOg@mail.gmail.com>
 <CAG8k2+6m5N8TaV01wkUHyuVXExCzupVFqAUC9oxqz605-Z1sgQ@mail.gmail.com>
 <lijnrb$t6f$4@ger.gmane.org>
 <CAG8k2+7jw=3Z0FwfVKcvpn=T+-moWq=zOXKY3uScbOPe3pt51Q@mail.gmail.com>
Message-ID: <CACac1F_ongfMawa3h5wDJfQzkLQRZEnYusazu-PN6HvpMc2bAA@mail.gmail.com>

On 15 April 2014 18:09, Daniel Holth <dholth at gmail.com> wrote:
> For me Python's startup time (warm) takes about 1/4 of the hg startup
> time in the worst case. I expect to both notice and appreciate any
> speedups and encourage all optimizers to optimize.

My experience is essentially irrelevant (as a Windows user, the OS
process creation time makes any of the numbers people are quoting for
Unix little more than a pipe dream for me :-)) but it seems to me that
the key measure of "sluggishness" tends to be on the tiny query
operations (the things people put in their prompt). Nobody cares about
process startup time on a hg clone of a 1GB repo across the network.
But "hg status" to get details of the current repo to show in your
prompt has to be very fast, or people complain "it's slow" [1].

On that point, I suspect that git's approach of having a plethora of
tiny focused commands each of which does only what it needs to,
probably makes for a better "simple things are fast" experience than a
single-application architecture like hg. (On the other hand, it
utterly kills git's performance on Windows, so you win some, you lose
some).

So improving Python startup time will probably help with Mercurial's
*perceived* speed - but architecture improvements focusing on running
enquiry type commands really fast may help as much if not more (I
don't know how much they've optimised for that case in the past).

On 15 April 2014 18:29, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Well, if we optimize 11% out of that 1/4, I don't expect you to notice the
> speedup at all ;-)

Yeah, in reality no one thing is ever going to be *that* perceptible.
But as they say, "every little helps".

Paul

[1] Windows process startup times affect this *a lot*. My powershell
prompt function directly reads the files in .hg/.git because running
the actual commands is far too slow for a prompt function.

From dholth at gmail.com  Tue Apr 15 19:32:43 2014
From: dholth at gmail.com (Daniel Holth)
Date: Tue, 15 Apr 2014 13:32:43 -0400
Subject: [Python-Dev] Mercurial sluggishness (was: this is what happens
 if you freeze all the modules required for startup)
In-Reply-To: <lijqac$jgk$1@ger.gmane.org>
References: <CANc-5UwL++W0_d8KXVF2+aG+_mcRC6FgvR+heTNYrw1-XmvpOg@mail.gmail.com>
 <CAG8k2+6m5N8TaV01wkUHyuVXExCzupVFqAUC9oxqz605-Z1sgQ@mail.gmail.com>
 <lijnrb$t6f$4@ger.gmane.org>
 <CAG8k2+7jw=3Z0FwfVKcvpn=T+-moWq=zOXKY3uScbOPe3pt51Q@mail.gmail.com>
 <lijqac$jgk$1@ger.gmane.org>
Message-ID: <CAG8k2+5QFwg4fx1K9Tt13KPO8-AGdNVUYAObDOTudj_TrG4NFw@mail.gmail.com>

On Tue, Apr 15, 2014 at 1:29 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le 15/04/2014 19:09, Daniel Holth a ?crit :
>
>> In case you were wondering, I'm using Ubuntu's "2.7.5+" and "3.3.2+".
>>
>> My feeling has long been that the speed of getting at the "--help"
>> option or any initial user feedback from Mercurial or git is a big
>> driver in perceived speed as opposed to how long the entire operation
>> might take. But for me any initial speed improvements from git are
>> fully offset by the feeling of irritation afterwards. /troll
>>
>> For me Python's startup time (warm) takes about 1/4 of the hg startup
>> time in the worst case. I expect to both notice and appreciate any
>> speedups and encourage all optimizers to optimize.
>
>
> Well, if we optimize 11% out of that 1/4, I don't expect you to notice the
> speedup at all ;-)
>
>
> Regards
>
> Antoine.

No one expects the Spanish Inquisition.

From breamoreboy at yahoo.co.uk  Tue Apr 15 19:46:46 2014
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 15 Apr 2014 18:46:46 +0100
Subject: [Python-Dev] Mercurial sluggishness (was: this is what happens
 if you freeze all the modules required for startup)
In-Reply-To: <CAG8k2+5QFwg4fx1K9Tt13KPO8-AGdNVUYAObDOTudj_TrG4NFw@mail.gmail.com>
References: <CANc-5UwL++W0_d8KXVF2+aG+_mcRC6FgvR+heTNYrw1-XmvpOg@mail.gmail.com>
 <CAG8k2+6m5N8TaV01wkUHyuVXExCzupVFqAUC9oxqz605-Z1sgQ@mail.gmail.com>
 <lijnrb$t6f$4@ger.gmane.org>
 <CAG8k2+7jw=3Z0FwfVKcvpn=T+-moWq=zOXKY3uScbOPe3pt51Q@mail.gmail.com>
 <lijqac$jgk$1@ger.gmane.org>
 <CAG8k2+5QFwg4fx1K9Tt13KPO8-AGdNVUYAObDOTudj_TrG4NFw@mail.gmail.com>
Message-ID: <lijra8$107$1@ger.gmane.org>

On 15/04/2014 18:32, Daniel Holth wrote:
> On Tue, Apr 15, 2014 at 1:29 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> Le 15/04/2014 19:09, Daniel Holth a ?crit :
>>
>>> In case you were wondering, I'm using Ubuntu's "2.7.5+" and "3.3.2+".
>>>
>>> My feeling has long been that the speed of getting at the "--help"
>>> option or any initial user feedback from Mercurial or git is a big
>>> driver in perceived speed as opposed to how long the entire operation
>>> might take. But for me any initial speed improvements from git are
>>> fully offset by the feeling of irritation afterwards. /troll
>>>
>>> For me Python's startup time (warm) takes about 1/4 of the hg startup
>>> time in the worst case. I expect to both notice and appreciate any
>>> speedups and encourage all optimizers to optimize.
>>
>>
>> Well, if we optimize 11% out of that 1/4, I don't expect you to notice the
>> speedup at all ;-)
>>
>>
>> Regards
>>
>> Antoine.
>
> No one expects the Spanish Inquisition.

Except those who expect Python 2.8?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.

Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com



From tjreedy at udel.edu  Tue Apr 15 20:28:34 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 15 Apr 2014 14:28:34 -0400
Subject: [Python-Dev] Appeal for reviews
In-Reply-To: <87lhv6fu0y.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <87sipiwf1w.fsf@vostro.rath.org>
 <87ppkmgjg5.fsf@uwakimon.sk.tsukuba.ac.jp> <87fvlh10wt.fsf@vostro.rath.org>
 <87fvlghej1.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140414040557.Horde.FRrDDDnMkJvhmQHf-0NFHg1@webmail.df.eu>
 <877g6sh2w6.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CADiSq7fdJSuhsW0cuRVT-ZzqGjzQKMd7Vi8+vNbF0LOmNofxWA@mail.gmail.com>
 <20140414124054.133AA250CAE@webabinitio.net>
 <CADiSq7eucW-LgKYGe5H4faJqov19YaS3AU4M=yKmLhOHpOzMXQ@mail.gmail.com>
 <87lhv6fu0y.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <lijtp8$eli$1@ger.gmane.org>

On 4/15/2014 12:15 PM, Stephen J. Turnbull wrote:

> I've always really liked MvL's 5-reviews-to-get-1 approach.

> The only thing I don't like about it[3] is that it puts an explicit
> price on core developer time ("my time is worth 5x as much as
> yours").

Not really true since any of the 5 could be one or two sentence comments 
while his 1 might include committing, or a second review after revision. 
Besides which, a person accepting the offer got to choose both the 5 to 
review and the 1 to be reviewed. Choice has value.

> I fear that it may be a little off-putting.

Perhaps, but it is also attractive enough that there have been several 
requests for someone to renew the offer, and no core developer currently 
willing to make the offer (because giving up choice is a cost).

-- 
Terry Jan Reedy


From guido at python.org  Tue Apr 15 20:54:54 2014
From: guido at python.org (Guido van Rossum)
Date: Tue, 15 Apr 2014 14:54:54 -0400
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <CAPTjJmpPbm+Fu-iENN35DMKYQSpO88qQ1fGNp3_XWm2CczUQog@mail.gmail.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <CANc-5Uwanud0_CJ4ieeORyPjRz3z8=-UAzP2bVRXOXJjrDrbCg@mail.gmail.com>
 <CAP1=2W7pRNcHMasszuSPa0KU4GLHNh4F2r27xjQZxq8tsVZeqQ@mail.gmail.com>
 <CAPTjJmoJXr-HLz7FvXndc7ix9n-MwEbZ2-8=WiLNKOcMRKWFPw@mail.gmail.com>
 <lijnei$t6f$1@ger.gmane.org>
 <CAPTjJmpPbm+Fu-iENN35DMKYQSpO88qQ1fGNp3_XWm2CczUQog@mail.gmail.com>
Message-ID: <CAP7+vJLw0V4-cus-vRK3rsa4jNAcQK0pJcSNNWpDnus5e7qN5A@mail.gmail.com>

Can we please stop the argument about Hg vs. Git?


On Tue, Apr 15, 2014 at 12:54 PM, Chris Angelico <rosuav at gmail.com> wrote:

> On Wed, Apr 16, 2014 at 2:40 AM, Antoine Pitrou <solipsis at pitrou.net>
> wrote:
> > Le 15/04/2014 09:45, Chris Angelico a ?crit :
> >
> >>
> >> Specific use-case that I can see: Mercurial. In a git vs hg shoot-out,
> >> git will usually win on performance, and hg is using Py2;
> >
> >
> > Keep in mind those shoot-outs usually rely on large repositories and/or
> > non-trivial operations, so startup time is not necessarily a significant
> > contributor in Mercurial "being slower" (when it actually is slower than
> > git, which may not be all the time).
>
> I'm talking also about the feel of actual daily use, partly on big
> repos like git (git), CPython (hg), and Pike (git), and partly on some
> smaller ones. Whether it's startup cost or operational cost I don't
> know, but if I want it consistently fast, I generally go for git.
>
> ChrisA
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140415/783dbb51/attachment-0001.html>

From rosuav at gmail.com  Tue Apr 15 20:56:44 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 16 Apr 2014 04:56:44 +1000
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <CAP7+vJLw0V4-cus-vRK3rsa4jNAcQK0pJcSNNWpDnus5e7qN5A@mail.gmail.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <CANc-5Uwanud0_CJ4ieeORyPjRz3z8=-UAzP2bVRXOXJjrDrbCg@mail.gmail.com>
 <CAP1=2W7pRNcHMasszuSPa0KU4GLHNh4F2r27xjQZxq8tsVZeqQ@mail.gmail.com>
 <CAPTjJmoJXr-HLz7FvXndc7ix9n-MwEbZ2-8=WiLNKOcMRKWFPw@mail.gmail.com>
 <lijnei$t6f$1@ger.gmane.org>
 <CAPTjJmpPbm+Fu-iENN35DMKYQSpO88qQ1fGNp3_XWm2CczUQog@mail.gmail.com>
 <CAP7+vJLw0V4-cus-vRK3rsa4jNAcQK0pJcSNNWpDnus5e7qN5A@mail.gmail.com>
Message-ID: <CAPTjJmq1D6ef+HtRS6XvJmtXHJ6-5ce1ybtKs1+LOsUKf61pGQ@mail.gmail.com>

On Wed, Apr 16, 2014 at 4:54 AM, Guido van Rossum <guido at python.org> wrote:
> Can we please stop the argument about Hg vs. Git?

My apologies. All I was saying was that hg is a use case where startup
performance really does matter, as opposed to the ones presented
earlier in the thread where a process stays in memory longer. It
wasn't meant to devolve like that.

ChrisA

From stephen at xemacs.org  Tue Apr 15 21:22:43 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 16 Apr 2014 04:22:43 +0900
Subject: [Python-Dev] Appeal for reviews
In-Reply-To: <lijtp8$eli$1@ger.gmane.org>
References: <87sipiwf1w.fsf@vostro.rath.org>
 <87ppkmgjg5.fsf@uwakimon.sk.tsukuba.ac.jp>
 <87fvlh10wt.fsf@vostro.rath.org>
 <87fvlghej1.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140414040557.Horde.FRrDDDnMkJvhmQHf-0NFHg1@webmail.df.eu>
 <877g6sh2w6.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CADiSq7fdJSuhsW0cuRVT-ZzqGjzQKMd7Vi8+vNbF0LOmNofxWA@mail.gmail.com>
 <20140414124054.133AA250CAE@webabinitio.net>
 <CADiSq7eucW-LgKYGe5H4faJqov19YaS3AU4M=yKmLhOHpOzMXQ@mail.gmail.com>
 <87lhv6fu0y.fsf@uwakimon.sk.tsukuba.ac.jp>
 <lijtp8$eli$1@ger.gmane.org>
Message-ID: <87fvlefldo.fsf@uwakimon.sk.tsukuba.ac.jp>

Terry Reedy writes:
 > On 4/15/2014 12:15 PM, Stephen J. Turnbull wrote:

 > > The only thing I don't like about it[3] is that it puts an
 > > explicit price on core developer time ("my time is worth 5x as
 > > much as yours").
 > 
 > Not really true

But that is *not* your call!  It's for the would-be contributor to
decide, because they're the one to decide whether to take the offer.

 > > I fear that it may be a little off-putting.
 > 
 > Perhaps, but it is also attractive enough that there have been
 > several requests for someone to renew the offer, and no core
 > developer currently willing to make the offer (because giving up
 > choice is a cost).

Sure.  That doesn't mean there wouldn't be more people to take up the
offer if it didn't implicitly deprecate the value of their time.

The issue here is not the reality of "value of choice", it's whether
would-be developers *perceive* the "price" negatively or not.  If they
don't, or can easily be convinced that it makes sense, great!

Unfortunately the point is moot at the moment.


From bcannon at gmail.com  Tue Apr 15 23:26:42 2014
From: bcannon at gmail.com (Brett Cannon)
Date: Tue, 15 Apr 2014 17:26:42 -0400
Subject: [Python-Dev] Timing breakdown of Py_InitializeEx_Private()
Message-ID: <CAP1=2W64X6YHT7OQZ4KQ6WOq2wV=rj848XprL091rQ7NN55aCA@mail.gmail.com>

To finish my timing work I decided to see where Py_InitializeEx_Private()
spends its time. The following is a breakdown measured in microseconds
running using -E:

INIT:
setlocale: 11
envvar: 2
random init: 2
interp creation: 15
thread creation: 6
GIL: 10
_Py_ReadyTypes(): 930
more types: 44
builtins: 141
exceptions: 287
sys: 258
_PyImport_Init: 15
import hooks: 4
_PyWarnings_Init(): 15
ENTERING import_init():
  PyImport_ImportFrozenModule(_frozen_importlib): 1186
  interp->importlib: 6
  PyInit_imp(): 15
  _imp: 3
  importlib._install(): 876
  _PyImportZip_Init(): 130
_PyFaulthandler_Init(): 13
time: 3
ENTERING initfsencoding():
  codec lookup: 2179
signals: 120
tracemalloc: 7
__main__: 13
initstdio(): 1568
warnings module: 4
initsite(): 9552
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140415/c27d7b33/attachment.html>

From guido at python.org  Tue Apr 15 23:40:54 2014
From: guido at python.org (Guido van Rossum)
Date: Tue, 15 Apr 2014 14:40:54 -0700
Subject: [Python-Dev] Timing breakdown of Py_InitializeEx_Private()
In-Reply-To: <CAP1=2W64X6YHT7OQZ4KQ6WOq2wV=rj848XprL091rQ7NN55aCA@mail.gmail.com>
References: <CAP1=2W64X6YHT7OQZ4KQ6WOq2wV=rj848XprL091rQ7NN55aCA@mail.gmail.com>
Message-ID: <CAP7+vJJrr9Qr+6EHj=rffbfbXztzRxcCX-Vy2=j-gYk3Zgh9TA@mail.gmail.com>

Are you going to post your code (or a link to it)?


On Tue, Apr 15, 2014 at 2:26 PM, Brett Cannon <bcannon at gmail.com> wrote:

> To finish my timing work I decided to see where Py_InitializeEx_Private()
> spends its time. The following is a breakdown measured in microseconds
> running using -E:
>
> INIT:
> setlocale: 11
> envvar: 2
> random init: 2
> interp creation: 15
> thread creation: 6
> GIL: 10
> _Py_ReadyTypes(): 930
> more types: 44
> builtins: 141
> exceptions: 287
> sys: 258
> _PyImport_Init: 15
> import hooks: 4
> _PyWarnings_Init(): 15
> ENTERING import_init():
>   PyImport_ImportFrozenModule(_frozen_importlib): 1186
>   interp->importlib: 6
>   PyInit_imp(): 15
>   _imp: 3
>   importlib._install(): 876
>   _PyImportZip_Init(): 130
> _PyFaulthandler_Init(): 13
> time: 3
> ENTERING initfsencoding():
>   codec lookup: 2179
> signals: 120
> tracemalloc: 7
> __main__: 13
> initstdio(): 1568
> warnings module: 4
> initsite(): 9552
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140415/36e1ff56/attachment.html>

From bcannon at gmail.com  Tue Apr 15 23:56:34 2014
From: bcannon at gmail.com (Brett Cannon)
Date: Tue, 15 Apr 2014 17:56:34 -0400
Subject: [Python-Dev] Timing breakdown of Py_InitializeEx_Private()
In-Reply-To: <CAP7+vJJrr9Qr+6EHj=rffbfbXztzRxcCX-Vy2=j-gYk3Zgh9TA@mail.gmail.com>
References: <CAP1=2W64X6YHT7OQZ4KQ6WOq2wV=rj848XprL091rQ7NN55aCA@mail.gmail.com>
 <CAP7+vJJrr9Qr+6EHj=rffbfbXztzRxcCX-Vy2=j-gYk3Zgh9TA@mail.gmail.com>
Message-ID: <CAP1=2W7R7UADkRpMHw-yTeExTHxo-y3ahCLJ=dvY0Jp0iN9N-A@mail.gmail.com>

On Tue, Apr 15, 2014 at 5:40 PM, Guido van Rossum <guido at python.org> wrote:

> Are you going to post your code (or a link to it)?
>

I basically wrote a function that uses gettimeofday() and just calculates
the delta between the calls. Greg Smith verified that I wasn't doing
anything stupid. =) Anyway, the diff can be found at
https://gist.github.com/brettcannon/9cd3960dd067bb7a45bd .

-Brett


>
>
> On Tue, Apr 15, 2014 at 2:26 PM, Brett Cannon <bcannon at gmail.com> wrote:
>
>> To finish my timing work I decided to see where Py_InitializeEx_Private()
>> spends its time. The following is a breakdown measured in microseconds
>> running using -E:
>>
>> INIT:
>> setlocale: 11
>> envvar: 2
>> random init: 2
>> interp creation: 15
>> thread creation: 6
>> GIL: 10
>> _Py_ReadyTypes(): 930
>> more types: 44
>> builtins: 141
>> exceptions: 287
>> sys: 258
>> _PyImport_Init: 15
>> import hooks: 4
>> _PyWarnings_Init(): 15
>> ENTERING import_init():
>>   PyImport_ImportFrozenModule(_frozen_importlib): 1186
>>   interp->importlib: 6
>>   PyInit_imp(): 15
>>   _imp: 3
>>   importlib._install(): 876
>>   _PyImportZip_Init(): 130
>> _PyFaulthandler_Init(): 13
>> time: 3
>> ENTERING initfsencoding():
>>   codec lookup: 2179
>> signals: 120
>> tracemalloc: 7
>> __main__: 13
>> initstdio(): 1568
>> warnings module: 4
>> initsite(): 9552
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140415/10643e1e/attachment.html>

From Steve.Dower at microsoft.com  Wed Apr 16 00:55:01 2014
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Tue, 15 Apr 2014 22:55:01 +0000
Subject: [Python-Dev] Python "2migr8"
In-Reply-To: <CAP7+vJKRpZ1KDxaVAFbJye++1CkZkbwp_pV43Rpje34uwuFf=w@mail.gmail.com>
References: <9f950fc6989849fbba2d604cdb7db0c2@BLUPR03MB389.namprd03.prod.outlook.com>
 <CAP7+vJKRpZ1KDxaVAFbJye++1CkZkbwp_pV43Rpje34uwuFf=w@mail.gmail.com>
Message-ID: <4db20bc6986e4439bb1662327dfb6cb0@BLUPR03MB389.namprd03.prod.outlook.com>

Apologies for the slow reply, I was travelling back from PyCon.

From Guido:
> - I'd prefer a name that plays on 2 and 3, not 2 and 8. :-)

I didn't really expect the name to stick, but Nick and I had such a good laugh that it would have been unfair not to share it :-) (though I laughed even more at 'migr2**3')

Really, I'd expect the name to be pretty boring - "Python Straddle", "Python 2 Strict" or "Python Migration Edition" (if anyone can live with Python ME ;-) ), since the aim is to discourage general use. Users can (and should) keep using Python 2.7 while the development team is using the stricter version - or however this looks for the organisations that want to use it.

> - Are you sure this isn't better directed to python-ideas first? Most ideas have to prove their worth in that list before python-dev will give them the light of day.

It was a fifty-fifty choice between here and there. Python-dev had the more recent discussion about this issue and it was also raised at the language summit.

> - When it comes to purely syntactic issues (e.g. "except x, y:") a linter or some other separate tool can handle this well (heck, you can build it into an import hook probably :-).

True, but that isn't quite the same forcing function that interpreter errors would give. Code reviews are great until all of your potential contributors start with commit rights (and you are stuck on a VCS that has no concept of commit hooks... yes, there is also a tooling issue for the people who want this, but there also isn't a "blessed" tool for this purpose).

Eric Snow suggested that this entire idea could probably be handled as an import hook, which may mean that the whole thing could just be a PyPI package that works with 2.7. If so, great! But it still needs the blessing of python-dev, and probably at least one contributor who is also a core developer.

> - When it's about backported modules, a sumo distribution is probably the way to go; when it's about renamed stdlib modules, six (perhaps an extended version) should cover you.

Agreed. The advantage of the sumo distribution is that not every enterprise finds it easy to add more dependencies (I think this discussion happened on distutils-sig) - when it's "just part of Python and Python is already approved by our lawyers/IT guys/whoever" things are significantly easier.

> - Regarding warning about the changed dict API, I wonder how you plan to implement that if you allow passing dict object back and forth between code that has opted in to single-source and code that hasn't yet. Please think through some specific examples before responding.

I haven't thought through specific examples, so instead I'll describe my vision for how this solves the issue (it finally dawned on my during a jet-lagged dream - let's see if it still makes sense :-) ):

Migrating from 2.7 to 3.3[1] causes a lot of pain - let's call it 100% pain. You have to do 100% of the migration work before it will work.

Phase 1: Migrating from 2.7 to the straddle-edition causes a small amount of unconditional warnings and performance hit (these would be for things considered "poor form" anyway, since they are errors in 3.3), but no behaviour changes. There is some pain in fixing these, let's say 10%, but it isn't pain-pain because your program still runs. You can defer this 10% pain entirely into Phase 2 if you want.

Phase 2: Migrating from straddle-edition files without the flag to files with the flag is 70-80% of pain, distributed over a long period of time, during which your program still runs.

Phase 3: Migrating from straddle-edition where every file is flagged to 3.3 covers the last 10-20% of pain. This pain is real, since your program will not run against 3.3 until it is fixed, but it is significantly smaller than going directly from 2.7.

The last phase includes the final pass of "should this string literal be bytes?" work, but even that could be partially moved into Phase 2 by failing comparisons between byte literals and strings (apologies for the simplistic idea that is probably close to impossible to implement - the fact that nobody has a good solution to this is really the core of the entire issue...).

Basically, this idea allows the migration pain to be distributed over a longer period of time.


(1: I keep using 3.3 as my target because 3.4 has no new syntax over 3.3 - the new features in 3.4 are basically irrelevant to porting. The target version is also basically irrelevant to this discussion.)

> - But the biggest issue is of course bytes vs. text. You would have to first do a careful analysis of the *whole* problem before you can even think about proposing a solution. Too many people think there is an easy solution for this; but almost everybody is focused on only part of the problem (the part that causes them immediate pain) without realizing that other people's pain may be different.

Agreed. Maybe there is a need for different levels of migration here ("use 2.7's model", "distinct bytes type that behaves like str under 2.7", "distinct bytes type that warns when mixed with str", "use 3's model") so that there are clear steps along the path to getting it right. 

> - As far as your assertion that it would have to come from python-dev because nobody outside is going to tackle it, I think it's the opposite: the core developers would prefer not to have to deal with this, while some folks outside the inner circles will not be discouraged by our opinions (e.g. Stackless is working on "Stackless 2.8").

I agree entirely, but I don't think the first part is the opposite of the second part. The "core CPython team" should probably not be the owner of this fork, but there will certainly be some overlap between the core developers and the straddle-edition team. 

> - Regarding open source projects having a reputation of "not taking contributions", I would guess that this is usually about those "contributions" violating the most basic rules of the project (and I don't mean the coding style). I don't [corrected - Steve] want to discourage discussions with users like the company you referred to, but I think it would be much more useful if they laid out their problems for us instead of expecting they can buy acceptance for a "solution" they develop in-house. We could then hopefully have a productive dialog over many months where we iterate over possible approaches that could be acceptable both to Python and to the customer. But it will take a significant investment of time on both sides -- there is no shortcut. And it's not a particularly interesting problem (for most people) to work on -- things like designing a notation for optional type declarations are always much more fun. :-)

Agreed, but python-dev has a great mix of those who are interested in making Python awesome in the future and those who want to make Python awesome now (this is especially obvious in the packaging discussions). 


From Terry:
> This should be a separate project from pydev, even if under the PSF umbrella.

Agree entirely, but as mentioned above, I'd be extremely surprised if pydev wanted to be in a position of complete denial about the project. If that were the case, I'd have concerns about using it at all.

> A reasonable request of pydev would be for python-coded stdlib modules to be updated as much as possible, if that has not already been done. No 'apply', no 'except SomeException, e'.

I think it was addressed later, but this only really helps those who copy code from the stdlib. I would not expect the stdlib files to get the "single-source only" flag - there's no reason it shouldn't match 2.7 exactly. (Or maybe there is and I haven't thought of it, but I don't think this is the reason.)

> For PSF/CPython, this is so untrue that it looks to me like an excuse to take without giving back. This might be 'unfair and incorrect', but it is my perception.

>>     > On 4/14/2014 11:32 AM, Steve Dower wrote:
>>     [...]
>>     >> However unfair
>>     >> and incorrect it may be, there is a perception in some businesses
>>     >> that open-source projects do not want contributions from them.
>
> I took this to mean that they think any 'do not want' is peculiar to them as from a business. Perhaps I was wrong to project that on them.

No, this is exactly what I meant.

Over the years there has developed a strong sense (at least on the closed-source side) of "us" versus "them", due to silly things that have been said by both sides (Microsoft was certainly guilty of this). The bridge has been burned and CPython is doing a great job of rebuilding it, but the big/established companies still need to get used to the idea that paid and unpaid developers are not enemies and we can all contribute to these projects. Slow progress, but we're getting there :-)



From Guido:
> I really hope to have a direct conversation with some companies in this situation, but unfortunately they didn't approach me at PyCon -- they only approached Steve (perhaps because he works for a brand they recognize and trust :-).

Quoting this here because it's directly relevant to the previous points. Companies that have never had dealings with open source see talking to Guido as akin to cold-calling a different company's CEO. Getting someone lurking on python-dev is a good invitation to start with, as is attending PyCon. (These companies understand conferences, especially when they have to pay to attend. Go figure...)

 
From Eric:
> This should be doable with an import hook.  For that matter it would be pretty straight-forward to provide a script around the Python binary that installs the import hook.  All this could be bundled up and distributed through the cheeseshop.

I love this solution. If it really is doable this way then it is even better than having a fork. My name suggestion for a PyPI packages would be "PyStraddle".

It'll probably be easier to get contributors to a project like this since there'll be a smaller gap between starting work and having something that runs (yes, that includes me, despite my declaration of non-interest in my initial post). 


And on that positive note I'll end this reply.

Cheers,
Steve

From tjreedy at udel.edu  Wed Apr 16 02:23:09 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 15 Apr 2014 20:23:09 -0400
Subject: [Python-Dev] Timing breakdown of Py_InitializeEx_Private()
In-Reply-To: <CAP1=2W64X6YHT7OQZ4KQ6WOq2wV=rj848XprL091rQ7NN55aCA@mail.gmail.com>
References: <CAP1=2W64X6YHT7OQZ4KQ6WOq2wV=rj848XprL091rQ7NN55aCA@mail.gmail.com>
Message-ID: <likii3$vr2$1@ger.gmane.org>

On 4/15/2014 5:26 PM, Brett Cannon wrote:
> To finish my timing work I decided to see
> where Py_InitializeEx_Private() spends its time. The following is a
> breakdown measured in microseconds running using -E:
>
> INIT:
> setlocale: 11
> envvar: 2
> random init: 2
> interp creation: 15
> thread creation: 6
> GIL: 10
> _Py_ReadyTypes(): 930
> more types: 44
> builtins: 141
> exceptions: 287
> sys: 258
> _PyImport_Init: 15
> import hooks: 4
> _PyWarnings_Init(): 15
> ENTERING import_init():
>    PyImport_ImportFrozenModule(_frozen_importlib): 1186
>    interp->importlib: 6
>    PyInit_imp(): 15
>    _imp: 3
>    importlib._install(): 876
>    _PyImportZip_Init(): 130
> _PyFaulthandler_Init(): 13
> time: 3
> ENTERING initfsencoding():
>    codec lookup: 2179
> signals: 120
> tracemalloc: 7
> __main__: 13
> initstdio(): 1568
> warnings module: 4
> initsite(): 9552

It looks like initsite takes half the time. Can it be sped up?


-- 
Terry Jan Reedy


From barry at python.org  Wed Apr 16 04:00:43 2014
From: barry at python.org (Barry Warsaw)
Date: Tue, 15 Apr 2014 22:00:43 -0400
Subject: [Python-Dev] Python "2migr8"
In-Reply-To: <4db20bc6986e4439bb1662327dfb6cb0@BLUPR03MB389.namprd03.prod.outlook.com>
References: <9f950fc6989849fbba2d604cdb7db0c2@BLUPR03MB389.namprd03.prod.outlook.com>
 <CAP7+vJKRpZ1KDxaVAFbJye++1CkZkbwp_pV43Rpje34uwuFf=w@mail.gmail.com>
 <4db20bc6986e4439bb1662327dfb6cb0@BLUPR03MB389.namprd03.prod.outlook.com>
Message-ID: <20140415220043.0295ed39@anarchist.localdomain>

On Apr 15, 2014, at 10:55 PM, Steve Dower wrote:

>Really, I'd expect the name to be pretty boring - "Python Straddle", "Python
>2 Strict" or "Python Migration Edition" (if anyone can live with Python ME
>;-) )

We often call code that works in both Python 2 and 3 from a single source
"bi-lingual".  Maybe we've just been mispronouncing "py-lingual" :).

-Barry

From ncoghlan at gmail.com  Wed Apr 16 04:08:29 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 15 Apr 2014 22:08:29 -0400
Subject: [Python-Dev] Python "2migr8"
In-Reply-To: <4db20bc6986e4439bb1662327dfb6cb0@BLUPR03MB389.namprd03.prod.outlook.com>
References: <9f950fc6989849fbba2d604cdb7db0c2@BLUPR03MB389.namprd03.prod.outlook.com>
 <CAP7+vJKRpZ1KDxaVAFbJye++1CkZkbwp_pV43Rpje34uwuFf=w@mail.gmail.com>
 <4db20bc6986e4439bb1662327dfb6cb0@BLUPR03MB389.namprd03.prod.outlook.com>
Message-ID: <CADiSq7cHXeC9GtajSoiq-dEDwqymAv6NcWHo=dQ5FhkyWpsmgQ@mail.gmail.com>

On 15 Apr 2014 18:56, "Steve Dower" <Steve.Dower at microsoft.com> wrote:
> From Eric:
> > This should be doable with an import hook.  For that matter it would be
pretty straight-forward to provide a script around the Python binary that
installs the import hook.  All this could be bundled up and distributed
through the cheeseshop.
>
> I love this solution. If it really is doable this way then it is even
better than having a fork. My name suggestion for a PyPI packages would be
"PyStraddle".

Eric and I spent some time discussing this today, and don't see any
insurmountable barriers to doing it as an import hook. It may mean doing a
2.7 compatible full importlib backport, but that's also useful for other
reasons.

We may also want to consider/discuss integrating this idea with the
python-future project rather than making it a new independent project, as
that provides ready access to Python 3 style builtins that a hook could
inject as globals in flagged files.

Regardless, I think we're now drifting off topic for python-dev - Eric and
I are still at the sprints, so we'll take a look at the feasibility of
proposing this to Ed and the other python-future devs (I also have some
ideas around using sitecustomize to put an entire Python installation into
enforcement mode for flagged files).

Any other related suggestions should probably be directed to python-ideas
at this point.

Cheers,
Nick.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140415/d8ecc436/attachment-0001.html>

From guido at python.org  Wed Apr 16 04:35:56 2014
From: guido at python.org (Guido van Rossum)
Date: Tue, 15 Apr 2014 19:35:56 -0700
Subject: [Python-Dev] Timing breakdown of Py_InitializeEx_Private()
In-Reply-To: <likii3$vr2$1@ger.gmane.org>
References: <CAP1=2W64X6YHT7OQZ4KQ6WOq2wV=rj848XprL091rQ7NN55aCA@mail.gmail.com>
 <likii3$vr2$1@ger.gmane.org>
Message-ID: <CAP7+vJKRBXho4KhDarvkCQWN2CsF499t8ERqTsyoxyp6qwVmwg@mail.gmail.com>

Well, that's the part that does "import site". Anything that speeds up the
code in Lib/site.py might help. :-)


On Tue, Apr 15, 2014 at 5:23 PM, Terry Reedy <tjreedy at udel.edu> wrote:

> On 4/15/2014 5:26 PM, Brett Cannon wrote:
>
>> To finish my timing work I decided to see
>> where Py_InitializeEx_Private() spends its time. The following is a
>> breakdown measured in microseconds running using -E:
>>
>> INIT:
>> setlocale: 11
>> envvar: 2
>> random init: 2
>> interp creation: 15
>> thread creation: 6
>> GIL: 10
>> _Py_ReadyTypes(): 930
>> more types: 44
>> builtins: 141
>> exceptions: 287
>> sys: 258
>> _PyImport_Init: 15
>> import hooks: 4
>> _PyWarnings_Init(): 15
>> ENTERING import_init():
>>    PyImport_ImportFrozenModule(_frozen_importlib): 1186
>>    interp->importlib: 6
>>    PyInit_imp(): 15
>>    _imp: 3
>>    importlib._install(): 876
>>    _PyImportZip_Init(): 130
>> _PyFaulthandler_Init(): 13
>> time: 3
>> ENTERING initfsencoding():
>>    codec lookup: 2179
>> signals: 120
>> tracemalloc: 7
>> __main__: 13
>> initstdio(): 1568
>> warnings module: 4
>> initsite(): 9552
>>
>
> It looks like initsite takes half the time. Can it be sped up?
>
>
> --
> Terry Jan Reedy
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140415/cb5de39c/attachment.html>

From stefan_ml at behnel.de  Wed Apr 16 07:28:55 2014
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 16 Apr 2014 07:28:55 +0200
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
Message-ID: <lil4en$ikk$1@ger.gmane.org>

Brett Cannon, 14.04.2014 23:51:
> It was realized during PyCon that since we are freezing importlib we could
> now consider freezing all the modules to cut out having to stat or read
> them from disk. So for day 1 of the sprints I I decided to hack up a
> proof-of-concept to see what kind of performance gain it would get.
> 
> Freezing everything except encodings.__init__, os, and _sysconfigdata, it
> speeds up startup by 11% compared to default. Compared to 2.7 it shaves 14%
> from the slowdown (27% slower vs. 41% slower). The full results are at the
> end of the email.
> 
> Now the question is whether the maintenance cost of having to rebuild
> Python for a select number of stdlib modules is enough to warrant putting
> in the effort to make this work. My guess is the best approach would be
> adding a Lib/_frozen directory where any modules that we treat like this
> would be kept to act as a reminder that you need to rebuild for them (I
> would probably move importlib/_boostrap.py as well to make this consistent).
> 
> Thoughts?

Alternatively, the modules could be compiled with Cython. That would not
only speed up the loading time but also the initialisation time and
runtime. And since you'd keep the original .py files next to the .so files,
you'd still get proper tracebacks.

Compiling the modules natively would also enable linking them right into
the interpreter core, BTW. But that would substantially increase its size.
Maybe some of them could still be worth being linked in.

Stefan



From tismer at stackless.com  Wed Apr 16 09:39:34 2014
From: tismer at stackless.com (Christian Tismer)
Date: Wed, 16 Apr 2014 09:39:34 +0200
Subject: [Python-Dev] Timing breakdown of Py_InitializeEx_Private()
In-Reply-To: <CAP7+vJKRBXho4KhDarvkCQWN2CsF499t8ERqTsyoxyp6qwVmwg@mail.gmail.com>
References: <CAP1=2W64X6YHT7OQZ4KQ6WOq2wV=rj848XprL091rQ7NN55aCA@mail.gmail.com>
 <likii3$vr2$1@ger.gmane.org>
 <CAP7+vJKRBXho4KhDarvkCQWN2CsF499t8ERqTsyoxyp6qwVmwg@mail.gmail.com>
Message-ID: <534E33B6.8040706@stackless.com>

Not trying to argue about Mercurial, but if a major amount
of startup time is spent in site.py:

I think in cases like hg command line scripts there is no need
to import site just for hg scripts.

Maybe that would improve things if those startup scripts avoid importing
site? Or do they, already?

cheers - chris


On 16.04.14 04:35, Guido van Rossum wrote:
> Well, that's the part that does "import site". Anything that speeds up
> the code in Lib/site.py might help. :-)
>
>
> On Tue, Apr 15, 2014 at 5:23 PM, Terry Reedy <tjreedy at udel.edu
> <mailto:tjreedy at udel.edu>> wrote:
>
>     On 4/15/2014 5:26 PM, Brett Cannon wrote:
>
>         To finish my timing work I decided to see
>         where Py_InitializeEx_Private() spends its time. The following
>         is a
>         breakdown measured in microseconds running using -E:
>
>         INIT:
>         setlocale: 11
>         envvar: 2
>         random init: 2
>         interp creation: 15
>         thread creation: 6
>         GIL: 10
>         _Py_ReadyTypes(): 930
>         more types: 44
>         builtins: 141
>         exceptions: 287
>         sys: 258
>         _PyImport_Init: 15
>         import hooks: 4
>         _PyWarnings_Init(): 15
>         ENTERING import_init():
>            PyImport_ImportFrozenModule(_frozen_importlib): 1186
>            interp->importlib: 6
>            PyInit_imp(): 15
>            _imp: 3
>            importlib._install(): 876
>            _PyImportZip_Init(): 130
>         _PyFaulthandler_Init(): 13
>         time: 3
>         ENTERING initfsencoding():
>            codec lookup: 2179
>         signals: 120
>         tracemalloc: 7
>         __main__: 13
>         initstdio(): 1568
>         warnings module: 4
>         initsite(): 9552
>
>
>     It looks like initsite takes half the time. Can it be sped up?
>
>
>     -- 
>     Terry Jan Reedy
>
>
>     _______________________________________________
>     Python-Dev mailing list
>     Python-Dev at python.org <mailto:Python-Dev at python.org>
>     https://mail.python.org/mailman/listinfo/python-dev
>     Unsubscribe:
>     https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>
>
>
> -- 
> --Guido van Rossum (python.org/~guido <http://python.org/%7Eguido>)
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/tismer%40stackless.com


-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
Software Consulting          :     Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
14482 Potsdam                :     PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776  fax +49 (30) 700143-0023
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140416/a86d7e77/attachment.html>

From christian at python.org  Wed Apr 16 09:46:55 2014
From: christian at python.org (Christian Heimes)
Date: Wed, 16 Apr 2014 09:46:55 +0200
Subject: [Python-Dev] Timing breakdown of Py_InitializeEx_Private()
In-Reply-To: <CAP7+vJKRBXho4KhDarvkCQWN2CsF499t8ERqTsyoxyp6qwVmwg@mail.gmail.com>
References: <CAP1=2W64X6YHT7OQZ4KQ6WOq2wV=rj848XprL091rQ7NN55aCA@mail.gmail.com>
 <likii3$vr2$1@ger.gmane.org>
 <CAP7+vJKRBXho4KhDarvkCQWN2CsF499t8ERqTsyoxyp6qwVmwg@mail.gmail.com>
Message-ID: <lilchf$uvi$1@ger.gmane.org>

On 16.04.2014 04:35, Guido van Rossum wrote:
> Well, that's the part that does "import site". Anything that speeds up
> the code in Lib/site.py might help. :-)

Antoine, Victor and me have implemented a couple of speed ups for
"import site" already. I removed several unnecessary imports by
rearranging some code. AFAIK imports on OSX haven't been optimized yet.

On Linux I get:

$ ./python -c "import sys; print(len(sys.modules))"
34
$ ./python -S -c "import sys; print(len(sys.modules))"
22

Brett, are you an OSX developer? :)

Christian


From bcannon at gmail.com  Wed Apr 16 14:29:27 2014
From: bcannon at gmail.com (Brett Cannon)
Date: Wed, 16 Apr 2014 08:29:27 -0400
Subject: [Python-Dev] Timing breakdown of Py_InitializeEx_Private()
In-Reply-To: <lilchf$uvi$1@ger.gmane.org>
References: <CAP1=2W64X6YHT7OQZ4KQ6WOq2wV=rj848XprL091rQ7NN55aCA@mail.gmail.com>
 <likii3$vr2$1@ger.gmane.org>
 <CAP7+vJKRBXho4KhDarvkCQWN2CsF499t8ERqTsyoxyp6qwVmwg@mail.gmail.com>
 <lilchf$uvi$1@ger.gmane.org>
Message-ID: <CAP1=2W4O0X8tCSegniHRBk78P03Yvv10Us-jd6gVte2A=8vBvQ@mail.gmail.com>

On Apr 16, 2014 3:47 AM, "Christian Heimes" <christian at python.org> wrote:
>
> On 16.04.2014 04:35, Guido van Rossum wrote:
> > Well, that's the part that does "import site". Anything that speeds up
> > the code in Lib/site.py might help. :-)
>
> Antoine, Victor and me have implemented a couple of speed ups for
> "import site" already. I removed several unnecessary imports by
> rearranging some code. AFAIK imports on OSX haven't been optimized yet.
>
> On Linux I get:
>
> $ ./python -c "import sys; print(len(sys.modules))"
> 34
> $ ./python -S -c "import sys; print(len(sys.modules))"
> 22
>
> Brett, are you an OSX developer? :)

Yep, my laptop is OS X.

-Brett

>
> Christian
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/brett%40python.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140416/c2d7053a/attachment.html>

From tjreedy at udel.edu  Wed Apr 16 15:49:01 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 16 Apr 2014 09:49:01 -0400
Subject: [Python-Dev] Timing breakdown of Py_InitializeEx_Private()
In-Reply-To: <lilchf$uvi$1@ger.gmane.org>
References: <CAP1=2W64X6YHT7OQZ4KQ6WOq2wV=rj848XprL091rQ7NN55aCA@mail.gmail.com>
 <likii3$vr2$1@ger.gmane.org>
 <CAP7+vJKRBXho4KhDarvkCQWN2CsF499t8ERqTsyoxyp6qwVmwg@mail.gmail.com>
 <lilchf$uvi$1@ger.gmane.org>
Message-ID: <lim1p4$i6m$1@ger.gmane.org>

On 4/16/2014 3:46 AM, Christian Heimes wrote:
> On 16.04.2014 04:35, Guido van Rossum wrote:
>> Well, that's the part that does "import site". Anything that speeds up
>> the code in Lib/site.py might help. :-)
>
> Antoine, Victor and me have implemented a couple of speed ups for
> "import site" already. I removed several unnecessary imports by
> rearranging some code. AFAIK imports on OSX haven't been optimized yet.
>
> On Linux I get:
>
> $ ./python -c "import sys; print(len(sys.modules))"
> 34
> $ ./python -S -c "import sys; print(len(sys.modules))"
> 22

With 3.4.0 installed on Windows, or 3.5 repository, 39 and 25

-- 
Terry Jan Reedy


From solipsis at pitrou.net  Wed Apr 16 16:35:06 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 16 Apr 2014 16:35:06 +0200
Subject: [Python-Dev] Timing breakdown of Py_InitializeEx_Private()
References: <CAP1=2W64X6YHT7OQZ4KQ6WOq2wV=rj848XprL091rQ7NN55aCA@mail.gmail.com>
 <likii3$vr2$1@ger.gmane.org>
 <CAP7+vJKRBXho4KhDarvkCQWN2CsF499t8ERqTsyoxyp6qwVmwg@mail.gmail.com>
 <534E33B6.8040706@stackless.com>
Message-ID: <20140416163506.203ae993@fsol>

On Wed, 16 Apr 2014 09:39:34 +0200
Christian Tismer <tismer at stackless.com> wrote:
> 
> I think in cases like hg command line scripts there is no need
> to import site just for hg scripts.

If you don't import site you won't be able to import Mercurial in most
cases.
Also, talking about "scripts" is a bit silly here - Mercurial is a
full-blown applications with many modules and tens of thousands of
lines of code.

Regards

Antoine.



From solipsis at pitrou.net  Wed Apr 16 16:39:13 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 16 Apr 2014 16:39:13 +0200
Subject: [Python-Dev] cpython (2.7): do not generate pipe names in the
	temporary dir
References: <3g6wCJ4vB9z7LjY@mail.python.org>
Message-ID: <20140416163913.727d087f@fsol>

On Mon, 14 Apr 2014 18:24:44 +0200 (CEST)
benjamin.peterson <python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/ea677e26dbeb
> changeset:   90269:ea677e26dbeb
> branch:      2.7
> parent:      90261:c095ff9b0e84
> user:        Benjamin Peterson <benjamin at python.org>
> date:        Mon Apr 14 12:24:37 2014 -0400
> summary:
>   do not generate pipe names in the temporary dir
> 
> files:
>   Lib/multiprocessing/connection.py |  2 +-
>   1 files changed, 1 insertions(+), 1 deletions(-)
> 
> 
> diff --git a/Lib/multiprocessing/connection.py b/Lib/multiprocessing/connection.py
> --- a/Lib/multiprocessing/connection.py
> +++ b/Lib/multiprocessing/connection.py
> @@ -90,7 +90,7 @@
>          return tempfile.mktemp(prefix='listener-', dir=get_temp_dir())
>      elif family == 'AF_PIPE':
>          return tempfile.mktemp(prefix=r'\\.\pipe\pyc-%d-%d-' %
> -                               (os.getpid(), _mmap_counter.next()))
> +                               (os.getpid(), _mmap_counter.next()), dir="")

This is some Windows-specific code, right? Did you witness problems
with this? Your commit message doesn't mention an issue number.

Regards

Antoine.



From jtaylor.debian at googlemail.com  Wed Apr 16 13:51:47 2014
From: jtaylor.debian at googlemail.com (Julian Taylor)
Date: Wed, 16 Apr 2014 13:51:47 +0200
Subject: [Python-Dev]  [numpy wishlist] PyMem_*Calloc
Message-ID: <CAK5FAtFFa7JmntPo24xTsh9m3eC7bh-kqx9zGnY6heXY5Fsm4Q@mail.gmail.com>

Hi,
In NumPy what we want is the tracing, not the exchangeable allocators.
I don't think it is a good idea for the core of a whole stack of
C-extension based modules to replace the default allocator or allowing
other modules to replace the allocator NumPy uses.

I think it would be more useful if Python provides functions to
register memory allocations and frees and the tracemalloc module
registers handlers for these register functions.
If no trace allocation tracer is registered the functions just return
immediately.
That way the tracemalloc can be used with arbitrary allocators as long
as they register their allocations with Python.

For example a hugepage allocator, which you would not want to use that
as the default allocator for all python objects, but you may still
want to trace its usage:

my_hugetlb_alloc(size)
    p = mmap('hugepagefs', ..., MAP_HUGETLB);
    PyMem_Register_Alloc(p, size, __func__, __line__);
    return p

my_hugetlb_free(p);
    PyMem_Register_Free(p, __func__, __line__);
    munmap(p, ...);

normally the registers are nops, but if tracemalloc did register
tracers the memory is tracked, e.g. tracemodule does this on start():
tracercontext.register_alloc = trace_alloc
tracercontext.register_free = trace_free
tracercontext.data = mycontext
PyMem_SetTracer(&tracercontext)

Regards,
Julian Taylor

From benjamin at python.org  Wed Apr 16 17:40:33 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 16 Apr 2014 08:40:33 -0700
Subject: [Python-Dev] cpython (2.7): do not generate pipe names in the
 temporary dir
In-Reply-To: <20140416163913.727d087f@fsol>
References: <3g6wCJ4vB9z7LjY@mail.python.org> <20140416163913.727d087f@fsol>
Message-ID: <1397662833.1116.107199377.73144B64@webmail.messagingengine.com>

On Wed, Apr 16, 2014, at 7:39, Antoine Pitrou wrote:
> On Mon, 14 Apr 2014 18:24:44 +0200 (CEST)
> benjamin.peterson <python-checkins at python.org> wrote:
> > http://hg.python.org/cpython/rev/ea677e26dbeb
> > changeset:   90269:ea677e26dbeb
> > branch:      2.7
> > parent:      90261:c095ff9b0e84
> > user:        Benjamin Peterson <benjamin at python.org>
> > date:        Mon Apr 14 12:24:37 2014 -0400
> > summary:
> >   do not generate pipe names in the temporary dir
> > 
> > files:
> >   Lib/multiprocessing/connection.py |  2 +-
> >   1 files changed, 1 insertions(+), 1 deletions(-)
> > 
> > 
> > diff --git a/Lib/multiprocessing/connection.py b/Lib/multiprocessing/connection.py
> > --- a/Lib/multiprocessing/connection.py
> > +++ b/Lib/multiprocessing/connection.py
> > @@ -90,7 +90,7 @@
> >          return tempfile.mktemp(prefix='listener-', dir=get_temp_dir())
> >      elif family == 'AF_PIPE':
> >          return tempfile.mktemp(prefix=r'\\.\pipe\pyc-%d-%d-' %
> > -                               (os.getpid(), _mmap_counter.next()))
> > +                               (os.getpid(), _mmap_counter.next()), dir="")
> 
> This is some Windows-specific code, right? Did you witness problems
> with this? Your commit message doesn't mention an issue number.

http://buildbot.python.org/all/builders/x86%20Windows7%202.7/builds/2549/steps/test/logs/stdio

I'd be interested to know how that ever worked.

From solipsis at pitrou.net  Wed Apr 16 17:53:06 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 16 Apr 2014 17:53:06 +0200
Subject: [Python-Dev] cpython (2.7): do not generate pipe names in the
	temporary dir
In-Reply-To: <1397662833.1116.107199377.73144B64@webmail.messagingengine.com>
References: <3g6wCJ4vB9z7LjY@mail.python.org> <20140416163913.727d087f@fsol>
 <1397662833.1116.107199377.73144B64@webmail.messagingengine.com>
Message-ID: <20140416175306.5ea78e58@fsol>

On Wed, 16 Apr 2014 08:40:33 -0700
Benjamin Peterson <benjamin at python.org> wrote:

> On Wed, Apr 16, 2014, at 7:39, Antoine Pitrou wrote:
> > On Mon, 14 Apr 2014 18:24:44 +0200 (CEST)
> > benjamin.peterson <python-checkins at python.org> wrote:
> > > http://hg.python.org/cpython/rev/ea677e26dbeb
> > > changeset:   90269:ea677e26dbeb
> > > branch:      2.7
> > > parent:      90261:c095ff9b0e84
> > > user:        Benjamin Peterson <benjamin at python.org>
> > > date:        Mon Apr 14 12:24:37 2014 -0400
> > > summary:
> > >   do not generate pipe names in the temporary dir
> > > 
> > > files:
> > >   Lib/multiprocessing/connection.py |  2 +-
> > >   1 files changed, 1 insertions(+), 1 deletions(-)
> > > 
> > > 
> > > diff --git a/Lib/multiprocessing/connection.py b/Lib/multiprocessing/connection.py
> > > --- a/Lib/multiprocessing/connection.py
> > > +++ b/Lib/multiprocessing/connection.py
> > > @@ -90,7 +90,7 @@
> > >          return tempfile.mktemp(prefix='listener-', dir=get_temp_dir())
> > >      elif family == 'AF_PIPE':
> > >          return tempfile.mktemp(prefix=r'\\.\pipe\pyc-%d-%d-' %
> > > -                               (os.getpid(), _mmap_counter.next()))
> > > +                               (os.getpid(), _mmap_counter.next()), dir="")
> > 
> > This is some Windows-specific code, right? Did you witness problems
> > with this? Your commit message doesn't mention an issue number.
> 
> http://buildbot.python.org/all/builders/x86%20Windows7%202.7/builds/2549/steps/test/logs/stdio
> 
> I'd be interested to know how that ever worked.

I don't know, but perhaps opening an issue would allow a Windows expert
(or Richard) to chime in.

Regards

Antoine.

From martin at v.loewis.de  Wed Apr 16 18:25:22 2014
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 16 Apr 2014 18:25:22 +0200
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
Message-ID: <534EAEF2.1000604@v.loewis.de>

Am 14.04.14 23:51, schrieb Brett Cannon:
> It was realized during PyCon that since we are freezing importlib we
> could now consider freezing all the modules to cut out having to stat or
> read them from disk.
[...]
> Thoughts?

They still get read from disk, except that it is the operating system
that does the reading. So what you really save is the access to many
tiny files; something that can also be achieved with the zipfile import.
So I wonder how your all-frozen binary compares to a standard binary
with a python35.zip.

If it is comparable, I'd rather extend on that route, i.e. promote
putting the standard library into a zip file in the default
installation, and also find a way where (say) /usr/bin/hg could
conveniently specify a zip file that will contain the Mercurial
byte code. For example, we could support a -Z option for the interpreter
which would allow to append a zip file to a script that gets put on
sys.path.

Regards,
Martin


From victor.stinner at gmail.com  Wed Apr 16 20:35:16 2014
From: victor.stinner at gmail.com (Victor Stinner)
Date: Wed, 16 Apr 2014 14:35:16 -0400
Subject: [Python-Dev] [numpy wishlist] PyMem_*Calloc
In-Reply-To: <CAK5FAtFFa7JmntPo24xTsh9m3eC7bh-kqx9zGnY6heXY5Fsm4Q@mail.gmail.com>
References: <CAK5FAtFFa7JmntPo24xTsh9m3eC7bh-kqx9zGnY6heXY5Fsm4Q@mail.gmail.com>
Message-ID: <CAMpsgwYBqwsBj4dO5PVbQ4fbwbJ7o6k+DF+epC5NeDrzXm5eVw@mail.gmail.com>

Hi,

2014-04-16 7:51 GMT-04:00 Julian Taylor <jtaylor.debian at googlemail.com>:
> In NumPy what we want is the tracing, not the exchangeable allocators.

Did you read the PEP 445? Using the new malloc API, in fact you can
have both: install new allocators and set up hooks on allocators.
http://legacy.python.org/dev/peps/pep-0445/

The PEP 445 has been implemented in Python 3.4, we don't plan to
rewrite it. So it's probably better to try to understand how it was
designed and why we chose this design.

See the talk I just have at Pycon Montreal for more information on how
tracemalloc works. Slides:
https://raw.githubusercontent.com/haypo/conf/master/2014-Pycon-Montreal/tracemalloc.pdf

Video:
http://pyvideo.org/video/2698/track-memory-leaks-in-python

> my_hugetlb_alloc(size)
>     p = mmap('hugepagefs', ..., MAP_HUGETLB);
>     PyMem_Register_Alloc(p, size, __func__, __line__);
>     return p
>
> my_hugetlb_free(p);
>     PyMem_Register_Free(p, __func__, __line__);
>     munmap(p, ...);

This is exactly how tracemalloc works. The advantage of the PEP 445 is
that you have a null overhead when tracemalloc is disabled. There is
no need to check if a trace function is present or not.

You can chain multiple hooks.

See also the calloc issue which was written for NumPy:
http://bugs.python.org/issue21233

Victor

From tjreedy at udel.edu  Wed Apr 16 20:56:35 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 16 Apr 2014 14:56:35 -0400
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <534EAEF2.1000604@v.loewis.de>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <534EAEF2.1000604@v.loewis.de>
Message-ID: <limjpp$bc7$1@ger.gmane.org>

On 4/16/2014 12:25 PM, "Martin v. L?wis" wrote:
> Am 14.04.14 23:51, schrieb Brett Cannon:
>> It was realized during PyCon that since we are freezing importlib we
>> could now consider freezing all the modules to cut out having to stat or
>> read them from disk.
> [...]
>> Thoughts?
>
> They still get read from disk, except that it is the operating system
> that does the reading. So what you really save is the access to many
> tiny files; something that can also be achieved with the zipfile import.
> So I wonder how your all-frozen binary compares to a standard binary
> with a python35.zip.
>
> If it is comparable, I'd rather extend on that route, i.e. promote
> putting the standard library into a zip file in the default
> installation, and also find a way where (say) /usr/bin/hg could
> conveniently specify a zip file that will contain the Mercurial
> byte code. For example, we could support a -Z option for the interpreter
> which would allow to append a zip file to a script that gets put on
> sys.path.

This could be useful for Idle also, as its startup is noticeably 
sluggish and could definitely stand to be zippier. About 50 Idle modules 
are imported in the user process and, I presume, at least as many in the 
Idle process.

PS. In the user process sys.modules, there are numerous null entries 
like these:
 >>> sys.modules['idlelib.os']
 >>> sys.modules['idlelib.tokenize']
 >>> sys.modules['idlelib.io']
 >>> <etcetera>

Does anyone know the most likely reason?

-- 
Terry Jan Reedy



From bcannon at gmail.com  Wed Apr 16 21:10:28 2014
From: bcannon at gmail.com (Dr. Brett Cannon)
Date: Wed, 16 Apr 2014 19:10:28 +0000
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <534EAEF2.1000604@v.loewis.de> <limjpp$bc7$1@ger.gmane.org>
Message-ID: <CAP1=2W6zZjy9JZ=NrGZtcgEuuLx04WJzk4U6Q37fbt0t1JdoMg@mail.gmail.com>

Is this Python 2 or 3? In Python 2 it means an attempt to perform a
relative import failed but an absolute in succeeded, e.g. from idlelib you
imported os, so import tried idlelib.is and then os. You should definitely
consider using a future import to guarantee absolute imports.

On Wednesday, April 16, 2014 2:57:35 PM, Terry Reedy <tjreedy at udel.edu>
wrote:

> On 4/16/2014 12:25 PM, "Martin v. L?wis" wrote:
> > Am 14.04.14 23:51, schrieb Brett Cannon:
> >> It was realized during PyCon that since we are freezing importlib we
> >> could now consider freezing all the modules to cut out having to stat or
> >> read them from disk.
> > [...]
> >> Thoughts?
> >
> > They still get read from disk, except that it is the operating system
> > that does the reading. So what you really save is the access to many
> > tiny files; something that can also be achieved with the zipfile import.
> > So I wonder how your all-frozen binary compares to a standard binary
> > with a python35.zip.
> >
> > If it is comparable, I'd rather extend on that route, i.e. promote
> > putting the standard library into a zip file in the default
> > installation, and also find a way where (say) /usr/bin/hg could
> > conveniently specify a zip file that will contain the Mercurial
> > byte code. For example, we could support a -Z option for the interpreter
> > which would allow to append a zip file to a script that gets put on
> > sys.path.
>
> This could be useful for Idle also, as its startup is noticeably
> sluggish and could definitely stand to be zippier. About 50 Idle modules
> are imported in the user process and, I presume, at least as many in the
> Idle process.
>
> PS. In the user process sys.modules, there are numerous null entries
> like these:
>  >>> sys.modules['idlelib.os']
>  >>> sys.modules['idlelib.tokenize']
>  >>> sys.modules['idlelib.io']
>  >>> <etcetera>
>
> Does anyone know the most likely reason?
>
> --
> Terry Jan Reedy
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140416/c4e6b293/attachment.html>

From tismer at stackless.com  Wed Apr 16 22:33:49 2014
From: tismer at stackless.com (Christian Tismer)
Date: Wed, 16 Apr 2014 22:33:49 +0200
Subject: [Python-Dev] Timing breakdown of Py_InitializeEx_Private()
In-Reply-To: <20140416163506.203ae993@fsol>
References: <CAP1=2W64X6YHT7OQZ4KQ6WOq2wV=rj848XprL091rQ7NN55aCA@mail.gmail.com>
 <likii3$vr2$1@ger.gmane.org>
 <CAP7+vJKRBXho4KhDarvkCQWN2CsF499t8ERqTsyoxyp6qwVmwg@mail.gmail.com>
 <534E33B6.8040706@stackless.com> <20140416163506.203ae993@fsol>
Message-ID: <534EE92D.4030204@stackless.com>

On 16/04/14 16:35, Antoine Pitrou wrote:
> On Wed, 16 Apr 2014 09:39:34 +0200
> Christian Tismer <tismer at stackless.com> wrote:
>>
>> I think in cases like hg command line scripts there is no need
>> to import site just for hg scripts.
> 
> If you don't import site you won't be able to import Mercurial in most
> cases.


Oh is that so?
I thought about hg as a set of command-line scripts with a defined
interface that should IMO not be dependent from any site-specific
settings.
Instead, it should be configured at install time, and after that
be completely independent from site settings.


> Also, talking about "scripts" is a bit silly here - Mercurial is a
> full-blown applications with many modules and tens of thousands of
> lines of code.


Thanks for regarding my comments as silly.

In my sense it would be silly if that full-blown app would suffer from
or relate to site at all, tho. Apps are meant to be self-contained.

regards - Chris

-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
Software Consulting          :     Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
14482 Potsdam                :     PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776  fax +49 (30) 700143-0023
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From ncoghlan at gmail.com  Wed Apr 16 22:33:55 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 16 Apr 2014 16:33:55 -0400
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <534EAEF2.1000604@v.loewis.de>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <534EAEF2.1000604@v.loewis.de>
Message-ID: <CADiSq7dug1J_xF7OiBejbVrXgVG7V2bYdD0MCwQx5iW9nu+=Bw@mail.gmail.com>

On 16 April 2014 12:25, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Am 14.04.14 23:51, schrieb Brett Cannon:
>> It was realized during PyCon that since we are freezing importlib we
>> could now consider freezing all the modules to cut out having to stat or
>> read them from disk.
> [...]
>> Thoughts?
>
> They still get read from disk, except that it is the operating system
> that does the reading. So what you really save is the access to many
> tiny files; something that can also be achieved with the zipfile import.
> So I wonder how your all-frozen binary compares to a standard binary
> with a python35.zip.
>
> If it is comparable, I'd rather extend on that route, i.e. promote
> putting the standard library into a zip file in the default
> installation, and also find a way where (say) /usr/bin/hg could
> conveniently specify a zip file that will contain the Mercurial
> byte code. For example, we could support a -Z option for the interpreter
> which would allow to append a zip file to a script that gets put on
> sys.path.

Has anyone tried running mercurial as a zipfile with __main__.py and a
prepended shebang line rather than as a collection of independent
files?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Wed Apr 16 22:42:08 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 16 Apr 2014 16:42:08 -0400
Subject: [Python-Dev] Timing breakdown of Py_InitializeEx_Private()
In-Reply-To: <534EE92D.4030204@stackless.com>
References: <CAP1=2W64X6YHT7OQZ4KQ6WOq2wV=rj848XprL091rQ7NN55aCA@mail.gmail.com>
 <likii3$vr2$1@ger.gmane.org>
 <CAP7+vJKRBXho4KhDarvkCQWN2CsF499t8ERqTsyoxyp6qwVmwg@mail.gmail.com>
 <534E33B6.8040706@stackless.com> <20140416163506.203ae993@fsol>
 <534EE92D.4030204@stackless.com>
Message-ID: <CADiSq7f2wMnJXwJiLtPp=HZpWyCAkNJ4sRPOwvjMB=n8i8hQHA@mail.gmail.com>

On 16 April 2014 16:33, Christian Tismer <tismer at stackless.com> wrote:
>
> In my sense it would be silly if that full-blown app would suffer from
> or relate to site at all, tho. Apps are meant to be self-contained.

For better or for worse, Python applications like Mercurial are
currently considered an integrated part of Linux distros and hence
subject to the usual de-duplication rules (like not embedding their
own Python interpreter, and relying on shared system libraries when
appropriate). And yes, this does cause problems (but not integrating
them causes major security update management problems).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From taavi at taaviburns.ca  Wed Apr 16 22:42:26 2014
From: taavi at taaviburns.ca (Taavi Burns)
Date: Wed, 16 Apr 2014 16:42:26 -0400
Subject: [Python-Dev] Language Summit notes
Message-ID: <AC5D9A0C-0768-4859-8D93-95380B966549@taaviburns.ca>

I was the guy at the back taking pictures. I also took notes! :)


Language Summit
---------------

Intros
~~~~~~

Everyone starts by doing a 15 second intro. People are good and keep it
short, and we make it through all ~40 people. People start introduce
themselves saying "module X is my fault". Guido just says, "It's my
fault."

Release Management
~~~~~~~~~~~~~~~~~~

Larry Hastings talks about release versioning! How to use branches for
the 3.5 RC process. When RCs start, should default be locked down,
should it become 3.5.1, or should it become 3.6? Or something else?
After a bit of discussion, people asked to move the decision to email.

PyPy
~~~~

PyPy update: 7 iterations of stm! (compared to "The JIT took 5?") STM is
25-50% slower on a single-threaded workload, but shows nearly linear
speedups with more cores on non-conflicting workloads. Brett Cannon: "So
it's only HOW much faster than CPython?" It's not sure if there will be
multiple PyPy releases or everyone takes the hit and STM is default. STM
fundraiser II is active now.

PyPy oving to 2.7.6 stdlib soon. A bunch of effort has gone to
supporting CFFI porting in the community which aids porting of packages
to PyPy. Something about Python 3 support that I didn't quite catch.
Alex Gaynor says that most people who look to PyPy are looking for
Python 2 right now.

IronPython
~~~~~~~~~~

IronPython is active again! New contributors! Python 3 support! What can
the PSF do to help IronPython? IronPython should ask for support if it
would help.

Jython
~~~~~~

Nobody from the team was present, but they sent a report which Michael
Foord talked through with the group. Jython moving to Java 7 minimum
runtime. Added buffer protocol (memoryview etc), required for Python 3
support. PyPA tooling? CFFI backend is coming! EuroPython sprint in
July. Looking forward to Python 3 because of string types vs JVM. But
it's slow (work?).

Packaging
~~~~~~~~~

pip will suddenly need to become more user-friendly, now that it's
bundled with 3.4. Instead of being used by people who need it, it's
going to be presented to people who don't know what it is. Nick Coghlan:
"PyPI's security model: it didn't have one." Richard Jones says the new
PyPI code is nearly ready. It has some rough bits (OAuth, OpenID) but
they're rough anyway. Replacing old crufty code with nice code; then
moving forward with shiny new features. Oh, and wheels!

Want to be at the point where Windows and OSX users can download the
binary distro from python.org, and ``pip install django`` and have it
work. Numpy is apparently still a sore point; conda would still be a
better way for most people. Largely two kinds of packages: those that
are self-contained (may include c-extensions though), and those that
depend on system libraries (e.g. link to libmysql). Packaging has to
make both of those work, and if it's made easier to make distro packages
(i.e. RedHat packages, Debian packages, etc), it should be easier to
build conda packages. Nick doesn't want to cross the line where python
packaging (PyPI) does the distro side of things. Currently, "we do c
extensions, but not linking to C libraries". Also, "metadata 2" has been
put off by ~12 months, but is becoming the last thing standing before
dropping setup.py and having wheels for all? Bits of metdata 2 have been
pushed out as extensions or "decide later" bits, which has been
instructive in terms of "how far can we get".

Build chain in stdlib: bad idea, release schedules are different. Need a
build system that supports all the verisons in the wild. Setuptools is
now version-independent distutils.

Oh, and Apple screwed up python, building it with a compiler that's not
in xcode.

``pkg_resources`` has a dependency on ``setuptools`` which is WRONG,
needs to be the other way around. But ``pkg_resources`` does what it
does very well.

Technically pip itself doesn't ship with 3.4, but rather ensurepip. This
is so that new versions of pip can be brought in in point releases, and
for security updates. Windows and OSX installers run ensurepip.
``virtualenv`` installs pip by default at this point, too.

Someone asks, "Can we do that for SSL?" Everyone laughs.

Disussion about packaging continues. Glyph asks if the PSF could fund a
usability study on installing Python. People generally seem to think
it's a good idea.

Go to `packaging.python.org <http://packaging.python.org/en/latest/>`__!
Documentation! Quite a few sections are already filled in: please
contribute!

There is no "one installer" that has everything you need for 2.7 right
now.

Nick: "C development is stupidly complicated."

Nature of the old guard of Python programmers: you were already a C
developer, and just wanted Python to help make some things easier. Nick
got into Python for DSP testing. The demographics have changed. How do
we change the docs and ecosystem to avoid the assumption that Python
programmers already know how to program in C? "We have the wrong people
maintaining the docs."

"A distro is a pre-packaged group of binaries." Linux distros do it for
Linux; On OSX you have macports, fink, and homebrew (OSX isn't itself a
distro, except that it is because they do ship with *some* libraries,
which are often broken); and on Windows there's Enthought, Activestate,
and cygwin. Need SEO to get Windows users searching for "installers" to
find "packages".

Want to turn PyPI into "write metadata, upload source, PyPI does the
rest", i.e. building c-extensions. Compiling c-extensions is
particularly painful on Windows becuase Python 2.7 is build with Visual
Studio 2008. OSX has similar issues where the system Python isn't
compiled with what's in XCode.

Ways to get the python.org Python packages to include the required bits:
2.7 is missing pip, 3.4 ships with it. If you want a GUI?then maybe you
should just get Enthought or ActiveState? Python(x,y) is another option
on Windows.

Lunch
~~~~~

There was food!

Pyston
~~~~~~

Pyston: Python on LLVM, coming out of Dropbox. Rebuilding a complete VM
targetted for Python. Pro: optimize for Python, most custom you can get.
Drawback: most work you can do. A different kind of JIT from PyPy:
method-at-a-time. "Directly" from Python to machine code. Generate a
fastpath for the types that you're seeing to avoid dynamic overheads.

`speed.pypy.org <speed.pypy.org>`__ and
`speed.python.org <speed.python.org>`__: suggested for pyston to help
move it along. Benchmarks aren't just to show how good you are, they're
a guide to where you can improve. Unladen Swallow did great work
compiling a list of less synthetic benchmarks.

Pyston memory usage is higher (includes jit etc), object size is smaller
(inferred slots), function calls are bigger (than CPython).

There's an LLVM IR interpreter for pre-JIT execution. Looking to add a
"quick, IR to machine code" skipping the expensive bits. [Taavi: sounds
like Firefox's `collection of OdinMonkey, IonMonkey,
etc <https://blog.mozilla.org/javascript/2014/01/23/the-monkeys-in-2013/>`__,
where JS is JITted very quickly to not-particularly-efficient machine
code, and re-JITted more expensively as the hot codepaths are found.]

Optional static type checking
-----------------------------

Presentation by Jukka Lehtosalo on Mypy's static type checking, making
use of type annotations in Python 3:

-  Faster than unit tests
-  Compatible with existing VMs (no runtime checks, just compilation
   checks; errors are warnings at runtime)
-  Documentation
-  Tools (IDEs)
-  Mypy: import typing; then you get static type checks! All valid
   programs, using Python 3 type annotations.
-  Can always fall back: from typing import Dict, Any; d = Dict[str,
   Any]; d[x] = 'y'; d[x] += 2
-  `www.mypy-lang.org <http://www.mypy-lang.org>`__
-  See also `https://github.com/google/pytypedecl <www.mypy-lang.org>`__

Michael Foord: mock and autospec; having function return annotations
would be nice so an autospec'd mock can return more autospec'd mocks

PEP8 said "don't use annotations", but that was only intended for stdlib
(so that people were free to play with annotations in their own code).

It was said that stdlib ought to be a good example of how to write
Python code. But it isn't (right now, at least).

Education
---------

No docs on python.org for teachers (links). Mostly just "teach yourself"
tutorials. Want things more like lesson plans.

Having a new doc section for "absolute beginners" (Jessica McKellar
volunteers to write it!) would be helpful; beginners often download the
other python (2/3) from the tutorial they're trying to do.

AP exams are starting to allow Python, but it's 10% of the AP CS exams.
Jessica's working on making it Real. People on the APCS board already
want it, just need the numbers to show that it's the right choice to
teach to students.

"The packaging and install programs, oh god."

New users are good at contributing to docs. StackOverflow gets refd'd in
bug tickets "see this StackOverflow post for why this is a
problem/confusing".

Selena Deckelmann talked about having a list of mitigations for teachers
to help students using Windows and OSX. Lots of non-obvious "gotchas".

Lots of outreach things. Women In Tech Linux kernel thing received 144
patches before selecting anyone (because applicants had to submit a
patch to apply in the first place), and ended up adding an extra 1-2
people beyhond their planned 7 acceptees.

Read the docs website! "Write the docs" conference! Yay!

Learning to use the tooling currently required for contribution is a
high bar. Being able to submit doc changes without a pull request might
help.

`bugs.python.org <http://bugs.python.org>`__: Selena filed one for a doc
bug. But there is no "docs" type tag. Is a dead (404) link an
"enhancement"? "Bug"? Should github issues be used for python.org
website bugs?

We have too many bug trackers? One for bugs.pycon.org, for pypi, for
website, and a bugtracker for the bugtracker? Richard Jones is no longer
working on Roundup, and reduced pypi from 3 bug trackers to 2?!?

Too many places to report problems, and it's not clear how to even find
them all.

Terminology and vocabulary: if there are no words that seem to have
anything to do with their problem, new bug sumitters will walk away.
Being able to report a doc bug without specifying a type (because
they're more worried about using the wrong type) might help.

Jessica suggesting ("OPW"s: outreach programmes?) to talk about how to
fill the pipeline.

India and UK have re-done syllabi to include programming. India has put
Python in explicitly, but there's pushback from parents and teachers
because they don't know what this "strange language" is. In the UK,
there's no language mandate, but children 7 and over are to be taught
programming. The biggest problem is that the teachers are used to
teaching spreadsheets and word processing, not programming. Raspberry
PIs are common, ?25 version includes minecraft. Kids usually start with
Scratch and move to Python.

Selena says that some American parents are worried because they don't
know if their kids could get jobs [Taavi: loud surprise from audience].
Our industry knows the demand, but apparently it's not "common
knowledge". Bringing parents and kids together in workshops, meeting
people who have tech jobs and businesspeople who want to hire for tech
jobs could help educate parents who don't realise there really are jobs?

Pain points; why Mercurial isn't on Python 3
--------------------------------------------

Currently supports 2.4, 2.5, 2.6, 2.7. 2.4 sucks, but there's nothing
much extra in 2.5 to make dropping 2.4 worth it.

Mercurial will stop supporting 2.7 when RHEL7 is end-of-life. "We'll all
be dead, I'll be on a differnet project." -- Matt Mackall

Mercurial is bytestrings everywhere (intentionally), which isn't nearly
as nice in Python 3 at this time.

Most important benchmark: startup time. "hello world" in 2.7: 0.017s. In
3.4, 0.043s. git is 10x faster (Perl. Or bash.).

Estimate of 1 person full time for 1.5 years to get mercurial running on
Python 3, at half the speed of Python 2. It's not a pressing issue for
anyone (that Matt knows of). People ask about Python 3 support once
every few months.

Matt's asking for 2.8. "I want a path forward that's not end-of-life."
Looking for new features, bugfixes? Guido: "We can keep 2.7 alive for
longer. ? It's reasonable. 2.8 is [a can of worms]."

3/4 of pypi downloads are for 2.7 (biased sample, but of millions a
month)

Glyph: 2.8 would give an opportunity to be more aggressive, e.g. raise
warnings on implicit string coercion.

Guido disagrees (people will stay away from loud warnings that break
their code).

Nick: there are missing warnings for "-3", but they could be added in
2.7.

Guido: What's going to be in 2.8?

Tres: All the backports from pypi.

Guido: Can we prevent people from adding new features along with the
backports?

The biggest difference for 2 =< 3 is str/bytes.

Guido: There is not going to be a 2.8.

Another good reason for a 2.8: Windows. 2.7 on Windows is compiled with
Visual Studio 2008, and changing compiler is (apparently) a real
problem.

The total amount of churn any 2.8 release would cause is too much.
(packaging, porting, etc) Even if it's a no-op from 2.7.6 to 2.8.0.

3.5
---

Kevin Modzelewski: Don't do 3.5, call it 4.0, then you can tell 2.x
people they're TWO major versions behind! :)

Guido: Can we get the PSF to issue a press release that 2.7 support will
continue? (Matt wants to know that support is not ending.) Bring up a
list of things that would be useful in a 2.8, so we can see what to do
in 3.5. Glyph might start the list. Someone should also look through
existing PEPs, because stuff might already be done. Guido asked about
2.4ish =< 2.7-single-source. "modernize" was mentioned, but it's
abandoned. [Taavi: modernize and one other library were mentioned at the
Python 3 panel discussion at the Montr?al Python meetup on April 14th.
Video will probably show up `on
YouTube <https://www.youtube.com/user/MontrealPython/playlists>`__ as
MP-45 in a few weeks?]

Talk of the PSF paying experienced contractors to do certain bits of
work.

OpenStack is looking to do single-source Python 2/3.

Other helpful things: instead of saying "print: invalid syntax", say
something like "you need parentheses in Python 3". Adding AI to error
messaging so that it can provide helpful messages would be great!
Tooling to help port 2.x to 2.7+3-single-source would be great.

Action items
------------

-  Update pep to declare that 2.7 is not dead, and getting new
   maintenance updates.
-  Glyph will tell us what to do in 3.5.


-- 
taa
/*eof*/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140416/6c0cc37e/attachment.sig>

From rdmurray at bitdance.com  Wed Apr 16 22:46:24 2014
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 16 Apr 2014 16:46:24 -0400
Subject: [Python-Dev] New mailing list for workflow/workflow infrastructure
	discussion/tasks
Message-ID: <20140416204624.BEFEB250D09@webabinitio.net>

Based on a number of conversations at PyCon, we've created a new mailing list:

   https://mail.python.org/mailman/listinfo/core-workflow 

The purpose of this list is to facilitate the conversations and coordinate
the work that needs to happen to improve our development workflow.
Nick's PEP is one piece of this conversation, but there are many other
aspects to it as well.

Here is the list description:

    This is a place to discuss and work on improvements to the CPython core
    development workflow and the infrastructure that supports that workflow.
    This includes changes to the roundup interface and functionality, rietveld,
    mercurial, buildbots, and any other infrastructure we may add.  It also
    includes discussing how we use these tools, and most importantly how we use
    these tools to integrate the community beyond the core developers into the
    workflow that gets patches committed to the python repository.  This means
    that it also includes discussions of the process of bringing in new
    contributors, including how we use the core-mentorship list, as well as how
    we organize ticket triage, and how we make use of external resources such
    as openhatch.  Discussions of documentation and how we organize and
    maintain the documentation are also appropriate.

Anyone who is interested helping with this, or who wants to keep up with the
evolution of our thoughts on these topics, are invited to sign up to the
mailing list.

We will of course check in with python-dev and/or python-committers
as appropriate.

--David


From tjreedy at udel.edu  Wed Apr 16 22:52:34 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 16 Apr 2014 16:52:34 -0400
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <CAP1=2W6zZjy9JZ=NrGZtcgEuuLx04WJzk4U6Q37fbt0t1JdoMg@mail.gmail.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <534EAEF2.1000604@v.loewis.de> <limjpp$bc7$1@ger.gmane.org>
 <CAP1=2W6zZjy9JZ=NrGZtcgEuuLx04WJzk4U6Q37fbt0t1JdoMg@mail.gmail.com>
Message-ID: <limqj8$n3o$1@ger.gmane.org>

 > On Wednesday, April 16, 2014 2:57:35 PM, Terry Reedy <tjreedy at udel.edu
 > <mailto:tjreedy at udel.edu>> wrote:

 >     PS. In the user process sys.modules, there are numerous null
 >     entries like these:
 >       >>> sys.modules['idlelib.os']
 >       >>> sys.modules['idlelib.tokenize'__]
 >       >>> sys.modules['idlelib.io <http://idlelib.io>']
 >       >>> <etcetera>

On 4/16/2014 3:10 PM, Dr. Brett Cannon wrote:
> Is this Python 2 or 3?

Py 2. I should have said so. The entries do not appear in py3.

> In Python 2 it means an attempt to perform a relative import failed but
> an absolute in succeeded, e.g. from idlelib you imported os, so import
 > tried idlelib.is <http://idlelib.is> and then  os.

*I* have not done anything. For tokenize, for instance, the existing 
code just does what I though were absolute imports, in 2 files.
   import tokenize

Perhaps the extra entries have something to do with the fact that these 
startup imports are invisible to user code, just like those done by the 
interpreter itself on startup. 2.7 uses spawnv (and 3.4 uses subprocces) 
to run something like one of the following.
   python -c "__import__('idlelib.run').run.main(False)"
   python -c "__import__('run').main(False)"

run.py has several normal lines with
   import <stdlib module>
   from idlelib import <idlelib module>
and ditto for some of the imported idlelib modules.

 > You should definitely consider using a future import to guarantee
> absolute imports.

-- 
Terry Jan Reedy


From nad at acm.org  Wed Apr 16 23:15:34 2014
From: nad at acm.org (Ned Deily)
Date: Wed, 16 Apr 2014 14:15:34 -0700
Subject: [Python-Dev] New mailing list for workflow/workflow
	infrastructure discussion/tasks
References: <20140416204624.BEFEB250D09@webabinitio.net>
Message-ID: <nad-64A5FA.14153416042014@news.gmane.org>

In article <20140416204624.BEFEB250D09 at webabinitio.net>,
 "R. David Murray" <rdmurray at bitdance.com> wrote:

> https://mail.python.org/mailman/listinfo/core-workflow

FYI, I've submitted a request for it to be mirrored at 
gmane.comp.python.devel.core-workflow

-- 
 Ned Deily,
 nad at acm.org


From ncoghlan at gmail.com  Thu Apr 17 00:04:12 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 16 Apr 2014 18:04:12 -0400
Subject: [Python-Dev] PEP 466: Network security enhancements for Python 2.7.7
Message-ID: <CADiSq7caOVWvKLP0wErBrfO5Bx1fR9Kg9nbVBruvJTCW-k_msQ@mail.gmail.com>

I've reworded the PEP to make it clear it is now just about
backporting a specific set of enhancements to 2.7.7, as well as
switching to updating to new OpenSSL feature releases in the binary
installers.

The idea of an open ended backport policy is now listed as a rejected
variant. I believe that change addresses Guido's main remaining
concern, so I think this version is ready for pronouncement.

Regards,
Nick.

========================================
PEP: 466
Title: Network Security Enhancements for Python 2.7.7
Version: $Revision$
Last-Modified: $Date$
Author: Nick Coghlan <ncoghlan at gmail.com>,
Status: Draft
Type: Informational
Content-Type: text/x-rst
Created: 23-Mar-2014
Post-History: 23-Mar-2014, 24-Mar-2014, 25-Mar-2014, 26-Mar-2014, 16-Apr-2014


Abstract
========

Most CPython tracker issues are classified as errors in behaviour or
proposed enhancements. Most patches to fix behavioural errors are
applied to all active maintenance branches.  Enhancement patches are
restricted to the default branch that becomes the next Python version.

This cadence works reasonably well during Python's normal 18-24 month
feature release cycle, which is still applicable to the Python 3 series.
However, the age of the standard library in Python 2 has now reached a point
where it is sufficiently far behind the state of the art in network security
protocols for it to be causing real problems in use cases where upgrading to
Python 3 in the near term may not be feasible.

In recognition of the additional practical considerations that have arisen
during the 4+ year maintenance cycle for Python 2.7, this PEP allows a
critical set of network security related features to be backported from
Python 3.4 to the upcoming Python 2.7.7 maintenance release.

While this PEP does not make any changes to the core development team's
handling of security-fix-only branches that are no longer in active
maintenance, it *does* recommend that commercial redistributors providing
extended support periods for the Python standard library either backport
these features to their supported versions, or else explicitly disclaim
support for the use of older versions in roles that involve connecting
directly to the public internet.


New security related features in Python 2.7.7
=============================================

Under this proposal, the following features will be backported from Python
3.4 to the upcoming Python 2.7.7 maintenance release:

* in the ``os`` module:

  * persistent file descriptor for ``os.urandom()``.

* in the ``hmac`` module:

  * constant time comparison function (``hmac.compare_digest()``).

* in the ``hashlib`` module:

  * password hashing function (``hashlib.pbkdf2_hmac()``).
  * details of hash algorithm availability (``hashlib.algorithms_guaranteed``
    and ``hashlib.algorithms_available``).

* in the ``ssl`` module:

  * this module is almost entirely synchronised with its Python 3
    counterpart, bringing TLSv1.x settings, SSLContext manipulation, Server
    Name Indication, access to platform certificate stores, standard
    library support for peer hostname validation and more to the Python 2
    series.
  * the only ``ssl`` module features *not* backported under this policy are
    the ``ssl.RAND_*`` functions that provide access to OpenSSL's random
    number generation capabilities - use ``os.urandom()`` instead.

As a general change in maintenance policy, permission is also granted to
upgrade to newer feature releases of OpenSSL when preparing the binary
installers for new maintenance releases of Python 2.7.

This PEP does NOT propose a general exception for backporting new features
to Python 2.7 - every new feature proposed for backporting will still need
to be justified independently. In particular, it will need to be explained
why relying on and independently updated backport on the Python Package Index
instead is not an acceptable solution.


Backwards compatibility considerations
======================================

As in the Python 3 series, the backported ``ssl.create_default_context()``
API is granted a backwards compatibility exemption that permits the
protocol, options, cipher and other settings of the created SSL context to
be updated in maintenance releases to use higher default security settings.
This allows them to appropriately balance compatibility and security at the
time of the maintenance release, rather than at the time of the original
feature release.

This PEP does *not* grant any other exemptions to the usual backwards
compatibility policy for maintenance releases. Instead, by explicitly
encouraging the use of feature based checks, it is designed to make it easier
to write more secure cross-version compatible Python software, while still
limiting the risk of breaking currently working software when upgrading to
a new Python 2.7 maintenance release.

In all cases where this proposal allows new features to be backported to
the Python 2.7 release series, it is possible to write cross-version
compatible code that operates by "feature detection" (for example, checking
for particular attributes in a module), without needing to explicitly check
the Python version.

It is then up to library and framework code to provide an appropriate warning
and fallback behaviour if a desired feature is found to be missing. While
some especially security sensitive software MAY fail outright if a desired
security feature is unavailable, most software SHOULD instead emit a warning
and continue operating using a slightly degraded security configuration.

The backported APIs allow library and application code to perform the
following actions after detecting the presence of a relevant
network security related feature:

* explicitly opt in to more secure settings (to allow the use of enhanced
  security features in older maintenance releases of Python with less
  secure default behaviour)
* explicitly opt in to less secure settings (to allow the use of newer Python
  feature releases in lower security environments)
* determine the default setting for the feature (this MAY require explicit
  Python version checks to determine the Python feature release, but DOES
  NOT require checking for a specific maintenance release)

Security related changes to other modules (such as higher level networking
libraries and data format processing libraries) will continue to be made
available as backports and new modules on the Python Package Index, as
independent distribution remains the preferred approach to handling
software that must continue to evolve to handle changing development
requirements independently of the Python 2 standard library. Refer to
the `Motivation and Rationale`_ section for a review of the characteristics
that make the secure networking infrastructure worthy of special
consideration.


OpenSSL compatibility
---------------------

Under this proposal, OpenSSL may be upgraded to more recent feature releases
in Python 2.7 maintenance releases. On Linux and most other POSIX systems,
the specific version of OpenSSL used already varies, as CPython dynamically
links to the system provided OpenSSL library by default.

For the Windows binary installers, the ``_ssl`` and ``_hashlib`` modules are
statically linked with OpenSSL and the associated symbols are not exported.
Marc-Andre Lemburg indicates that updating to newer OpenSSL releases in the
``egenix-pyopenssl`` binaries has not resulted in any reported compatibility
issues [3]_

The Mac OS X binary installers historically followed the same policy as
other POSIX installations and dynamically linked to the Apple provided
OpenSSL libraries. However, Apple has now ceased updating these
cross-platform libraries, instead requiring that even cross-platform
developers adopt Mac OS X specific interfaces to access up to date security
infrastructure on their platform. Accordingly, and independently of this
PEP, the Mac OS X binary installers were already going to be switched to
statically linker newer versions of OpenSSL [4]_


Other Considerations
====================

Maintainability
---------------

A number of developers, including Alex Gaynor and Donald Stufft, have
expressed interest in carrying out the feature backports covered by this
policy, and assisting with any additional maintenance burdens that arise
in the Python 2 series as a result.

Steve Dower and Brian Curtin have offered to help with the creation of the
Windows installers, allowing Martin von L?wis the opportunity to step back
from the task of maintaining the 2.7 Windows installer.

This PEP is primarily about establishing the consensus needed to allow them
to carry out this work. For other core developers, this policy change
shouldn't impose any additional effort beyond potentially reviewing the
resulting patches for those developers specifically interested in the
affected modules.


Security releases
-----------------

This PEP does not propose any changes to the handling of security
releases - those will continue to be source only releases that
include only critical security fixes.

However, the recommendations for library and application developers are
deliberately designed to accommodate commercial redistributors that choose
to apply these changes to additional Python release series that are either
in security fix only mode, or have been declared "end of life" by the core
development team.

Whether or not redistributors choose to exercise that option will be up
to the individual redistributor.


Integration testing
-------------------

Third party integration testing services should offer users the ability
to test against multiple Python 2.7 maintenance releases (at least 2.7.6
and 2.7.7+), to ensure that libraries, frameworks and applications can still
test their handling of the legacy security infrastructure correctly (either
failing or degrading gracefully, depending on the security sensitivity of
the software), even after the features covered in this proposal have been
backported to the Python 2.7 series.


Handling lower security environments with low risk tolerance
------------------------------------------------------------

For better or for worse (mostly worse), there are some environments where
the risk of latent security defects is more tolerated than even a slightly
increased risk of regressions in maintenance releases. This proposal largely
excludes these environments from consideration where the modules covered by
the exemption are concerned - this approach is entirely inappropriate for
software connected to the public internet, and defence in depth security
principles suggest that it is not appropriate for most private networks
either.

Downstream redistributors may still choose to cater to such environments,
but they will need to handle the process of downgrading the security
related modules and doing the associated regression testing themselves.
The main CPython continuous integration infrastructure will not cover this
scenario.


Motivation and Rationale
========================

The creation of this PEP was prompted primarily by the aging SSL support in
the Python 2 series. As of March 2014, the Python 2.7 SSL module is
approaching four years of age, and the SSL support in the still popular
Python 2.6 release had its feature set locked six years ago.

These are simply too old to provide a foundation that can be recommended
in good conscience for secure networking software that operates over the
public internet, especially in an era where it is becoming quite clearly
evident that advanced persistent security threats are even more widespread
and more indiscriminate in their targeting than had previously been
understood. While they represented reasonable security infrastructure in
their time, the state of the art has moved on, and we need to investigate
mechanisms for effectively providing more up to date network security
infrastructure for users that, for whatever reason, are not currently in
a position to migrate to Python 3.

While the use of the system OpenSSL installation addresses many of these
concerns on Linux platforms, it doesn't address all of them (in particular,
it is still difficult for sotware to explicitly require some higher level
security settings). The standard library support can be bypassed by using a
third party library like PyOpenSSL or Pycurl, but this still results in a
security problem, as these can be difficult dependencies to deploy, and many
users will remain unaware that they might want them. Rather than explaining
to potentially naive users how to obtain and use these libraries, it seems
better to just fix the included batteries.

In the case of the binary installers for Windows and Mac OS X that are
published on python.org, the version of OpenSSL used is entirely within
the control of the Python core development team, but is currently limited
to OpenSSL maintenance releases for the version initially shipped with the
corresponding Python feature release.

With increased popularity comes increased responsibility, and this proposal
aims to acknowledge the fact that Python's popularity and adoption is at a
sufficiently high level that some of our design and policy decisions have
significant implications beyond the Python development community.

As one example, the Python 2 ``ssl`` module does not support the Server
Name Indication standard. While it is possible to obtain SNI support
by using the third party ``requests`` client library, actually doing so
currently requires using not only ``requests`` and its embedded dependencies,
but also half a dozen or more additional libraries. The lack of support
in the Python 2 series thus serves as an impediment to making effective
use of SNI on servers, as Python 2 clients will frequently fail to handle
it correctly.

Another more critical example is the lack of SSL hostname matching in the
Python 2 standard library - it is currently necessary to rely on a third
party library, such as ``requests`` or ``backports.ssl_match_hostname`` to
obtain that functionality in Python 2.

The Python 2 series also remains more vulnerable to remote timing attacks
on security sensitive comparisons than the Python 3 series, as it lacks a
standard library equivalent to the timing attack resistant
``hmac.compare_digest()`` function. While appropriate secure comparison
functions can be implemented in third party extensions, many users don't
even consider the issue and use ordinary equality comparisons instead
- while a standard library solution doesn't automatically fix that problem,
it *does* make the barrier to resolution much lower once the problem is
pointed out.

Python 2.7 represents the only long term maintenance release the core
development team has provided, and it is natural that there will be things
that worked over a historically shorter maintenance lifespan that don't work
over this longer support period. In the specific case of the problem
described in this PEP, the simplest available solution is to acknowledge
that long term maintenance of network security related modules *requires*
the ability to add new features, even while retaining backwards compatibility
for existing interfaces.

For those familiar with it, it is worth comparing the approach described in
this PEP with Red Hat's handling of its long term open source support
commitments: it isn't the RHEL 6.0 release itself that receives 10 years
worth of support, but the overall RHEL 6 *series*. The individual RHEL 6.x
point releases within the series then receive a wide variety of new
features, including security enhancements, all while meeting strict
backwards compatibility guarantees for existing software. The proposal
covered in this PEP brings our approach to long term maintenance more into
line with this precedent - we retain our strict backwards compatibility
requirements, but make an exception to the restriction against adding new
features.

To date, downstream redistributors have respected our upstream policy of
"no new features in Python maintenance releases". This PEP explicitly
accepts that a more nuanced policy is appropriate in the case of network
security related features, and the specific change it describes is
deliberately designed such that it is potentially suitable for Red Hat
Enterprise Linux and its downstream derivatives.


Rejected alternative: just advise developers to migrate to Python 3
-------------------------------------------------------------------

This alternative represents the status quo. Unfortunately, it has proven
to be unworkable in practice, as the backwards compatibility implications
mean that this is a non-trivial migration process for large applications
and integration projects. While the tools for migration have evolved to
a point where it is possible to migrate even large applications
opportunistically and incrementally (rather than all at once) by updating
code to run in the large common subset of Python 2 and Python 3, using the
most recent technology often isn't a priority in commercial environments.

Previously, this was considered an acceptable harm, as while it was an
unfortunate problem for the affected developers to have to face, it was
seen as an issue between them and their management chain to make the case
for infrastructure modernisation, and this case would become naturally
more compelling as the Python 3 series evolved.

However, now that we're fully aware of the impact the limitations of the
Python 2 standard library may be having on the evolution of internet
security standards, I no longer believe that it is reasonable to expect
platform and application developers to resolve all of the latent defects
in an application's Unicode correctness solely in order to gain access to
the network security enhancements already available in Python 3.

While Ubuntu (and to some extent Debian as well) are committed to porting all
default system services and scripts to Python 3, and to removing Python 2
from its default distribution images (but not from its archives), this is
a mammoth task and won't be completed for the Ubuntu 14.04 LTS release
(at least for the desktop image - it may be achieved for the mobile and
server images).

Fedora has even more work to do to migrate, and it will take a non-trivial
amount of time to migrate the relevant infrastructure components. While
Red Hat are also actively working to make it easier for users to use more
recent versions of Python on our stable platforms, it's going to take time
for those efforts to start having an impact on end users' choice of version,
and any such changes also don't benefit the core platform infrastructure
that runs in the integrated system Python by necessity.

The OpenStack migration to Python 3 is also still in its infancy, and even
though that's a project with an extensive and relatively robust automated
test suite, it's still large enough that it is going to take quite some time
to migrate fully to a Python 2/3 compatible code base.

And that's just three of the highest profile open source projects that
make heavy use of Python. Given the likely existence of large amounts of
legacy code that lacks the kind of automated regression test suite needed
to help support a migration from Python 2 to Python 3, there are likely to
be many cases where reimplementation (perhaps even in Python 3) proves
easier than migration. The key point of this PEP is that those situations
affect more people than just the developers and users of the affected
application: the existence of clients and servers with outdated network
security infrastructure becomes something that developers of secure
networked services need to take into account as part of their security
design, and that's a problem that inhibits the adoption of better security
standards.

As Terry Reedy noted, if we try to persist with the status quo, the likely
outcome is that commercial redistributors will attempt to do something
like this on behalf of their customers *anyway*, but in a potentially
inconsistent and ad hoc manner. By drawing the scope definition process
into the upstream project we are in a better position to influence the
approach taken to address the situation and to help ensure some consistency
across redistributors.

The problem is real, so *something* needs to change, and this PEP describes
my preferred approach to addressing the situation.


Rejected alternative: create and release Python 2.8
---------------------------------------------------

With sufficient corporate support, it likely *would* be possible to create
and release Python 2.8 (it's highly unlikely such a project would garner
enough interest to be achievable with only volunteers). However, this
wouldn't actually solve the problem, as the aim is to provide a *relatively
low impact* way to incorporate enhanced security features into integrated
products and deployments that make use of Python 2.

Upgrading to a new Python feature release would mean both more work for the
core development team, as well as a more disruptive update that most
potential end users would likely just skip entirely.

Attempting to create a Python 2.8 release would also bring in suggestions
to backport many additional features from Python 3 (such as ``tracemalloc``
and the improved coroutine support), making the migration from Python 2.7
to this hypothetical 2.8 release even riskier and more disruptive.

This is not a recommended approach, as it would involve substantial
additional work for a result that is actually less effective in achieving
the original aim (which is to eliminate the current widespread use of the
aging network security infrastructure in the Python 2 series).

Furthermore, while I can't make any commitments to actually addressing
this issue on Red Hat platforms, I *can* categorically rule out the idea
of a Python 2.8 being of any use to me in even attempting to get it
addressed.


Rejected alternative: distribute the security enhancements via PyPI
-------------------------------------------------------------------

While this initially appears to be an attractive and easier to manage
approach, it actually suffers from several significant problems.

Firstly, this is complex, low level, cross-platform code that integrates
with the underlying operating system across a variety of POSIX platforms
(including Mac OS X) and Windows. The CPython BuildBot fleet is already set
up to handle continuous integration in that context, but most of the
freely available continuous integration services just offer Linux, and
perhaps paid access to Windows. Those services work reasonably well for
software that largely runs on the abstraction layers offered by Python and
other dynamic languages, as well as the more comprehensive abstraction
offered by the JVM, but won't suffice for the kind of code involved here.

The OpenSSL dependency for the network security support also qualifies as
the kind of "complex binary dependency" that isn't yet handled well by the
``pip`` based software distribution ecosystem. Relying on a third party
binary dependency also creates potential compatibility problems for ``pip``
when running on other interpreters like ``PyPy``.

Another practical problem with the idea is the fact that ``pip`` itself
relies on the ``ssl`` support in the standard library (with some additional
support from a bundled copy of ``requests``, which in turn bundles
``backport.ssl_match_hostname``), and hence would require any replacement
module to also be bundled within ``pip``. This wouldn't pose any
insurmountable difficulties (it's just another dependency to vendor), but
it *would* mean yet another copy of OpenSSL to keep up to date.

This approach also has the same flaw as all other "improve security by
renaming things" approaches: they completely miss the users who most need
help, and raise significant barriers against being able to encourage users
to do the right thing when their infrastructure supports it (since
"use this other module" is a much higher impact change than "turn on this
higher security setting"). Deprecating the aging SSL infrastructure in the
standard library in favour of an external module would be even more user
hostile than accepting the slightly increased risk of regressions associated
with upgrading it in place.

Last, but certainly not least, this approach suffers from the same problem
as the idea of doing a Python 2.8 release: likely not solving the actual
problem. Commercial redistributors of Python are set up to redistribute
*Python*, and a pre-existing set of additional packages. Getting new
packages added to the pre-existing set *can* be done, but means approaching
each and every redistributor and asking them to update their
repackaging process accordingly. By contrast, the approach described in
this PEP would require redistributors to deliberately *opt out* of the
security enhancements by deliberately downgrading the provided network
security infrastructure, which most of them are unlikely to do.


Rejected variant: provide a "legacy SSL infrastructure" branch
--------------------------------------------------------------

Earlier versions of this PEP included the concept of a ``2.7-legacy-ssl``
branch that preserved the exact feature set of the Python 2.7.6 network
security infrastructure.

In my opinion, anyone that actually wants this is almost certainly making a
mistake, and if they insist they really do want it in their specific
situation, they're welcome to either make it themselves or arrange for a
downstream redistributor to make it for them.

If they are made publicly available, any such rebuilds should be referred to
as "Python 2.7 with Legacy SSL" to clearly distinguish them from the official
Python 2.7 releases that include more up to date network security
infrastructure.

After the first Python 2.7 maintenance release that implements this PEP, it
would also be appropriate to refer to Python 2.7.6 and earlier releases as
"Python 2.7 with Legacy SSL".


Rejected variant: synchronise particular modules entirely with Python 3
-----------------------------------------------------------------------

Earlier versions of this PEP suggested synchronising the ``hmac``,
``hashlib`` and ``ssl`` modules entirely with their Python 3 counterparts.

This approach proved too vague to build a compelling case for the exception,
and has thus been replaced by the current more explicit proposal.


Rejected variant: open ended backport policy
--------------------------------------------

Earlier versions of this PEP suggested a general policy change related to
future Python 3 enhancements that impact the general security of the
internet.

That approach created unnecessary uncertainty, so it has been simplified to
propose backport a specific concrete set of changes. Future feature
backport proposals can refer back to this PEP as precedent, but it will
still be necessary to make a specific case for each feature addition to
the Python 2.7 long term support release.


Disclosure of Interest
======================

The author of this PEP currently works for Red Hat on test automation tools.
If this proposal is accepted, I will be strongly encouraging Red Hat to take
advantage of the resulting opportunity to help improve the overall security
of the Python ecosystem. However, I do not speak for Red Hat in this matter,
and cannot make any commitments on Red Hat's behalf.


Acknowledgements
================

Thanks to Christian Heimes and other for their efforts in greatly improving
Python's SSL support in the Python 3 series, and a variety of members of
the Python community for helping me to better understand the implications
of the default settings we provide in our SSL modules, and the impact that
tolerating the use of SSL infrastructure that was defined in 2010
(Python 2.7) or even 2008 (Python 2.6) potentially has for the security
of the web as a whole.

Thanks to Donald Stufft and Alex Gaynor for identifying a more limited set
of essential security features that allowed the proposal to be made more
fine-grained than backporting entire modules from Python 3.4 [7,8]_.

Christian and Donald also provided valuable feedback on a preliminary
draft of this proposal.

Thanks also to participants in the python-dev mailing list threads
[1,2,5,6]_, as well as the various folks I discussed this issue with at
PyCon 2014 in Montreal.


References
==========

.. [1] PEP 466 discussion (round 1)
   (https://mail.python.org/pipermail/python-dev/2014-March/133334.html)

.. [2] PEP 466 discussion (round 2)
   (https://mail.python.org/pipermail/python-dev/2014-March/133389.html)

.. [3] Marc-Andre Lemburg's OpenSSL feedback for Windows
   (https://mail.python.org/pipermail/python-dev/2014-March/133438.html)

.. [4] Ned Deily's OpenSSL feedback for Mac OS X
   (https://mail.python.org/pipermail/python-dev/2014-March/133347.html)

.. [5] PEP 466 discussion (round 3)
   (https://mail.python.org/pipermail/python-dev/2014-March/133442.html)

.. [6] PEP 466 discussion (round 4)
   (https://mail.python.org/pipermail/python-dev/2014-March/133472.html)

.. [7] Donald Stufft's recommended set of backported features
   (https://mail.python.org/pipermail/python-dev/2014-March/133500.html)

.. [8] Alex Gaynor's recommended set of backported features
   (https://mail.python.org/pipermail/python-dev/2014-March/133503.html)



Copyright
=========

This document has been placed in the public domain.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From solipsis at pitrou.net  Thu Apr 17 00:14:17 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 17 Apr 2014 00:14:17 +0200
Subject: [Python-Dev] Timing breakdown of Py_InitializeEx_Private()
In-Reply-To: <534EE92D.4030204@stackless.com>
References: <CAP1=2W64X6YHT7OQZ4KQ6WOq2wV=rj848XprL091rQ7NN55aCA@mail.gmail.com>
 <likii3$vr2$1@ger.gmane.org>
 <CAP7+vJKRBXho4KhDarvkCQWN2CsF499t8ERqTsyoxyp6qwVmwg@mail.gmail.com>
 <534E33B6.8040706@stackless.com> <20140416163506.203ae993@fsol>
 <534EE92D.4030204@stackless.com>
Message-ID: <20140417001417.241c0330@fsol>

On Wed, 16 Apr 2014 22:33:49 +0200
Christian Tismer <tismer at stackless.com> wrote:
> On 16/04/14 16:35, Antoine Pitrou wrote:
> > On Wed, 16 Apr 2014 09:39:34 +0200
> > Christian Tismer <tismer at stackless.com> wrote:
> >>
> >> I think in cases like hg command line scripts there is no need
> >> to import site just for hg scripts.
> > 
> > If you don't import site you won't be able to import Mercurial in most
> > cases.
> 
> 
> Oh is that so?
> I thought about hg as a set of command-line scripts with a defined
> interface that should IMO not be dependent from any site-specific
> settings.

Well perhaps you should take a look. Mercurial is actually a package
(in the Python sense) named "mercurial" with a bunch of modules inside
it. When it gets installed, it's typically installed into
site-packages, which means site.py is necessary for the install
location to crop up into sys.path.

Regards

Antoine.

From solipsis at pitrou.net  Thu Apr 17 00:26:21 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 17 Apr 2014 00:26:21 +0200
Subject: [Python-Dev] Language Summit notes
References: <AC5D9A0C-0768-4859-8D93-95380B966549@taaviburns.ca>
Message-ID: <20140417002621.1907c95c@fsol>


Hi Taavi,

Thanks for the report!

> Disussion about packaging continues. Glyph asks if the PSF could fund a
> usability study on installing Python. People generally seem to think
> it's a good idea.

What does this mean exactly? Under OS X and Linux, Python is typically
installed by default. And under Windows, it's a simple installer that
even non-Windows users like me have no problem executing. So what is
the problem this is trying to solve?

> There is no "one installer" that has everything you need for 2.7 right
> now.

Neither for 3.x, for that record.

> Lunch
> ~~~~~
> 
> There was food!

Good to know nobody starved to death :-)

> AP exams are starting to allow Python, but it's 10% of the AP CS exams.

"AP"?
(I thought that was me, but it sounds unlikely :-))

> Selena says that some American parents are worried because they don't
> know if their kids could get jobs [Taavi: loud surprise from audience].

?? That's not exclusively American.

> Estimate of 1 person full time for 1.5 years to get mercurial running on
> Python 3, at half the speed of Python 2.

That sounds exagerated, IMO.

> Guido: Can we get the PSF to issue a press release that 2.7 support will
> continue?

What does "support" mean exactly here?

> -  Glyph will tell us what to do in 3.5.

Thank you, Glyph!

Regards

Antoine.



From benjamin at python.org  Thu Apr 17 00:38:21 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 16 Apr 2014 15:38:21 -0700
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <20140417002621.1907c95c@fsol>
References: <AC5D9A0C-0768-4859-8D93-95380B966549@taaviburns.ca>
 <20140417002621.1907c95c@fsol>
Message-ID: <1397687901.12460.107348329.3EDAD203@webmail.messagingengine.com>

On Wed, Apr 16, 2014, at 15:26, Antoine Pitrou wrote:
> 
> Hi Taavi,
> 
> Thanks for the report!
> 
> > Disussion about packaging continues. Glyph asks if the PSF could fund a
> > usability study on installing Python. People generally seem to think
> > it's a good idea.
> 
> What does this mean exactly? Under OS X and Linux, Python is typically
> installed by default. And under Windows, it's a simple installer that
> even non-Windows users like me have no problem executing. So what is
> the problem this is trying to solve?

The installers might be hard to find on the website or hard to use.

> 
> > There is no "one installer" that has everything you need for 2.7 right
> > now.
> 
> Neither for 3.x, for that record.
> 
> > Lunch
> > ~~~~~
> > 
> > There was food!
> 
> Good to know nobody starved to death :-)
> 
> > AP exams are starting to allow Python, but it's 10% of the AP CS exams.
> 
> "AP"?
> (I thought that was me, but it sounds unlikely :-))

https://en.wikipedia.org/wiki/Advanced_Placement

From donald at stufft.io  Thu Apr 17 00:42:48 2014
From: donald at stufft.io (Donald Stufft)
Date: Wed, 16 Apr 2014 18:42:48 -0400
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <1397687901.12460.107348329.3EDAD203@webmail.messagingengine.com>
References: <AC5D9A0C-0768-4859-8D93-95380B966549@taaviburns.ca>
 <20140417002621.1907c95c@fsol>
 <1397687901.12460.107348329.3EDAD203@webmail.messagingengine.com>
Message-ID: <13CE5EB2-AB92-4D48-A5ED-E499F206B754@stufft.io>


On Apr 16, 2014, at 6:38 PM, Benjamin Peterson <benjamin at python.org> wrote:

> On Wed, Apr 16, 2014, at 15:26, Antoine Pitrou wrote:
>> 
>> Hi Taavi,
>> 
>> Thanks for the report!
>> 
>>> Disussion about packaging continues. Glyph asks if the PSF could fund a
>>> usability study on installing Python. People generally seem to think
>>> it's a good idea.
>> 
>> What does this mean exactly? Under OS X and Linux, Python is typically
>> installed by default. And under Windows, it's a simple installer that
>> even non-Windows users like me have no problem executing. So what is
>> the problem this is trying to solve?
> 
> The installers might be hard to find on the website or hard to use.
> 
>> 
>>> There is no "one installer" that has everything you need for 2.7 right
>>> now.
>> 
>> Neither for 3.x, for that record.
>> 
>>> Lunch
>>> ~~~~~
>>> 
>>> There was food!
>> 
>> Good to know nobody starved to death :-)
>> 
>>> AP exams are starting to allow Python, but it's 10% of the AP CS exams.
>> 
>> "AP"?
>> (I thought that was me, but it sounds unlikely :-))
> 
> https://en.wikipedia.org/wiki/Advanced_Placement
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/donald%40stufft.io

Possibly Glyph meant installing a Python *stack*, which likely includes setuptools and
pip in order to actually get other things installable. Possibly also a compiler setup
for installing C things.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140416/2b35555a/attachment.sig>

From rdmurray at bitdance.com  Thu Apr 17 03:01:54 2014
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 16 Apr 2014 21:01:54 -0400
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <1397687901.12460.107348329.3EDAD203@webmail.messagingengine.com>
References: <AC5D9A0C-0768-4859-8D93-95380B966549@taaviburns.ca>
 <20140417002621.1907c95c@fsol>
 <1397687901.12460.107348329.3EDAD203@webmail.messagingengine.com>
Message-ID: <20140417010154.EFF23B14086@webabinitio.net>

On Wed, 16 Apr 2014 15:38:21 -0700, Benjamin Peterson <benjamin at python.org> wrote:
> On Wed, Apr 16, 2014, at 15:26, Antoine Pitrou wrote:
> > 
> > Hi Taavi,
> > 
> > Thanks for the report!
> > 
> > > Disussion about packaging continues. Glyph asks if the PSF could fund a
> > > usability study on installing Python. People generally seem to think
> > > it's a good idea.
> > 
> > What does this mean exactly? Under OS X and Linux, Python is typically
> > installed by default. And under Windows, it's a simple installer that
> > even non-Windows users like me have no problem executing. So what is
> > the problem this is trying to solve?
> 
> The installers might be hard to find on the website or hard to use.

Exactly.  As Glyph said, the usability study starts from giving the
user the task "install Python on your computer", or "get X working
on your computer" where X is something that is written in Python.
Running the installer is just one step in that process, and not one of
the difficult ones.

Let's give it a whirl:

Googling for 'python' got me python.org as the top hit.  So far so
good.

Clicking on python.org, I looked for a download link.  There wasn't a
button, but there was a header that says 'Download' with a nice download
icon next to it, just below the big blue area that comprises most of
the screen.  I couldn't click on 'download', though.  Nor on the little
download icon.  Then I saw the 'click here' text below it, so I clicked
on that.  That took me to an old-website-style page that was a wall of
text that starts out talking about python 3.0 being released in 2008.
(No, I didn't read the text before the link that explained where I was
going to be sent: I'm looking for a download link.)

Going back to the main page, I noticed that below the 'click here'
there was another link labeled "Python 3.4.0 - Python 2.7.6" (yes,
I know that's really two links, but no that isn't obvious, so someone
clicking on "it" is likely to get one or the other of them more or
less at random, although maybe they'll notice as the link changes
color when their cursor moves over it...).  So clicking on that, I
get another slightly less intimidating wall of text with a lot of "PEP
NNN" lines.  Scrolling down, after a section labeled 'More Resources',
there is a "Download" header...which is again not clickable.  Oh but look,
there's a 'download page' link in the middle of the paragraph after that
"Download" header.

Clicking on *that*, the first couple links on the page are 'Release Notes'
and 'Detailed Release Information'.  Below that we have a Files section.
It starts out with Mac OSX installer, then XZ compressed source tarball,
then gzipped source tarball, then, finally, a Windows installer...followed
by something about a debug information file, a help information file,
another debug information file, and another Windows installer file.
Oh, yeah, that's 64 bit vs 32 bit.  (Are normal windows users
comfortable about picking the right one of those? I assume they are.)

You will note that I missed the 'downloads' link in the menu bar in the
middle of the big blue section on the front page.  If we rewind, and I
hover there, I see a big "Python Source" with buttons for 3.4 and 2.7.
Guess what I'm going to do next if I'm a naive user?  Hmm.  "xxx.xz".
That doesn't look like an installer or executable, so cancel that.
Oh yeah, there's a 'windows' link on the left.  If I click that...I'm
back on that wall-of-peps page.

Googling 'install python' will get you to python.org as the top hit, with
some secondary links that take you to one of the links I found above,
or other less useful ones...but none of the top google links are to a
page that directly contains a link for downloading a windows installer,
not even the one labeled 'Windows'.

There is an 'Installing Python on Windows' link further down the google
results that links to a fairly good page from python-guide.org, whose
first link lets you download the 2.7.6 msi.  I guess that's the 32
bit version.  But it then tells you go to python.org to make sure you
get the latest release, and tells you to click on a link that doesn't
exist any more (the "windows installer" link).

So, yeah.

Usability.

--David

PS: the old website was worse in many ways, and was even more of a wall of
text, but at least once you see the "download now" link in that wall of
text it only takes one click to get to the windows installer (or zero,
if you happen to notice the 'windows installer link in the left hand
navigation bar), and the two windows installers are at the top of the
list on that download page.

PPS: I believe I've heard rumors that the download pages are a work
in progress, so I'm sure they will get *better*...but I think
the above indicates why a usability study isn't a bad idea :)

From tjreedy at udel.edu  Thu Apr 17 07:23:13 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 17 Apr 2014 01:23:13 -0400
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <20140417002621.1907c95c@fsol>
References: <AC5D9A0C-0768-4859-8D93-95380B966549@taaviburns.ca>
 <20140417002621.1907c95c@fsol>
Message-ID: <linogo$huk$1@ger.gmane.org>

On 4/16/2014 6:26 PM, Antoine Pitrou wrote:

>> AP exams are starting to allow Python, but it's 10% of the AP CS exams.
>
> "AP"?
> (I thought that was me, but it sounds unlikely :-))

AP = Advanced Placement. US and Canadian high school students who have 
taken advanced (AP) courses equivalent to American college freshman 
courses can take AP exams to demonstrate that they learned and retained 
enough that they should get college credit for the course.
https://en.wikipedia.org/wiki/Advanced_Placement

I believe there is a committee for each subject that sets out a syllabus 
describing the subject that may be tested. The CS exam originally tested 
knowledge of Pascal (1984-1999) and switched to C++ (1999-2003) and then 
Java (2004-date).
https://en.wikipedia.org/wiki/Advanced_Placement_Computer_Science#AP_Computer_Science_A

The report is that Python is creeping in, though I an not sure exactly 
what the report above means. Python replacing Java as the AP CS language 
would be a fairly big deal here.

-- 
Terry Jan Reedy


From rdmurray at bitdance.com  Thu Apr 17 16:58:35 2014
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 17 Apr 2014 10:58:35 -0400
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <linogo$huk$1@ger.gmane.org>
References: <AC5D9A0C-0768-4859-8D93-95380B966549@taaviburns.ca>
 <20140417002621.1907c95c@fsol> <linogo$huk$1@ger.gmane.org>
Message-ID: <20140417145835.CEC11250D09@webabinitio.net>

On Thu, 17 Apr 2014 01:23:13 -0400, Terry Reedy <tjreedy at udel.edu> wrote:
> On 4/16/2014 6:26 PM, Antoine Pitrou wrote:
> 
> >> AP exams are starting to allow Python, but it's 10% of the AP CS exams.
> >
> > "AP"?
> > (I thought that was me, but it sounds unlikely :-))
> 
> AP = Advanced Placement. US and Canadian high school students who have 
> taken advanced (AP) courses equivalent to American college freshman 
> courses can take AP exams to demonstrate that they learned and retained 
> enough that they should get college credit for the course.
> https://en.wikipedia.org/wiki/Advanced_Placement
> 
> I believe there is a committee for each subject that sets out a syllabus 
> describing the subject that may be tested. The CS exam originally tested 
> knowledge of Pascal (1984-1999) and switched to C++ (1999-2003) and then 
> Java (2004-date).
> https://en.wikipedia.org/wiki/Advanced_Placement_Computer_Science#AP_Computer_Science_A
> 
> The report is that Python is creeping in, though I an not sure exactly 
> what the report above means. Python replacing Java as the AP CS language 
> would be a fairly big deal here.

As I understand it, there is a *new* (pilot?) comp-sci related AP test
that is using Python.  Jessica expects the existing compsci AP tests
may move to Python at some point, but her timeframe was fairly long for
that one.

--David

From jmao at rocketsoftware.com  Thu Apr 17 17:34:26 2014
From: jmao at rocketsoftware.com (Jianfeng Mao)
Date: Thu, 17 Apr 2014 15:34:26 +0000
Subject: [Python-Dev] is the concept of 'reference ownership' no long
 applicable in Python 3.4?
Message-ID: <3e20acf3c75847da9dea85cd03d7c9a5@wal-vm-mbx1.rocketsoftware.com>

Hi,

I noticed the following changes in the C API manuals from 3.3.5 (and earlier versions) to 3.4. I don't know if these changes are deliberate and imply that we C extension developers no longer need to care about 'reference ownership' because of some improvements in 3.4. Could anyone clarify it?

Thanks,
Jianfeng


--- 3.4  C API  reference manual ----
PyObject<https://docs.python.org/3/c-api/structures.html#c.PyObject>* PySequence_GetItem(PyObject<https://docs.python.org/3/c-api/structures.html#c.PyObject> *o, Py_ssize_t i)?<https://docs.python.org/3/c-api/sequence.html#c.PySequence_GetItem>

Return the ith element of o, or NULL on failure. This is the equivalent of the Python expression o[i].
PyObject<https://docs.python.org/3.4/c-api/structures.html#c.PyObject>* PyList_GetItem(PyObject<https://docs.python.org/3.4/c-api/structures.html#c.PyObject> *list, Py_ssize_t index)

Return the object at position index in the list pointed to by list. The position must be positive, indexing from the end of the list is not supported. If index is out of bounds, return NULL and set an IndexError<https://docs.python.org/3.4/library/exceptions.html#IndexError> exception.

--- 3.3.5 C API reference manual ---
PyObject<https://docs.python.org/3.3/c-api/structures.html#PyObject>* PySequence_GetItem(PyObject<https://docs.python.org/3.3/c-api/structures.html#PyObject> *o, Py_ssize_t i)?<https://docs.python.org/3.3/c-api/sequence.html#PySequence_GetItem>
Return value: New reference.

Return the ith element of o, or NULL on failure. This is the equivalent of the Python expression o[i].

PyObject<https://docs.python.org/3.3/c-api/structures.html#PyObject>* PyList_GetItem(PyObject<https://docs.python.org/3.3/c-api/structures.html#PyObject> *list, Py_ssize_t index)
Return value: Borrowed reference.

Return the object at position index in the list pointed to by list. The position must be positive, indexing from the end of the list is not supported. If index is out of bounds, return NULL and set an IndexError<https://docs.python.org/3.3/library/exceptions.html#IndexError> exception.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140417/bb53bfe1/attachment.html>

From bcannon at gmail.com  Thu Apr 17 18:01:22 2014
From: bcannon at gmail.com (Brett Cannon)
Date: Thu, 17 Apr 2014 16:01:22 +0000
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <534EAEF2.1000604@v.loewis.de> <limjpp$bc7$1@ger.gmane.org>
 <CAP1=2W6zZjy9JZ=NrGZtcgEuuLx04WJzk4U6Q37fbt0t1JdoMg@mail.gmail.com>
 <limqj8$n3o$1@ger.gmane.org>
Message-ID: <CAP1=2W6-ACqQ6runj6pru-iL8n2Xf1bNwHzYeULzi4NTJBRZnA@mail.gmail.com>

On Wed Apr 16 2014 at 4:53:25 PM, Terry Reedy <tjreedy at udel.edu> wrote:

>  > On Wednesday, April 16, 2014 2:57:35 PM, Terry Reedy <tjreedy at udel.edu
>  > <mailto:tjreedy at udel.edu>> wrote:
>
>  >     PS. In the user process sys.modules, there are numerous null
>  >     entries like these:
>  >       >>> sys.modules['idlelib.os']
>  >       >>> sys.modules['idlelib.tokenize'__]
>  >       >>> sys.modules['idlelib.io <http://idlelib.io>']
>  >       >>> <etcetera>
>
> On 4/16/2014 3:10 PM, Dr. Brett Cannon wrote:
> > Is this Python 2 or 3?
>
> Py 2. I should have said so. The entries do not appear in py3.
>
> > In Python 2 it means an attempt to perform a relative import failed but
> > an absolute in succeeded, e.g. from idlelib you imported os, so import
>  > tried idlelib.os and then  os.
>
> *I* have not done anything. For tokenize, for instance, the existing
> code just does what I though were absolute imports, in 2 files.
>    import tokenize
>

That's not an absolute import if it's within a package and you didn't
declare `from __future__ import absolute_import`.


>
> Perhaps the extra entries have something to do with the fact that these
> startup imports are invisible to user code, just like those done by the
> interpreter itself on startup. 2.7 uses spawnv (and 3.4 uses subprocces)
> to run something like one of the following.
>    python -c "__import__('idlelib.run').run.main(False)"
>    python -c "__import__('run').main(False)"
>

Nope, it has to simply do with how Python 2 does implicit relative imports.
Add the __future__ statement and they will go away.

-Brett


>
> run.py has several normal lines with
>    import <stdlib module>
>    from idlelib import <idlelib module>
> and ditto for some of the imported idlelib modules.
>
>  > You should definitely consider using a future import to guarantee
> > absolute imports.
>
> --
> Terry Jan Reedy
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140417/9bb4ce2a/attachment.html>

From dickinsm at gmail.com  Thu Apr 17 18:06:16 2014
From: dickinsm at gmail.com (Mark Dickinson)
Date: Thu, 17 Apr 2014 17:06:16 +0100
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <20140417002621.1907c95c@fsol>
References: <AC5D9A0C-0768-4859-8D93-95380B966549@taaviburns.ca>
 <20140417002621.1907c95c@fsol>
Message-ID: <CAAu3qLXnfY7gXy10xzfDagROxodX5_G9YTuAMb-=5aN9DvC0SQ@mail.gmail.com>

On Wed, Apr 16, 2014 at 11:26 PM, Antoine Pitrou <solipsis at pitrou.net>wrote:

> What does this mean exactly? Under OS X and Linux, Python is typically
> installed by default.


Under OS X, at least, I think there are valid reasons to not want to use
the system-supplied Python.  On my up-to-date OS X 10.9.2 machine, I see
Python 2.7.5, NumPy 1.6.2, Matplotlib 1.1.1 and Twisted 12.2.0.  For at
least Matplotlib and NumPy, those versions are pretty old (mid 2012), and
I'd be wary of updating them on the *system* Python: I have no idea what I
might or might not break.

-- 
Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140417/27f1af59/attachment.html>

From njs at pobox.com  Thu Apr 17 18:12:21 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Thu, 17 Apr 2014 17:12:21 +0100
Subject: [Python-Dev] [numpy wishlist] PyMem_*Calloc
In-Reply-To: <CAMpsgwYBqwsBj4dO5PVbQ4fbwbJ7o6k+DF+epC5NeDrzXm5eVw@mail.gmail.com>
References: <CAK5FAtFFa7JmntPo24xTsh9m3eC7bh-kqx9zGnY6heXY5Fsm4Q@mail.gmail.com>
 <CAMpsgwYBqwsBj4dO5PVbQ4fbwbJ7o6k+DF+epC5NeDrzXm5eVw@mail.gmail.com>
Message-ID: <CAPJVwBn1wmdaNK0dxS1+MLAuyEYhDqt44ofZUY6RrmOJ0uWV6w@mail.gmail.com>

On Wed, Apr 16, 2014 at 7:35 PM, Victor Stinner
<victor.stinner at gmail.com> wrote:
> Hi,
>
> 2014-04-16 7:51 GMT-04:00 Julian Taylor <jtaylor.debian at googlemail.com>:
>> In NumPy what we want is the tracing, not the exchangeable allocators.
>
> Did you read the PEP 445? Using the new malloc API, in fact you can
> have both: install new allocators and set up hooks on allocators.
> http://legacy.python.org/dev/peps/pep-0445/

The context here is that there's been some followup discussion on the
numpy list about whether there are cases where we need even more
exotic memory allocators than calloc(), and what to do about it if so.

(Thread: http://mail.scipy.org/pipermail/numpy-discussion/2014-April/069935.html
)

One case that has come up is when efficient use of SIMD instructions
requires better-than-default alignment (e.g. malloc() usually gives
something like 8 byte alignment, but if you're using an instruction
that operates on 32 bytes at once you might need your array to have 32
byte alignment). Most (all?) OSes provide an extended version of
malloc that allows one to request more alignment (posix_memalign on
POSIX, _aligned_malloc on windows), and C11 standardizes this as
aligned_alloc. An important feature of these functions is that they
allocate from the same heap that 'malloc' does, i.e., when done with
the aligned memory you call free() -- there's no such thing as
aligned_free(). This means that if your program uses these functions
then swapping out malloc/free without swapping out aligned_alloc will
produce undesireable results.

Numpy does not currently use aligned allocation, and it's not clear
how important it is -- on older x86 it matters, but not so much on
current CPUs, but when the next round of x86 SIMD instructions are
released next year it might matter again, and apparently on popular
IBM supercomputers it matters (but less on newer versions)[1,2], and
who knows what will happen with ARM. It's a bit of a mess. But if
we're messing about with APIs it seems worth thinking about.

[1] http://mail.scipy.org/pipermail/numpy-discussion/2014-April/069965.html
[2] http://mail.scipy.org/pipermail/numpy-discussion/2014-April/069967.html

A second possible use case is:

>> my_hugetlb_alloc(size)
>>     p = mmap('hugepagefs', ..., MAP_HUGETLB);
>>     PyMem_Register_Alloc(p, size, __func__, __line__);
>>     return p
>>
>> my_hugetlb_free(p);
>>     PyMem_Register_Free(p, __func__, __line__);
>>     munmap(p, ...);
>
> This is exactly how tracemalloc works. The advantage of the PEP 445 is
> that you have a null overhead when tracemalloc is disabled. There is
> no need to check if a trace function is present or not.

I think the key thing about this example is that you would *never*
want to use MAP_HUGETLB as a generic replacement for malloc(). Huge
pages can have all kinds of weird quirky limitations, and are
certainly unsuited for small allocations. BUT they can provide huge
speed wins if used for certain specific allocations in certain
programs. (In case anyone needs a reminder what "huge pages" even are:
http://lwn.net/Articles/374424/)

If I wrote a Python library to make it easy to use huge pages with
numpy, then I might well want the allocations I was making to be
visible to tracemalloc, even though they would not be going through
malloc/free.

(For that matter -- should calls to os.mmap be calling some
tracemalloc hook in general? There are lots of cases where mmap is
really doing memory allocation -- it's very useful for shared memory
and stuff too.)

---

My current impression is something like:

- From the bug report discussion it sounds like calloc() is useful
even in core Python, so it makes sense to go ahead with that
regardless.
- Now that aligned_alloc has been standardized, it might make sense to
add it to the PyMemAllocator struct too.
- And it might also make sense to have an API by which a Python
library can say to tracemalloc: "hey FYI I just allocated something
using my favorite weird exotic method", like in the huge pages
example. This is a fully generic mechanism, so it could act as a kind
of "safety valve" for future weirdnesses.

All numpy *needs* to support its current and immediately foreseeable
usage is calloc(). But I'm a bit nervous about getting trapped -- if
the PyMem_* machinery implements calloc(), and we switch to using it
and advertise tracemalloc support to our users, and then later it
turns out that we need aligned_alloc or similar, then we'll be stuck
unless and until we can get at least one of these other changes into
CPython upstream, and that will suck for all of us.

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

From njs at pobox.com  Thu Apr 17 18:16:08 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Thu, 17 Apr 2014 17:16:08 +0100
Subject: [Python-Dev] [numpy wishlist] PyMem_*Calloc
In-Reply-To: <CAK5FAtFFa7JmntPo24xTsh9m3eC7bh-kqx9zGnY6heXY5Fsm4Q@mail.gmail.com>
References: <CAK5FAtFFa7JmntPo24xTsh9m3eC7bh-kqx9zGnY6heXY5Fsm4Q@mail.gmail.com>
Message-ID: <CAPJVwBmF9wYSwJV8cvEq2HVc9tnq2M2PoE_XenCw7N=fjkvt5g@mail.gmail.com>

On Wed, Apr 16, 2014 at 12:51 PM, Julian Taylor
<jtaylor.debian at googlemail.com> wrote:
> Hi,
> In NumPy what we want is the tracing, not the exchangeable allocators.
> I don't think it is a good idea for the core of a whole stack of
> C-extension based modules to replace the default allocator or allowing
> other modules to replace the allocator NumPy uses.

I don't think modules are ever supposed to replace the underlying
allocator itself -- and it'd be very difficult to do this safely,
since by the time any modules are imported there are already active
allocations floating around. I think the allocator replacement
functionality is designed to be used by applications embedding Python,
who can set it up a special allocator before the interpreter starts.

I'm not sure what exactly why one would need to swap out malloc and
friends for something else, so I can't really judge, but it does at
least seem plausible that if someone is taking the trouble to swap out
the allocator like this then numpy should respect that and use the new
allocator.

-n

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

From dickinsm at gmail.com  Thu Apr 17 18:33:38 2014
From: dickinsm at gmail.com (Mark Dickinson)
Date: Thu, 17 Apr 2014 17:33:38 +0100
Subject: [Python-Dev] is the concept of 'reference ownership' no long
 applicable in Python 3.4?
In-Reply-To: <3e20acf3c75847da9dea85cd03d7c9a5@wal-vm-mbx1.rocketsoftware.com>
References: <3e20acf3c75847da9dea85cd03d7c9a5@wal-vm-mbx1.rocketsoftware.com>
Message-ID: <CAAu3qLVdz8YsA_LLd0jNBh1M+h492nWxh=0zQys-gS7sUk29Gw@mail.gmail.com>

On Thu, Apr 17, 2014 at 4:34 PM, Jianfeng Mao <jmao at rocketsoftware.com>wrote:

>  I noticed the following changes in the C API manuals from 3.3.5 (and
> earlier versions) to 3.4. I don?t know if these changes are deliberate and
> imply that we C extension developers no longer need to care about
> ?reference ownership? because of some improvements in 3.4. Could anyone
> clarify it?
>

AFAIK there's been no deliberate change to the notion of reference
ownership.  Moreover, any such change would break existing C extensions, so
it's highly unlikely that anything's changed here, behaviour-wise.

This looks like a doc build issue: when I build the documentation locally
for the default branch, I still see the expected "Return value: New
reference." lines.  Maybe something went wrong with refcounts.dat or the
Sphinx refcounting extension when building the 3.4 documentation?  Larry:
any ideas?

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140417/0165bd5a/attachment.html>

From dickinsm at gmail.com  Thu Apr 17 18:39:48 2014
From: dickinsm at gmail.com (Mark Dickinson)
Date: Thu, 17 Apr 2014 17:39:48 +0100
Subject: [Python-Dev] is the concept of 'reference ownership' no long
 applicable in Python 3.4?
In-Reply-To: <CAAu3qLVdz8YsA_LLd0jNBh1M+h492nWxh=0zQys-gS7sUk29Gw@mail.gmail.com>
References: <3e20acf3c75847da9dea85cd03d7c9a5@wal-vm-mbx1.rocketsoftware.com>
 <CAAu3qLVdz8YsA_LLd0jNBh1M+h492nWxh=0zQys-gS7sUk29Gw@mail.gmail.com>
Message-ID: <CAAu3qLUFpYY_xak4ySDj3i-vGN9GiO79VkcwZrimvDxwnHfOzA@mail.gmail.com>

On Thu, Apr 17, 2014 at 5:33 PM, Mark Dickinson <dickinsm at gmail.com> wrote:

> This looks like a doc build issue: when I build the documentation locally
> for the default branch, I still see the expected "Return value: New
> reference." lines.
>

Opened http://bugs.python.org/issue21286 for this issue.

-- 
Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140417/ee207a7d/attachment.html>

From benjamin at python.org  Thu Apr 17 18:42:31 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 17 Apr 2014 09:42:31 -0700
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.4 -> default):
	Merge 3.4
In-Reply-To: <3g8lQZ4rQQz7Lkh@mail.python.org>
References: <3g8lQZ4rQQz7Lkh@mail.python.org>
Message-ID: <1397752951.29059.107630649.01472991@webmail.messagingengine.com>

On Thu, Apr 17, 2014, at 8:55, matthias.klose wrote:
> http://hg.python.org/cpython/rev/1d1aefd00f07
> changeset:   90382:1d1aefd00f07
> parent:      90380:517de1983677
> parent:      90381:1a00e04a233d
> user:        doko at ubuntu.com
> date:        Thu Apr 17 17:55:03 2014 +0200
> summary:
>   Merge 3.4
> 
> files:
>   Misc/NEWS |   6 ++++++
>   setup.py  |  15 +++++++++++++--
>   2 files changed, 19 insertions(+), 2 deletions(-)
> 
> 
> diff --git a/Misc/NEWS b/Misc/NEWS
> --- a/Misc/NEWS
> +++ b/Misc/NEWS
> @@ -244,8 +244,14 @@
>  Build
>  -----
>  
> +<<<<<<< local
>  - Issue #17861: Tools/scripts/generate_opcode_h.py automatically
>  regenerates
>    Include/opcode.h from Lib/opcode.py if the later gets any change.
> +=======
> +- Issue #15234: For BerkelyDB and Sqlite, only add the found library and
> +  include directories if they aren't already being searched. This avoids
> +  an explicit runtime library dependency.
> +>>>>>>> other

You might want to actually resolve this conflict.

From jurko.gospodnetic at pke.hr  Thu Apr 17 19:33:35 2014
From: jurko.gospodnetic at pke.hr (=?UTF-8?B?SnVya28gR29zcG9kbmV0acSH?=)
Date: Thu, 17 Apr 2014 19:33:35 +0200
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
Message-ID: <lip39g$jnr$1@ger.gmane.org>

   Hi.

On 14.4.2014. 23:51, Brett Cannon wrote:
> Now the question is whether the maintenance cost of having to rebuild
> Python for a select number of stdlib modules is enough to warrant
> putting in the effort to make this work.

   I would really love to have better startup times in production, but I 
would also really hate to lose the ability to hack around in stdlib 
sources during development just to get better startup performance.

   In general, what I really like about using Python for software 
development is the ability to open any stdlib file and easily go poking 
around using stuff like 'import pdb;pdb.set_trace()' or simple print 
statements. Researching mysterious behaviour is generally much much 
MUCH! easier (read: takes less hours/days/weeks) if it ends up leading 
into a stdlib Python module than if it takes you down into the bowels of 
some C module (think zipimport.c *grin*). Not to mention the effect that 
being able to quickly resolve a mystery by hacking on some Python 
internals leaves you feeling very satisfied, while having to entrench 
yourself in those internals for a long time just to find out you've made 
something foolish on your end leaves you feeling exhausted at best.

   Not considering the zipped stdlib technique mentioned in other posts, 
would it perhaps be better to support two different CPython builds:
   - one with all the needed stdlib parts frozen - to be used in production
   - one with only the minimal needed number of stdlib parts frozen - to 
have as much of the stdlib sources readily accessible to application 
developers as possible

   The installer could then perhaps install both executables, or the 
frozen stdlib parts could perhaps be built as a separate DLL to be 
loaded at runtime instead of its content being used from their Python 
sources.

   OK... just my 2 cents worth... :-)

   Best regards,
     Jurko Gospodneti?



From guido at python.org  Thu Apr 17 19:57:20 2014
From: guido at python.org (Guido van Rossum)
Date: Thu, 17 Apr 2014 10:57:20 -0700
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <lip39g$jnr$1@ger.gmane.org>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <lip39g$jnr$1@ger.gmane.org>
Message-ID: <CAP7+vJ+42=Eti-wD1pAtx-FF4aEZ3qKUFF05Oadcm9xjSFmqfA@mail.gmail.com>

On Thu, Apr 17, 2014 at 10:33 AM, Jurko Gospodneti? <
jurko.gospodnetic at pke.hr> wrote:

>   I would really love to have better startup times in production,


What's your use case? I understand why startup time is important for Hg,
but I'd like to understand what other situations occur frequently enough to
worry about it.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140417/c59d4a08/attachment.html>

From bcannon at gmail.com  Thu Apr 17 20:09:22 2014
From: bcannon at gmail.com (Brett Cannon)
Date: Thu, 17 Apr 2014 18:09:22 +0000
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <lip39g$jnr$1@ger.gmane.org>
Message-ID: <CAP1=2W4hHLn6rPS36bokKXgKnECDsFgKnp=TGJS_2MgvvcLmhA@mail.gmail.com>

On Thu Apr 17 2014 at 1:34:23 PM, Jurko Gospodneti? <
jurko.gospodnetic at pke.hr> wrote:

>    Hi.
>
> On 14.4.2014. 23:51, Brett Cannon wrote:
> > Now the question is whether the maintenance cost of having to rebuild
> > Python for a select number of stdlib modules is enough to warrant
> > putting in the effort to make this work.
>
>    I would really love to have better startup times in production, but I
> would also really hate to lose the ability to hack around in stdlib
> sources during development just to get better startup performance.
>
>    In general, what I really like about using Python for software
> development is the ability to open any stdlib file and easily go poking
> around using stuff like 'import pdb;pdb.set_trace()' or simple print
> statements. Researching mysterious behaviour is generally much much
> MUCH! easier (read: takes less hours/days/weeks) if it ends up leading
> into a stdlib Python module than if it takes you down into the bowels of
> some C module (think zipimport.c *grin*). Not to mention the effect that
> being able to quickly resolve a mystery by hacking on some Python
> internals leaves you feeling very satisfied, while having to entrench
> yourself in those internals for a long time just to find out you've made
> something foolish on your end leaves you feeling exhausted at best.
>

Freezing modules does not affect the ability to use gdb. And as long as you
set the appropriate __file__ values then tracebacks will contain even the
file line and location.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140417/bc35d1a7/attachment.html>

From marky1991 at gmail.com  Thu Apr 17 20:15:51 2014
From: marky1991 at gmail.com (Mark Young)
Date: Thu, 17 Apr 2014 14:15:51 -0400
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <CAP1=2W4hHLn6rPS36bokKXgKnECDsFgKnp=TGJS_2MgvvcLmhA@mail.gmail.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <lip39g$jnr$1@ger.gmane.org>
 <CAP1=2W4hHLn6rPS36bokKXgKnECDsFgKnp=TGJS_2MgvvcLmhA@mail.gmail.com>
Message-ID: <CAG3cHab=WE-GW7Po5SVLpq3K31WmH2+57M1o00HCqZ8k8nBPXQ@mail.gmail.com>

I think he meant modifying the source files themselves for debugging
purposes (e.g. putting print statements in itertools.py).


2014-04-17 14:09 GMT-04:00 Brett Cannon <bcannon at gmail.com>:

>
>
> On Thu Apr 17 2014 at 1:34:23 PM, Jurko Gospodneti? <
> jurko.gospodnetic at pke.hr> wrote:
>
>>    Hi.
>>
>> On 14.4.2014. 23:51, Brett Cannon wrote:
>> > Now the question is whether the maintenance cost of having to rebuild
>> > Python for a select number of stdlib modules is enough to warrant
>> > putting in the effort to make this work.
>>
>>    I would really love to have better startup times in production, but I
>> would also really hate to lose the ability to hack around in stdlib
>> sources during development just to get better startup performance.
>>
>>    In general, what I really like about using Python for software
>> development is the ability to open any stdlib file and easily go poking
>> around using stuff like 'import pdb;pdb.set_trace()' or simple print
>> statements. Researching mysterious behaviour is generally much much
>> MUCH! easier (read: takes less hours/days/weeks) if it ends up leading
>> into a stdlib Python module than if it takes you down into the bowels of
>> some C module (think zipimport.c *grin*). Not to mention the effect that
>> being able to quickly resolve a mystery by hacking on some Python
>> internals leaves you feeling very satisfied, while having to entrench
>> yourself in those internals for a long time just to find out you've made
>> something foolish on your end leaves you feeling exhausted at best.
>>
>
> Freezing modules does not affect the ability to use gdb. And as long as
> you set the appropriate __file__ values then tracebacks will contain even
> the file line and location.
>
> -Brett
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/marky1991%40gmail.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140417/043335b5/attachment.html>

From solipsis at pitrou.net  Thu Apr 17 20:17:52 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 17 Apr 2014 20:17:52 +0200
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <lip39g$jnr$1@ger.gmane.org>
 <CAP1=2W4hHLn6rPS36bokKXgKnECDsFgKnp=TGJS_2MgvvcLmhA@mail.gmail.com>
Message-ID: <20140417201752.4fc9fafe@fsol>

On Thu, 17 Apr 2014 18:09:22 +0000
Brett Cannon <bcannon at gmail.com> wrote:
> >
> >    I would really love to have better startup times in production, but I
> > would also really hate to lose the ability to hack around in stdlib
> > sources during development just to get better startup performance.
> >
> >    In general, what I really like about using Python for software
> > development is the ability to open any stdlib file and easily go poking
> > around using stuff like 'import pdb;pdb.set_trace()' or simple print
> > statements. Researching mysterious behaviour is generally much much
> > MUCH! easier (read: takes less hours/days/weeks) if it ends up leading
> > into a stdlib Python module than if it takes you down into the bowels of
> > some C module (think zipimport.c *grin*). Not to mention the effect that
> > being able to quickly resolve a mystery by hacking on some Python
> > internals leaves you feeling very satisfied, while having to entrench
> > yourself in those internals for a long time just to find out you've made
> > something foolish on your end leaves you feeling exhausted at best.
> >
> 
> Freezing modules does not affect the ability to use gdb. And as long as you
> set the appropriate __file__ values then tracebacks will contain even the
> file line and location.

I sympathize with Jurko's opinion. Being able to poke inside stdlib
source files makes Python more approachable. I'm sure several of us got
into Python that way.

Regards

Antoine.



From guido at python.org  Thu Apr 17 20:21:41 2014
From: guido at python.org (Guido van Rossum)
Date: Thu, 17 Apr 2014 11:21:41 -0700
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <20140417201752.4fc9fafe@fsol>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <lip39g$jnr$1@ger.gmane.org>
 <CAP1=2W4hHLn6rPS36bokKXgKnECDsFgKnp=TGJS_2MgvvcLmhA@mail.gmail.com>
 <20140417201752.4fc9fafe@fsol>
Message-ID: <CAP7+vJLUsXgY4cVvUGgnPqGp8Rhc9fTtU7UhtYDOZaCYYz-m-g@mail.gmail.com>

I still do that!


On Thu, Apr 17, 2014 at 11:17 AM, Antoine Pitrou <solipsis at pitrou.net>wrote:

> On Thu, 17 Apr 2014 18:09:22 +0000
> Brett Cannon <bcannon at gmail.com> wrote:
> > >
> > >    I would really love to have better startup times in production, but
> I
> > > would also really hate to lose the ability to hack around in stdlib
> > > sources during development just to get better startup performance.
> > >
> > >    In general, what I really like about using Python for software
> > > development is the ability to open any stdlib file and easily go poking
> > > around using stuff like 'import pdb;pdb.set_trace()' or simple print
> > > statements. Researching mysterious behaviour is generally much much
> > > MUCH! easier (read: takes less hours/days/weeks) if it ends up leading
> > > into a stdlib Python module than if it takes you down into the bowels
> of
> > > some C module (think zipimport.c *grin*). Not to mention the effect
> that
> > > being able to quickly resolve a mystery by hacking on some Python
> > > internals leaves you feeling very satisfied, while having to entrench
> > > yourself in those internals for a long time just to find out you've
> made
> > > something foolish on your end leaves you feeling exhausted at best.
> > >
> >
> > Freezing modules does not affect the ability to use gdb. And as long as
> you
> > set the appropriate __file__ values then tracebacks will contain even the
> > file line and location.
>
> I sympathize with Jurko's opinion. Being able to poke inside stdlib
> source files makes Python more approachable. I'm sure several of us got
> into Python that way.
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140417/1909cd27/attachment.html>

From jurko.gospodnetic at pke.hr  Thu Apr 17 20:25:23 2014
From: jurko.gospodnetic at pke.hr (=?UTF-8?B?SnVya28gR29zcG9kbmV0acSH?=)
Date: Thu, 17 Apr 2014 20:25:23 +0200
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <CAG3cHab=WE-GW7Po5SVLpq3K31WmH2+57M1o00HCqZ8k8nBPXQ@mail.gmail.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <lip39g$jnr$1@ger.gmane.org>
 <CAP1=2W4hHLn6rPS36bokKXgKnECDsFgKnp=TGJS_2MgvvcLmhA@mail.gmail.com>
 <CAG3cHab=WE-GW7Po5SVLpq3K31WmH2+57M1o00HCqZ8k8nBPXQ@mail.gmail.com>
Message-ID: <lip6ak$3l5$1@ger.gmane.org>

   Hi.

On 17.4.2014. 20:15, Mark Young wrote:
> I think he meant modifying the source files themselves for debugging
> purposes (e.g. putting print statements in itertools.py).

   Exactly! :-)

   Best regards,
     Jurko Gospodneti?



From ethan at stoneleaf.us  Thu Apr 17 19:46:23 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 17 Apr 2014 10:46:23 -0700
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <lip39g$jnr$1@ger.gmane.org>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <lip39g$jnr$1@ger.gmane.org>
Message-ID: <5350136F.2090100@stoneleaf.us>

On 04/17/2014 10:33 AM, Jurko Gospodneti? wrote:
>
>    In general, what I really like about using Python for software development is the ability to open any stdlib file and
> easily go poking around using stuff like 'import pdb;pdb.set_trace()' or simple print statements.

+1

--
~Ethan~

From g.rodola at gmail.com  Thu Apr 17 20:41:02 2014
From: g.rodola at gmail.com (Giampaolo Rodola')
Date: Thu, 17 Apr 2014 20:41:02 +0200
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <20140417201752.4fc9fafe@fsol>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <lip39g$jnr$1@ger.gmane.org>
 <CAP1=2W4hHLn6rPS36bokKXgKnECDsFgKnp=TGJS_2MgvvcLmhA@mail.gmail.com>
 <20140417201752.4fc9fafe@fsol>
Message-ID: <CAFYqXL9sJnBG=bZnCR9FOZPqJgrGurzMMmEUBVcX8FOvLU6VGg@mail.gmail.com>

On Thu, Apr 17, 2014 at 8:17 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Thu, 17 Apr 2014 18:09:22 +0000
> Brett Cannon <bcannon at gmail.com> wrote:
> > >
> > >    I would really love to have better startup times in production, but
> I
> > > would also really hate to lose the ability to hack around in stdlib
> > > sources during development just to get better startup performance.
> > >
> > >    In general, what I really like about using Python for software
> > > development is the ability to open any stdlib file and easily go poking
> > > around using stuff like 'import pdb;pdb.set_trace()' or simple print
> > > statements. Researching mysterious behaviour is generally much much
> > > MUCH! easier (read: takes less hours/days/weeks) if it ends up leading
> > > into a stdlib Python module than if it takes you down into the bowels
> of
> > > some C module (think zipimport.c *grin*). Not to mention the effect
> that
> > > being able to quickly resolve a mystery by hacking on some Python
> > > internals leaves you feeling very satisfied, while having to entrench
> > > yourself in those internals for a long time just to find out you've
> made
> > > something foolish on your end leaves you feeling exhausted at best.
> > >
> >
> > Freezing modules does not affect the ability to use gdb. And as long as
> you
> > set the appropriate __file__ values then tracebacks will contain even the
> > file line and location.
>
> I sympathize with Jurko's opinion. Being able to poke inside stdlib
> source files makes Python more approachable. I'm sure several of us got
> into Python that way.
>
> Regards
>
> Antoine.


I also wouldn't want that to be the default but Martin also suggested a -Z
cmdline option which sounds like an interesting idea to me.
...Or maybe simply use the existent -O option, which doesn't really
optimize much AFAIK.


-- 
Giampaolo - http://grodola.blogspot.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140417/b8b5adf0/attachment.html>

From leandropls at cpti.cetuc.puc-rio.br  Thu Apr 17 20:32:45 2014
From: leandropls at cpti.cetuc.puc-rio.br (Leandro Pereira de Lima e Silva)
Date: Thu, 17 Apr 2014 15:32:45 -0300
Subject: [Python-Dev] List vs Tuple / Homogeneous vs Heterogeneous / Mutable
	vs Immutable
Message-ID: <CAEaFV_iasbNn0_jbN6XaR8+oNq5m+9vi1TfwTZcuHp7OnbqLSg@mail.gmail.com>

Hello there!

I've stumbled upon this discussion on python-dev about what the choice
between using a list or a tuple is all about in 2003:
1. https://mail.python.org/pipermail/python-dev/2003-March/033962.html
2. https://mail.python.org/pipermail/python-dev/2003-March/034029.html

There's a vague comment about it on python documentation but afaik there
the discussion hasn't made into any PEPs. Is there an understanding about
it?

Cheers, Leandro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140417/b82aafd2/attachment.html>

From jurko.gospodnetic at pke.hr  Thu Apr 17 20:23:52 2014
From: jurko.gospodnetic at pke.hr (=?UTF-8?B?SnVya28gR29zcG9kbmV0acSH?=)
Date: Thu, 17 Apr 2014 20:23:52 +0200
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <CAP7+vJ+42=Eti-wD1pAtx-FF4aEZ3qKUFF05Oadcm9xjSFmqfA@mail.gmail.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <lip39g$jnr$1@ger.gmane.org>
 <CAP7+vJ+42=Eti-wD1pAtx-FF4aEZ3qKUFF05Oadcm9xjSFmqfA@mail.gmail.com>
Message-ID: <53501C38.10202@pke.hr>

   Hi.

On 17.4.2014. 19:57, Guido van Rossum wrote:
> On Thu, Apr 17, 2014 at 10:33 AM, Jurko Gospodneti?
> <jurko.gospodnetic at pke.hr <mailto:jurko.gospodnetic at pke.hr>> wrote:
>
>        I would really love to have better startup times in production,
>
> What's your use case? I understand why startup time is important for Hg,
> but I'd like to understand what other situations occur frequently enough
> to worry about it.

   The first one that pops to mind is scripting when automating 
different system administration tasks.

   When you automate something that ends up calling lots of different 
Python scripts - the startup times add up. Yes, I know you can update 
the system so that the scripts get called inside a single Python 
process, but that often requires major refactoring, e.g.:
   - you have to refactor those scripts to be importable while they were 
originally prepared to be used as 'stand-alone executables'
   - you either have to use Python as your external automation tool or 
you need to implement some sort of a Python based tool runner daemon process

   Another example is the speed at which some automated test suits run 
that need to call external Python scripts. Such suites often call 
thousands of such scripts so their startup times add up to such numbers 
that Python gets a bad rep. And shaving off unnecessarily wasted seconds 
or minutes in a test suite is always good, as it speeds up the whole 
develop/test cycle. :-)

   I've been in situations where I got a request to 'convert those 
Python scripts to batch files so they would run faster'. :-) And, while 
I really love Python as a development language, simple scripts 
implemented in it often do make the system feel kind of sluggish. :-(

   And with that in mind, the effect of systems becoming 'even more 
sluggish' when upgrading them to use the new 'Python 3' version, even if 
that slowdown is not all startup related, often comes as an additional 
slap in the face. :-(

   Best regards,
     Jurko Gospodneti?


From bcannon at gmail.com  Thu Apr 17 20:47:52 2014
From: bcannon at gmail.com (Brett Cannon)
Date: Thu, 17 Apr 2014 18:47:52 +0000
Subject: [Python-Dev] this is what happens if you freeze all the modules
	required for startup
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
Message-ID: <CAP1=2W5Oj_mtigyQxDi-EyQNXhuNikEuQA2L+kfpkFBLpm1ssQ@mail.gmail.com>

Because people keep bringing it up, below is the results of hacking up the
interpreter to include a sys.path entry for ./python35.zip instead of
hard-coding to /usr/lib/python35.zip and simply zipped up Lib/ recursively.
TL;DR, zipimport performance no longer measures up (probably because of
stat caching and such that importlib introduced).

### normal_startup ###

Min: 0.510211 -> 2.667958: 5.23x slower

Avg: 0.521073 -> 2.694876: 5.17x slower

Significant (t=-1129.54)

Stddev: 0.00478 -> 0.01274: 2.6681x larger


### startup_nosite ###

Min: 0.304090 -> 0.908059: 2.99x slower

Avg: 0.312374 -> 0.921807: 2.95x slower

Significant (t=-797.79)

Stddev: 0.00372 -> 0.00667: 1.7956x larger



-Brett

On Mon Apr 14 2014 at 5:51:23 PM, Brett Cannon <bcannon at gmail.com> wrote:

> It was realized during PyCon that since we are freezing importlib we could
> now consider freezing all the modules to cut out having to stat or read
> them from disk. So for day 1 of the sprints I I decided to hack up a
> proof-of-concept to see what kind of performance gain it would get.
>
> Freezing everything except encodings.__init__, os, and _sysconfigdata, it
> speeds up startup by 11% compared to default. Compared to 2.7 it shaves 14%
> from the slowdown (27% slower vs. 41% slower). The full results are at the
> end of the email.
>
> Now the question is whether the maintenance cost of having to rebuild
> Python for a select number of stdlib modules is enough to warrant putting
> in the effort to make this work. My guess is the best approach would be
> adding a Lib/_frozen directory where any modules that we treat like this
> would be kept to act as a reminder that you need to rebuild for them (I
> would probably move importlib/_boostrap.py as well to make this consistent).
>
> Thoughts?
>
>
> --------------------------------------
>
> default vs the freezing:
>
> ### normal_startup ###
>
> Min: 0.524812 -> 0.473339: 1.11x faster
>
> Avg: 0.534403 -> 0.481245: 1.11x faster
>
> Significant (t=61.80)
>
> Stddev: 0.00466 -> 0.00391: 1.1909x smaller
>
>
> ### startup_nosite ###
>
> Min: 0.307359 -> 0.291939: 1.05x faster
>
> Avg: 0.317667 -> 0.300156: 1.06x faster
>
> Significant (t=26.29)
>
> Stddev: 0.00543 -> 0.00385: 1.4099x smaller
>
> ---------
>
> 2.7 vs the freezing:
>
> ### normal_startup ###
>
> Min: 0.367571 -> 0.465264: 1.27x slower
>
> Avg: 0.374404 -> 0.476662: 1.27x slower
>
> Significant (t=-90.26)
>
> Stddev: 0.00313 -> 0.00738: 2.3603x larger
>
>
> ### startup_nosite ###
>
> Min: 0.164510 -> 0.290544: 1.77x slower
>
> Avg: 0.169833 -> 0.301109: 1.77x slower
>
> Significant (t=-286.30)
>
> Stddev: 0.00211 -> 0.00407: 1.9310x larger
>
> ---------
>
> As a baseline, 2.7 vs default:
>
> ### normal_startup ###
>
> Min: 0.368916 -> 0.521758: 1.41x slower
>
> Avg: 0.376784 -> 0.531883: 1.41x slower
>
> Significant (t=-172.82)
>
> Stddev: 0.00423 -> 0.00474: 1.1207x larger
>
>
> ### startup_nosite ###
>
> Min: 0.165156 -> 0.309090: 1.87x slower
>
> Avg: 0.171516 -> 0.319004: 1.86x slower
>
> Significant (t=-283.45)
>
> Stddev: 0.00334 -> 0.00399: 1.1948x larger
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140417/98fbc7ff/attachment.html>

From bcannon at gmail.com  Thu Apr 17 20:49:21 2014
From: bcannon at gmail.com (Brett Cannon)
Date: Thu, 17 Apr 2014 18:49:21 +0000
Subject: [Python-Dev] List vs Tuple / Homogeneous vs Heterogeneous / Mutable
 vs Immutable
References: <CAEaFV_iasbNn0_jbN6XaR8+oNq5m+9vi1TfwTZcuHp7OnbqLSg@mail.gmail.com>
Message-ID: <CAP1=2W6NMJdtnVr22=gqx8bOwoY7JM7uhk78V6D4iU_WBVF=cA@mail.gmail.com>

On Thu Apr 17 2014 at 2:43:35 PM, Leandro Pereira de Lima e Silva <
leandropls at cpti.cetuc.puc-rio.br> wrote:

> Hello there!
>
> I've stumbled upon this discussion on python-dev about what the choice
> between using a list or a tuple is all about in 2003:
> 1. https://mail.python.org/pipermail/python-dev/2003-March/033962.html
> 2. https://mail.python.org/pipermail/python-dev/2003-March/034029.html
>
> There's a vague comment about it on python documentation but afaik there
> the discussion hasn't made into any PEPs. Is there an understanding about
> it?
>

Think of tuples like a struct in C, lists like an array. That's just out of
Guido's head so I don't think we have ever bothered to write it down
somewhere as an important distinction of the initial design that should be
emphasized.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140417/c70d4a86/attachment.html>

From bcannon at gmail.com  Thu Apr 17 21:10:46 2014
From: bcannon at gmail.com (Brett Cannon)
Date: Thu, 17 Apr 2014 19:10:46 +0000
Subject: [Python-Dev] this is what happens if you freeze all the modules
	required for startup
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <CAP1=2W5Oj_mtigyQxDi-EyQNXhuNikEuQA2L+kfpkFBLpm1ssQ@mail.gmail.com>
Message-ID: <CAP1=2W5yB5QnyDFq-mqjtOdKdgnu3VC-PM1yPZOJR242gXOpPg@mail.gmail.com>

Actually ignore this data, I think I may have messed something up. I'll
reply after I check something

On Thu Apr 17 2014 at 2:47:52 PM, Brett Cannon <bcannon at gmail.com> wrote:

> Because people keep bringing it up, below is the results of hacking up the
> interpreter to include a sys.path entry for ./python35.zip instead of
> hard-coding to /usr/lib/python35.zip and simply zipped up Lib/ recursively.
> TL;DR, zipimport performance no longer measures up (probably because of
> stat caching and such that importlib introduced).
>
> ### normal_startup ###
>
> Min: 0.510211 -> 2.667958: 5.23x slower
>
> Avg: 0.521073 -> 2.694876: 5.17x slower
>
> Significant (t=-1129.54)
>
> Stddev: 0.00478 -> 0.01274: 2.6681x larger
>
>
> ### startup_nosite ###
>
> Min: 0.304090 -> 0.908059: 2.99x slower
>
> Avg: 0.312374 -> 0.921807: 2.95x slower
>
> Significant (t=-797.79)
>
> Stddev: 0.00372 -> 0.00667: 1.7956x larger
>
>
>
> -Brett
>
> On Mon Apr 14 2014 at 5:51:23 PM, Brett Cannon <bcannon at gmail.com> wrote:
>
>> It was realized during PyCon that since we are freezing importlib we
>> could now consider freezing all the modules to cut out having to stat or
>> read them from disk. So for day 1 of the sprints I I decided to hack up a
>> proof-of-concept to see what kind of performance gain it would get.
>>
>> Freezing everything except encodings.__init__, os, and _sysconfigdata, it
>> speeds up startup by 11% compared to default. Compared to 2.7 it shaves 14%
>> from the slowdown (27% slower vs. 41% slower). The full results are at the
>> end of the email.
>>
>> Now the question is whether the maintenance cost of having to rebuild
>> Python for a select number of stdlib modules is enough to warrant putting
>> in the effort to make this work. My guess is the best approach would be
>> adding a Lib/_frozen directory where any modules that we treat like this
>> would be kept to act as a reminder that you need to rebuild for them (I
>> would probably move importlib/_boostrap.py as well to make this consistent).
>>
>> Thoughts?
>>
>>
>> --------------------------------------
>>
>> default vs the freezing:
>>
>> ### normal_startup ###
>>
>> Min: 0.524812 -> 0.473339: 1.11x faster
>>
>> Avg: 0.534403 -> 0.481245: 1.11x faster
>>
>> Significant (t=61.80)
>>
>> Stddev: 0.00466 -> 0.00391: 1.1909x smaller
>>
>>
>> ### startup_nosite ###
>>
>> Min: 0.307359 -> 0.291939: 1.05x faster
>>
>> Avg: 0.317667 -> 0.300156: 1.06x faster
>>
>> Significant (t=26.29)
>>
>> Stddev: 0.00543 -> 0.00385: 1.4099x smaller
>>
>> ---------
>>
>> 2.7 vs the freezing:
>>
>> ### normal_startup ###
>>
>> Min: 0.367571 -> 0.465264: 1.27x slower
>>
>> Avg: 0.374404 -> 0.476662: 1.27x slower
>>
>> Significant (t=-90.26)
>>
>> Stddev: 0.00313 -> 0.00738: 2.3603x larger
>>
>>
>> ### startup_nosite ###
>>
>> Min: 0.164510 -> 0.290544: 1.77x slower
>>
>> Avg: 0.169833 -> 0.301109: 1.77x slower
>>
>> Significant (t=-286.30)
>>
>> Stddev: 0.00211 -> 0.00407: 1.9310x larger
>>
>> ---------
>>
>> As a baseline, 2.7 vs default:
>>
>> ### normal_startup ###
>>
>> Min: 0.368916 -> 0.521758: 1.41x slower
>>
>> Avg: 0.376784 -> 0.531883: 1.41x slower
>>
>> Significant (t=-172.82)
>>
>> Stddev: 0.00423 -> 0.00474: 1.1207x larger
>>
>>
>> ### startup_nosite ###
>>
>> Min: 0.165156 -> 0.309090: 1.87x slower
>>
>> Avg: 0.171516 -> 0.319004: 1.86x slower
>>
>> Significant (t=-283.45)
>>
>> Stddev: 0.00334 -> 0.00399: 1.1948x larger
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140417/2b49f9e5/attachment.html>

From bcannon at gmail.com  Thu Apr 17 21:14:06 2014
From: bcannon at gmail.com (Brett Cannon)
Date: Thu, 17 Apr 2014 19:14:06 +0000
Subject: [Python-Dev] this is what happens if you freeze all the modules
	required for startup
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <CAP1=2W5Oj_mtigyQxDi-EyQNXhuNikEuQA2L+kfpkFBLpm1ssQ@mail.gmail.com>
 <CAP1=2W5yB5QnyDFq-mqjtOdKdgnu3VC-PM1yPZOJR242gXOpPg@mail.gmail.com>
Message-ID: <CAP1=2W6KjhjVjR-CfFj8S7c_qFuV4DfjOmuWUfOUo0fd5QjvbQ@mail.gmail.com>

On Thu Apr 17 2014 at 3:10:46 PM, Brett Cannon <bcannon at gmail.com> wrote:

> Actually ignore this data, I think I may have messed something up. I'll
> reply after I check something
>

Unfortunately my check says the data is accurate, so zip startup is really
just slow.

-Brett


>
> On Thu Apr 17 2014 at 2:47:52 PM, Brett Cannon <bcannon at gmail.com> wrote:
>
>> Because people keep bringing it up, below is the results of hacking up
>> the interpreter to include a sys.path entry for ./python35.zip instead of
>> hard-coding to /usr/lib/python35.zip and simply zipped up Lib/ recursively.
>> TL;DR, zipimport performance no longer measures up (probably because of
>> stat caching and such that importlib introduced).
>>
>> ### normal_startup ###
>>
>> Min: 0.510211 -> 2.667958: 5.23x slower
>>
>> Avg: 0.521073 -> 2.694876: 5.17x slower
>>
>> Significant (t=-1129.54)
>>
>> Stddev: 0.00478 -> 0.01274: 2.6681x larger
>>
>>
>> ### startup_nosite ###
>>
>> Min: 0.304090 -> 0.908059: 2.99x slower
>>
>> Avg: 0.312374 -> 0.921807: 2.95x slower
>>
>> Significant (t=-797.79)
>>
>> Stddev: 0.00372 -> 0.00667: 1.7956x larger
>>
>>
>>
>> -Brett
>>
>> On Mon Apr 14 2014 at 5:51:23 PM, Brett Cannon <bcannon at gmail.com> wrote:
>>
>>> It was realized during PyCon that since we are freezing importlib we
>>> could now consider freezing all the modules to cut out having to stat or
>>> read them from disk. So for day 1 of the sprints I I decided to hack up a
>>> proof-of-concept to see what kind of performance gain it would get.
>>>
>>> Freezing everything except encodings.__init__, os, and _sysconfigdata,
>>> it speeds up startup by 11% compared to default. Compared to 2.7 it shaves
>>> 14% from the slowdown (27% slower vs. 41% slower). The full results are at
>>> the end of the email.
>>>
>>> Now the question is whether the maintenance cost of having to rebuild
>>> Python for a select number of stdlib modules is enough to warrant putting
>>> in the effort to make this work. My guess is the best approach would be
>>> adding a Lib/_frozen directory where any modules that we treat like this
>>> would be kept to act as a reminder that you need to rebuild for them (I
>>> would probably move importlib/_boostrap.py as well to make this consistent).
>>>
>>> Thoughts?
>>>
>>>
>>> --------------------------------------
>>>
>>> default vs the freezing:
>>>
>>> ### normal_startup ###
>>>
>>> Min: 0.524812 -> 0.473339: 1.11x faster
>>>
>>> Avg: 0.534403 -> 0.481245: 1.11x faster
>>>
>>> Significant (t=61.80)
>>>
>>> Stddev: 0.00466 -> 0.00391: 1.1909x smaller
>>>
>>>
>>> ### startup_nosite ###
>>>
>>> Min: 0.307359 -> 0.291939: 1.05x faster
>>>
>>> Avg: 0.317667 -> 0.300156: 1.06x faster
>>>
>>> Significant (t=26.29)
>>>
>>> Stddev: 0.00543 -> 0.00385: 1.4099x smaller
>>>
>>> ---------
>>>
>>> 2.7 vs the freezing:
>>>
>>> ### normal_startup ###
>>>
>>> Min: 0.367571 -> 0.465264: 1.27x slower
>>>
>>> Avg: 0.374404 -> 0.476662: 1.27x slower
>>>
>>> Significant (t=-90.26)
>>>
>>> Stddev: 0.00313 -> 0.00738: 2.3603x larger
>>>
>>>
>>> ### startup_nosite ###
>>>
>>> Min: 0.164510 -> 0.290544: 1.77x slower
>>>
>>> Avg: 0.169833 -> 0.301109: 1.77x slower
>>>
>>> Significant (t=-286.30)
>>>
>>> Stddev: 0.00211 -> 0.00407: 1.9310x larger
>>>
>>> ---------
>>>
>>> As a baseline, 2.7 vs default:
>>>
>>> ### normal_startup ###
>>>
>>> Min: 0.368916 -> 0.521758: 1.41x slower
>>>
>>> Avg: 0.376784 -> 0.531883: 1.41x slower
>>>
>>> Significant (t=-172.82)
>>>
>>> Stddev: 0.00423 -> 0.00474: 1.1207x larger
>>>
>>>
>>> ### startup_nosite ###
>>>
>>> Min: 0.165156 -> 0.309090: 1.87x slower
>>>
>>> Avg: 0.171516 -> 0.319004: 1.86x slower
>>>
>>> Significant (t=-283.45)
>>>
>>> Stddev: 0.00334 -> 0.00399: 1.1948x larger
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140417/bf76e564/attachment-0001.html>

From guido at python.org  Thu Apr 17 21:21:29 2014
From: guido at python.org (Guido van Rossum)
Date: Thu, 17 Apr 2014 12:21:29 -0700
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <CAP1=2W5Oj_mtigyQxDi-EyQNXhuNikEuQA2L+kfpkFBLpm1ssQ@mail.gmail.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <CAP1=2W5Oj_mtigyQxDi-EyQNXhuNikEuQA2L+kfpkFBLpm1ssQ@mail.gmail.com>
Message-ID: <CAP7+vJ+6LE7QYw9NHVt87KSn30NZFsD+pYex+kxuVU9db+Oz2g@mail.gmail.com>

I'm sorry to keep asking dumb questions, but your description didn't job my
understanding of what you are comparing here. What is slower than what?


On Thu, Apr 17, 2014 at 11:47 AM, Brett Cannon <bcannon at gmail.com> wrote:

> Because people keep bringing it up, below is the results of hacking up the
> interpreter to include a sys.path entry for ./python35.zip instead of
> hard-coding to /usr/lib/python35.zip and simply zipped up Lib/ recursively.
> TL;DR, zipimport performance no longer measures up (probably because of
> stat caching and such that importlib introduced).
>
> ### normal_startup ###
>
> Min: 0.510211 -> 2.667958: 5.23x slower
>
> Avg: 0.521073 -> 2.694876: 5.17x slower
>
> Significant (t=-1129.54)
>
> Stddev: 0.00478 -> 0.01274: 2.6681x larger
>
>
> ### startup_nosite ###
>
> Min: 0.304090 -> 0.908059: 2.99x slower
>
> Avg: 0.312374 -> 0.921807: 2.95x slower
>
> Significant (t=-797.79)
>
> Stddev: 0.00372 -> 0.00667: 1.7956x larger
>
>
>
> -Brett
>
> On Mon Apr 14 2014 at 5:51:23 PM, Brett Cannon <bcannon at gmail.com> wrote:
>
>> It was realized during PyCon that since we are freezing importlib we
>> could now consider freezing all the modules to cut out having to stat or
>> read them from disk. So for day 1 of the sprints I I decided to hack up a
>> proof-of-concept to see what kind of performance gain it would get.
>>
>> Freezing everything except encodings.__init__, os, and _sysconfigdata, it
>> speeds up startup by 11% compared to default. Compared to 2.7 it shaves 14%
>> from the slowdown (27% slower vs. 41% slower). The full results are at the
>> end of the email.
>>
>> Now the question is whether the maintenance cost of having to rebuild
>> Python for a select number of stdlib modules is enough to warrant putting
>> in the effort to make this work. My guess is the best approach would be
>> adding a Lib/_frozen directory where any modules that we treat like this
>> would be kept to act as a reminder that you need to rebuild for them (I
>> would probably move importlib/_boostrap.py as well to make this consistent).
>>
>> Thoughts?
>>
>>
>> --------------------------------------
>>
>> default vs the freezing:
>>
>> ### normal_startup ###
>>
>> Min: 0.524812 -> 0.473339: 1.11x faster
>>
>> Avg: 0.534403 -> 0.481245: 1.11x faster
>>
>> Significant (t=61.80)
>>
>> Stddev: 0.00466 -> 0.00391: 1.1909x smaller
>>
>>
>> ### startup_nosite ###
>>
>> Min: 0.307359 -> 0.291939: 1.05x faster
>>
>> Avg: 0.317667 -> 0.300156: 1.06x faster
>>
>> Significant (t=26.29)
>>
>> Stddev: 0.00543 -> 0.00385: 1.4099x smaller
>>
>> ---------
>>
>> 2.7 vs the freezing:
>>
>> ### normal_startup ###
>>
>> Min: 0.367571 -> 0.465264: 1.27x slower
>>
>> Avg: 0.374404 -> 0.476662: 1.27x slower
>>
>> Significant (t=-90.26)
>>
>> Stddev: 0.00313 -> 0.00738: 2.3603x larger
>>
>>
>> ### startup_nosite ###
>>
>> Min: 0.164510 -> 0.290544: 1.77x slower
>>
>> Avg: 0.169833 -> 0.301109: 1.77x slower
>>
>> Significant (t=-286.30)
>>
>> Stddev: 0.00211 -> 0.00407: 1.9310x larger
>>
>> ---------
>>
>> As a baseline, 2.7 vs default:
>>
>> ### normal_startup ###
>>
>> Min: 0.368916 -> 0.521758: 1.41x slower
>>
>> Avg: 0.376784 -> 0.531883: 1.41x slower
>>
>> Significant (t=-172.82)
>>
>> Stddev: 0.00423 -> 0.00474: 1.1207x larger
>>
>>
>> ### startup_nosite ###
>>
>> Min: 0.165156 -> 0.309090: 1.87x slower
>>
>> Avg: 0.171516 -> 0.319004: 1.86x slower
>>
>> Significant (t=-283.45)
>>
>> Stddev: 0.00334 -> 0.00399: 1.1948x larger
>>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140417/f3066a37/attachment.html>

From guido at python.org  Thu Apr 17 21:23:42 2014
From: guido at python.org (Guido van Rossum)
Date: Thu, 17 Apr 2014 12:23:42 -0700
Subject: [Python-Dev] List vs Tuple / Homogeneous vs Heterogeneous /
 Mutable vs Immutable
In-Reply-To: <CAP1=2W6NMJdtnVr22=gqx8bOwoY7JM7uhk78V6D4iU_WBVF=cA@mail.gmail.com>
References: <CAEaFV_iasbNn0_jbN6XaR8+oNq5m+9vi1TfwTZcuHp7OnbqLSg@mail.gmail.com>
 <CAP1=2W6NMJdtnVr22=gqx8bOwoY7JM7uhk78V6D4iU_WBVF=cA@mail.gmail.com>
Message-ID: <CAP7+vJKMyJ8sm3a3Hce3zngcmDz0SG+V4e31K6qFP38_-2LpYg@mail.gmail.com>

It's definitely something that should be put in some documentation,
probably at the point when people have learned enough to be designing their
own programs where this issue comes up -- before they're wizards but well
after they have learned the semantic differences between lists and tuples.


On Thu, Apr 17, 2014 at 11:49 AM, Brett Cannon <bcannon at gmail.com> wrote:

>
>
> On Thu Apr 17 2014 at 2:43:35 PM, Leandro Pereira de Lima e Silva <
> leandropls at cpti.cetuc.puc-rio.br> wrote:
>
>> Hello there!
>>
>> I've stumbled upon this discussion on python-dev about what the choice
>> between using a list or a tuple is all about in 2003:
>> 1. https://mail.python.org/pipermail/python-dev/2003-March/033962.html
>> 2. https://mail.python.org/pipermail/python-dev/2003-March/034029.html
>>
>> There's a vague comment about it on python documentation but afaik there
>> the discussion hasn't made into any PEPs. Is there an understanding about
>> it?
>>
>
> Think of tuples like a struct in C, lists like an array. That's just out
> of Guido's head so I don't think we have ever bothered to write it down
> somewhere as an important distinction of the initial design that should be
> emphasized.
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140417/00c1913c/attachment.html>

From donald at stufft.io  Thu Apr 17 21:49:48 2014
From: donald at stufft.io (Donald Stufft)
Date: Thu, 17 Apr 2014 15:49:48 -0400
Subject: [Python-Dev] this is what happens if you freeze all the modules
	required for startup
In-Reply-To: <53501C38.10202@pke.hr>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <lip39g$jnr$1@ger.gmane.org>
 <CAP7+vJ+42=Eti-wD1pAtx-FF4aEZ3qKUFF05Oadcm9xjSFmqfA@mail.gmail.com>
 <53501C38.10202@pke.hr>
Message-ID: <268EE724-04D1-4347-9155-92D992E55E90@stufft.io>


On Apr 17, 2014, at 2:23 PM, Jurko Gospodneti? <jurko.gospodnetic at pke.hr> wrote:

>  Hi.
> 
> On 17.4.2014. 19:57, Guido van Rossum wrote:
>> On Thu, Apr 17, 2014 at 10:33 AM, Jurko Gospodneti?
>> <jurko.gospodnetic at pke.hr <mailto:jurko.gospodnetic at pke.hr>> wrote:
>> 
>>       I would really love to have better startup times in production,
>> 
>> What's your use case? I understand why startup time is important for Hg,
>> but I'd like to understand what other situations occur frequently enough
>> to worry about it.
> 
>  The first one that pops to mind is scripting when automating different system administration tasks.
> 
>  When you automate something that ends up calling lots of different Python scripts - the startup times add up. Yes, I know you can update the system so that the scripts get called inside a single Python process, but that often requires major refactoring, e.g.:
>  - you have to refactor those scripts to be importable while they were originally prepared to be used as 'stand-alone executables'
>  - you either have to use Python as your external automation tool or you need to implement some sort of a Python based tool runner daemon process
> 
>  Another example is the speed at which some automated test suits run that need to call external Python scripts. Such suites often call thousands of such scripts so their startup times add up to such numbers that Python gets a bad rep. And shaving off unnecessarily wasted seconds or minutes in a test suite is always good, as it speeds up the whole develop/test cycle. :-)

pip invokes a ton of pythons in a subprocess in it?s test suite, and the ?install from sdist? stuff tends to invoke 1-3 python?s per thing you install too. So any speed up there would make installing stuff faster.

> 
>  I've been in situations where I got a request to 'convert those Python scripts to batch files so they would run faster'. :-) And, while I really love Python as a development language, simple scripts implemented in it often do make the system feel kind of sluggish. :-(
> 
>  And with that in mind, the effect of systems becoming 'even more sluggish' when upgrading them to use the new 'Python 3' version, even if that slowdown is not all startup related, often comes as an additional slap in the face. :-(
> 
>  Best regards,
>    Jurko Gospodneti?
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/donald%40stufft.io


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140417/d7ea376a/attachment.sig>

From guido at python.org  Thu Apr 17 21:59:19 2014
From: guido at python.org (Guido van Rossum)
Date: Thu, 17 Apr 2014 12:59:19 -0700
Subject: [Python-Dev] List vs Tuple / Homogeneous vs Heterogeneous /
 Mutable vs Immutable
In-Reply-To: <CAEaFV_jWbG+u6O5Zzy0+TvYc6otJ11dXPqYD7EXb0VhV+mdPFg@mail.gmail.com>
References: <CAEaFV_iasbNn0_jbN6XaR8+oNq5m+9vi1TfwTZcuHp7OnbqLSg@mail.gmail.com>
 <CAP1=2W6NMJdtnVr22=gqx8bOwoY7JM7uhk78V6D4iU_WBVF=cA@mail.gmail.com>
 <CAP7+vJKMyJ8sm3a3Hce3zngcmDz0SG+V4e31K6qFP38_-2LpYg@mail.gmail.com>
 <CAEaFV_jWbG+u6O5Zzy0+TvYc6otJ11dXPqYD7EXb0VhV+mdPFg@mail.gmail.com>
Message-ID: <CAP7+vJ+Odk6g+xYS_XJt8dPgJ4Dfj-9-mNETYZCTFQPfSiAs=Q@mail.gmail.com>

No, I don't think it belongs in the style guide. It is not about code
formatting or naming, it is about data structure design and API design.


On Thu, Apr 17, 2014 at 12:49 PM, Leandro Pereira de Lima e Silva <
leandropls at cpti.cetuc.puc-rio.br> wrote:

> This looks like an issue to be addressed at PEP-8 since it looks like a
> styling issue.
>
> I haven't seen any other recommendations there on how to use a certain
> data structure, though.
>
> Cheers, Leandro
> Em 17/04/2014 16:24, "Guido van Rossum" <guido at python.org> escreveu:
>
> It's definitely something that should be put in some documentation,
>> probably at the point when people have learned enough to be designing their
>> own programs where this issue comes up -- before they're wizards but well
>> after they have learned the semantic differences between lists and tuples.
>>
>>
>> On Thu, Apr 17, 2014 at 11:49 AM, Brett Cannon <bcannon at gmail.com> wrote:
>>
>>>
>>>
>>> On Thu Apr 17 2014 at 2:43:35 PM, Leandro Pereira de Lima e Silva <
>>> leandropls at cpti.cetuc.puc-rio.br> wrote:
>>>
>>>> Hello there!
>>>>
>>>> I've stumbled upon this discussion on python-dev about what the choice
>>>> between using a list or a tuple is all about in 2003:
>>>> 1. https://mail.python.org/pipermail/python-dev/2003-March/033962.html
>>>> 2. https://mail.python.org/pipermail/python-dev/2003-March/034029.html
>>>>
>>>> There's a vague comment about it on python documentation but afaik
>>>> there the discussion hasn't made into any PEPs. Is there an understanding
>>>> about it?
>>>>
>>>
>>> Think of tuples like a struct in C, lists like an array. That's just out
>>> of Guido's head so I don't think we have ever bothered to write it down
>>> somewhere as an important distinction of the initial design that should be
>>> emphasized.
>>>
>>> _______________________________________________
>>> Python-Dev mailing list
>>> Python-Dev at python.org
>>> https://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe:
>>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>>
>>>
>>
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>>
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140417/47ab8828/attachment.html>

From bcannon at gmail.com  Thu Apr 17 22:31:14 2014
From: bcannon at gmail.com (Brett Cannon)
Date: Thu, 17 Apr 2014 20:31:14 +0000
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <CAP1=2W5Oj_mtigyQxDi-EyQNXhuNikEuQA2L+kfpkFBLpm1ssQ@mail.gmail.com>
 <CAP7+vJ+6LE7QYw9NHVt87KSn30NZFsD+pYex+kxuVU9db+Oz2g@mail.gmail.com>
Message-ID: <CAP1=2W6+4wh6EhAh5To6+L-F81-a5dk8cZSOSOHFqmYPep66Tg@mail.gmail.com>

On Thu Apr 17 2014 at 3:21:49 PM, Guido van Rossum <guido at python.org> wrote:

> I'm sorry to keep asking dumb questions, but your description didn't job
> my understanding of what you are comparing here. What is slower than what?
>

Startup where the stdlib is entirely in a zip file is slower than the
status quo of reading from files. IOW it looks like speeding up startup
from an import perspective requires either freezing modules -- for about a
10% boost -- or some fundamental change in import that no one has thought
of yet.

-Brett


>
>
> On Thu, Apr 17, 2014 at 11:47 AM, Brett Cannon <bcannon at gmail.com> wrote:
>
>> Because people keep bringing it up, below is the results of hacking up
>> the interpreter to include a sys.path entry for ./python35.zip instead of
>> hard-coding to /usr/lib/python35.zip and simply zipped up Lib/ recursively.
>> TL;DR, zipimport performance no longer measures up (probably because of
>> stat caching and such that importlib introduced).
>>
>> ### normal_startup ###
>>
>> Min: 0.510211 -> 2.667958: 5.23x slower
>>
>> Avg: 0.521073 -> 2.694876: 5.17x slower
>>
>> Significant (t=-1129.54)
>>
>> Stddev: 0.00478 -> 0.01274: 2.6681x larger
>>
>>
>> ### startup_nosite ###
>>
>> Min: 0.304090 -> 0.908059: 2.99x slower
>>
>> Avg: 0.312374 -> 0.921807: 2.95x slower
>>
>> Significant (t=-797.79)
>>
>> Stddev: 0.00372 -> 0.00667: 1.7956x larger
>>
>>
>>
>> -Brett
>>
>> On Mon Apr 14 2014 at 5:51:23 PM, Brett Cannon <bcannon at gmail.com> wrote:
>>
>>> It was realized during PyCon that since we are freezing importlib we
>>> could now consider freezing all the modules to cut out having to stat or
>>> read them from disk. So for day 1 of the sprints I I decided to hack up a
>>> proof-of-concept to see what kind of performance gain it would get.
>>>
>>> Freezing everything except encodings.__init__, os, and _sysconfigdata,
>>> it speeds up startup by 11% compared to default. Compared to 2.7 it shaves
>>> 14% from the slowdown (27% slower vs. 41% slower). The full results are at
>>> the end of the email.
>>>
>>> Now the question is whether the maintenance cost of having to rebuild
>>> Python for a select number of stdlib modules is enough to warrant putting
>>> in the effort to make this work. My guess is the best approach would be
>>> adding a Lib/_frozen directory where any modules that we treat like this
>>> would be kept to act as a reminder that you need to rebuild for them (I
>>> would probably move importlib/_boostrap.py as well to make this consistent).
>>>
>>> Thoughts?
>>>
>>>
>>> --------------------------------------
>>>
>>> default vs the freezing:
>>>
>>> ### normal_startup ###
>>>
>>> Min: 0.524812 -> 0.473339: 1.11x faster
>>>
>>> Avg: 0.534403 -> 0.481245: 1.11x faster
>>>
>>> Significant (t=61.80)
>>>
>>> Stddev: 0.00466 -> 0.00391: 1.1909x smaller
>>>
>>>
>>> ### startup_nosite ###
>>>
>>> Min: 0.307359 -> 0.291939: 1.05x faster
>>>
>>> Avg: 0.317667 -> 0.300156: 1.06x faster
>>>
>>> Significant (t=26.29)
>>>
>>> Stddev: 0.00543 -> 0.00385: 1.4099x smaller
>>>
>>> ---------
>>>
>>> 2.7 vs the freezing:
>>>
>>> ### normal_startup ###
>>>
>>> Min: 0.367571 -> 0.465264: 1.27x slower
>>>
>>> Avg: 0.374404 -> 0.476662: 1.27x slower
>>>
>>> Significant (t=-90.26)
>>>
>>> Stddev: 0.00313 -> 0.00738: 2.3603x larger
>>>
>>>
>>> ### startup_nosite ###
>>>
>>> Min: 0.164510 -> 0.290544: 1.77x slower
>>>
>>> Avg: 0.169833 -> 0.301109: 1.77x slower
>>>
>>> Significant (t=-286.30)
>>>
>>> Stddev: 0.00211 -> 0.00407: 1.9310x larger
>>>
>>> ---------
>>>
>>> As a baseline, 2.7 vs default:
>>>
>>> ### normal_startup ###
>>>
>>> Min: 0.368916 -> 0.521758: 1.41x slower
>>>
>>> Avg: 0.376784 -> 0.531883: 1.41x slower
>>>
>>> Significant (t=-172.82)
>>>
>>> Stddev: 0.00423 -> 0.00474: 1.1207x larger
>>>
>>>
>>> ### startup_nosite ###
>>>
>>> Min: 0.165156 -> 0.309090: 1.87x slower
>>>
>>> Avg: 0.171516 -> 0.319004: 1.86x slower
>>>
>>> Significant (t=-283.45)
>>>
>>> Stddev: 0.00334 -> 0.00399: 1.1948x larger
>>>
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>>
> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140417/b1b06ac0/attachment-0001.html>

From guido at python.org  Thu Apr 17 22:38:17 2014
From: guido at python.org (Guido van Rossum)
Date: Thu, 17 Apr 2014 13:38:17 -0700
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <CAP1=2W6+4wh6EhAh5To6+L-F81-a5dk8cZSOSOHFqmYPep66Tg@mail.gmail.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <CAP1=2W5Oj_mtigyQxDi-EyQNXhuNikEuQA2L+kfpkFBLpm1ssQ@mail.gmail.com>
 <CAP7+vJ+6LE7QYw9NHVt87KSn30NZFsD+pYex+kxuVU9db+Oz2g@mail.gmail.com>
 <CAP1=2W6+4wh6EhAh5To6+L-F81-a5dk8cZSOSOHFqmYPep66Tg@mail.gmail.com>
Message-ID: <CAP7+vJKwp6TAwJJsL+gXm1isH4Pj=ta2dZRtpJhF8hFdaYWPDg@mail.gmail.com>

On Thu, Apr 17, 2014 at 1:31 PM, Brett Cannon <bcannon at gmail.com> wrote:


>
> On Thu Apr 17 2014 at 3:21:49 PM, Guido van Rossum <guido at python.org>
> wrote:
>
>> I'm sorry to keep asking dumb questions, but your description didn't job
>> my understanding of what you are comparing here. What is slower than what?
>>
>
> Startup where the stdlib is entirely in a zip file is slower than the
> status quo of reading from files.
>

That deserves more research. I'm not sure I believe we understand exactly
what goes on in each case -- perhaps our zip reading code isn't as
efficient as it could be? It would also be interesting to compare different
platforms.


> IOW it looks like speeding up startup from an import perspective requires
> either freezing modules -- for about a 10% boost -- or some fundamental
> change in import that no one has thought of yet.
>

And it's probably premature. (Unless you already have a prototype and it
shows a solid speedup.)

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140417/07b53e66/attachment.html>

From leandropls at cpti.cetuc.puc-rio.br  Thu Apr 17 21:49:13 2014
From: leandropls at cpti.cetuc.puc-rio.br (Leandro Pereira de Lima e Silva)
Date: Thu, 17 Apr 2014 16:49:13 -0300
Subject: [Python-Dev] List vs Tuple / Homogeneous vs Heterogeneous /
 Mutable vs Immutable
In-Reply-To: <CAP7+vJKMyJ8sm3a3Hce3zngcmDz0SG+V4e31K6qFP38_-2LpYg@mail.gmail.com>
References: <CAEaFV_iasbNn0_jbN6XaR8+oNq5m+9vi1TfwTZcuHp7OnbqLSg@mail.gmail.com>
 <CAP1=2W6NMJdtnVr22=gqx8bOwoY7JM7uhk78V6D4iU_WBVF=cA@mail.gmail.com>
 <CAP7+vJKMyJ8sm3a3Hce3zngcmDz0SG+V4e31K6qFP38_-2LpYg@mail.gmail.com>
Message-ID: <CAEaFV_jWbG+u6O5Zzy0+TvYc6otJ11dXPqYD7EXb0VhV+mdPFg@mail.gmail.com>

This looks like an issue to be addressed at PEP-8 since it looks like a
styling issue.

I haven't seen any other recommendations there on how to use a certain data
structure, though.

Cheers, Leandro
Em 17/04/2014 16:24, "Guido van Rossum" <guido at python.org> escreveu:

> It's definitely something that should be put in some documentation,
> probably at the point when people have learned enough to be designing their
> own programs where this issue comes up -- before they're wizards but well
> after they have learned the semantic differences between lists and tuples.
>
>
> On Thu, Apr 17, 2014 at 11:49 AM, Brett Cannon <bcannon at gmail.com> wrote:
>
>>
>>
>> On Thu Apr 17 2014 at 2:43:35 PM, Leandro Pereira de Lima e Silva <
>> leandropls at cpti.cetuc.puc-rio.br> wrote:
>>
>>> Hello there!
>>>
>>> I've stumbled upon this discussion on python-dev about what the choice
>>> between using a list or a tuple is all about in 2003:
>>> 1. https://mail.python.org/pipermail/python-dev/2003-March/033962.html
>>> 2. https://mail.python.org/pipermail/python-dev/2003-March/034029.html
>>>
>>> There's a vague comment about it on python documentation but afaik there
>>> the discussion hasn't made into any PEPs. Is there an understanding about
>>> it?
>>>
>>
>> Think of tuples like a struct in C, lists like an array. That's just out
>> of Guido's head so I don't think we have ever bothered to write it down
>> somewhere as an important distinction of the initial design that should be
>> emphasized.
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140417/c40a9d6a/attachment.html>

From martin at v.loewis.de  Thu Apr 17 23:18:52 2014
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 17 Apr 2014 23:18:52 +0200
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <CAP1=2W5Oj_mtigyQxDi-EyQNXhuNikEuQA2L+kfpkFBLpm1ssQ@mail.gmail.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <CAP1=2W5Oj_mtigyQxDi-EyQNXhuNikEuQA2L+kfpkFBLpm1ssQ@mail.gmail.com>
Message-ID: <5350453C.80102@v.loewis.de>

Am 17.04.14 20:47, schrieb Brett Cannon:
> Because people keep bringing it up, below is the results of hacking up
> the interpreter to include a sys.path entry for ./python35.zip instead
> of hard-coding to /usr/lib/python35.zip and simply zipped up Lib/
> recursively. TL;DR, zipimport performance no longer measures up
> (probably because of stat caching and such that importlib introduced).
> 
> ### normal_startup ###
> 
> Min: 0.510211 -> 2.667958: 5.23x slower
> 

Not sure how to interpret this: what is 5.23x slower than what?

Regards,
Martin



From martin at v.loewis.de  Thu Apr 17 23:21:16 2014
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 17 Apr 2014 23:21:16 +0200
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <CAP1=2W5Oj_mtigyQxDi-EyQNXhuNikEuQA2L+kfpkFBLpm1ssQ@mail.gmail.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <CAP1=2W5Oj_mtigyQxDi-EyQNXhuNikEuQA2L+kfpkFBLpm1ssQ@mail.gmail.com>
Message-ID: <535045CC.5080402@v.loewis.de>

Am 17.04.14 20:47, schrieb Brett Cannon:
> Because people keep bringing it up, below is the results of hacking up
> the interpreter to include a sys.path entry for ./python35.zip instead
> of hard-coding to /usr/lib/python35.zip and simply zipped up Lib/
> recursively. TL;DR, zipimport performance no longer measures up
> (probably because of stat caching and such that importlib introduced).

[I found the answer on what is being compared in replies]

So how did you create the zip file? Any chance that you may have
compressed the pyc files?

Regards,
Martin



From bcannon at gmail.com  Fri Apr 18 00:06:09 2014
From: bcannon at gmail.com (Brett Cannon)
Date: Thu, 17 Apr 2014 22:06:09 +0000
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <CAP1=2W5Oj_mtigyQxDi-EyQNXhuNikEuQA2L+kfpkFBLpm1ssQ@mail.gmail.com>
 <535045CC.5080402@v.loewis.de>
Message-ID: <CAP1=2W5kxv4dcHjLEjya_aSckQKM9KXkRqkaEt8YC4e-UnEZwg@mail.gmail.com>

On Thu Apr 17 2014 at 5:21:14 PM, "Martin v. L?wis" <martin at v.loewis.de>
wrote:

> Am 17.04.14 20:47, schrieb Brett Cannon:
> > Because people keep bringing it up, below is the results of hacking up
> > the interpreter to include a sys.path entry for ./python35.zip instead
> > of hard-coding to /usr/lib/python35.zip and simply zipped up Lib/
> > recursively. TL;DR, zipimport performance no longer measures up
> > (probably because of stat caching and such that importlib introduced).
>
> [I found the answer on what is being compared in replies]
>

Yeah, I did it in under 5 minutes on a whim so I wasn't entirely thinking
when I posted the numbers.


>
> So how did you create the zip file?


zip ../python35.zip -r .


> Any chance that you may have
> compressed the pyc files?
>

Yes.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140417/2eb2d6ef/attachment.html>

From tseaver at palladion.com  Fri Apr 18 00:18:27 2014
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 17 Apr 2014 18:18:27 -0400
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <CAP1=2W5kxv4dcHjLEjya_aSckQKM9KXkRqkaEt8YC4e-UnEZwg@mail.gmail.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <CAP1=2W5Oj_mtigyQxDi-EyQNXhuNikEuQA2L+kfpkFBLpm1ssQ@mail.gmail.com>
 <535045CC.5080402@v.loewis.de>
 <CAP1=2W5kxv4dcHjLEjya_aSckQKM9KXkRqkaEt8YC4e-UnEZwg@mail.gmail.com>
Message-ID: <lipjvk$s4k$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 04/17/2014 06:06 PM, Brett Cannon wrote:
> On Thu Apr 17 2014 at 5:21:14 PM, "Martin v. L?wis"
> <martin at v.loewis.de> wrote:
> 
>> Am 17.04.14 20:47, schrieb Brett Cannon:
>>> Because people keep bringing it up, below is the results of
>>> hacking up the interpreter to include a sys.path entry for
>>> ./python35.zip instead of hard-coding to /usr/lib/python35.zip and
>>> simply zipped up Lib/ recursively. TL;DR, zipimport performance no
>>> longer measures up (probably because of stat caching and such that
>>> importlib introduced).
>> 
>> [I found the answer on what is being compared in replies]
>> 
> 
> Yeah, I did it in under 5 minutes on a whim so I wasn't entirely
> thinking when I posted the numbers.
> 
> 
>> 
>> So how did you create the zip file?
> 
> 
> zip ../python35.zip -r .
> 
> 
>> Any chance that you may have compressed the pyc files?

I think you want 'zip -0' to avoid the compression.


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlNQUzMACgkQ+gerLs4ltQ53XACcCihQVdb9h4RSnOphhkzu8AjU
JsAAoJXClEcf4/McqA610Lh5SDdeHdhW
=6pNL
-----END PGP SIGNATURE-----


From ja.py at farowl.co.uk  Fri Apr 18 10:21:41 2014
From: ja.py at farowl.co.uk (Jeff Allen)
Date: Fri, 18 Apr 2014 09:21:41 +0100
Subject: [Python-Dev] List vs Tuple / Homogeneous vs Heterogeneous /
 Mutable vs Immutable
In-Reply-To: <CAEaFV_jWbG+u6O5Zzy0+TvYc6otJ11dXPqYD7EXb0VhV+mdPFg@mail.gmail.com>
References: <CAEaFV_iasbNn0_jbN6XaR8+oNq5m+9vi1TfwTZcuHp7OnbqLSg@mail.gmail.com>
 <CAP1=2W6NMJdtnVr22=gqx8bOwoY7JM7uhk78V6D4iU_WBVF=cA@mail.gmail.com>
 <CAP7+vJKMyJ8sm3a3Hce3zngcmDz0SG+V4e31K6qFP38_-2LpYg@mail.gmail.com>
 <CAEaFV_jWbG+u6O5Zzy0+TvYc6otJ11dXPqYD7EXb0VhV+mdPFg@mail.gmail.com>
Message-ID: <5350E095.3070705@farowl.co.uk>


The "think of tuples like a struct in C" explanation immediately 
reminded me that ...

On 16/04/2014 21:42, Taavi Burns wrote (in his excellent notes from the 
language summit):
>   The demographics have changed. How do
> we change the docs and ecosystem to avoid the assumption that Python
> programmers already know how to program in C?
Good question. My version was going to be that if you are dealing with 
tuples of mixed data like (name, age, shoesize), inserting something or 
sorting, in the way a list can, would confuse your code. A list, you 
almost always iterate over, to do the same thing with each member, and 
that only works if they are the same type of thing.

Then I realised David Beazley had explained this (but better), starting 
in the Tuples section of his "Python Essential Reference". With 
permission, this could perhaps be adopted wherever it best fits in the 
documentation.

Jeff Allen

On 17/04/2014 20:49, Leandro Pereira de Lima e Silva wrote:
>
> This looks like an issue to be addressed at PEP-8 since it looks like 
> a styling issue.
>
> I haven't seen any other recommendations there on how to use a certain 
> data structure, though.
>
> Cheers, Leandro
>
> Em 17/04/2014 16:24, "Guido van Rossum" <guido at python.org 
> <mailto:guido at python.org>> escreveu:
>
>     It's definitely something that should be put in some
>     documentation, probably at the point when people have learned
>     enough to be designing their own programs where this issue comes
>     up -- before they're wizards but well after they have learned the
>     semantic differences between lists and tuples.
>
>
>     On Thu, Apr 17, 2014 at 11:49 AM, Brett Cannon <bcannon at gmail.com
>     <mailto:bcannon at gmail.com>> wrote:
>
>
>
>         On Thu Apr 17 2014 at 2:43:35 PM, Leandro Pereira de Lima e
>         Silva <leandropls at cpti.cetuc.puc-rio.br
>         <mailto:leandropls at cpti.cetuc.puc-rio.br>> wrote:
>
>             Hello there!
>
>             I've stumbled upon this discussion on python-dev about
>             what the choice between using a list or a tuple is all
>             about in 2003:
>             1.
>             https://mail.python.org/pipermail/python-dev/2003-March/033962.html
>             2.
>             https://mail.python.org/pipermail/python-dev/2003-March/034029.html
>
>             There's a vague comment about it on python documentation
>             but afaik there the discussion hasn't made into any PEPs.
>             Is there an understanding about it?
>
>
>         Think of tuples like a struct in C, lists like an array.
>         That's just out of Guido's head so I don't think we have ever
>         bothered to write it down somewhere as an important
>         distinction of the initial design that should be emphasized.
>
>         _______________________________________________
>         Python-Dev mailing list
>         Python-Dev at python.org <mailto:Python-Dev at python.org>
>         https://mail.python.org/mailman/listinfo/python-dev
>         Unsubscribe:
>         https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>
>
>
>     -- 
>     --Guido van Rossum (python.org/~guido <http://python.org/%7Eguido>)
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140418/136d8fa6/attachment.html>

From kristjan at ccpgames.com  Fri Apr 18 14:04:10 2014
From: kristjan at ccpgames.com (=?utf-8?B?S3Jpc3Rqw6FuIFZhbHVyIErDs25zc29u?=)
Date: Fri, 18 Apr 2014 12:04:10 +0000
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
Message-ID: <EFE3877620384242A686D52278B7CCD3A52D8EB0@RKV-IT-EXCH104.ccp.ad.local>

Here, a week later, are some of my thoughts from the summit, for the record:

2.8:
The issue of a hyptothetical 2.8 never fails to entertain.  However, I noticed that there seem to be at least two distinct missions of such a thing.

1.       An aid in the conversion from 2.x series to 3.x series.  Enabling a bunch of warnings and such by default.  Perhaps allowing 3.x syntax in some places without fuss.  The problem with this idea is that it is pointless.  Why would anyone want to upgrade from 2.7 to 2.8 if all they get is some new warnings for 3.x?  If people are willing to make a version upgrade just to get new warnings (i.e. no immediate feature benefit) they might as well go directly to 3.x and be done with it.

2.       Feature enhancement to 2.8.  Take a robust and popular version of python and add some of the language goodies that have been added to 3.x and that don?t have an inherent 3.x aspect.  Yield from.  New exception model.  Stdlib enhancements such as futures.   The argument goes like this:  We have a very popular platform out there with lots of momentum.  People want incremental enhancements to it.  Why not give them what they want?  Bread and games and all that?  A Rockband cannot stay cooped up in a studio producing experimental concept albums all the time.  That is death.  Sometimes it needs to go on tour and play old hits for the fans!
3.5 features
When asked what should we aim for in 3.5, there were mostly some very minor incremental changes suggested, IIRC.  In my opinion, the reason 3.x has not caught on is that there is no real carrot there.  There is no new vision, no killer feature.  Nothing that a programmer sees and makes him say ?Yeah! I want to program my next project using this feature, it will be super!?.
In my opinion we should be thinking more boldly.  Either for 3.x or for a version 4.  We should be taking the language to a new level.  Thinking about evolving the language.  New paradigms.   Look at what C# is doing, with each language revision.  Look at Go.  I?m no CS but here are some ideas on stuff we could visit:

1.       Code blocks as a core language construct.  Re-implement context managers as block executors.  We shouldn?t let details such as syntax questions distract us.   That?s like saying that we can?t  eat spaghetti because our Italian is so poor.  Proper code blocks would open up new avenues for exploration of expressability and paradigms.

2.       Concurrency primitives built into the language.  Again, see C# with its ?async? keyword  (a feature we?ve experimented with in stacklesslib, see e.g. stacklesslib.async in https://bitbucket.org/stackless-dev/stacklesslib ).  Look at Go with its channels and more importantly, the select feature.  ( see goless, http://goless.readthedocs.org/en/latest/index.html a 2014 sprint project).  Don?t get distracted by the GIL.  Concurrency is as much about orchestration of operations as it is about parallel execution of code.  Let?s learn from the success of stackless, gevent, go, and build on top of it by absorbing tried and tested research from more than 30 years of CS.
These are the immediate ideas rolling off the top of my head.  Notice how I don?t mention ?removing the GIL? here since that is not a ?language feature? as such, not something inspiring new thinking and invention.  Of course a non-GIL implementation is also desirable, even if it would involve completely rethinking the C API.  For a version 4 of python.  But I think we thinking beyond that, even.

Let?s keep on truckin?!

K


From: Python-Dev [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Guido van Rossum
Sent: 10. apr?l 2014 01:08
To: Python-Dev
Subject: [Python-Dev] Language Summit notes

To anyone who took notes at the language summit at PyCon today, even if you took them just for yourself, would you mind posting them here? It would be good to have some kind of (informal!) as soon as possible, before we collectively forget. You won't be held responsible for correctness.
)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140418/db0473f9/attachment.html>

From ncoghlan at gmail.com  Fri Apr 18 17:58:59 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 18 Apr 2014 11:58:59 -0400
Subject: [Python-Dev] Software integrators vs end users (was Re: Language
 Summit notes)
Message-ID: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>

On 16 Apr 2014 21:03, "R. David Murray" <rdmurray at bitdance.com> wrote:
> There is an 'Installing Python on Windows' link further down the google
> results that links to a fairly good page from python-guide.org, whose
> first link lets you download the 2.7.6 msi.  I guess that's the 32
> bit version.  But it then tells you go to python.org to make sure you
> get the latest release, and tells you to click on a link that doesn't
> exist any more (the "windows installer" link).
>
> So, yeah.
>
> Usability.

As part of thrashing out the respective distribution ecosystem roles
of pip and conda (still a work in progress), we're at least converging
on the notion that there are actually now *two* main ways of consuming
Python: as a "software integrator" (the way most of us have
traditionally consumed it, and the way that dominates most project
documentation outside the scientific Python community) and as an "end
user" (the way Linux system administrators have long consumed it, and
the way scientists, financial analysts and folks just learning Python
are likely best off consuming it).

Making these different personas explicit is a process that has barely
begun (this email is mostly based on some conversations I had in
person at PyCon and via email during the sprints), but here's the gist
(based on listing examples):

Software integrators:

* Linux distributions and other operating system vendors
* Sumo redistributions (commercial or otherwise)
* "Python based environments" (PTVS, Enthought Canopy, wakari.io,
Python Anywhere, etc)
* Software-as-a-Service developers
* Device manufacturers
* PC OEMs
* creators of corporate "Standard Operating Environment" definitions
* System integrators (IBM, Boeing et al)
* Application developers (from simple CLI tools to OpenStack)

End users:
* system administrators
* scientists (whether in academia or corporations)
* financial analysts
* engineers
* data miners
* graphic artists
* animation designers
* <<< Folks trying out Python for the first time >>>

For end users, Python is likely consumed as *part of something else*.
For Linux admins, it's a way of scripting the system, for graphic
artists and animators, it's likely embedded as part of a tool like
Blender or Maya, for scientists, financial analysts, engineers and
data miners, it likely makes sense to use it as part of a larger
visualisation environment like IPython Notebook, Python (x, y),
Enthought Canopy, or a hosted system like Wakari or Python Anywhere.

Folks just learning Python are also in the latter list, and we
currently ask them (on the python.org download pages) to jump straight
into the "software integrator" role to get their environment set up,
rather than setting out to impress them by recommending one of the
pre-integrated sumo distributions that offers quick and easy access to
powerful visualisation and data analysis tools. While providing access
to the CPython default interactive prompt in the browser is cool, it's
less impressive as the default experience we provide after someone has
downloaded and installed the interpreter. Instead, we likely want to
*really* impress them by making it easy for them to explore the full
power of things like IPython Notebooks. My own current preferred
approach for that is the fully open source "Anaconda" distribution
from Continuum Analytics, specifically because it *is* fully open
source, and you can "pip install conda" to bootstrap their package
manager in other contexts. And, importantly, because conda
environments can *manage the Python runtime itself*, it is able to
work more consistently across different platforms than the upstream
tools by reliably isolating itself from the system Python on POSIX
platforms. It should even by possible to get conda to manage alternate
Python implementations like Stackless, PyPy, Jython, IronPython, etc
(although I don't believe anyone is using it that way as yet).

However, conda *isn't* optimised for the software integrator use case
- the additional work it does to improve cross platform consistency
actually *gets in the way*, when you're trying to integrate Python
closely with a larger system (like a Linux distribution), and "a large
set of automatically provided libraries" isn't actually a feature in
that context. While conda does offer some options (like miniconda) to
make that kind of integration task easier, I strongly believe that
realm is better handled by consuming CPython and the standard library,
along with pip and related tools, directly, and accepting the
additional platform specific details that come along with that.

Making this "software integrator" and "end user" split deliberately
and consciously, and pushing the former toward consuming things in a
more DIY fashion, and the latter towards a pre-assembled sumo
distribution should help greatly in allowing us to better balance the
starkly different needs of the two groups, and provide an extremely
high quality "out of the box" experience for new users, while still
allowing software integrators to dive in and customise things to suit
their own needs.

>From a Python 2->3 migration perspective, blessing at least one sumo
distribution also provides a way to make it easy to gain ready access
to backports from the Python 3 standard library (for example, Anaconda
currently includes at least the backports.ssl_match_hostname module).

Regards,
Nick.

From status at bugs.python.org  Fri Apr 18 18:07:47 2014
From: status at bugs.python.org (Python tracker)
Date: Fri, 18 Apr 2014 18:07:47 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20140418160747.79A6A56A25@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2014-04-11 - 2014-04-18)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    4579 (+12)
  closed 28493 (+88)
  total  33072 (+100)

Open issues with patches: 2118 


Issues opened (68)
==================

#17160: test_urllib2net fails
http://bugs.python.org/issue17160  reopened by ddvento at ucar.edu

#17861: put opcode information in one place
http://bugs.python.org/issue17861  reopened by twouters

#21201: Uninformative error message in multiprocessing.Manager()
http://bugs.python.org/issue21201  opened by wojtekwalczak

#21202: Naming a file` io.py` causes cryptic error message
http://bugs.python.org/issue21202  opened by madison.may

#21204: multiprocessing example does not work on Windows
http://bugs.python.org/issue21204  opened by jmaki

#21205: Unable to make decorated generator object to inherit generator
http://bugs.python.org/issue21205  opened by SzieberthAdam

#21207: urandom persistent fd - not re-openned after fd close
http://bugs.python.org/issue21207  opened by kwirk

#21211: pkgutil.find_loader() raises ImportError instead of returning 
http://bugs.python.org/issue21211  opened by eric.snow

#21213: Memory bomb by incorrect custom serializer to json.dumps
http://bugs.python.org/issue21213  opened by saaj

#21216: getaddrinfo is wrongly considered thread safe on linux
http://bugs.python.org/issue21216  opened by Julien.Palard

#21217: inspect.getsourcelines finds wrong lines when lambda used argu
http://bugs.python.org/issue21217  opened by ballingt

#21220: Enhance obmalloc allocation strategy
http://bugs.python.org/issue21220  opened by kristjan.jonsson

#21221: Minor struct_time documentation bug
http://bugs.python.org/issue21221  opened by lurchman

#21224: BaseHTTPRequestHandler, update the protocol version to http 1.
http://bugs.python.org/issue21224  opened by matrixise

#21225: io.py: Improve docstrings for classes
http://bugs.python.org/issue21225  opened by akuchling

#21226: PyImport_ExecCodeModuleObject not setting module attributes
http://bugs.python.org/issue21226  opened by trevor3

#21227: Decimal class error messages for integer division aren't good
http://bugs.python.org/issue21227  opened by leewz

#21228: Missing enumeration of HTTPResponse Objects methods of urllib.
http://bugs.python.org/issue21228  opened by EvensF

#21230: imghdr does not accept adobe photoshop mime type
http://bugs.python.org/issue21230  opened by faiz

#21231: Issue a python 3 warning when old style classes are defined.
http://bugs.python.org/issue21231  opened by alex

#21232: Use of '1' instead of 'True' as 'splitlines' argument in diffl
http://bugs.python.org/issue21232  opened by jayanthkoushik

#21233: Add *Calloc functions to CPython memory allocation API
http://bugs.python.org/issue21233  opened by njs

#21235: importlib's spec module create algorithm is not exposed
http://bugs.python.org/issue21235  opened by aronacher

#21236: patch to use cabinet.lib instead of fci.lib (fixes build with 
http://bugs.python.org/issue21236  opened by mst

#21237: Update Python 2/3 porting HOWTO's suggestion for dealing with 
http://bugs.python.org/issue21237  opened by brett.cannon

#21240: Add an abstactmethod directive to the Python ReST domain
http://bugs.python.org/issue21240  opened by eric.snow

#21242: Generalize configure check for working Python executable
http://bugs.python.org/issue21242  opened by brett.cannon

#21243: Auto-generate exceptions.c from a Python file
http://bugs.python.org/issue21243  opened by brett.cannon

#21247: test_asyncio: test_subprocess_send_signal hangs on Fedora buil
http://bugs.python.org/issue21247  opened by opoplawski

#21249: removing pythonXY.zip from sys.path results in additional test
http://bugs.python.org/issue21249  opened by doko

#21250: sqlite3 doesn't have unit tests for 'insert or [algorithm]' fu
http://bugs.python.org/issue21250  opened by Alex.Lord

#21251: Standard library trace module crashes with exception
http://bugs.python.org/issue21251  opened by mkolman

#21252: Lib/asyncio/events.py has tons of docstrings which are just "X
http://bugs.python.org/issue21252  opened by alex

#21253: Difflib.compare() crashes when sequences contain little or no 
http://bugs.python.org/issue21253  opened by nnja

#21254: PropertyMock refuses to raise AttributeErrror as a side effect
http://bugs.python.org/issue21254  opened by michael.foord

#21255: Attaching a PropertyMock records calls
http://bugs.python.org/issue21255  opened by michael.foord

#21256: Sort keyword arguments in mock _format_call_signature
http://bugs.python.org/issue21256  opened by michael.foord

#21257: Document parse_headers function of http.client
http://bugs.python.org/issue21257  opened by orsenthil

#21258: Add __iter__ support for mock_open
http://bugs.python.org/issue21258  opened by michael.foord

#21259: replace "except: pass" by "except Exception: pass"
http://bugs.python.org/issue21259  opened by matrixise

#21261: Teach IDLE to Autocomplete dictionary keys
http://bugs.python.org/issue21261  opened by rhettinger

#21263: test_gdb failures on os x 10.9.2
http://bugs.python.org/issue21263  opened by sam.kimbrel

#21264: test_compileall fails to build in the installed location
http://bugs.python.org/issue21264  opened by doko

#21265: ConfigParser allows "get(*, raw=True), but no corresponding "s
http://bugs.python.org/issue21265  opened by metagriffin

#21266: test_zipfile fails to run in the installed location
http://bugs.python.org/issue21266  opened by doko

#21269: Provide args and kwargs attributes on mock call objects
http://bugs.python.org/issue21269  opened by michael.foord

#21270: unittest.mock.call object has inherited count method
http://bugs.python.org/issue21270  opened by michael.foord

#21271: reset_mock needs parameters to also reset return_value and sid
http://bugs.python.org/issue21271  opened by michael.foord

#21272: use _sysconfigdata to itinialize distutils.sysconfig
http://bugs.python.org/issue21272  opened by doko

#21273: don't defined socket constants which are not implemented for G
http://bugs.python.org/issue21273  opened by doko

#21277: don't try to link _ctypes with a ffi_convenience library
http://bugs.python.org/issue21277  opened by doko

#21278: Running the test suite with -v makes the test_ctypes and the t
http://bugs.python.org/issue21278  opened by ddvento at ucar.edu

#21279: str.translate documentation incomplete
http://bugs.python.org/issue21279  opened by bgailer

#21280: shutil.make_archive() with default "root_dir" parameter raises
http://bugs.python.org/issue21280  opened by vadmium

#21281: DEBUGGING: Simultaneous stopping of all threads on breakpoint 
http://bugs.python.org/issue21281  opened by azyr

#21282: setup.py: More informative error msg for modules which built b
http://bugs.python.org/issue21282  opened by Lukas.Vacek

#21284: IDLE reformat tests fail in presence of non-default FormatPara
http://bugs.python.org/issue21284  opened by rhettinger

#21287: Better support for AF_PACKET on opensolaris (illumos)
http://bugs.python.org/issue21287  opened by igor.pashev

#21288: hashlib.pbkdf2_hmac Hash Constructor
http://bugs.python.org/issue21288  opened by aronacher

#21289: make.bat not building documentation
http://bugs.python.org/issue21289  opened by dsawyer

#21291: subprocess Popen objects are not thread safe w.r.t. wait() and
http://bugs.python.org/issue21291  opened by gregory.p.smith

#21292: C API in debug fails
http://bugs.python.org/issue21292  opened by Banger

#21293: Remove "capsule hack" from object.c?
http://bugs.python.org/issue21293  opened by larry

#21295: Python 3.4 gives wrong col_offset for Call nodes returned from
http://bugs.python.org/issue21295  opened by Aivar.Annamaa

#21296: smtplib Sends Commands in Lower-Case
http://bugs.python.org/issue21296  opened by luiji

#21297: skipinitialspace in the csv module only skips spaces, not "whi
http://bugs.python.org/issue21297  opened by Daniel.Andersson

#21298: Cheese shop registration stopped working for me recently
http://bugs.python.org/issue21298  opened by tlevine

#21300: Docs (incorrectly) suggest email.policy.default is the default
http://bugs.python.org/issue21300  opened by valhallasw



Most recent 15 issues with no replies (15)
==========================================

#21300: Docs (incorrectly) suggest email.policy.default is the default
http://bugs.python.org/issue21300

#21298: Cheese shop registration stopped working for me recently
http://bugs.python.org/issue21298

#21297: skipinitialspace in the csv module only skips spaces, not "whi
http://bugs.python.org/issue21297

#21296: smtplib Sends Commands in Lower-Case
http://bugs.python.org/issue21296

#21293: Remove "capsule hack" from object.c?
http://bugs.python.org/issue21293

#21287: Better support for AF_PACKET on opensolaris (illumos)
http://bugs.python.org/issue21287

#21284: IDLE reformat tests fail in presence of non-default FormatPara
http://bugs.python.org/issue21284

#21282: setup.py: More informative error msg for modules which built b
http://bugs.python.org/issue21282

#21281: DEBUGGING: Simultaneous stopping of all threads on breakpoint 
http://bugs.python.org/issue21281

#21280: shutil.make_archive() with default "root_dir" parameter raises
http://bugs.python.org/issue21280

#21278: Running the test suite with -v makes the test_ctypes and the t
http://bugs.python.org/issue21278

#21271: reset_mock needs parameters to also reset return_value and sid
http://bugs.python.org/issue21271

#21270: unittest.mock.call object has inherited count method
http://bugs.python.org/issue21270

#21269: Provide args and kwargs attributes on mock call objects
http://bugs.python.org/issue21269

#21266: test_zipfile fails to run in the installed location
http://bugs.python.org/issue21266



Most recent 15 issues waiting for review (15)
=============================================

#21300: Docs (incorrectly) suggest email.policy.default is the default
http://bugs.python.org/issue21300

#21293: Remove "capsule hack" from object.c?
http://bugs.python.org/issue21293

#21291: subprocess Popen objects are not thread safe w.r.t. wait() and
http://bugs.python.org/issue21291

#21289: make.bat not building documentation
http://bugs.python.org/issue21289

#21287: Better support for AF_PACKET on opensolaris (illumos)
http://bugs.python.org/issue21287

#21279: str.translate documentation incomplete
http://bugs.python.org/issue21279

#21273: don't defined socket constants which are not implemented for G
http://bugs.python.org/issue21273

#21272: use _sysconfigdata to itinialize distutils.sysconfig
http://bugs.python.org/issue21272

#21265: ConfigParser allows "get(*, raw=True), but no corresponding "s
http://bugs.python.org/issue21265

#21259: replace "except: pass" by "except Exception: pass"
http://bugs.python.org/issue21259

#21258: Add __iter__ support for mock_open
http://bugs.python.org/issue21258

#21256: Sort keyword arguments in mock _format_call_signature
http://bugs.python.org/issue21256

#21249: removing pythonXY.zip from sys.path results in additional test
http://bugs.python.org/issue21249

#21240: Add an abstactmethod directive to the Python ReST domain
http://bugs.python.org/issue21240

#21233: Add *Calloc functions to CPython memory allocation API
http://bugs.python.org/issue21233



Top 10 most discussed issues (10)
=================================

#12916: Add inspect.splitdoc
http://bugs.python.org/issue12916  21 msgs

#21233: Add *Calloc functions to CPython memory allocation API
http://bugs.python.org/issue21233  20 msgs

#20438: inspect: Deprecate getfullargspec?
http://bugs.python.org/issue20438  14 msgs

#21199: Python on 64-bit Windows uses signed 32-bit type for read leng
http://bugs.python.org/issue21199  14 msgs

#21259: replace "except: pass" by "except Exception: pass"
http://bugs.python.org/issue21259  14 msgs

#21227: Decimal class error messages for integer division aren't good
http://bugs.python.org/issue21227  11 msgs

#20434: Fix error handler of _PyString_Resize() on allocation failure
http://bugs.python.org/issue20434  10 msgs

#20578: BufferedIOBase.readinto1 is missing
http://bugs.python.org/issue20578  10 msgs

#17861: put opcode information in one place
http://bugs.python.org/issue17861   9 msgs

#21220: Enhance obmalloc allocation strategy
http://bugs.python.org/issue21220   9 msgs



Issues closed (81)
==================

#5420: Queue deprecation warning patch
http://bugs.python.org/issue5420  closed by rhettinger

#6490: os.popen documentation is probably wrong
http://bugs.python.org/issue6490  closed by akuchling

#6727: ImportError when package is symlinked on Windows
http://bugs.python.org/issue6727  closed by r.david.murray

#8060: PEP 3101 string formatting missing engineering presentation ty
http://bugs.python.org/issue8060  closed by eric.smith

#8931: '#' has no effect with 'c' type
http://bugs.python.org/issue8931  closed by eric.smith

#10224: Build 3.x documentation using python3.x
http://bugs.python.org/issue10224  closed by georg.brandl

#10481: subprocess PIPEs are byte streams
http://bugs.python.org/issue10481  closed by akuchling

#10983: Errors in http.client.HTTPConnection class (python3)
http://bugs.python.org/issue10983  closed by orsenthil

#11316: RFC822 header parsing API inconsistencies between httplib.HTTP
http://bugs.python.org/issue11316  closed by r.david.murray

#12546: builtin __format__ methods cannot fill with \x00 char
http://bugs.python.org/issue12546  closed by eric.smith

#13244: WebSocket schemes in urllib.parse
http://bugs.python.org/issue13244  closed by orsenthil

#14216: ImportError: No module named binascii
http://bugs.python.org/issue14216  closed by akuchling

#15234: avoid runtime library path for extensions found in system dire
http://bugs.python.org/issue15234  closed by doko

#15370: test_runpy should include namespace package tests
http://bugs.python.org/issue15370  closed by akuchling

#15840: Ambiguity with regard to the effect of accessing a closed IOBa
http://bugs.python.org/issue15840  closed by akuchling

#15916: change doctest DocTestSuite not to raise ValueError if no docs
http://bugs.python.org/issue15916  closed by r.david.murray

#16236: Doc/Makefile should have $PYTHON=python2
http://bugs.python.org/issue16236  closed by berker.peksag

#17009: "Thread Programming With Python" should be removed
http://bugs.python.org/issue17009  closed by r.david.murray

#17498: error responses from server are masked in smtplib when server 
http://bugs.python.org/issue17498  closed by r.david.murray

#17660: mock.patch could whitelist builtins to not need create=True
http://bugs.python.org/issue17660  closed by python-dev

#17826: Setting a side_effect on mock from create_autospec doesn't wor
http://bugs.python.org/issue17826  closed by michael.foord

#18321: Multivolume support in tarfile module
http://bugs.python.org/issue18321  closed by lars.gustaebel

#18566: In unittest.TestCase docs for setUp() and tearDown() don't men
http://bugs.python.org/issue18566  closed by terry.reedy

#18628: Better index entry for encoding declarations
http://bugs.python.org/issue18628  closed by r.david.murray

#18650: intermittent test_pydoc failure on 3.4.0a1
http://bugs.python.org/issue18650  closed by ned.deily

#20103: Documentation of itertools.accumulate is confused
http://bugs.python.org/issue20103  closed by akuchling

#20307: Android's failure to expose SYS_* system call constants causes
http://bugs.python.org/issue20307  closed by gregory.p.smith

#20480: Add ipaddress property to get name of reverse DNS PTR record
http://bugs.python.org/issue20480  closed by eric.smith

#20624: Clarify recommendation to inherit from Exception
http://bugs.python.org/issue20624  closed by mark.dickinson

#20636: Better repr for tkinter widgets
http://bugs.python.org/issue20636  closed by serhiy.storchaka

#20701: warning in compileall.rst
http://bugs.python.org/issue20701  closed by Arfrever

#20874: Tutorial section on starting python is out of date
http://bugs.python.org/issue20874  closed by r.david.murray

#20904: HAVE_PY_SET_53BIT_PRECISION for m68k
http://bugs.python.org/issue20904  closed by python-dev

#20956: tokenize module claims tokenize.tokenize returns namedtuple, b
http://bugs.python.org/issue20956  closed by python-dev

#20968: mock.MagicMock does not mock __truediv__
http://bugs.python.org/issue20968  closed by michael.foord

#21028: ElementTree objects should support all the same methods as Ele
http://bugs.python.org/issue21028  closed by rhettinger

#21164: print unicode in Windows console
http://bugs.python.org/issue21164  closed by terry.reedy

#21169: getpass.getpass() fails with non-ASCII characters in prompt
http://bugs.python.org/issue21169  closed by serhiy.storchaka

#21170: Incorrect signature for unittest.TestResult.startTestRun(), .s
http://bugs.python.org/issue21170  closed by terry.reedy

#21171: Outdated usage str.encode('rot-13') in rot13 codec
http://bugs.python.org/issue21171  closed by benjamin.peterson

#21177: ValueError: byte must be in range(0, 256)
http://bugs.python.org/issue21177  closed by pitrou

#21188: Broken link
http://bugs.python.org/issue21188  closed by loewis

#21193: pow(a, b, c) should not raise TypeError when b is negative and
http://bugs.python.org/issue21193  closed by mark.dickinson

#21194: json.dumps with ensure_ascii=False doesn't escape control char
http://bugs.python.org/issue21194  closed by ned.deily

#21197: venv does not create lib64 directory and appropriate symlinks
http://bugs.python.org/issue21197  closed by python-dev

#21203: logging configurators ignoring documented options
http://bugs.python.org/issue21203  closed by python-dev

#21206: Provide legit HTTPS certificate for http://bugs.python.org/, r
http://bugs.python.org/issue21206  closed by ned.deily

#21208: Change default behavior of arguments with type bool when optio
http://bugs.python.org/issue21208  closed by r.david.murray

#21209: q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1
http://bugs.python.org/issue21209  closed by python-dev

#21210: Warnings in Doc/library/json.rst
http://bugs.python.org/issue21210  closed by python-dev

#21212: Documentation of octal representation
http://bugs.python.org/issue21212  closed by python-dev

#21214: PEP8 doesn't verifies last line.
http://bugs.python.org/issue21214  closed by mark.dickinson

#21215: build-deps instructions for Ubuntu
http://bugs.python.org/issue21215  closed by r.david.murray

#21218: Test failure for test_ssl.test_default_ecdh_curve on OS X
http://bugs.python.org/issue21218  closed by ned.deily

#21219: WARNING: Inline literal start-string without end-string.
http://bugs.python.org/issue21219  closed by orsenthil

#21222: Mock create_autospec with name argument fails
http://bugs.python.org/issue21222  closed by python-dev

#21223: fix test_site/test_startup_imports when some of the extensions
http://bugs.python.org/issue21223  closed by doko

#21229: Path used for HTTP PUT request doesn't match the description
http://bugs.python.org/issue21229  closed by python-dev

#21234: __contains__ and friends should check "is" for all elements fi
http://bugs.python.org/issue21234  closed by rhettinger

#21238: unittest.mock.Mock should not allow you to use non-existent as
http://bugs.python.org/issue21238  closed by python-dev

#21239: unittest.mock.patch.stopall intermittently doesn't work when t
http://bugs.python.org/issue21239  closed by python-dev

#21241: Variable name with number causes interactive console to crash
http://bugs.python.org/issue21241  closed by ned.deily

#21244: distutils fails to build C extensions with XCode 5.1 and OS X 
http://bugs.python.org/issue21244  closed by ned.deily

#21245: Logging Logger.exception documentation
http://bugs.python.org/issue21245  closed by python-dev

#21246: test_ssl handshake failure
http://bugs.python.org/issue21246  closed by pitrou

#21248: BROWSER env var described inaccurately in webbrowser docs
http://bugs.python.org/issue21248  closed by python-dev

#21260: python malloc mach_vm_map failed
http://bugs.python.org/issue21260  closed by ned.deily

#21262: assert_not_called method for mocks
http://bugs.python.org/issue21262  closed by python-dev

#21267: mktime_tz may return wrong result for past dates before Python
http://bugs.python.org/issue21267  closed by akira

#21268: Update pydoc module docstring
http://bugs.python.org/issue21268  closed by eric.araujo

#21274: define PATH_MAX for GNU/Hurd in Python/pythonrun.c
http://bugs.python.org/issue21274  closed by doko

#21275: fix a socket test on KFreeBSD
http://bugs.python.org/issue21275  closed by doko

#21276: don't define USE_XATTRS on kfreebsd and the Hurd
http://bugs.python.org/issue21276  closed by doko

#21283: A escape character is used when a REGEXP is an argument of "st
http://bugs.python.org/issue21283  closed by eric.smith

#21285: refactor and fix curses configure checks
http://bugs.python.org/issue21285  closed by doko

#21286: Refcounting information missing in docs for Python 3.4 and abo
http://bugs.python.org/issue21286  closed by python-dev

#21290: imaplib.error when importing email package
http://bugs.python.org/issue21290  closed by r.david.murray

#21294: len wrong help
http://bugs.python.org/issue21294  closed by python-dev

#21299: Spam
http://bugs.python.org/issue21299  closed by berker.peksag

#984870: curses: getmaxyx() breaks when the window shrinks
http://bugs.python.org/issue984870  closed by akuchling

#1521196: smtplib login fails with aol smtp server
http://bugs.python.org/issue1521196  closed by r.david.murray

From guido at python.org  Fri Apr 18 18:31:04 2014
From: guido at python.org (Guido van Rossum)
Date: Fri, 18 Apr 2014 09:31:04 -0700
Subject: [Python-Dev] Software integrators vs end users (was Re:
 Language Summit notes)
In-Reply-To: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>
References: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>
Message-ID: <CAP7+vJ+5EcEdofo6d4dL6mhePRovBDkJ1+yJotbXYHOB_-2BEA@mail.gmail.com>

Could I summarize that as "software integrators build from source, while
end users use an installer"? And the rest of the discussion is about which
installer (in the widest sense of the word) to recommend, where the choices
include Linux vendor distros, sumo Python distros, Mac/Win installers, as
well as fine-grained tools like apt-get (Linux vendor specific) and pip and
conda (Python specific)? And perhaps conda is pushing the boundaries
because it also tries to manage things that aren't strictly Python, but
useful for scientists?


On Fri, Apr 18, 2014 at 8:58 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 16 Apr 2014 21:03, "R. David Murray" <rdmurray at bitdance.com> wrote:
> > There is an 'Installing Python on Windows' link further down the google
> > results that links to a fairly good page from python-guide.org, whose
> > first link lets you download the 2.7.6 msi.  I guess that's the 32
> > bit version.  But it then tells you go to python.org to make sure you
> > get the latest release, and tells you to click on a link that doesn't
> > exist any more (the "windows installer" link).
> >
> > So, yeah.
> >
> > Usability.
>
> As part of thrashing out the respective distribution ecosystem roles
> of pip and conda (still a work in progress), we're at least converging
> on the notion that there are actually now *two* main ways of consuming
> Python: as a "software integrator" (the way most of us have
> traditionally consumed it, and the way that dominates most project
> documentation outside the scientific Python community) and as an "end
> user" (the way Linux system administrators have long consumed it, and
> the way scientists, financial analysts and folks just learning Python
> are likely best off consuming it).
>
> Making these different personas explicit is a process that has barely
> begun (this email is mostly based on some conversations I had in
> person at PyCon and via email during the sprints), but here's the gist
> (based on listing examples):
>
> Software integrators:
>
> * Linux distributions and other operating system vendors
> * Sumo redistributions (commercial or otherwise)
> * "Python based environments" (PTVS, Enthought Canopy, wakari.io,
> Python Anywhere, etc)
> * Software-as-a-Service developers
> * Device manufacturers
> * PC OEMs
> * creators of corporate "Standard Operating Environment" definitions
> * System integrators (IBM, Boeing et al)
> * Application developers (from simple CLI tools to OpenStack)
>
> End users:
> * system administrators
> * scientists (whether in academia or corporations)
> * financial analysts
> * engineers
> * data miners
> * graphic artists
> * animation designers
> * <<< Folks trying out Python for the first time >>>
>
> For end users, Python is likely consumed as *part of something else*.
> For Linux admins, it's a way of scripting the system, for graphic
> artists and animators, it's likely embedded as part of a tool like
> Blender or Maya, for scientists, financial analysts, engineers and
> data miners, it likely makes sense to use it as part of a larger
> visualisation environment like IPython Notebook, Python (x, y),
> Enthought Canopy, or a hosted system like Wakari or Python Anywhere.
>
> Folks just learning Python are also in the latter list, and we
> currently ask them (on the python.org download pages) to jump straight
> into the "software integrator" role to get their environment set up,
> rather than setting out to impress them by recommending one of the
> pre-integrated sumo distributions that offers quick and easy access to
> powerful visualisation and data analysis tools. While providing access
> to the CPython default interactive prompt in the browser is cool, it's
> less impressive as the default experience we provide after someone has
> downloaded and installed the interpreter. Instead, we likely want to
> *really* impress them by making it easy for them to explore the full
> power of things like IPython Notebooks. My own current preferred
> approach for that is the fully open source "Anaconda" distribution
> from Continuum Analytics, specifically because it *is* fully open
> source, and you can "pip install conda" to bootstrap their package
> manager in other contexts. And, importantly, because conda
> environments can *manage the Python runtime itself*, it is able to
> work more consistently across different platforms than the upstream
> tools by reliably isolating itself from the system Python on POSIX
> platforms. It should even by possible to get conda to manage alternate
> Python implementations like Stackless, PyPy, Jython, IronPython, etc
> (although I don't believe anyone is using it that way as yet).
>
> However, conda *isn't* optimised for the software integrator use case
> - the additional work it does to improve cross platform consistency
> actually *gets in the way*, when you're trying to integrate Python
> closely with a larger system (like a Linux distribution), and "a large
> set of automatically provided libraries" isn't actually a feature in
> that context. While conda does offer some options (like miniconda) to
> make that kind of integration task easier, I strongly believe that
> realm is better handled by consuming CPython and the standard library,
> along with pip and related tools, directly, and accepting the
> additional platform specific details that come along with that.
>
> Making this "software integrator" and "end user" split deliberately
> and consciously, and pushing the former toward consuming things in a
> more DIY fashion, and the latter towards a pre-assembled sumo
> distribution should help greatly in allowing us to better balance the
> starkly different needs of the two groups, and provide an extremely
> high quality "out of the box" experience for new users, while still
> allowing software integrators to dive in and customise things to suit
> their own needs.
>
> From a Python 2->3 migration perspective, blessing at least one sumo
> distribution also provides a way to make it easy to gain ready access
> to backports from the Python 3 standard library (for example, Anaconda
> currently includes at least the backports.ssl_match_hostname module).
>
> Regards,
> Nick.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140418/30fbf209/attachment.html>

From ncoghlan at gmail.com  Fri Apr 18 18:48:10 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 18 Apr 2014 12:48:10 -0400
Subject: [Python-Dev] List vs Tuple / Homogeneous vs Heterogeneous /
 Mutable vs Immutable
In-Reply-To: <5350E095.3070705@farowl.co.uk>
References: <CAEaFV_iasbNn0_jbN6XaR8+oNq5m+9vi1TfwTZcuHp7OnbqLSg@mail.gmail.com>
 <CAP1=2W6NMJdtnVr22=gqx8bOwoY7JM7uhk78V6D4iU_WBVF=cA@mail.gmail.com>
 <CAP7+vJKMyJ8sm3a3Hce3zngcmDz0SG+V4e31K6qFP38_-2LpYg@mail.gmail.com>
 <CAEaFV_jWbG+u6O5Zzy0+TvYc6otJ11dXPqYD7EXb0VhV+mdPFg@mail.gmail.com>
 <5350E095.3070705@farowl.co.uk>
Message-ID: <CADiSq7fAbqVZ78a+UJjOMh9cPKDG+BS9G71f3rxR=y3+=H_cYg@mail.gmail.com>

On 18 April 2014 04:21, Jeff Allen <ja.py at farowl.co.uk> wrote:
>
> The "think of tuples like a struct in C" explanation immediately reminded me
> that ...
>
> On 16/04/2014 21:42, Taavi Burns wrote (in his excellent notes from the
> language summit):
>
>  The demographics have changed. How do
> we change the docs and ecosystem to avoid the assumption that Python
> programmers already know how to program in C?
>
> Good question. My version was going to be that if you are dealing with
> tuples of mixed data like (name, age, shoesize), inserting something or
> sorting, in the way a list can, would confuse your code. A list, you almost
> always iterate over, to do the same thing with each member, and that only
> works if they are the same type of thing.
>
> Then I realised David Beazley had explained this (but better), starting in
> the Tuples section of his "Python Essential Reference". With permission,
> this could perhaps be adopted wherever it best fits in the documentation.

This is the kind of "competence to mastery" transition my "Python
User's Reference" was designed to cover, but the book contract fell
through, and I was never able to interest anyone in rescuing the
material from the ODF manuscript and migrating it to Sphinx after I
donated it to the PSF:
http://svn.python.org/view/sandbox/trunk/userref/

(It's also all 2.5 era docs, so would need a fair bit of work to bring
it up to date for 2.7 and 3.4)

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From p.f.moore at gmail.com  Fri Apr 18 18:55:02 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 18 Apr 2014 17:55:02 +0100
Subject: [Python-Dev] Software integrators vs end users (was Re:
 Language Summit notes)
In-Reply-To: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>
References: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>
Message-ID: <CACac1F9k_kDN0UspOUs8BUxkbr=tZnLTorkFf2fJGJUfBR9P+g@mail.gmail.com>

On 18 April 2014 16:58, Nick Coghlan <ncoghlan at gmail.com> wrote:
> As part of thrashing out the respective distribution ecosystem roles
> of pip and conda (still a work in progress), we're at least converging
> on the notion that there are actually now *two* main ways of consuming
> Python: as a "software integrator" (the way most of us have
> traditionally consumed it, and the way that dominates most project
> documentation outside the scientific Python community) and as an "end
> user" (the way Linux system administrators have long consumed it, and
> the way scientists, financial analysts and folks just learning Python
> are likely best off consuming it).
>
> Making these different personas explicit is a process that has barely
> begun (this email is mostly based on some conversations I had in
> person at PyCon and via email during the sprints), but here's the gist
> (based on listing examples):

Interesting perspective. However, I'm not convinced it's complete.
Specifically, there's one group of people who I encounter relatively
often, who don't seem to me to fit well into either category you're
proposing. That is, (Windows in my experience, but maybe Linux as
well) users who want to write "simple scripts" and for whom batch
files or similar are not sufficient. Such people typically don't have
the sort of "single application area" focus that your "end user"
category seems to imply, but on the other hand they don't really fit
the "software integrators" role in the sense of necessarily being
comfortable setting up their own development environment.

I worry that your classification risks ignoring that group (maybe
because Unix users are well served with other alternatives than Python
for this type of task, or because on Unix "use the system Python" is
the right answer).

Your list of "end user" targeted distributions seem to be limited to:

  - Linux distribution vendors
  - Vendors focused on the essentially scientific community (in the
broadest sense)
  - Embedded Python

That's very far from being complete coverage of all the people *I'd*
like to be able to recommend Python to. Specifically, unless we're not
interested in "generic" Windows users, I think we need to offer *some*
form of equivalent of the OS-packaged Python on Linux for Windows
users. That's what the python.org builds, plus pip, wheels and PyPI,
give for Windows users now. Hmm, if we assume that supporting that
remains a priority, is what you're really saying that we *don't* try
to extend that to work for Linux/OSX, as doing so competes with the OS
vendors - but rather we see python.org binaries and binary
infrastructure like wheels as being focused on the Windows user
experience?

(I wish I'd been at PyCon, this would have been a very interesting
discussion to have face to face. Email isn't ideal for this...)

Paul.

From dholth at gmail.com  Fri Apr 18 19:14:33 2014
From: dholth at gmail.com (Daniel Holth)
Date: Fri, 18 Apr 2014 13:14:33 -0400
Subject: [Python-Dev] Software integrators vs end users (was Re:
 Language Summit notes)
In-Reply-To: <CACac1F9k_kDN0UspOUs8BUxkbr=tZnLTorkFf2fJGJUfBR9P+g@mail.gmail.com>
References: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>
 <CACac1F9k_kDN0UspOUs8BUxkbr=tZnLTorkFf2fJGJUfBR9P+g@mail.gmail.com>
Message-ID: <CAG8k2+6qOEG70vzA99vTBtxkDSAh2V=HSuFO4c53DsyFxCmRFA@mail.gmail.com>

On Fri, Apr 18, 2014 at 12:55 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> On 18 April 2014 16:58, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> As part of thrashing out the respective distribution ecosystem roles
>> of pip and conda (still a work in progress), we're at least converging
>> on the notion that there are actually now *two* main ways of consuming
>> Python: as a "software integrator" (the way most of us have
>> traditionally consumed it, and the way that dominates most project
>> documentation outside the scientific Python community) and as an "end
>> user" (the way Linux system administrators have long consumed it, and
>> the way scientists, financial analysts and folks just learning Python
>> are likely best off consuming it).
>>
>> Making these different personas explicit is a process that has barely
>> begun (this email is mostly based on some conversations I had in
>> person at PyCon and via email during the sprints), but here's the gist
>> (based on listing examples):
>
> Interesting perspective. However, I'm not convinced it's complete.
> Specifically, there's one group of people who I encounter relatively
> often, who don't seem to me to fit well into either category you're
> proposing. That is, (Windows in my experience, but maybe Linux as
> well) users who want to write "simple scripts" and for whom batch
> files or similar are not sufficient. Such people typically don't have
> the sort of "single application area" focus that your "end user"
> category seems to imply, but on the other hand they don't really fit
> the "software integrators" role in the sense of necessarily being
> comfortable setting up their own development environment.
>
> I worry that your classification risks ignoring that group (maybe
> because Unix users are well served with other alternatives than Python
> for this type of task, or because on Unix "use the system Python" is
> the right answer).
>
> Your list of "end user" targeted distributions seem to be limited to:
>
>   - Linux distribution vendors
>   - Vendors focused on the essentially scientific community (in the
> broadest sense)
>   - Embedded Python
>
> That's very far from being complete coverage of all the people *I'd*
> like to be able to recommend Python to. Specifically, unless we're not
> interested in "generic" Windows users, I think we need to offer *some*
> form of equivalent of the OS-packaged Python on Linux for Windows
> users. That's what the python.org builds, plus pip, wheels and PyPI,
> give for Windows users now. Hmm, if we assume that supporting that
> remains a priority, is what you're really saying that we *don't* try
> to extend that to work for Linux/OSX, as doing so competes with the OS
> vendors - but rather we see python.org binaries and binary
> infrastructure like wheels as being focused on the Windows user
> experience?
>
> (I wish I'd been at PyCon, this would have been a very interesting
> discussion to have face to face. Email isn't ideal for this...)

One more group that I find interesting is application users. These
people should not need to notice that Python is in use at all, in
contrast to the "build a virtualenv / install / pass through fire and
death / use" workflow that is sometimes promoted. They are well served
by good tools that make single-file zip distributions or py2exe etc.
from a collection of wheels or sdists. Application users are using
Python because a best-in-class program is written in Python and not
because a Python program integrates better with other Python
libraries.


Conda is interesting because it is a system package manager, except it
installs everything into what they sometimes describe as "C-level
virtualenvs" rather than /. They've come up with a design that works
well with Python programs but isn't particularly Python-specific at
all.

From ncoghlan at gmail.com  Fri Apr 18 20:22:31 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 18 Apr 2014 14:22:31 -0400
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <EFE3877620384242A686D52278B7CCD3A52D8EB0@RKV-IT-EXCH104.ccp.ad.local>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
 <EFE3877620384242A686D52278B7CCD3A52D8EB0@RKV-IT-EXCH104.ccp.ad.local>
Message-ID: <CADiSq7cnPt4Z1gYybXfezHiDqyPZF7yaBZdYmoiofKiWsw32JQ@mail.gmail.com>

On 18 April 2014 08:04, Kristj?n Valur J?nsson <kristjan at ccpgames.com> wrote:
> Here, a week later, are some of my thoughts from the summit, for the record:
> 1.       An aid in the conversion from 2.x series to 3.x series.  Enabling a
> bunch of warnings and such by default.  Perhaps allowing 3.x syntax in some
> places without fuss.  The problem with this idea is that it is pointless.
> Why would anyone want to upgrade from 2.7 to 2.8 if all they get is some new
> warnings for 3.x?  If people are willing to make a version upgrade just to
> get new warnings (i.e. no immediate feature benefit) they might as well go
> directly to 3.x and be done with it.

Right, this is much better handled through tools like linters, import
hooks, custom interpreters, additional support modules, etc. I think
http://python-future.org/ represents the current state of the art, but
there are still a lot of opportunities for improvements here.

> 2.       Feature enhancement to 2.8.  Take a robust and popular version of
> python and add some of the language goodies that have been added to 3.x and
> that don?t have an inherent 3.x aspect.  Yield from.  New exception model.
> Stdlib enhancements such as futures.   The argument goes like this:  We have
> a very popular platform out there with lots of momentum.  People want
> incremental enhancements to it.  Why not give them what they want?  Bread
> and games and all that?  A Rockband cannot stay cooped up in a studio
> producing experimental concept albums all the time.  That is death.
> Sometimes it needs to go on tour and play old hits for the fans!

Do you know how much work a new Python 2.x release creates for people?
All the redistributors have to update, books get outdated, a new
wrinkle gets added to the compatibility matrix for everyone. A new
Python 2.x release is simple untenable at this point in the transition
- it's a *massively* expensive way to achieve things that can be
achieved more cheaply in other ways.

Take yield from, for example. Hy is able to compile *LISP* syntax to
Python AST structures. PEP 380 includes a semantic expansion of yield
from in terms of yield. Is it *really* impossible to get "yield from"
based code running in Python 2.6? Or have people just assumed it's not
possible and never even tried, because the idea of using import hooks
to backport syntax to earlier feature releases is too novel?

(importlib includes tools to make this kind of thing relatively
straightforward, and there's nothing inherently impossible about using
importlib to write import hooks in Python 2 - it's just a matter of
having to do the backport first, and Eric Snow is now seriously
considering doing exactly that. Hit us up on import-sig if you think
that might be interesting to you)

> 3.5 features
>
> When asked what should we aim for in 3.5, there were mostly some very minor
> incremental changes suggested, IIRC.  In my opinion, the reason 3.x has not
> caught on is that there is no real carrot there.  There is no new vision, no
> killer feature.  Nothing that a programmer sees and makes him say ?Yeah! I
> want to program my next project using this feature, it will be super!?.

I *really* wish folks from North America, Europe and other regions
where 8-bit encodings can handle their native language and where
Anglicisation of terms to fit them into the ASCII identifier
restriction poses no barrier to communication would stop trotting out
this "no killer feature in Python 3" canard. While it is *possible* to
write internationalised and localised applications in it, Python 2's
Unicode support is so broken that some people can't even run the
interpreter from their home directory because it can't cope with their
username. Python 3 makes Unicode support in the interpreter pervasive
(just as it is in the JVM and CLR), and we've spent the last 5+ years
dealing with the latent Unicode handling defects this uncovered in the
standard library, fixing mistakes we made in the initial transition,
and figuring out how to get the new system to play nice with the POSIX
"text is just bytes in a particular encoding" model, as well as
figuring out what we really lost in the transition when it comes to
binary data manipulation.

If anyone is *ever* tempted to utter the words "Python 3 has no killer
feature" without immediately following it up with the "for me"
qualifier, please go read this post about the creation of a Portuguese
version of Stack Overflow:
http://blog.stackoverflow.com/2014/02/cant-we-all-be-reasonable-and-speak-english/

Late last year, Alex Gaynor made the insightful observation that we
write software either for the users we have or for the users we want.
Python 3 is the software we're writing for the users we want. Python
2.7 long term maintenance, PyPy, standard library backports,
transition libraries and tools, packaging ecosystem improvements,
Python 3 features designed primarily to easy migration from Python 2,
PyPI modules that run on both Python 2 & 3 - that's the software we
and many other people are writing for the users we have.

Sure, there's not yet a killer Python 3 feature for many developers
that *were already happy with Python 2.7* (although matrix
multiplication syntax in 3.5 will likely reach that bar for a subset
of users, and chained exceptions are already pretty amazing for
debugging obscure failures that then run into broken error handlers).
That's fine - that's why Python 2.7 is now being supported upstream
out to 2020. It's also why lots of things are already available as
backports on PyPI (if any of us as core devs identify a Python 3
library that *we* think is a killer feature, we'll often maintain a
backport for it, since many of us are still maintaining Python 2
systems in our day jobs), and we're going to work on ensuring that
those are easier to access for new users. Two of the best projects
people can actually work on that front are actually
https://github.com/python/pythondotorg and
http://packaging.python.org/, to help enable them to become effective
gateways to the entire Python *ecosystem*, rather than having people
downloading CPython and assuming the reference interpreter and the
standard library are all they have available to them.

> In my opinion we should be thinking more boldly.  Either for 3.x or for a
> version 4.  We should be taking the language to a new level.  Thinking about
> evolving the language.  New paradigms.   Look at what C# is doing, with each
> language revision.  Look at Go.  I?m no CS but here are some ideas on stuff
> we could visit:
>
> 1.       Code blocks as a core language construct.  Re-implement context
> managers as block executors.  We shouldn?t let details such as syntax
> questions distract us.   That?s like saying that we can?t  eat spaghetti
> because our Italian is so poor.

When it comes to blocks, the syntax isn't a distraction - it's the
fundamental framing of what the construct is and how it should be
used. How does a block get used to define a sorting key function? How
does it get broadcast over a NumPy array? How does it get distributed
to multiple threads, processes or greenlets, or passed to an IO
scheduler? How does it differ from an ordinary closure? How does it
relate to loops and other constructs?

There is a large existing constellation of concepts that blocks need
to be fitted into, and any successful proposal for adding blocks to
the language will require giving that process the respect and
attention it deserves.

> Proper code blocks would open up new
> avenues for exploration of expressability and paradigms.

This won't happen without a full usability study, but I've been
speaking to Fernando Perez and Greg Wilson about some vague concepts
for doing exactly that. I hope to be able to explore that possibility
further at SciPy in July - Greg laments the lack of "evidence based
language design" and I think the strong scientific presence in the
Python community provides us with a genuine opportunity to move in
that direction when it comes to controversial changes like code
blocks.

> 2.       Concurrency primitives built into the language.  Again, see C# with
> its ?async? keyword  (a feature we?ve experimented with in stacklesslib, see
> e.g. stacklesslib.async in https://bitbucket.org/stackless-dev/stacklesslib
> ).  Look at Go with its channels and more importantly, the select feature.
> ( see goless, http://goless.readthedocs.org/en/latest/index.html a 2014
> sprint project).  Don?t get distracted by the GIL.  Concurrency is as much
> about orchestration of operations as it is about parallel execution of code.
> Let?s learn from the success of stackless, gevent, go, and build on top of
> it by absorbing tried and tested research from more than 30 years of CS.

I'm personally letting Armin Rigo spearhead that effort (via his STM
work), as I believe PyPy provides a better foundation for that
exploration.

It also doesn't make sense to me to do this at a language level until
we resolve the blocks question, since that opens up many new
possibilities for parallelisation that won't even need new syntax at
that point.

> These are the immediate ideas rolling off the top of my head.  Notice how I
> don?t mention ?removing the GIL? here since that is not a ?language feature?
> as such, not something inspiring new thinking and invention.  Of course a
> non-GIL implementation is also desirable, even if it would involve
> completely rethinking the C API.  For a version 4 of python.  But I think we
> thinking beyond that, even.

If people are genuinely interested in building and supporting the
Python *community*, stop thinking about tinkering with the *language*.
Tinkering with the language is *fun* (that's why I still do it), but
the most interesting innovations in the Python world with the largest
near term impact *aren't going to happen in the reference
interpreter*. Our cycle times are too long, especially once commercial
redistributors are taken into account, and the ongoing popularity of
Python 2.7 shows that the language itself reached "good enough" status
for most use cases years ago. As technologists we like to tinker with
out toys, but it's important to remember that *Excel spreadsheets* are
far more dominant as a programming paradigm than any open source
programming language.

So folks looking to core development as the engine room of Python's
growth are *looking in the wrong place*. We laid the foundation (and
will continue to sustain and evolve it), but the real growth drivers
have now moved further out to things like the work the PyPA is doing
to make upstream packaging easier to consume, the work Travis Oliphant
and others are doing to make the scientific Python stack easier to
consume, the work Fernando Perez is coordinating around IPython and
IPython notebook, the work Titus Brown et al are doing around
reproducable science, the work Greg Wilson and Mozilla Science Labs
are doing around Software Carpentry, the work Van Lindberg and others
are putting into opening up the PSF, the work Jesse Noller has been
driving around opening up python.org to more contributors, the work
Jessica McKellar and more are doing around getting Python into high
school curricula in the US, the work the Raspberry Pi Foundation are
doing in the UK, the work around the core mentorship program, GSoC and
the Gnome Outreach Program for Women, the work to build the regional
network of Python conferences (I'm told we're currently running at a
rate of something like 2 new PyCons per week!), the work folks like
me, Guido, Steve Dower, Jesse and more are doing to better engage with
corporate users and inviting them to more directly support the
sustainability of the upstream Python community (on *our* terms), the
work PyLadies is doing around education and outreach, the work Steve
Holden is now doing around improving the ready availability of Python
training courses, and on, and on, and on.

We are *far* past the point where any specific language feature will
be the defining aspect of language adoption - it's all about the
ecosystem, and has been for years. Discoverability and usability and
building relationships with external groups - that is how you make the
leap across the chasm from early adopters to mass market popularity.
Not tinkering with syntactic details - do the latter for fun and to
help people years down the track, but do other things (like helping
with packaging or the core development workflow tools or
communications projects like Mailman 3 and the HyperKitty archiver) if
the long game doesn't excite you and you would like to make a more
immediate impact on the world.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From solipsis at pitrou.net  Fri Apr 18 20:29:23 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 18 Apr 2014 20:29:23 +0200
Subject: [Python-Dev] devguide: Add note about Kushal Das' privs
References: <3g72kV00hjz7LjZ@mail.python.org>
Message-ID: <20140418202923.46f20653@fsol>

On Mon, 14 Apr 2014 23:18:42 +0200 (CEST)
brett.cannon <python-checkins at python.org> wrote:
> http://hg.python.org/devguide/rev/c14c8a195fec
> changeset:   686:c14c8a195fec
> user:        Brett Cannon <brett at python.org>
> date:        Mon Apr 14 17:18:37 2014 -0400
> summary:
>   Add note about Kushal Das' privs

I have no objection to Kushal getting commit rights, but is there a
public record of the discussion leading to this decision? I can't find
anything on either python-dev or python-committers (perhaps my search
skills are failing me).

Also, welcome on board, Kushal!

Regards

Antoine.



From benjamin at python.org  Fri Apr 18 20:34:57 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 18 Apr 2014 11:34:57 -0700
Subject: [Python-Dev] devguide: Add note about Kushal Das' privs
In-Reply-To: <20140418202923.46f20653@fsol>
References: <3g72kV00hjz7LjZ@mail.python.org> <20140418202923.46f20653@fsol>
Message-ID: <1397846097.5285.107984529.11878B72@webmail.messagingengine.com>

On Fri, Apr 18, 2014, at 11:29, Antoine Pitrou wrote:
> On Mon, 14 Apr 2014 23:18:42 +0200 (CEST)
> brett.cannon <python-checkins at python.org> wrote:
> > http://hg.python.org/devguide/rev/c14c8a195fec
> > changeset:   686:c14c8a195fec
> > user:        Brett Cannon <brett at python.org>
> > date:        Mon Apr 14 17:18:37 2014 -0400
> > summary:
> >   Add note about Kushal Das' privs
> 
> I have no objection to Kushal getting commit rights, but is there a
> public record of the discussion leading to this decision? I can't find
> anything on either python-dev or python-committers (perhaps my search
> skills are failing me).

Probably in person conversion at PyCon sprints.

From solipsis at pitrou.net  Fri Apr 18 20:31:23 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 18 Apr 2014 20:31:23 +0200
Subject: [Python-Dev] Language Summit notes
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
 <EFE3877620384242A686D52278B7CCD3A52D8EB0@RKV-IT-EXCH104.ccp.ad.local>
Message-ID: <20140418203123.2197bef9@fsol>

On Fri, 18 Apr 2014 12:04:10 +0000
Kristj?n Valur J?nsson <kristjan at ccpgames.com> wrote:
> 
> 2.       Feature enhancement to 2.8.  Take a robust and popular version of python and add some of the language goodies that have been added to 3.x and that don?t have an inherent 3.x aspect.  Yield from.  New exception model.  Stdlib enhancements such as futures.   The argument goes like this:  We have a very popular platform out there with lots of momentum.  People want incremental enhancements to it.  Why not give them what they want?  Bread and games and all that?  A Rockband cannot stay cooped up in a studio producing experimental concept albums all the time.  That is death.  Sometimes it needs to go on tour and play old hits for the fans!

I don't think we have recent download numbers since the Website
overhaul (do we?), but Python 3 isn't an "experimental concept
language" anymore (it hasn't been since 3.3 or 3.2, I'd say).

Regards

Antoine.



From bcannon at gmail.com  Fri Apr 18 20:54:41 2014
From: bcannon at gmail.com (Brett Cannon)
Date: Fri, 18 Apr 2014 14:54:41 -0400
Subject: [Python-Dev] devguide: Add note about Kushal Das' privs
In-Reply-To: <1397846097.5285.107984529.11878B72@webmail.messagingengine.com>
References: <3g72kV00hjz7LjZ@mail.python.org> <20140418202923.46f20653@fsol>
 <1397846097.5285.107984529.11878B72@webmail.messagingengine.com>
Message-ID: <CAP1=2W4J8fFndzkjAsQzzr59kPF5eu9rL0qGLCBK4CUsaE7xUw@mail.gmail.com>

On Friday, April 18, 2014 2:35:32 PM, Benjamin Peterson <benjamin at python.org>
wrote:
>
> On Fri, Apr 18, 2014, at 11:29, Antoine Pitrou wrote:
> > On Mon, 14 Apr 2014 23:18:42 +0200 (CEST)
> > brett.cannon <python-checkins at python.org> wrote:
> > > http://hg.python.org/devguide/rev/c14c8a195fec
> > > changeset:   686:c14c8a195fec
> > > user:        Brett Cannon <brett at python.org>
> > > date:        Mon Apr 14 17:18:37 2014 -0400
> > > summary:
> > >   Add note about Kushal Das' privs
> >
> > I have no objection to Kushal getting commit rights, but is there a
> > public record of the discussion leading to this decision? I can't find
> > anything on either python-dev or python-committers (perhaps my search
> > skills are failing me).
>
> Probably in person conversion at PyCon sprints

What Benjamin said; discussions at PyCon amongst several of us.

-Brett

>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/brett%40python.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140418/182ed20e/attachment.html>

From zachary.ware+pydev at gmail.com  Fri Apr 18 21:10:34 2014
From: zachary.ware+pydev at gmail.com (Zachary Ware)
Date: Fri, 18 Apr 2014 14:10:34 -0500
Subject: [Python-Dev] devguide: Add note about Kushal Das' privs
In-Reply-To: <CAP1=2W4J8fFndzkjAsQzzr59kPF5eu9rL0qGLCBK4CUsaE7xUw@mail.gmail.com>
References: <3g72kV00hjz7LjZ@mail.python.org> <20140418202923.46f20653@fsol>
 <1397846097.5285.107984529.11878B72@webmail.messagingengine.com>
 <CAP1=2W4J8fFndzkjAsQzzr59kPF5eu9rL0qGLCBK4CUsaE7xUw@mail.gmail.com>
Message-ID: <CAKJDb-M2MOXwzX-uyGZ0V8A8hniXat0hp3p5fvVVJVzLAaQQ9Q@mail.gmail.com>

On Fri, Apr 18, 2014 at 1:54 PM, Brett Cannon <bcannon at gmail.com> wrote:
> On Friday, April 18, 2014 2:35:32 PM, Benjamin Peterson
> <benjamin at python.org> wrote:
>> On Fri, Apr 18, 2014, at 11:29, Antoine Pitrou wrote:
>> > On Mon, 14 Apr 2014 23:18:42 +0200 (CEST)
>> > brett.cannon <python-checkins at python.org> wrote:
>> > > http://hg.python.org/devguide/rev/c14c8a195fec
>> > > changeset:   686:c14c8a195fec
>> > > user:        Brett Cannon <brett at python.org>
>> > > date:        Mon Apr 14 17:18:37 2014 -0400
>> > > summary:
>> > >   Add note about Kushal Das' privs
>> >
>> > I have no objection to Kushal getting commit rights, but is there a
>> > public record of the discussion leading to this decision? I can't find
>> > anything on either python-dev or python-committers (perhaps my search
>> > skills are failing me).
>>
>> Probably in person conversion at PyCon sprints
>
> What Benjamin said; discussions at PyCon amongst several of us.

Looking at that page, it doesn't look like I was ever added.  Brett,
was it you that enabled me?

Also, welcome Kushal!

-- 
Zach

From donald at stufft.io  Fri Apr 18 21:18:11 2014
From: donald at stufft.io (Donald Stufft)
Date: Fri, 18 Apr 2014 15:18:11 -0400
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <20140418203123.2197bef9@fsol>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
 <EFE3877620384242A686D52278B7CCD3A52D8EB0@RKV-IT-EXCH104.ccp.ad.local>
 <20140418203123.2197bef9@fsol>
Message-ID: <643EAF53-A851-4C08-A81A-6C1BDC90EE28@stufft.io>


On Apr 18, 2014, at 2:31 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Fri, 18 Apr 2014 12:04:10 +0000
> Kristj?n Valur J?nsson <kristjan at ccpgames.com> wrote:
>> 
>> 2.       Feature enhancement to 2.8.  Take a robust and popular version of python and add some of the language goodies that have been added to 3.x and that don?t have an inherent 3.x aspect.  Yield from.  New exception model.  Stdlib enhancements such as futures.   The argument goes like this:  We have a very popular platform out there with lots of momentum.  People want incremental enhancements to it.  Why not give them what they want?  Bread and games and all that?  A Rockband cannot stay cooped up in a studio producing experimental concept albums all the time.  That is death.  Sometimes it needs to go on tour and play old hits for the fans!
> 
> I don't think we have recent download numbers since the Website
> overhaul (do we?), but Python 3 isn't an "experimental concept
> language" anymore (it hasn't been since 3.3 or 3.2, I'd say).
> 
> Regards
> 
> Antoine.
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/donald%40stufft.io

Fastly logs are stored in Dreamhost so we could make those numbers if they aren?t already available.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140418/1de4bc0f/attachment.sig>

From ncoghlan at gmail.com  Fri Apr 18 21:18:36 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 18 Apr 2014 15:18:36 -0400
Subject: [Python-Dev] Software integrators vs end users (was Re:
 Language Summit notes)
In-Reply-To: <CACac1F9k_kDN0UspOUs8BUxkbr=tZnLTorkFf2fJGJUfBR9P+g@mail.gmail.com>
References: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>
 <CACac1F9k_kDN0UspOUs8BUxkbr=tZnLTorkFf2fJGJUfBR9P+g@mail.gmail.com>
Message-ID: <CADiSq7d1FWdM2g658dPzt+JptEpDc6B_DFnsVVxHPcDCDUECuw@mail.gmail.com>

On 18 April 2014 12:55, Paul Moore <p.f.moore at gmail.com> wrote:
> On 18 April 2014 16:58, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> As part of thrashing out the respective distribution ecosystem roles
>> of pip and conda (still a work in progress), we're at least converging
>> on the notion that there are actually now *two* main ways of consuming
>> Python: as a "software integrator" (the way most of us have
>> traditionally consumed it, and the way that dominates most project
>> documentation outside the scientific Python community) and as an "end
>> user" (the way Linux system administrators have long consumed it, and
>> the way scientists, financial analysts and folks just learning Python
>> are likely best off consuming it).
>>
>> Making these different personas explicit is a process that has barely
>> begun (this email is mostly based on some conversations I had in
>> person at PyCon and via email during the sprints), but here's the gist
>> (based on listing examples):
>
> Interesting perspective. However, I'm not convinced it's complete.

Personas are never complete - however, they represent sufficiently
useful archetypes that help guide design decisions. Where alarm bells
need to start ringing is when two personas are far enough apart in
interests and skill levels that it is highly unlikely that one tool
will be able to satisfy both groups of users.

I think that's the situation we find ourselves in with respect to
folks that are already professional programmers and folks that just
need to do some programming as part of another activity.

> Specifically, there's one group of people who I encounter relatively
> often, who don't seem to me to fit well into either category you're
> proposing. That is, (Windows in my experience, but maybe Linux as
> well) users who want to write "simple scripts" and for whom batch
> files or similar are not sufficient. Such people typically don't have
> the sort of "single application area" focus that your "end user"
> category seems to imply, but on the other hand they don't really fit
> the "software integrators" role in the sense of necessarily being
> comfortable setting up their own development environment.

The "single application area" in that case is "Windows scripting", and
it *would* make sense to have a sumo distribution that provided
PyWin32 and other components. That distribution may include other
things as well, though.

> I worry that your classification risks ignoring that group (maybe
> because Unix users are well served with other alternatives than Python
> for this type of task, or because on Unix "use the system Python" is
> the right answer).

I think the conda "end user" approach handles that case pretty well
(while it certainly started with the scientific Python stack, conda
certainly isn't limited to that). So the entry can be "system
administrators" in general, rather than being specific to Linux.

The "lack of integration" issues on Linux and Mac OS X show up because
there *is* a system Python for conda to be separate from. We don't
have that problem on Windows (although integrating with the Python
launcher may be an issue).

> Your list of "end user" targeted distributions seem to be limited to:
>
>   - Linux distribution vendors

As noted above, this category is broader than just Linux - it really
covers any system scripting context.

>   - Vendors focused on the essentially scientific community (in the
> broadest sense)

I'd say "analytical", rather than specifically "scientific". From a
user experience perspective though, the important point is that many
of the folks being targeted *aren't* programmers in the traditional
sense - they just need to do some programming as part of their actual
job. It turns out that's also a useful usability proxy for people just
starting out with programming :)

>   - Embedded Python
>
> That's very far from being complete coverage of all the people *I'd*
> like to be able to recommend Python to. Specifically, unless we're not
> interested in "generic" Windows users,

I think conda covers that "generic Windows user" case as well - that's
why I emphasised the fully open source nature of it. We've also been
having some interesting conversations with Travis about potentially
improving the level of collaboration between the PyPA and conda devs.

> I think we need to offer *some*
> form of equivalent of the OS-packaged Python on Linux for Windows
> users. That's what the python.org builds, plus pip, wheels and PyPI,
> give for Windows users now. Hmm, if we assume that supporting that
> remains a priority, is what you're really saying that we *don't* try
> to extend that to work for Linux/OSX, as doing so competes with the OS
> vendors - but rather we see python.org binaries and binary
> infrastructure like wheels as being focused on the Windows user
> experience?

I'd still like to improve the wheel story for Linux (there are various
benefits of that which accrue to system integrators as well), but I
think the key benefit of promoting both pip *and* conda is that it
lets us hit *two* different points on the flexibility vs usability
curve, rather than being limited to one.

pip needs to preserve a lot of flexibility, because it's already the
upstream for a *lot* of different use cases. However, that flexibility
often comes at a cost in usability, especially for new users.

Since conda *doesn't* play that upstream role the way pip does, the
flexibility can be dialled down a bit in favour of increased "out of
the box" usability, making it easier to offer a compelling new user
experience.

Ideally we'd like to make the two tools share a common back end to
reduce duplicated effort (and distlib may end up filling that role),
but I think such a split makes a lot of sense from a front end
perspective.

> (I wish I'd been at PyCon, this would have been a very interesting
> discussion to have face to face. Email isn't ideal for this...)

We didn't even get far with it at PyCon - there's certainly no
consensus around these characterisations at this point. The discussion
was mostly focused on pip vs conda, since the requirements for the two
tools are very, very similar. My contention is that it *doesn't* make
sense to try to make conda a replacement for pip (or vice-versa)
because the two tools are focused on different *audiences*, even
though an awful lot of the backend engineering requirements are the
same.

At this point, however, I'm mainly looking for consensus that there
*are* two different problems to be solved here, and solving them both
well in a single tool is likely to be nigh on impossible. (I'm aware
we're really on the wrong list for that discussion, but I also think
there's value in getting some broader python-dev awareness of this
particular topic)

Regards,
Nick.

>
> Paul.



-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From donald at stufft.io  Fri Apr 18 21:39:13 2014
From: donald at stufft.io (Donald Stufft)
Date: Fri, 18 Apr 2014 15:39:13 -0400
Subject: [Python-Dev] Software integrators vs end users (was Re:
	Language Summit notes)
In-Reply-To: <CADiSq7d1FWdM2g658dPzt+JptEpDc6B_DFnsVVxHPcDCDUECuw@mail.gmail.com>
References: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>
 <CACac1F9k_kDN0UspOUs8BUxkbr=tZnLTorkFf2fJGJUfBR9P+g@mail.gmail.com>
 <CADiSq7d1FWdM2g658dPzt+JptEpDc6B_DFnsVVxHPcDCDUECuw@mail.gmail.com>
Message-ID: <0334A003-E9FF-4C62-B3AF-2E21488955B8@stufft.io>


On Apr 18, 2014, at 3:18 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> At this point, however, I'm mainly looking for consensus that there
> *are* two different problems to be solved here, and solving them both
> well in a single tool is likely to be nigh on impossible. (I'm aware
> we're really on the wrong list for that discussion, but I also think
> there's value in getting some broader python-dev awareness of this
> particular topic)

I?m not sure about this? I mean yes those are two different areas, but I?m
not sure about the split between Conda and pip here. As far as I can tell
Conda is useful in the same way apt-get or yum is useful, you get a non
Python specific set of packages (because sometimes things aren?t pure
python) and you also remove a little bit of thinking about versions (although
honestly I think it?s possible to remove most of that for consumers of
packages).

To be quite frank, a lot of the benefit of Conda outside of the ?I need something
that isn?t strictly Python) is in the fact they can bootstrap compiled packages
whereas pip/wheel/PyPI combination we need to convince authors to upload
their own binary packages (or at least develop something to make it easier
like a build farm).

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140418/725cb14c/attachment.sig>

From ezio.melotti at gmail.com  Fri Apr 18 21:59:41 2014
From: ezio.melotti at gmail.com (Ezio Melotti)
Date: Fri, 18 Apr 2014 22:59:41 +0300
Subject: [Python-Dev] List vs Tuple / Homogeneous vs Heterogeneous /
 Mutable vs Immutable
In-Reply-To: <CAP7+vJKMyJ8sm3a3Hce3zngcmDz0SG+V4e31K6qFP38_-2LpYg@mail.gmail.com>
References: <CAEaFV_iasbNn0_jbN6XaR8+oNq5m+9vi1TfwTZcuHp7OnbqLSg@mail.gmail.com>
 <CAP1=2W6NMJdtnVr22=gqx8bOwoY7JM7uhk78V6D4iU_WBVF=cA@mail.gmail.com>
 <CAP7+vJKMyJ8sm3a3Hce3zngcmDz0SG+V4e31K6qFP38_-2LpYg@mail.gmail.com>
Message-ID: <CACBhJdGPBPOaScoLAgWuGvfg24c-6eEzpjFQCSOLo05yiHAuNg@mail.gmail.com>

Hi,

On Thu, Apr 17, 2014 at 10:23 PM, Guido van Rossum <guido at python.org> wrote:
> It's definitely something that should be put in some documentation,

see http://bugs.python.org/issue14840 and
https://docs.python.org/3/tutorial/datastructures.html#tuples-and-sequences
:
"""
Though tuples may seem similar to lists, they are often used in
different situations and for different purposes. Tuples are immutable,
and usually contain an heterogeneous sequence of elements that are
accessed via unpacking (see later in this section) or indexing (or
even by attribute in the case of namedtuples). Lists are mutable, and
their elements are usually homogeneous and are accessed by iterating
over the list.
"""

Best Regards,
Ezio Melotti

> probably
> at the point when people have learned enough to be designing their own
> programs where this issue comes up -- before they're wizards but well after
> they have learned the semantic differences between lists and tuples.
>
>
> On Thu, Apr 17, 2014 at 11:49 AM, Brett Cannon <bcannon at gmail.com> wrote:
>>
>>
>>
>> On Thu Apr 17 2014 at 2:43:35 PM, Leandro Pereira de Lima e Silva
>> <leandropls at cpti.cetuc.puc-rio.br> wrote:
>>>
>>> Hello there!
>>>
>>> I've stumbled upon this discussion on python-dev about what the choice
>>> between using a list or a tuple is all about in 2003:
>>> 1. https://mail.python.org/pipermail/python-dev/2003-March/033962.html
>>> 2. https://mail.python.org/pipermail/python-dev/2003-March/034029.html
>>>
>>> There's a vague comment about it on python documentation but afaik there
>>> the discussion hasn't made into any PEPs. Is there an understanding about
>>> it?
>>
>>
>> Think of tuples like a struct in C, lists like an array. That's just out
>> of Guido's head so I don't think we have ever bothered to write it down
>> somewhere as an important distinction of the initial design that should be
>> emphasized.
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/ezio.melotti%40gmail.com
>

From ncoghlan at gmail.com  Fri Apr 18 22:22:49 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 18 Apr 2014 16:22:49 -0400
Subject: [Python-Dev] Software integrators vs end users (was Re:
 Language Summit notes)
In-Reply-To: <0334A003-E9FF-4C62-B3AF-2E21488955B8@stufft.io>
References: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>
 <CACac1F9k_kDN0UspOUs8BUxkbr=tZnLTorkFf2fJGJUfBR9P+g@mail.gmail.com>
 <CADiSq7d1FWdM2g658dPzt+JptEpDc6B_DFnsVVxHPcDCDUECuw@mail.gmail.com>
 <0334A003-E9FF-4C62-B3AF-2E21488955B8@stufft.io>
Message-ID: <CADiSq7funpGBCOzuye41sJUxBZWA9ZtmqWv=M=Z-+NSNhCsP9w@mail.gmail.com>

On 18 April 2014 15:39, Donald Stufft <donald at stufft.io> wrote:
>
> On Apr 18, 2014, at 3:18 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
>> At this point, however, I'm mainly looking for consensus that there
>> *are* two different problems to be solved here, and solving them both
>> well in a single tool is likely to be nigh on impossible. (I'm aware
>> we're really on the wrong list for that discussion, but I also think
>> there's value in getting some broader python-dev awareness of this
>> particular topic)
>
> I?m not sure about this? I mean yes those are two different areas, but I?m
> not sure about the split between Conda and pip here. As far as I can tell
> Conda is useful in the same way apt-get or yum is useful, you get a non
> Python specific set of packages (because sometimes things aren?t pure
> python) and you also remove a little bit of thinking about versions (although
> honestly I think it?s possible to remove most of that for consumers of
> packages).

You also get the ability to use that same system to update *Python
itself*, regardless of platform. Being notified of and consuming
CPython updates on Windows, as well as consuming alternate versions on
Linux distros, is currently not a well solved problem - with conda,
it's no more complex than updating a PyPI package.

That's one of the most attractive aspects for me - making Python 3.4,
pypy, etc as easy to update with consistent cross platform
instructions as Python packages are.

> To be quite frank, a lot of the benefit of Conda outside of the ?I need something
> that isn?t strictly Python) is in the fact they can bootstrap compiled packages
> whereas pip/wheel/PyPI combination we need to convince authors to upload
> their own binary packages (or at least develop something to make it easier
> like a build farm).

Yep, that's a large part of why I think "divide and conquer" is the
way to go here. While it isn't completely accurate (as most SaaS
developers don't want to build C extensions from source) I think
Guido's "build from source" vs "install a pre-built binary"
distinction is still a reasonably good way to characterise it. For a
distro like Fedora (or, even more so, RHEL), we're not going to trust
a binary created by someone else if we can possibly avoid it, so
upstream binaries aren't useful to us, but pip's ability to abstract
away the vagaries of the upcoming metadata 2.0 migration is incredibly
helpful. The other distros are in the same situation (we'll always
feed source tarballs into our own build systems), and ditto for the
conda folks. We need that for all sorts of reasons that potential new
Python users don't care about, but continuing to meet our
requirements, along with the free-for-all that is PyPI makes handling
the binary distribution problem *much* harder for pip.

By contrast, like any other distro, conda doesn't need to boil the
ocean - it just needs to provide a filtered, up to date set of core
packages that work well together. The advantage it has over other
distros is that it is *cross-platform* - it works essentially the same
way on Windows, Mac OS X. Most other package management systems are
either platform specific or can't handle arbitrary binary
dependencies. By being Python-centric (even if not Python specific),
there's also a strong focus on updating the core packages more often
than the Linux distros do.

There's no "always use conda instead of pip" competition here - we
need pip, we need wheels. But I see those as software integrator
focused tools that would need a lot of additional work to provide a
truly spectacularly compelling out of the box experience for new
users. I don't think that's a useful way to expend effort - I think it
makes more sense to work on a separate "here's the fully assembled
environment for using Python as a tool to develop ideas" introduction,
while pushing the "here's how to build your own custom environment
from upstream parts" as a later step in a new user's journey towards
full Python mastery (if they're interested in that path). Making sure
that "pip install foo" does the right thing inside conda environments
(if it doesn't already) should be all that is needed to ensure that
random installation instructions on the internet still work.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From p.f.moore at gmail.com  Fri Apr 18 22:27:51 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 18 Apr 2014 21:27:51 +0100
Subject: [Python-Dev] Software integrators vs end users (was Re:
 Language Summit notes)
In-Reply-To: <CADiSq7d1FWdM2g658dPzt+JptEpDc6B_DFnsVVxHPcDCDUECuw@mail.gmail.com>
References: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>
 <CACac1F9k_kDN0UspOUs8BUxkbr=tZnLTorkFf2fJGJUfBR9P+g@mail.gmail.com>
 <CADiSq7d1FWdM2g658dPzt+JptEpDc6B_DFnsVVxHPcDCDUECuw@mail.gmail.com>
Message-ID: <CACac1F9wn17PjbkDRJd6RZT6Sm92e6xmDBfs-OPhLAT1Vbfb-g@mail.gmail.com>

On 18 April 2014 20:18, Nick Coghlan <ncoghlan at gmail.com> wrote:
> At this point, however, I'm mainly looking for consensus that there
> *are* two different problems to be solved here, and solving them both
> well in a single tool is likely to be nigh on impossible. (I'm aware
> we're really on the wrong list for that discussion, but I also think
> there's value in getting some broader python-dev awareness of this
> particular topic)

I'm not entirely sure what you're proposing here.

Obviously there are multiple classes of users (whether it's as simple
as just two, or whether there's more is less important).
Clearly there is a case for curated stacks and OS distributions, and
clearly some people use those stacks and distributions.
Things aren't perfect - CPython and projects like pip need to be aware
of, and respond to, the differing needs of people who use Python in
different ways.

But what are you suggesting python-dev needs to *do* about this? What
precisely is wrong with how we deal with the multiple types of user
that exist at the moment?

Without wanting to sound like I'm taking things personally, it feels
like there's an intention to suggest to *some* people that they would
be better served by a curated stack. I don't know how to answer that
except from a personal perspective[1], and it's hard to do that
without knowing whether I'm in a group that you'd see as benefiting
from a curated stack.

One thing I *would* suggest is that a lot of "corporate" use of Python
(by which I mean semi-informal scripting and automation done as part
of the infrastructure of larger projects written in more "enterprise"
tools like Java or higher level CRM/eBusiness/etc packages) is not
suitable for a curated stack (corporate IT policies would see that as
a "3rd party tool" where the python.org distribution is just a
project-internal utility). But the staff involved are typically not
familiar with open source or its culture, and struggle to deal with
things like package management (this is *not* the "legal approval"
issue, as cut and paste of code from websites is common in this
culture - it's "internal use only"). Within the context of your two
categories, this may well be a third one (unless you stretch
"application developers" way beyond what I think you are intending).

Paul

[1] By which I mean "looking at what I use Python for, how I see it
used by others in my organisation, and how I would expect to promote
Python to people who don't currently use it but whom I feel would
benefit from it".

From donald at stufft.io  Fri Apr 18 22:50:44 2014
From: donald at stufft.io (Donald Stufft)
Date: Fri, 18 Apr 2014 16:50:44 -0400
Subject: [Python-Dev] Software integrators vs end users (was Re:
	Language Summit notes)
In-Reply-To: <CADiSq7funpGBCOzuye41sJUxBZWA9ZtmqWv=M=Z-+NSNhCsP9w@mail.gmail.com>
References: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>
 <CACac1F9k_kDN0UspOUs8BUxkbr=tZnLTorkFf2fJGJUfBR9P+g@mail.gmail.com>
 <CADiSq7d1FWdM2g658dPzt+JptEpDc6B_DFnsVVxHPcDCDUECuw@mail.gmail.com>
 <0334A003-E9FF-4C62-B3AF-2E21488955B8@stufft.io>
 <CADiSq7funpGBCOzuye41sJUxBZWA9ZtmqWv=M=Z-+NSNhCsP9w@mail.gmail.com>
Message-ID: <8068621A-C864-4E16-BE7A-2A358C478EC5@stufft.io>


On Apr 18, 2014, at 4:22 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 18 April 2014 15:39, Donald Stufft <donald at stufft.io> wrote:
>> 
>> On Apr 18, 2014, at 3:18 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> 
>>> At this point, however, I'm mainly looking for consensus that there
>>> *are* two different problems to be solved here, and solving them both
>>> well in a single tool is likely to be nigh on impossible. (I'm aware
>>> we're really on the wrong list for that discussion, but I also think
>>> there's value in getting some broader python-dev awareness of this
>>> particular topic)
>> 
>> I?m not sure about this? I mean yes those are two different areas, but I?m
>> not sure about the split between Conda and pip here. As far as I can tell
>> Conda is useful in the same way apt-get or yum is useful, you get a non
>> Python specific set of packages (because sometimes things aren?t pure
>> python) and you also remove a little bit of thinking about versions (although
>> honestly I think it?s possible to remove most of that for consumers of
>> packages).
> 
> You also get the ability to use that same system to update *Python
> itself*, regardless of platform. Being notified of and consuming
> CPython updates on Windows, as well as consuming alternate versions on
> Linux distros, is currently not a well solved problem - with conda,
> it's no more complex than updating a PyPI package.
> 
> That's one of the most attractive aspects for me - making Python 3.4,
> pypy, etc as easy to update with consistent cross platform
> instructions as Python packages are.

Ah right. I would never want that personally so I forgot about it. (I also don?t use
the system Python for development but I user a different tool for managing it).

> 
>> To be quite frank, a lot of the benefit of Conda outside of the ?I need something
>> that isn?t strictly Python) is in the fact they can bootstrap compiled packages
>> whereas pip/wheel/PyPI combination we need to convince authors to upload
>> their own binary packages (or at least develop something to make it easier
>> like a build farm).
> 
> Yep, that's a large part of why I think "divide and conquer" is the
> way to go here. While it isn't completely accurate (as most SaaS
> developers don't want to build C extensions from source) I think
> Guido's "build from source" vs "install a pre-built binary"
> distinction is still a reasonably good way to characterise it. For a
> distro like Fedora (or, even more so, RHEL), we're not going to trust
> a binary created by someone else if we can possibly avoid it, so
> upstream binaries aren't useful to us, but pip's ability to abstract
> away the vagaries of the upcoming metadata 2.0 migration is incredibly
> helpful. The other distros are in the same situation (we'll always
> feed source tarballs into our own build systems), and ditto for the
> conda folks. We need that for all sorts of reasons that potential new
> Python users don't care about, but continuing to meet our
> requirements, along with the free-for-all that is PyPI makes handling
> the binary distribution problem *much* harder for pip.
> 
> By contrast, like any other distro, conda doesn't need to boil the
> ocean - it just needs to provide a filtered, up to date set of core
> packages that work well together. The advantage it has over other
> distros is that it is *cross-platform* - it works essentially the same
> way on Windows, Mac OS X. Most other package management systems are
> either platform specific or can't handle arbitrary binary
> dependencies. By being Python-centric (even if not Python specific),
> there's also a strong focus on updating the core packages more often
> than the Linux distros do.
> 
> There's no "always use conda instead of pip" competition here - we
> need pip, we need wheels. But I see those as software integrator
> focused tools that would need a lot of additional work to provide a
> truly spectacularly compelling out of the box experience for new
> users. I don't think that's a useful way to expend effort - I think it
> makes more sense to work on a separate "here's the fully assembled
> environment for using Python as a tool to develop ideas" introduction,
> while pushing the "here's how to build your own custom environment
> from upstream parts" as a later step in a new user's journey towards
> full Python mastery (if they're interested in that path). Making sure
> that "pip install foo" does the right thing inside conda environments
> (if it doesn't already) should be all that is needed to ensure that
> random installation instructions on the internet still work.

So I?m not really worried about a competition or anything. I?m mostly worried
about confusion of users. What you?re suggestion we give to use is *two* ways
to install Python packages (and 2 or 3 ways to virtualize a Python instance).
That provides extra cognitive burden for people who are new to Python. They
have to both understand that the packages you install from Conda are different
than the ones you install from pip, and that they come from different places.
If a new user is reasonably likely to have to use ``pip install`` when they are
using pip, then I think that provides a worse experience than only having to
use one tool to manage your packages. This confusion is going to be worse when
new users find a library they want to use and it tells them to use
``pip install`` (or even easy_install!) even if Anaconda itself has a package
inside of it.

So I think this could potentially in the short term make things easier for
people who need nothing but what conda provides (because of the bootstrapped
pre-compiled binaries), but I think ultimately you?re going to confuse them
until they know enough about the ecosystem to grasp the difference.

I?ve *personally* seen this first hand with the Linux and other OSs who
distribute stuff where people get really confused about what the difference is
between pip and apt-get/yum/portmaster and when they?d use one over the other,
adding another column in the matrix for new users seems to be something that is
only going to cause more confusion and pain.

As you said the difference is often "precompiled vs source: and I think the
answer to that is to make it easier to use precompiled stuff in pip and PyPI
and not to push people to an entirely different stack and invalidate the vast
bulk of documentation (which most of it already recommends pip) that those
users are likely to encounter.

Baring implementation problems (bad defaults, ux, etc) the fundamental
difference between conda and pip/easy_install is that conda manages more than
just Python packages, much like apt-get/yum) does and any discussion about what
to recommend where should focus on fundamental differences as implementation
problems in *either* solution can be addressed (lack of binary/ux on pip,
security issues on Conda, etc).

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140418/28d301c9/attachment.sig>

From donald at stufft.io  Fri Apr 18 22:51:49 2014
From: donald at stufft.io (Donald Stufft)
Date: Fri, 18 Apr 2014 16:51:49 -0400
Subject: [Python-Dev] Software integrators vs end users (was Re:
	Language Summit notes)
In-Reply-To: <8068621A-C864-4E16-BE7A-2A358C478EC5@stufft.io>
References: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>
 <CACac1F9k_kDN0UspOUs8BUxkbr=tZnLTorkFf2fJGJUfBR9P+g@mail.gmail.com>
 <CADiSq7d1FWdM2g658dPzt+JptEpDc6B_DFnsVVxHPcDCDUECuw@mail.gmail.com>
 <0334A003-E9FF-4C62-B3AF-2E21488955B8@stufft.io>
 <CADiSq7funpGBCOzuye41sJUxBZWA9ZtmqWv=M=Z-+NSNhCsP9w@mail.gmail.com>
 <8068621A-C864-4E16-BE7A-2A358C478EC5@stufft.io>
Message-ID: <4638E6A3-E34A-4233-A001-578B28A1D160@stufft.io>


On Apr 18, 2014, at 4:50 PM, Donald Stufft <donald at stufft.io> wrote:

> So I?m not really worried about a competition or anything. I?m mostly worried
> about confusion of users. What you?re suggestion we give to use is *two* ways
> to install Python packages (and 2 or 3 ways to virtualize a Python instance).
> That provides extra cognitive burden for people who are new to Python. They
> have to both understand that the packages you install from Conda are different
> than the ones you install from pip, and that they come from different places.
> If a new user is reasonably likely to have to use ``pip install`` when they are
> using pip, then I think that provides a worse experience than only having to
> use one tool to manage your packages. This confusion is going to be worse when
> new users find a library they want to use and it tells them to use
> ``pip install`` (or even easy_install!) even if Anaconda itself has a package
> inside of it.

Typo - When they are using conda.
-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140418/fd455423/attachment.sig>

From ncoghlan at gmail.com  Fri Apr 18 22:59:35 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 18 Apr 2014 16:59:35 -0400
Subject: [Python-Dev] Software integrators vs end users (was Re:
 Language Summit notes)
In-Reply-To: <CACac1F9wn17PjbkDRJd6RZT6Sm92e6xmDBfs-OPhLAT1Vbfb-g@mail.gmail.com>
References: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>
 <CACac1F9k_kDN0UspOUs8BUxkbr=tZnLTorkFf2fJGJUfBR9P+g@mail.gmail.com>
 <CADiSq7d1FWdM2g658dPzt+JptEpDc6B_DFnsVVxHPcDCDUECuw@mail.gmail.com>
 <CACac1F9wn17PjbkDRJd6RZT6Sm92e6xmDBfs-OPhLAT1Vbfb-g@mail.gmail.com>
Message-ID: <CADiSq7ffagXWBdfTsGr4=07Km0T=oCGOwO2LHGP9y94on5rheA@mail.gmail.com>

On 18 April 2014 16:27, Paul Moore <p.f.moore at gmail.com> wrote:
> On 18 April 2014 20:18, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> At this point, however, I'm mainly looking for consensus that there
>> *are* two different problems to be solved here, and solving them both
>> well in a single tool is likely to be nigh on impossible. (I'm aware
>> we're really on the wrong list for that discussion, but I also think
>> there's value in getting some broader python-dev awareness of this
>> particular topic)
>
> I'm not entirely sure what you're proposing here.
>
> Obviously there are multiple classes of users (whether it's as simple
> as just two, or whether there's more is less important).
> Clearly there is a case for curated stacks and OS distributions, and
> clearly some people use those stacks and distributions.
> Things aren't perfect - CPython and projects like pip need to be aware
> of, and respond to, the differing needs of people who use Python in
> different ways.
>
> But what are you suggesting python-dev needs to *do* about this? What
> precisely is wrong with how we deal with the multiple types of user
> that exist at the moment?

When you go to python.org, you are currently advised to download one
of the following:

- a source tarball (Linux)
- a bare bones binary installer (Windows & Mac OS X)

What I am advocating for is that *we are currently doing it wrong*, as
these are unlikely to be the best thing to install for most new Python
users. Now, we could make things even *more* confusing by instead
presenting the prospective user with *even more* choices, like "oh,
you might want to get it from your Linux distro, or maybe homebrew, or
MacPorts", but that's not helping, because it's asking new users to
make choices that they may not be in a position to make.

>From a usability perspective, I don't believe we should be advertising
a minimalist installation as the preferred entry point for new users -
we should be advertising a full featured sumo distribution that lets
new users quickly experience the full power and flexibility of things
like IPython notebooks. I like Anaconda, because it's fully open
source, abstracts away the underlying operating system more than the
standard installers and the same mechanism that you use to update
packages can also be used to update the Python interpreter itself.

Everything else we do today should remain available for those that
want them - I'm not advocating taking anything away. But we need to
start getting more opinionated on behalf of new users that don't yet
have the experience they need to start challenging our judgement and
make their own choices about which tools to use out of the vast array
of choices the Python ecosystem provides.

> Without wanting to sound like I'm taking things personally, it feels
> like there's an intention to suggest to *some* people that they would
> be better served by a curated stack. I don't know how to answer that
> except from a personal perspective[1], and it's hard to do that
> without knowing whether I'm in a group that you'd see as benefiting
> from a curated stack.

No, I don't think you'd benefit substantially from a curated stack - I
don't think any experienced Python user would. But think through your
own personal stack. Start from a Google search for "Python tutorial"
and see how easily you can figure out how and where to obtain all
those components.

Putting a sumo distribution front and centre on the website
drastically improves the out of the box experience for new users,
*without* introducing a huge maintainability issue for the standard
library. New users gain quick access to a much larger fraction of the
overall Python ecosystem, while on the back end software *production*
side of things, everything stays decoupled.

It also provides a clearer timeline for adoption of new versions of
Python - while the reference interpreter would go up immediately, the
sumo version would only be updated once all the contained projects had
been checked for compatibility with the new version.

> One thing I *would* suggest is that a lot of "corporate" use of Python
> (by which I mean semi-informal scripting and automation done as part
> of the infrastructure of larger projects written in more "enterprise"
> tools like Java or higher level CRM/eBusiness/etc packages) is not
> suitable for a curated stack (corporate IT policies would see that as
> a "3rd party tool" where the python.org distribution is just a
> project-internal utility). But the staff involved are typically not
> familiar with open source or its culture, and struggle to deal with
> things like package management (this is *not* the "legal approval"
> issue, as cut and paste of code from websites is common in this
> culture - it's "internal use only"). Within the context of your two
> categories, this may well be a third one (unless you stretch
> "application developers" way beyond what I think you are intending).

No, that is the case covered by 'creators of corporate "Standard
Operating Environment" definitions'. That's explicitly in the software
integrator category - whether those users formally have the
responsibility of defining an SOE, that's what they're doing in
practice.

Cheers,
Nick.

>
> Paul
>
> [1] By which I mean "looking at what I use Python for, how I see it
> used by others in my organisation, and how I would expect to promote
> Python to people who don't currently use it but whom I feel would
> benefit from it".



-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Fri Apr 18 23:08:56 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 18 Apr 2014 17:08:56 -0400
Subject: [Python-Dev] Software integrators vs end users (was Re:
 Language Summit notes)
In-Reply-To: <8068621A-C864-4E16-BE7A-2A358C478EC5@stufft.io>
References: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>
 <CACac1F9k_kDN0UspOUs8BUxkbr=tZnLTorkFf2fJGJUfBR9P+g@mail.gmail.com>
 <CADiSq7d1FWdM2g658dPzt+JptEpDc6B_DFnsVVxHPcDCDUECuw@mail.gmail.com>
 <0334A003-E9FF-4C62-B3AF-2E21488955B8@stufft.io>
 <CADiSq7funpGBCOzuye41sJUxBZWA9ZtmqWv=M=Z-+NSNhCsP9w@mail.gmail.com>
 <8068621A-C864-4E16-BE7A-2A358C478EC5@stufft.io>
Message-ID: <CADiSq7dH4HAOBEYe7KLyGE5OtfZF+BQ=PRD_23b7BR6npZCpQg@mail.gmail.com>

On 18 April 2014 16:50, Donald Stufft <donald at stufft.io> wrote:
> So I?m not really worried about a competition or anything. I?m mostly worried
> about confusion of users. What you?re suggestion we give to use is *two* ways
> to install Python packages (and 2 or 3 ways to virtualize a Python instance).

Note that one of my requirements was that "pip install foo" *must* do
the right thing in conda environments (whatever we decide the "right
thing" means in that context). It was buried at the end of a long
email though, so it would have been easy to miss.

That means the instructions to new users can be simple and consistent
- use pip commands to manage Python things, conda commands to manage
other stuff. They'll likely discover in fairly short order that the
conda commands also work for Python things, but it can be explained
that not all environments are conda environments, and hence pip works
in more situations than conda does, but at the cost of being specific
to Python packages.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ezio.melotti at gmail.com  Fri Apr 18 23:03:33 2014
From: ezio.melotti at gmail.com (Ezio Melotti)
Date: Sat, 19 Apr 2014 00:03:33 +0300
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <CAP1=2W4hHLn6rPS36bokKXgKnECDsFgKnp=TGJS_2MgvvcLmhA@mail.gmail.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <lip39g$jnr$1@ger.gmane.org>
 <CAP1=2W4hHLn6rPS36bokKXgKnECDsFgKnp=TGJS_2MgvvcLmhA@mail.gmail.com>
Message-ID: <CACBhJdGw5K2+ZzhMW16QS5t49eSm+o3GRxsmRe=haWWVWGtbdA@mail.gmail.com>

Hi,

On Thu, Apr 17, 2014 at 9:09 PM, Brett Cannon <bcannon at gmail.com> wrote:
>
>
> On Thu Apr 17 2014 at 1:34:23 PM, Jurko Gospodneti?
> <jurko.gospodnetic at pke.hr> wrote:
>>
>>    Hi.
>>
>> On 14.4.2014. 23:51, Brett Cannon wrote:
>> > Now the question is whether the maintenance cost of having to rebuild
>> > Python for a select number of stdlib modules is enough to warrant
>> > putting in the effort to make this work.
>>
>>    I would really love to have better startup times in production, but I
>> would also really hate to lose the ability to hack around in stdlib
>> sources during development just to get better startup performance.
>>
>>    In general, what I really like about using Python for software
>> development is the ability to open any stdlib file and easily go poking
>> around using stuff like 'import pdb;pdb.set_trace()' or simple print
>> statements. Researching mysterious behaviour is generally much much
>> MUCH! easier (read: takes less hours/days/weeks) if it ends up leading
>> into a stdlib Python module than if it takes you down into the bowels of
>> some C module (think zipimport.c *grin*). Not to mention the effect that
>> being able to quickly resolve a mystery by hacking on some Python
>> internals leaves you feeling very satisfied, while having to entrench
>> yourself in those internals for a long time just to find out you've made
>> something foolish on your end leaves you feeling exhausted at best.
>
>
> Freezing modules does not affect the ability to use gdb. And as long as you
> set the appropriate __file__ values then tracebacks will contain even the
> file line and location.
>

Will the tracebacks only contain the line number or the line of code too?

I've seen tracebacks involving importlib,_bootstrap that didn't
include the code, and I'm wondering if we will get something similar
for all the other modules you are freezing:

Traceback (most recent call last):
  File "/tmp/foo.py", line 7, in <module>
    import email
  File "<frozen importlib._bootstrap>", line 1561, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1519, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1473, in _find_module
  File "<frozen importlib._bootstrap>", line 1308, in find_module
  File "<frozen importlib._bootstrap>", line 1284, in _get_loader
  File "<frozen importlib._bootstrap>", line 1273, in _path_importer_cache
  File "<frozen importlib._bootstrap>", line 1254, in _path_hooks
TypeError: 'NoneType' object is not iterable

Best Regards,
Ezio Melotti



> -Brett
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/ezio.melotti%40gmail.com
>

From p.f.moore at gmail.com  Fri Apr 18 23:19:24 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 18 Apr 2014 22:19:24 +0100
Subject: [Python-Dev] Software integrators vs end users (was Re:
 Language Summit notes)
In-Reply-To: <CADiSq7ffagXWBdfTsGr4=07Km0T=oCGOwO2LHGP9y94on5rheA@mail.gmail.com>
References: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>
 <CACac1F9k_kDN0UspOUs8BUxkbr=tZnLTorkFf2fJGJUfBR9P+g@mail.gmail.com>
 <CADiSq7d1FWdM2g658dPzt+JptEpDc6B_DFnsVVxHPcDCDUECuw@mail.gmail.com>
 <CACac1F9wn17PjbkDRJd6RZT6Sm92e6xmDBfs-OPhLAT1Vbfb-g@mail.gmail.com>
 <CADiSq7ffagXWBdfTsGr4=07Km0T=oCGOwO2LHGP9y94on5rheA@mail.gmail.com>
Message-ID: <CACac1F_etDCODb6-0oAkCSpZ2dB4aW3gcy-Un=nBOgY1wgEJMQ@mail.gmail.com>

On 18 April 2014 21:59, Nick Coghlan <ncoghlan at gmail.com> wrote:
> What I am advocating for is that *we are currently doing it wrong*, as
> these are unlikely to be the best thing to install for most new Python
> users.

For Windows users at least, I disagree. I have directed a lot of
people to the python.org Windows installers, and they have got up and
running without any issues (ignoring website usability "find the right
download" questions). Most of the time, they need nothing more than
the interpreter and the stdlib to become productive.

The downside to directing them at (say) conda is that 99.99% of the
documentation on the internet is not directed at conda users. Most of
the time, that's fine, because it's all generic stuff, but it's a trap
waiting to happen. I can't tell people to use conda myself as I'm not
a conda user, so I can't help them if they hit problems.

> No, I don't think you'd benefit substantially from a curated stack - I
> don't think any experienced Python user would. But think through your
> own personal stack. Start from a Google search for "Python tutorial"
> and see how easily you can figure out how and where to obtain all
> those components.

Core python, plus a standalone copy of virtualenv (and on 3.4+ I don't
even need that). Everything else is installed from PyPI as needed.
Agreed that starting from a Google search won't get that at the
moment, but that's what the packaging initiatives are trying to solve
- get the infrastructure there so that more and more people say the
same things, and so google searches become more accurate.

My vision of the ideal is:

1. Install Python from python.org
2. py -m venv foo
3. foo\Scripts\activate
4. pip install <anything>

(Note that item 4 relies on pervasive availability of wheels for
packages like numpy, scipy, lxml, pywin32, etc)

Oh, and everyone needs to switch to Windows so this works :-)

More seriously, I genuinely think that this is the ideal solution for
*all* Windows users who don't have the sort of specialised needs that
mean a curated stack is right for them, and they know about it from
word of mouth via colleagues in the same situation.

I have no idea, and no opinion, on how practical it is to achieve this
level of simplicity on Unix or OSX. But I do think it's the right goal
to aim for for any cases where using "the system Python" is the wrong
answer.

>> One thing I *would* suggest is that a lot of "corporate" use of Python
>> (by which I mean semi-informal scripting and automation done as part
>> of the infrastructure of larger projects written in more "enterprise"
>> tools like Java or higher level CRM/eBusiness/etc packages) is not
>> suitable for a curated stack (corporate IT policies would see that as
>> a "3rd party tool" where the python.org distribution is just a
>> project-internal utility). But the staff involved are typically not
>> familiar with open source or its culture, and struggle to deal with
>> things like package management (this is *not* the "legal approval"
>> issue, as cut and paste of code from websites is common in this
>> culture - it's "internal use only"). Within the context of your two
>> categories, this may well be a third one (unless you stretch
>> "application developers" way beyond what I think you are intending).
>
> No, that is the case covered by 'creators of corporate "Standard
> Operating Environment" definitions'. That's explicitly in the software
> integrator category - whether those users formally have the
> responsibility of defining an SOE, that's what they're doing in
> practice.

OK. In which case that category is far broader than I anticipated, and
includes people who cannot even write a Windows batch file or use a
command line without instruction (I can name names!!! ;-)) If that's
your view, though, then all I can say is that I have no experience of
anyone who isn't in the "software integrator" category, so I'll take
your word that there is only one other category :-)

Paul

From donald at stufft.io  Fri Apr 18 23:22:36 2014
From: donald at stufft.io (Donald Stufft)
Date: Fri, 18 Apr 2014 17:22:36 -0400
Subject: [Python-Dev] Software integrators vs end users (was Re:
	Language Summit notes)
In-Reply-To: <CADiSq7dH4HAOBEYe7KLyGE5OtfZF+BQ=PRD_23b7BR6npZCpQg@mail.gmail.com>
References: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>
 <CACac1F9k_kDN0UspOUs8BUxkbr=tZnLTorkFf2fJGJUfBR9P+g@mail.gmail.com>
 <CADiSq7d1FWdM2g658dPzt+JptEpDc6B_DFnsVVxHPcDCDUECuw@mail.gmail.com>
 <0334A003-E9FF-4C62-B3AF-2E21488955B8@stufft.io>
 <CADiSq7funpGBCOzuye41sJUxBZWA9ZtmqWv=M=Z-+NSNhCsP9w@mail.gmail.com>
 <8068621A-C864-4E16-BE7A-2A358C478EC5@stufft.io>
 <CADiSq7dH4HAOBEYe7KLyGE5OtfZF+BQ=PRD_23b7BR6npZCpQg@mail.gmail.com>
Message-ID: <8AF38D6F-0E7A-48EB-AB70-F19666E7B372@stufft.io>


On Apr 18, 2014, at 5:08 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 18 April 2014 16:50, Donald Stufft <donald at stufft.io> wrote:
>> So I?m not really worried about a competition or anything. I?m mostly worried
>> about confusion of users. What you?re suggestion we give to use is *two* ways
>> to install Python packages (and 2 or 3 ways to virtualize a Python instance).
> 
> Note that one of my requirements was that "pip install foo" *must* do
> the right thing in conda environments (whatever we decide the "right
> thing" means in that context). It was buried at the end of a long
> email though, so it would have been easy to miss.
> 
> That means the instructions to new users can be simple and consistent
> - use pip commands to manage Python things, conda commands to manage
> other stuff. They'll likely discover in fairly short order that the
> conda commands also work for Python things, but it can be explained
> that not all environments are conda environments, and hence pip works
> in more situations than conda does, but at the cost of being specific
> to Python packages.
> 
> Cheers,
> Nick.
> 
> -- 
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

Thinking about that and the implications of it.

Next question, where even is the code for ?Anaconda?? I tried to download it
from their website and it?s behind an email form, I saw a link for github issues
but it?s in a dedicated ?anaconda-issues? repo which doesn?t have any code
associated with it.

Also to be honest i?m a little uncomfortable with the idea of Python.org pushing
a platform where the company that develops the platform sells Add-ons to that
platform. So while Anaconda itself may be free and open source the fact that
the Anaconda distribution is a gateway to a particular company?s paid add ons
makes me feel a bit like a government sponsored monopoly kind of thing? I?m
not using good words here to describe what I mean, but it feels kind of icky to me.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140418/3665e449/attachment.sig>

From p.f.moore at gmail.com  Fri Apr 18 23:40:19 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 18 Apr 2014 22:40:19 +0100
Subject: [Python-Dev] Software integrators vs end users (was Re:
 Language Summit notes)
In-Reply-To: <CADiSq7dH4HAOBEYe7KLyGE5OtfZF+BQ=PRD_23b7BR6npZCpQg@mail.gmail.com>
References: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>
 <CACac1F9k_kDN0UspOUs8BUxkbr=tZnLTorkFf2fJGJUfBR9P+g@mail.gmail.com>
 <CADiSq7d1FWdM2g658dPzt+JptEpDc6B_DFnsVVxHPcDCDUECuw@mail.gmail.com>
 <0334A003-E9FF-4C62-B3AF-2E21488955B8@stufft.io>
 <CADiSq7funpGBCOzuye41sJUxBZWA9ZtmqWv=M=Z-+NSNhCsP9w@mail.gmail.com>
 <8068621A-C864-4E16-BE7A-2A358C478EC5@stufft.io>
 <CADiSq7dH4HAOBEYe7KLyGE5OtfZF+BQ=PRD_23b7BR6npZCpQg@mail.gmail.com>
Message-ID: <CACac1F8h-f=BRyrKF0yNLi9VHWr1tAo+25LspQf6XYe_AFy1xA@mail.gmail.com>

On 18 April 2014 22:08, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Note that one of my requirements was that "pip install foo" *must* do
> the right thing in conda environments (whatever we decide the "right
> thing" means in that context).

Is this specifically a requirement for conda? Or do you expect the
same to be true for ActivePython, Python(X,Y) and whatever other
distributions exist? Assuming conda isn't a special case, is it down
to pip to ensure that or to the distributions to play nicely with pip?

It's starting to feel as if conda *is* being treated as somehow a
special case. (At least in the sense that you're advocating that we
promote a non-minimalist distribution on the python.org website, and
you've explicitly stated that you personally like conda). I'm strongly
against the official python.org website promoting *any* external
distribution as the recommended solution. I'm not against having a
simple list of "alternative distributions" if you think that'd help,
but that's a far cry from "advertising a full featured sumo
distribution that lets new users quickly experience the full power and
flexibility of things like IPython notebooks".

On the other hand, it might be good to have an explicit advertising
page - "See the power of Python" which *did* direct users to stacks
that showcase particularly impressive uses of Python. There could be a
section about the IPython notebook (which mentions that the conda
Python stack comes with IPython preinstalled) and another about Django
(hmm, what's the curated stack for web developers?) and another
showing the Visual Stiudio Tools for Python (hmm, stack for Windows
developers/admins?).

As a comparison, java.com points you straight at the standard Java SE
download. You don't see JBoss or Weblogic promoted on there, even
though those are likely to be how most developers interact with Java.
To get anything other than the minimal distribution, you need to know
you want it and search for it.

Paul

From ncoghlan at gmail.com  Fri Apr 18 23:40:46 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 18 Apr 2014 17:40:46 -0400
Subject: [Python-Dev] Software integrators vs end users (was Re:
 Language Summit notes)
In-Reply-To: <8AF38D6F-0E7A-48EB-AB70-F19666E7B372@stufft.io>
References: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>
 <CACac1F9k_kDN0UspOUs8BUxkbr=tZnLTorkFf2fJGJUfBR9P+g@mail.gmail.com>
 <CADiSq7d1FWdM2g658dPzt+JptEpDc6B_DFnsVVxHPcDCDUECuw@mail.gmail.com>
 <0334A003-E9FF-4C62-B3AF-2E21488955B8@stufft.io>
 <CADiSq7funpGBCOzuye41sJUxBZWA9ZtmqWv=M=Z-+NSNhCsP9w@mail.gmail.com>
 <8068621A-C864-4E16-BE7A-2A358C478EC5@stufft.io>
 <CADiSq7dH4HAOBEYe7KLyGE5OtfZF+BQ=PRD_23b7BR6npZCpQg@mail.gmail.com>
 <8AF38D6F-0E7A-48EB-AB70-F19666E7B372@stufft.io>
Message-ID: <CADiSq7fbmMNy=Gp2wLc9g7Uxb6t0wXnm6oAUm2m70Ejpmz2=uw@mail.gmail.com>

On 18 April 2014 17:22, Donald Stufft <donald at stufft.io> wrote:
>
> On Apr 18, 2014, at 5:08 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
>> On 18 April 2014 16:50, Donald Stufft <donald at stufft.io> wrote:
>>> So I?m not really worried about a competition or anything. I?m mostly worried
>>> about confusion of users. What you?re suggestion we give to use is *two* ways
>>> to install Python packages (and 2 or 3 ways to virtualize a Python instance).
>>
>> Note that one of my requirements was that "pip install foo" *must* do
>> the right thing in conda environments (whatever we decide the "right
>> thing" means in that context). It was buried at the end of a long
>> email though, so it would have been easy to miss.
>>
>> That means the instructions to new users can be simple and consistent
>> - use pip commands to manage Python things, conda commands to manage
>> other stuff. They'll likely discover in fairly short order that the
>> conda commands also work for Python things, but it can be explained
>> that not all environments are conda environments, and hence pip works
>> in more situations than conda does, but at the cost of being specific
>> to Python packages.
>>
>> Cheers,
>> Nick.
>>
>> --
>> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>
> Thinking about that and the implications of it.
>
> Next question, where even is the code for ?Anaconda?? I tried to download it
> from their website and it?s behind an email form, I saw a link for github issues
> but it?s in a dedicated ?anaconda-issues? repo which doesn?t have any code
> associated with it.
>
> Also to be honest i?m a little uncomfortable with the idea of Python.org pushing
> a platform where the company that develops the platform sells Add-ons to that
> platform. So while Anaconda itself may be free and open source the fact that
> the Anaconda distribution is a gateway to a particular company?s paid add ons
> makes me feel a bit like a government sponsored monopoly kind of thing? I?m
> not using good words here to describe what I mean, but it feels kind of icky to me.

I actually agree and it wouldn't necessarily make sense for *Anaconda*
itself to be the promoted sumo distribution. I'm more interested in
the idea of making a sumo distribution the default recommended entry
point, and handling the issues of independent curation and
redistribution that would come with that, as well as the mechanics of
how to actually handle installation and updates from the end user
perspective.

Perhaps we can get the "pip install ipython" experience to a good
place faster than I currently expect, and we can duck this entire
question (at least for Windows and Mac OS X).

Regardless, I think we've drifted far enough into speculative
territory to be off-topic for python-dev. This isn't a question we're
going to resolve in a hurry, and the near term focus should likely be
on improving the numpy+wheel story, since that's valuable regardless
of what happens with the python.org download pages. While that is
ongoing, people can mull over the various possibilities independently.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From jimjjewett at gmail.com  Fri Apr 18 23:46:35 2014
From: jimjjewett at gmail.com (Jim J. Jewett)
Date: Fri, 18 Apr 2014 17:46:35 -0400
Subject: [Python-Dev] dict and required hashing
Message-ID: <CA+OGgf4Asw_VZ-jTtVxVfhygBfc5nZggnKJoS_wyyR1Jm2Wr6g@mail.gmail.com>

(1)  I believe the recent consensus was that the number of comparisons
made in a dict lookup is an implementation detail.  (Please correct me
if I am wrong.)

(2)  Is "the item will be hashed at least once" a language guarantee?

For small mappings, it might well be more efficient to just store the
2-3 key/value pairs and skip the bucket calculation.

On the other hand, if a key is not hashable, discovering that long
after it has already been added to the dict is suboptimal.

Of course, that sort of delayed exception can already happen if it is
the __eq__ method that is messed up ...

-jJ

From p.f.moore at gmail.com  Fri Apr 18 23:48:09 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 18 Apr 2014 22:48:09 +0100
Subject: [Python-Dev] Software integrators vs end users (was Re:
 Language Summit notes)
In-Reply-To: <CADiSq7fbmMNy=Gp2wLc9g7Uxb6t0wXnm6oAUm2m70Ejpmz2=uw@mail.gmail.com>
References: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>
 <CACac1F9k_kDN0UspOUs8BUxkbr=tZnLTorkFf2fJGJUfBR9P+g@mail.gmail.com>
 <CADiSq7d1FWdM2g658dPzt+JptEpDc6B_DFnsVVxHPcDCDUECuw@mail.gmail.com>
 <0334A003-E9FF-4C62-B3AF-2E21488955B8@stufft.io>
 <CADiSq7funpGBCOzuye41sJUxBZWA9ZtmqWv=M=Z-+NSNhCsP9w@mail.gmail.com>
 <8068621A-C864-4E16-BE7A-2A358C478EC5@stufft.io>
 <CADiSq7dH4HAOBEYe7KLyGE5OtfZF+BQ=PRD_23b7BR6npZCpQg@mail.gmail.com>
 <8AF38D6F-0E7A-48EB-AB70-F19666E7B372@stufft.io>
 <CADiSq7fbmMNy=Gp2wLc9g7Uxb6t0wXnm6oAUm2m70Ejpmz2=uw@mail.gmail.com>
Message-ID: <CACac1F_XwUAwv2AsbKUJg7XvLHjxCnBA4dHWkFyDk-u8OkbuKg@mail.gmail.com>

On 18 April 2014 22:40, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Perhaps we can get the "pip install ipython" experience to a good
> place faster than I currently expect, and we can duck this entire
> question (at least for Windows and Mac OS X).

Huh? Last time I tried, it was pretty trivial.

pip install pyzmq pyreadline tornado ipython

(I might have missed a couple, as I did this from memory, not by
looking at the website to check). There's nothing that needs a
compiler apart from pyzmq and that has wheels for Windows these days,
and builds out of the box with no dependencies in any case. The rest
is all pure Python.

I've not tried with 2.0, but I did try one of the betas and it was
really that simple. (Windows 7 64-bit, Python 3.4 IIRC). And the 1.x
versions were just as simple.

Paul

From benjamin at python.org  Fri Apr 18 23:57:55 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 18 Apr 2014 14:57:55 -0700
Subject: [Python-Dev] dict and required hashing
In-Reply-To: <CA+OGgf4Asw_VZ-jTtVxVfhygBfc5nZggnKJoS_wyyR1Jm2Wr6g@mail.gmail.com>
References: <CA+OGgf4Asw_VZ-jTtVxVfhygBfc5nZggnKJoS_wyyR1Jm2Wr6g@mail.gmail.com>
Message-ID: <1397858275.17357.108033801.64457FB7@webmail.messagingengine.com>

On Fri, Apr 18, 2014, at 14:46, Jim J. Jewett wrote:
> (1)  I believe the recent consensus was that the number of comparisons
> made in a dict lookup is an implementation detail.  (Please correct me
> if I am wrong.)

Absolutely.

> 
> (2)  Is "the item will be hashed at least once" a language guarantee?

No. (Would that be useful at all?)

From donald at stufft.io  Fri Apr 18 23:57:56 2014
From: donald at stufft.io (Donald Stufft)
Date: Fri, 18 Apr 2014 17:57:56 -0400
Subject: [Python-Dev] Software integrators vs end users (was Re:
	Language Summit notes)
In-Reply-To: <CACac1F_XwUAwv2AsbKUJg7XvLHjxCnBA4dHWkFyDk-u8OkbuKg@mail.gmail.com>
References: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>
 <CACac1F9k_kDN0UspOUs8BUxkbr=tZnLTorkFf2fJGJUfBR9P+g@mail.gmail.com>
 <CADiSq7d1FWdM2g658dPzt+JptEpDc6B_DFnsVVxHPcDCDUECuw@mail.gmail.com>
 <0334A003-E9FF-4C62-B3AF-2E21488955B8@stufft.io>
 <CADiSq7funpGBCOzuye41sJUxBZWA9ZtmqWv=M=Z-+NSNhCsP9w@mail.gmail.com>
 <8068621A-C864-4E16-BE7A-2A358C478EC5@stufft.io>
 <CADiSq7dH4HAOBEYe7KLyGE5OtfZF+BQ=PRD_23b7BR6npZCpQg@mail.gmail.com>
 <8AF38D6F-0E7A-48EB-AB70-F19666E7B372@stufft.io>
 <CADiSq7fbmMNy=Gp2wLc9g7Uxb6t0wXnm6oAUm2m70Ejpmz2=uw@mail.gmail.com>
 <CACac1F_XwUAwv2AsbKUJg7XvLHjxCnBA4dHWkFyDk-u8OkbuKg@mail.gmail.com>
Message-ID: <BC567E5D-D19C-4737-AADA-AB2C3DCFEEFF@stufft.io>


On Apr 18, 2014, at 5:48 PM, Paul Moore <p.f.moore at gmail.com> wrote:

> On 18 April 2014 22:40, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Perhaps we can get the "pip install ipython" experience to a good
>> place faster than I currently expect, and we can duck this entire
>> question (at least for Windows and Mac OS X).
> 
> Huh? Last time I tried, it was pretty trivial.
> 
> pip install pyzmq pyreadline tornado ipython
> 
> (I might have missed a couple, as I did this from memory, not by
> looking at the website to check). There's nothing that needs a
> compiler apart from pyzmq and that has wheels for Windows these days,
> and builds out of the box with no dependencies in any case. The rest
> is all pure Python.
> 
> I've not tried with 2.0, but I did try one of the betas and it was
> really that simple. (Windows 7 64-bit, Python 3.4 IIRC). And the 1.x
> versions were just as simple.
> 
> Paul

Maybe Nick meant ``pip install ipython[all]`` but I don?t actually know what that
includes. I?ve never used ipython except for the console.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140418/37601f5c/attachment.sig>

From greg.ewing at canterbury.ac.nz  Sat Apr 19 00:16:50 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 19 Apr 2014 10:16:50 +1200
Subject: [Python-Dev] Software integrators vs end users (was Re:
 Language Summit notes)
In-Reply-To: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>
References: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>
Message-ID: <5351A452.5020106@canterbury.ac.nz>

Nick Coghlan wrote:
> there are actually now *two* main ways of consuming
> Python:

Really? We'd better do something about that. We don't want
anyone consuming Python -- we want some left over for the
rest of us!

(I'm making a serious point -- it's annoying when people use
the word "consume" as though it were a synonym for "use".
It's not.)

When-I-consume-a-word-said-Humpty-Dumpty-ly,
Greg

From p.f.moore at gmail.com  Sat Apr 19 00:17:27 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 18 Apr 2014 23:17:27 +0100
Subject: [Python-Dev] Software integrators vs end users (was Re:
 Language Summit notes)
In-Reply-To: <BC567E5D-D19C-4737-AADA-AB2C3DCFEEFF@stufft.io>
References: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>
 <CACac1F9k_kDN0UspOUs8BUxkbr=tZnLTorkFf2fJGJUfBR9P+g@mail.gmail.com>
 <CADiSq7d1FWdM2g658dPzt+JptEpDc6B_DFnsVVxHPcDCDUECuw@mail.gmail.com>
 <0334A003-E9FF-4C62-B3AF-2E21488955B8@stufft.io>
 <CADiSq7funpGBCOzuye41sJUxBZWA9ZtmqWv=M=Z-+NSNhCsP9w@mail.gmail.com>
 <8068621A-C864-4E16-BE7A-2A358C478EC5@stufft.io>
 <CADiSq7dH4HAOBEYe7KLyGE5OtfZF+BQ=PRD_23b7BR6npZCpQg@mail.gmail.com>
 <8AF38D6F-0E7A-48EB-AB70-F19666E7B372@stufft.io>
 <CADiSq7fbmMNy=Gp2wLc9g7Uxb6t0wXnm6oAUm2m70Ejpmz2=uw@mail.gmail.com>
 <CACac1F_XwUAwv2AsbKUJg7XvLHjxCnBA4dHWkFyDk-u8OkbuKg@mail.gmail.com>
 <BC567E5D-D19C-4737-AADA-AB2C3DCFEEFF@stufft.io>
Message-ID: <CACac1F8ohmCDR9SUBef=qVAx52hjoKYOWtA7LRaMxdzixwQpHA@mail.gmail.com>

On 18 April 2014 22:57, Donald Stufft <donald at stufft.io> wrote:
> Maybe Nick meant ``pip install ipython[all]`` but I don?t actually know what that
> includes. I?ve never used ipython except for the console.

The hard bit is the QT Console, but that's because there aren't wheels
for PySide AFAICT.

Paul

From ncoghlan at gmail.com  Sat Apr 19 00:21:07 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 18 Apr 2014 18:21:07 -0400
Subject: [Python-Dev] Software integrators vs end users (was Re:
 Language Summit notes)
In-Reply-To: <5351A452.5020106@canterbury.ac.nz>
References: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>
 <5351A452.5020106@canterbury.ac.nz>
Message-ID: <CADiSq7c_g=0dQ2qaGu8MsX7PLbGxdOw2FQt4Saf0tFgYR9PjMQ@mail.gmail.com>

On 18 April 2014 18:16, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Nick Coghlan wrote:
>>
>> there are actually now *two* main ways of consuming
>> Python:
>
>
> Really? We'd better do something about that. We don't want
> anyone consuming Python -- we want some left over for the
> rest of us!
>
> (I'm making a serious point -- it's annoying when people use
> the word "consume" as though it were a synonym for "use".
> It's not.)

I don't consider "use" a synonym for "consume" in this case, as
consuming an upstream project covers redistribution as well. Now, I
could write "use or redistribute" everywhere, but "consume" is
shorter.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Sat Apr 19 00:24:38 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 18 Apr 2014 18:24:38 -0400
Subject: [Python-Dev] Software integrators vs end users (was Re:
 Language Summit notes)
In-Reply-To: <CACac1F8ohmCDR9SUBef=qVAx52hjoKYOWtA7LRaMxdzixwQpHA@mail.gmail.com>
References: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>
 <CACac1F9k_kDN0UspOUs8BUxkbr=tZnLTorkFf2fJGJUfBR9P+g@mail.gmail.com>
 <CADiSq7d1FWdM2g658dPzt+JptEpDc6B_DFnsVVxHPcDCDUECuw@mail.gmail.com>
 <0334A003-E9FF-4C62-B3AF-2E21488955B8@stufft.io>
 <CADiSq7funpGBCOzuye41sJUxBZWA9ZtmqWv=M=Z-+NSNhCsP9w@mail.gmail.com>
 <8068621A-C864-4E16-BE7A-2A358C478EC5@stufft.io>
 <CADiSq7dH4HAOBEYe7KLyGE5OtfZF+BQ=PRD_23b7BR6npZCpQg@mail.gmail.com>
 <8AF38D6F-0E7A-48EB-AB70-F19666E7B372@stufft.io>
 <CADiSq7fbmMNy=Gp2wLc9g7Uxb6t0wXnm6oAUm2m70Ejpmz2=uw@mail.gmail.com>
 <CACac1F_XwUAwv2AsbKUJg7XvLHjxCnBA4dHWkFyDk-u8OkbuKg@mail.gmail.com>
 <BC567E5D-D19C-4737-AADA-AB2C3DCFEEFF@stufft.io>
 <CACac1F8ohmCDR9SUBef=qVAx52hjoKYOWtA7LRaMxdzixwQpHA@mail.gmail.com>
Message-ID: <CADiSq7d32BnpKOB=8r465X7Bdv-VzH6UABHo8t9-+sEamPog5Q@mail.gmail.com>

On 18 April 2014 18:17, Paul Moore <p.f.moore at gmail.com> wrote:
> On 18 April 2014 22:57, Donald Stufft <donald at stufft.io> wrote:
>> Maybe Nick meant ``pip install ipython[all]`` but I don?t actually know what that
>> includes. I?ve never used ipython except for the console.
>
> The hard bit is the QT Console, but that's because there aren't wheels
> for PySide AFAICT.

IPython, matplotlib, scikit-learn, NumPy, nltk, etc. The things that
let you break programming out of the low level box of controlling the
computer, and connect it directly to the more universal high level
task of understanding and visualising the world.

Regards,
Nick.

>
> Paul



-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From donald at stufft.io  Sat Apr 19 00:28:34 2014
From: donald at stufft.io (Donald Stufft)
Date: Fri, 18 Apr 2014 18:28:34 -0400
Subject: [Python-Dev] Software integrators vs end users (was Re:
	Language Summit notes)
In-Reply-To: <CADiSq7d32BnpKOB=8r465X7Bdv-VzH6UABHo8t9-+sEamPog5Q@mail.gmail.com>
References: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>
 <CACac1F9k_kDN0UspOUs8BUxkbr=tZnLTorkFf2fJGJUfBR9P+g@mail.gmail.com>
 <CADiSq7d1FWdM2g658dPzt+JptEpDc6B_DFnsVVxHPcDCDUECuw@mail.gmail.com>
 <0334A003-E9FF-4C62-B3AF-2E21488955B8@stufft.io>
 <CADiSq7funpGBCOzuye41sJUxBZWA9ZtmqWv=M=Z-+NSNhCsP9w@mail.gmail.com>
 <8068621A-C864-4E16-BE7A-2A358C478EC5@stufft.io>
 <CADiSq7dH4HAOBEYe7KLyGE5OtfZF+BQ=PRD_23b7BR6npZCpQg@mail.gmail.com>
 <8AF38D6F-0E7A-48EB-AB70-F19666E7B372@stufft.io>
 <CADiSq7fbmMNy=Gp2wLc9g7Uxb6t0wXnm6oAUm2m70Ejpmz2=uw@mail.gmail.com>
 <CACac1F_XwUAwv2AsbKUJg7XvLHjxCnBA4dHWkFyDk-u8OkbuKg@mail.gmail.com>
 <BC567E5D-D19C-4737-AADA-AB2C3DCFEEFF@stufft.io>
 <CACac1F8ohmCDR9SUBef=qVAx52hjoKYOWtA7LRaMxdzixwQpHA@mail.gmail.com>
 <CADiSq7d32BnpKOB=8r465X7Bdv-VzH6UABHo8t9-+sEamPog5Q@mail.gmail.com>
Message-ID: <54CE2ED8-C2C9-481B-9509-CB5F4A5F52D1@stufft.io>


On Apr 18, 2014, at 6:24 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 18 April 2014 18:17, Paul Moore <p.f.moore at gmail.com> wrote:
>> On 18 April 2014 22:57, Donald Stufft <donald at stufft.io> wrote:
>>> Maybe Nick meant ``pip install ipython[all]`` but I don?t actually know what that
>>> includes. I?ve never used ipython except for the console.
>> 
>> The hard bit is the QT Console, but that's because there aren't wheels
>> for PySide AFAICT.
> 
> IPython, matplotlib, scikit-learn, NumPy, nltk, etc. The things that
> let you break programming out of the low level box of controlling the
> computer, and connect it directly to the more universal high level
> task of understanding and visualising the world.
> 
> Regards,
> Nick.
> 
>> 
>> Paul
> 
> 
> 
> -- 
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

FWIW It?s been David Cournapeau?s opinion (on Twitter at least) that some/all/most
(I?m not sure exactly which) of these can be handled by Wheels (they just aren?t right now!).

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140418/64c8859b/attachment.sig>

From ncoghlan at gmail.com  Sat Apr 19 00:37:07 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 18 Apr 2014 18:37:07 -0400
Subject: [Python-Dev] Software integrators vs end users (was Re:
 Language Summit notes)
In-Reply-To: <54CE2ED8-C2C9-481B-9509-CB5F4A5F52D1@stufft.io>
References: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>
 <CACac1F9k_kDN0UspOUs8BUxkbr=tZnLTorkFf2fJGJUfBR9P+g@mail.gmail.com>
 <CADiSq7d1FWdM2g658dPzt+JptEpDc6B_DFnsVVxHPcDCDUECuw@mail.gmail.com>
 <0334A003-E9FF-4C62-B3AF-2E21488955B8@stufft.io>
 <CADiSq7funpGBCOzuye41sJUxBZWA9ZtmqWv=M=Z-+NSNhCsP9w@mail.gmail.com>
 <8068621A-C864-4E16-BE7A-2A358C478EC5@stufft.io>
 <CADiSq7dH4HAOBEYe7KLyGE5OtfZF+BQ=PRD_23b7BR6npZCpQg@mail.gmail.com>
 <8AF38D6F-0E7A-48EB-AB70-F19666E7B372@stufft.io>
 <CADiSq7fbmMNy=Gp2wLc9g7Uxb6t0wXnm6oAUm2m70Ejpmz2=uw@mail.gmail.com>
 <CACac1F_XwUAwv2AsbKUJg7XvLHjxCnBA4dHWkFyDk-u8OkbuKg@mail.gmail.com>
 <BC567E5D-D19C-4737-AADA-AB2C3DCFEEFF@stufft.io>
 <CACac1F8ohmCDR9SUBef=qVAx52hjoKYOWtA7LRaMxdzixwQpHA@mail.gmail.com>
 <CADiSq7d32BnpKOB=8r465X7Bdv-VzH6UABHo8t9-+sEamPog5Q@mail.gmail.com>
 <54CE2ED8-C2C9-481B-9509-CB5F4A5F52D1@stufft.io>
Message-ID: <CADiSq7fRdZhza1vYsRBaVWbXjbjpzVrOXKja0Xe5t_xVJ2dOig@mail.gmail.com>

On 18 April 2014 18:28, Donald Stufft <donald at stufft.io> wrote:
>
> On Apr 18, 2014, at 6:24 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
>> On 18 April 2014 18:17, Paul Moore <p.f.moore at gmail.com> wrote:
>>> On 18 April 2014 22:57, Donald Stufft <donald at stufft.io> wrote:
>>>> Maybe Nick meant ``pip install ipython[all]`` but I don?t actually know what that
>>>> includes. I?ve never used ipython except for the console.
>>>
>>> The hard bit is the QT Console, but that's because there aren't wheels
>>> for PySide AFAICT.
>>
>> IPython, matplotlib, scikit-learn, NumPy, nltk, etc. The things that
>> let you break programming out of the low level box of controlling the
>> computer, and connect it directly to the more universal high level
>> task of understanding and visualising the world.
>>
>> Regards,
>> Nick.
>>
>>>
>>> Paul
>>
>>
>>
>> --
>> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>
> FWIW It?s been David Cournapeau?s opinion (on Twitter at least) that some/all/most
> (I?m not sure exactly which) of these can be handled by Wheels (they just aren?t right now!).

Yeah, I think they're fixable too. And after thinking through the
implications of recommending a specific sumo distribution, that
actually does seem to be a more straightforward path as a "default
entry point".

I still see merit in working with the conda folks to make it easier
for Windows and Mac OS folks to keep their Python installations up to
date, and for Linux users to stay out of the system Python in a distro
independent manner, but that's a separate discussion from the
python.org download pages one.

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From donald at stufft.io  Sat Apr 19 00:59:36 2014
From: donald at stufft.io (Donald Stufft)
Date: Fri, 18 Apr 2014 18:59:36 -0400
Subject: [Python-Dev] Software integrators vs end users (was Re:
	Language Summit notes)
In-Reply-To: <CADiSq7fRdZhza1vYsRBaVWbXjbjpzVrOXKja0Xe5t_xVJ2dOig@mail.gmail.com>
References: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>
 <CACac1F9k_kDN0UspOUs8BUxkbr=tZnLTorkFf2fJGJUfBR9P+g@mail.gmail.com>
 <CADiSq7d1FWdM2g658dPzt+JptEpDc6B_DFnsVVxHPcDCDUECuw@mail.gmail.com>
 <0334A003-E9FF-4C62-B3AF-2E21488955B8@stufft.io>
 <CADiSq7funpGBCOzuye41sJUxBZWA9ZtmqWv=M=Z-+NSNhCsP9w@mail.gmail.com>
 <8068621A-C864-4E16-BE7A-2A358C478EC5@stufft.io>
 <CADiSq7dH4HAOBEYe7KLyGE5OtfZF+BQ=PRD_23b7BR6npZCpQg@mail.gmail.com>
 <8AF38D6F-0E7A-48EB-AB70-F19666E7B372@stufft.io>
 <CADiSq7fbmMNy=Gp2wLc9g7Uxb6t0wXnm6oAUm2m70Ejpmz2=uw@mail.gmail.com>
 <CACac1F_XwUAwv2AsbKUJg7XvLHjxCnBA4dHWkFyDk-u8OkbuKg@mail.gmail.com>
 <BC567E5D-D19C-4737-AADA-AB2C3DCFEEFF@stufft.io>
 <CACac1F8ohmCDR9SUBef=qVAx52hjoKYOWtA7LRaMxdzixwQpHA@mail.gmail.com>
 <CADiSq7d32BnpKOB=8r465X7Bdv-VzH6UABHo8t9-+sEamPog5Q@mail.gmail.com>
 <54CE2ED8-C2C9-481B-9509-CB5F4A5F52D1@stufft.io>
 <CADiSq7fRdZhza1vYsRBaVWbXjbjpzVrOXKja0Xe5t_xVJ2dOig@mail.gmail.com>
Message-ID: <F28B23F8-A242-42D8-9A70-11321B9311D9@stufft.io>


On Apr 18, 2014, at 6:37 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 18 April 2014 18:28, Donald Stufft <donald at stufft.io> wrote:
>> 
>> On Apr 18, 2014, at 6:24 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> 
>>> On 18 April 2014 18:17, Paul Moore <p.f.moore at gmail.com> wrote:
>>>> On 18 April 2014 22:57, Donald Stufft <donald at stufft.io> wrote:
>>>>> Maybe Nick meant ``pip install ipython[all]`` but I don?t actually know what that
>>>>> includes. I?ve never used ipython except for the console.
>>>> 
>>>> The hard bit is the QT Console, but that's because there aren't wheels
>>>> for PySide AFAICT.
>>> 
>>> IPython, matplotlib, scikit-learn, NumPy, nltk, etc. The things that
>>> let you break programming out of the low level box of controlling the
>>> computer, and connect it directly to the more universal high level
>>> task of understanding and visualising the world.
>>> 
>>> Regards,
>>> Nick.
>>> 
>>>> 
>>>> Paul
>>> 
>>> 
>>> 
>>> --
>>> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>> 
>> FWIW It?s been David Cournapeau?s opinion (on Twitter at least) that some/all/most
>> (I?m not sure exactly which) of these can be handled by Wheels (they just aren?t right now!).
> 
> Yeah, I think they're fixable too. And after thinking through the
> implications of recommending a specific sumo distribution, that
> actually does seem to be a more straightforward path as a "default
> entry point".
> 
> I still see merit in working with the conda folks to make it easier
> for Windows and Mac OS folks to keep their Python installations up to
> date, and for Linux users to stay out of the system Python in a distro
> independent manner, but that's a separate discussion from the
> python.org download pages one.

Sure, and for *nix based ones there?s also pyenv which I personally use
and like :)

> 
> Cheers,
> Nick.
> 
> 
> -- 
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140418/732f60cf/attachment.sig>

From guido at python.org  Sat Apr 19 01:34:50 2014
From: guido at python.org (Guido van Rossum)
Date: Fri, 18 Apr 2014 16:34:50 -0700
Subject: [Python-Dev] PEP 466: Network security enhancements for Python
	2.7.7
In-Reply-To: <CADiSq7caOVWvKLP0wErBrfO5Bx1fR9Kg9nbVBruvJTCW-k_msQ@mail.gmail.com>
References: <CADiSq7caOVWvKLP0wErBrfO5Bx1fR9Kg9nbVBruvJTCW-k_msQ@mail.gmail.com>
Message-ID: <CAP7+vJJVvf8-NG5yqEwmMyU-U20SXzXC+B9BJibSHBi3H9znhg@mail.gmail.com>

Thanks, Nick. I hereby approve this PEP. You can update the status
yourself. Congrats!


On Wed, Apr 16, 2014 at 3:04 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> I've reworded the PEP to make it clear it is now just about
> backporting a specific set of enhancements to 2.7.7, as well as
> switching to updating to new OpenSSL feature releases in the binary
> installers.
>
> The idea of an open ended backport policy is now listed as a rejected
> variant. I believe that change addresses Guido's main remaining
> concern, so I think this version is ready for pronouncement.
>
> Regards,
> Nick.
>
> ========================================
> PEP: 466
> Title: Network Security Enhancements for Python 2.7.7
> Version: $Revision$
> Last-Modified: $Date$
> Author: Nick Coghlan <ncoghlan at gmail.com>,
> Status: Draft
> Type: Informational
> Content-Type: text/x-rst
> Created: 23-Mar-2014
> Post-History: 23-Mar-2014, 24-Mar-2014, 25-Mar-2014, 26-Mar-2014,
> 16-Apr-2014
>
>
> Abstract
> ========
>
> Most CPython tracker issues are classified as errors in behaviour or
> proposed enhancements. Most patches to fix behavioural errors are
> applied to all active maintenance branches.  Enhancement patches are
> restricted to the default branch that becomes the next Python version.
>
> This cadence works reasonably well during Python's normal 18-24 month
> feature release cycle, which is still applicable to the Python 3 series.
> However, the age of the standard library in Python 2 has now reached a
> point
> where it is sufficiently far behind the state of the art in network
> security
> protocols for it to be causing real problems in use cases where upgrading
> to
> Python 3 in the near term may not be feasible.
>
> In recognition of the additional practical considerations that have arisen
> during the 4+ year maintenance cycle for Python 2.7, this PEP allows a
> critical set of network security related features to be backported from
> Python 3.4 to the upcoming Python 2.7.7 maintenance release.
>
> While this PEP does not make any changes to the core development team's
> handling of security-fix-only branches that are no longer in active
> maintenance, it *does* recommend that commercial redistributors providing
> extended support periods for the Python standard library either backport
> these features to their supported versions, or else explicitly disclaim
> support for the use of older versions in roles that involve connecting
> directly to the public internet.
>
>
> New security related features in Python 2.7.7
> =============================================
>
> Under this proposal, the following features will be backported from Python
> 3.4 to the upcoming Python 2.7.7 maintenance release:
>
> * in the ``os`` module:
>
>   * persistent file descriptor for ``os.urandom()``.
>
> * in the ``hmac`` module:
>
>   * constant time comparison function (``hmac.compare_digest()``).
>
> * in the ``hashlib`` module:
>
>   * password hashing function (``hashlib.pbkdf2_hmac()``).
>   * details of hash algorithm availability
> (``hashlib.algorithms_guaranteed``
>     and ``hashlib.algorithms_available``).
>
> * in the ``ssl`` module:
>
>   * this module is almost entirely synchronised with its Python 3
>     counterpart, bringing TLSv1.x settings, SSLContext manipulation, Server
>     Name Indication, access to platform certificate stores, standard
>     library support for peer hostname validation and more to the Python 2
>     series.
>   * the only ``ssl`` module features *not* backported under this policy are
>     the ``ssl.RAND_*`` functions that provide access to OpenSSL's random
>     number generation capabilities - use ``os.urandom()`` instead.
>
> As a general change in maintenance policy, permission is also granted to
> upgrade to newer feature releases of OpenSSL when preparing the binary
> installers for new maintenance releases of Python 2.7.
>
> This PEP does NOT propose a general exception for backporting new features
> to Python 2.7 - every new feature proposed for backporting will still need
> to be justified independently. In particular, it will need to be explained
> why relying on and independently updated backport on the Python Package
> Index
> instead is not an acceptable solution.
>
>
> Backwards compatibility considerations
> ======================================
>
> As in the Python 3 series, the backported ``ssl.create_default_context()``
> API is granted a backwards compatibility exemption that permits the
> protocol, options, cipher and other settings of the created SSL context to
> be updated in maintenance releases to use higher default security settings.
> This allows them to appropriately balance compatibility and security at the
> time of the maintenance release, rather than at the time of the original
> feature release.
>
> This PEP does *not* grant any other exemptions to the usual backwards
> compatibility policy for maintenance releases. Instead, by explicitly
> encouraging the use of feature based checks, it is designed to make it
> easier
> to write more secure cross-version compatible Python software, while still
> limiting the risk of breaking currently working software when upgrading to
> a new Python 2.7 maintenance release.
>
> In all cases where this proposal allows new features to be backported to
> the Python 2.7 release series, it is possible to write cross-version
> compatible code that operates by "feature detection" (for example, checking
> for particular attributes in a module), without needing to explicitly check
> the Python version.
>
> It is then up to library and framework code to provide an appropriate
> warning
> and fallback behaviour if a desired feature is found to be missing. While
> some especially security sensitive software MAY fail outright if a desired
> security feature is unavailable, most software SHOULD instead emit a
> warning
> and continue operating using a slightly degraded security configuration.
>
> The backported APIs allow library and application code to perform the
> following actions after detecting the presence of a relevant
> network security related feature:
>
> * explicitly opt in to more secure settings (to allow the use of enhanced
>   security features in older maintenance releases of Python with less
>   secure default behaviour)
> * explicitly opt in to less secure settings (to allow the use of newer
> Python
>   feature releases in lower security environments)
> * determine the default setting for the feature (this MAY require explicit
>   Python version checks to determine the Python feature release, but DOES
>   NOT require checking for a specific maintenance release)
>
> Security related changes to other modules (such as higher level networking
> libraries and data format processing libraries) will continue to be made
> available as backports and new modules on the Python Package Index, as
> independent distribution remains the preferred approach to handling
> software that must continue to evolve to handle changing development
> requirements independently of the Python 2 standard library. Refer to
> the `Motivation and Rationale`_ section for a review of the characteristics
> that make the secure networking infrastructure worthy of special
> consideration.
>
>
> OpenSSL compatibility
> ---------------------
>
> Under this proposal, OpenSSL may be upgraded to more recent feature
> releases
> in Python 2.7 maintenance releases. On Linux and most other POSIX systems,
> the specific version of OpenSSL used already varies, as CPython dynamically
> links to the system provided OpenSSL library by default.
>
> For the Windows binary installers, the ``_ssl`` and ``_hashlib`` modules
> are
> statically linked with OpenSSL and the associated symbols are not exported.
> Marc-Andre Lemburg indicates that updating to newer OpenSSL releases in the
> ``egenix-pyopenssl`` binaries has not resulted in any reported
> compatibility
> issues [3]_
>
> The Mac OS X binary installers historically followed the same policy as
> other POSIX installations and dynamically linked to the Apple provided
> OpenSSL libraries. However, Apple has now ceased updating these
> cross-platform libraries, instead requiring that even cross-platform
> developers adopt Mac OS X specific interfaces to access up to date security
> infrastructure on their platform. Accordingly, and independently of this
> PEP, the Mac OS X binary installers were already going to be switched to
> statically linker newer versions of OpenSSL [4]_
>
>
> Other Considerations
> ====================
>
> Maintainability
> ---------------
>
> A number of developers, including Alex Gaynor and Donald Stufft, have
> expressed interest in carrying out the feature backports covered by this
> policy, and assisting with any additional maintenance burdens that arise
> in the Python 2 series as a result.
>
> Steve Dower and Brian Curtin have offered to help with the creation of the
> Windows installers, allowing Martin von L?wis the opportunity to step back
> from the task of maintaining the 2.7 Windows installer.
>
> This PEP is primarily about establishing the consensus needed to allow them
> to carry out this work. For other core developers, this policy change
> shouldn't impose any additional effort beyond potentially reviewing the
> resulting patches for those developers specifically interested in the
> affected modules.
>
>
> Security releases
> -----------------
>
> This PEP does not propose any changes to the handling of security
> releases - those will continue to be source only releases that
> include only critical security fixes.
>
> However, the recommendations for library and application developers are
> deliberately designed to accommodate commercial redistributors that choose
> to apply these changes to additional Python release series that are either
> in security fix only mode, or have been declared "end of life" by the core
> development team.
>
> Whether or not redistributors choose to exercise that option will be up
> to the individual redistributor.
>
>
> Integration testing
> -------------------
>
> Third party integration testing services should offer users the ability
> to test against multiple Python 2.7 maintenance releases (at least 2.7.6
> and 2.7.7+), to ensure that libraries, frameworks and applications can
> still
> test their handling of the legacy security infrastructure correctly (either
> failing or degrading gracefully, depending on the security sensitivity of
> the software), even after the features covered in this proposal have been
> backported to the Python 2.7 series.
>
>
> Handling lower security environments with low risk tolerance
> ------------------------------------------------------------
>
> For better or for worse (mostly worse), there are some environments where
> the risk of latent security defects is more tolerated than even a slightly
> increased risk of regressions in maintenance releases. This proposal
> largely
> excludes these environments from consideration where the modules covered by
> the exemption are concerned - this approach is entirely inappropriate for
> software connected to the public internet, and defence in depth security
> principles suggest that it is not appropriate for most private networks
> either.
>
> Downstream redistributors may still choose to cater to such environments,
> but they will need to handle the process of downgrading the security
> related modules and doing the associated regression testing themselves.
> The main CPython continuous integration infrastructure will not cover this
> scenario.
>
>
> Motivation and Rationale
> ========================
>
> The creation of this PEP was prompted primarily by the aging SSL support in
> the Python 2 series. As of March 2014, the Python 2.7 SSL module is
> approaching four years of age, and the SSL support in the still popular
> Python 2.6 release had its feature set locked six years ago.
>
> These are simply too old to provide a foundation that can be recommended
> in good conscience for secure networking software that operates over the
> public internet, especially in an era where it is becoming quite clearly
> evident that advanced persistent security threats are even more widespread
> and more indiscriminate in their targeting than had previously been
> understood. While they represented reasonable security infrastructure in
> their time, the state of the art has moved on, and we need to investigate
> mechanisms for effectively providing more up to date network security
> infrastructure for users that, for whatever reason, are not currently in
> a position to migrate to Python 3.
>
> While the use of the system OpenSSL installation addresses many of these
> concerns on Linux platforms, it doesn't address all of them (in particular,
> it is still difficult for sotware to explicitly require some higher level
> security settings). The standard library support can be bypassed by using a
> third party library like PyOpenSSL or Pycurl, but this still results in a
> security problem, as these can be difficult dependencies to deploy, and
> many
> users will remain unaware that they might want them. Rather than explaining
> to potentially naive users how to obtain and use these libraries, it seems
> better to just fix the included batteries.
>
> In the case of the binary installers for Windows and Mac OS X that are
> published on python.org, the version of OpenSSL used is entirely within
> the control of the Python core development team, but is currently limited
> to OpenSSL maintenance releases for the version initially shipped with the
> corresponding Python feature release.
>
> With increased popularity comes increased responsibility, and this proposal
> aims to acknowledge the fact that Python's popularity and adoption is at a
> sufficiently high level that some of our design and policy decisions have
> significant implications beyond the Python development community.
>
> As one example, the Python 2 ``ssl`` module does not support the Server
> Name Indication standard. While it is possible to obtain SNI support
> by using the third party ``requests`` client library, actually doing so
> currently requires using not only ``requests`` and its embedded
> dependencies,
> but also half a dozen or more additional libraries. The lack of support
> in the Python 2 series thus serves as an impediment to making effective
> use of SNI on servers, as Python 2 clients will frequently fail to handle
> it correctly.
>
> Another more critical example is the lack of SSL hostname matching in the
> Python 2 standard library - it is currently necessary to rely on a third
> party library, such as ``requests`` or ``backports.ssl_match_hostname`` to
> obtain that functionality in Python 2.
>
> The Python 2 series also remains more vulnerable to remote timing attacks
> on security sensitive comparisons than the Python 3 series, as it lacks a
> standard library equivalent to the timing attack resistant
> ``hmac.compare_digest()`` function. While appropriate secure comparison
> functions can be implemented in third party extensions, many users don't
> even consider the issue and use ordinary equality comparisons instead
> - while a standard library solution doesn't automatically fix that problem,
> it *does* make the barrier to resolution much lower once the problem is
> pointed out.
>
> Python 2.7 represents the only long term maintenance release the core
> development team has provided, and it is natural that there will be things
> that worked over a historically shorter maintenance lifespan that don't
> work
> over this longer support period. In the specific case of the problem
> described in this PEP, the simplest available solution is to acknowledge
> that long term maintenance of network security related modules *requires*
> the ability to add new features, even while retaining backwards
> compatibility
> for existing interfaces.
>
> For those familiar with it, it is worth comparing the approach described in
> this PEP with Red Hat's handling of its long term open source support
> commitments: it isn't the RHEL 6.0 release itself that receives 10 years
> worth of support, but the overall RHEL 6 *series*. The individual RHEL 6.x
> point releases within the series then receive a wide variety of new
> features, including security enhancements, all while meeting strict
> backwards compatibility guarantees for existing software. The proposal
> covered in this PEP brings our approach to long term maintenance more into
> line with this precedent - we retain our strict backwards compatibility
> requirements, but make an exception to the restriction against adding new
> features.
>
> To date, downstream redistributors have respected our upstream policy of
> "no new features in Python maintenance releases". This PEP explicitly
> accepts that a more nuanced policy is appropriate in the case of network
> security related features, and the specific change it describes is
> deliberately designed such that it is potentially suitable for Red Hat
> Enterprise Linux and its downstream derivatives.
>
>
> Rejected alternative: just advise developers to migrate to Python 3
> -------------------------------------------------------------------
>
> This alternative represents the status quo. Unfortunately, it has proven
> to be unworkable in practice, as the backwards compatibility implications
> mean that this is a non-trivial migration process for large applications
> and integration projects. While the tools for migration have evolved to
> a point where it is possible to migrate even large applications
> opportunistically and incrementally (rather than all at once) by updating
> code to run in the large common subset of Python 2 and Python 3, using the
> most recent technology often isn't a priority in commercial environments.
>
> Previously, this was considered an acceptable harm, as while it was an
> unfortunate problem for the affected developers to have to face, it was
> seen as an issue between them and their management chain to make the case
> for infrastructure modernisation, and this case would become naturally
> more compelling as the Python 3 series evolved.
>
> However, now that we're fully aware of the impact the limitations of the
> Python 2 standard library may be having on the evolution of internet
> security standards, I no longer believe that it is reasonable to expect
> platform and application developers to resolve all of the latent defects
> in an application's Unicode correctness solely in order to gain access to
> the network security enhancements already available in Python 3.
>
> While Ubuntu (and to some extent Debian as well) are committed to porting
> all
> default system services and scripts to Python 3, and to removing Python 2
> from its default distribution images (but not from its archives), this is
> a mammoth task and won't be completed for the Ubuntu 14.04 LTS release
> (at least for the desktop image - it may be achieved for the mobile and
> server images).
>
> Fedora has even more work to do to migrate, and it will take a non-trivial
> amount of time to migrate the relevant infrastructure components. While
> Red Hat are also actively working to make it easier for users to use more
> recent versions of Python on our stable platforms, it's going to take time
> for those efforts to start having an impact on end users' choice of
> version,
> and any such changes also don't benefit the core platform infrastructure
> that runs in the integrated system Python by necessity.
>
> The OpenStack migration to Python 3 is also still in its infancy, and even
> though that's a project with an extensive and relatively robust automated
> test suite, it's still large enough that it is going to take quite some
> time
> to migrate fully to a Python 2/3 compatible code base.
>
> And that's just three of the highest profile open source projects that
> make heavy use of Python. Given the likely existence of large amounts of
> legacy code that lacks the kind of automated regression test suite needed
> to help support a migration from Python 2 to Python 3, there are likely to
> be many cases where reimplementation (perhaps even in Python 3) proves
> easier than migration. The key point of this PEP is that those situations
> affect more people than just the developers and users of the affected
> application: the existence of clients and servers with outdated network
> security infrastructure becomes something that developers of secure
> networked services need to take into account as part of their security
> design, and that's a problem that inhibits the adoption of better security
> standards.
>
> As Terry Reedy noted, if we try to persist with the status quo, the likely
> outcome is that commercial redistributors will attempt to do something
> like this on behalf of their customers *anyway*, but in a potentially
> inconsistent and ad hoc manner. By drawing the scope definition process
> into the upstream project we are in a better position to influence the
> approach taken to address the situation and to help ensure some consistency
> across redistributors.
>
> The problem is real, so *something* needs to change, and this PEP describes
> my preferred approach to addressing the situation.
>
>
> Rejected alternative: create and release Python 2.8
> ---------------------------------------------------
>
> With sufficient corporate support, it likely *would* be possible to create
> and release Python 2.8 (it's highly unlikely such a project would garner
> enough interest to be achievable with only volunteers). However, this
> wouldn't actually solve the problem, as the aim is to provide a *relatively
> low impact* way to incorporate enhanced security features into integrated
> products and deployments that make use of Python 2.
>
> Upgrading to a new Python feature release would mean both more work for the
> core development team, as well as a more disruptive update that most
> potential end users would likely just skip entirely.
>
> Attempting to create a Python 2.8 release would also bring in suggestions
> to backport many additional features from Python 3 (such as ``tracemalloc``
> and the improved coroutine support), making the migration from Python 2.7
> to this hypothetical 2.8 release even riskier and more disruptive.
>
> This is not a recommended approach, as it would involve substantial
> additional work for a result that is actually less effective in achieving
> the original aim (which is to eliminate the current widespread use of the
> aging network security infrastructure in the Python 2 series).
>
> Furthermore, while I can't make any commitments to actually addressing
> this issue on Red Hat platforms, I *can* categorically rule out the idea
> of a Python 2.8 being of any use to me in even attempting to get it
> addressed.
>
>
> Rejected alternative: distribute the security enhancements via PyPI
> -------------------------------------------------------------------
>
> While this initially appears to be an attractive and easier to manage
> approach, it actually suffers from several significant problems.
>
> Firstly, this is complex, low level, cross-platform code that integrates
> with the underlying operating system across a variety of POSIX platforms
> (including Mac OS X) and Windows. The CPython BuildBot fleet is already set
> up to handle continuous integration in that context, but most of the
> freely available continuous integration services just offer Linux, and
> perhaps paid access to Windows. Those services work reasonably well for
> software that largely runs on the abstraction layers offered by Python and
> other dynamic languages, as well as the more comprehensive abstraction
> offered by the JVM, but won't suffice for the kind of code involved here.
>
> The OpenSSL dependency for the network security support also qualifies as
> the kind of "complex binary dependency" that isn't yet handled well by the
> ``pip`` based software distribution ecosystem. Relying on a third party
> binary dependency also creates potential compatibility problems for ``pip``
> when running on other interpreters like ``PyPy``.
>
> Another practical problem with the idea is the fact that ``pip`` itself
> relies on the ``ssl`` support in the standard library (with some additional
> support from a bundled copy of ``requests``, which in turn bundles
> ``backport.ssl_match_hostname``), and hence would require any replacement
> module to also be bundled within ``pip``. This wouldn't pose any
> insurmountable difficulties (it's just another dependency to vendor), but
> it *would* mean yet another copy of OpenSSL to keep up to date.
>
> This approach also has the same flaw as all other "improve security by
> renaming things" approaches: they completely miss the users who most need
> help, and raise significant barriers against being able to encourage users
> to do the right thing when their infrastructure supports it (since
> "use this other module" is a much higher impact change than "turn on this
> higher security setting"). Deprecating the aging SSL infrastructure in the
> standard library in favour of an external module would be even more user
> hostile than accepting the slightly increased risk of regressions
> associated
> with upgrading it in place.
>
> Last, but certainly not least, this approach suffers from the same problem
> as the idea of doing a Python 2.8 release: likely not solving the actual
> problem. Commercial redistributors of Python are set up to redistribute
> *Python*, and a pre-existing set of additional packages. Getting new
> packages added to the pre-existing set *can* be done, but means approaching
> each and every redistributor and asking them to update their
> repackaging process accordingly. By contrast, the approach described in
> this PEP would require redistributors to deliberately *opt out* of the
> security enhancements by deliberately downgrading the provided network
> security infrastructure, which most of them are unlikely to do.
>
>
> Rejected variant: provide a "legacy SSL infrastructure" branch
> --------------------------------------------------------------
>
> Earlier versions of this PEP included the concept of a ``2.7-legacy-ssl``
> branch that preserved the exact feature set of the Python 2.7.6 network
> security infrastructure.
>
> In my opinion, anyone that actually wants this is almost certainly making a
> mistake, and if they insist they really do want it in their specific
> situation, they're welcome to either make it themselves or arrange for a
> downstream redistributor to make it for them.
>
> If they are made publicly available, any such rebuilds should be referred
> to
> as "Python 2.7 with Legacy SSL" to clearly distinguish them from the
> official
> Python 2.7 releases that include more up to date network security
> infrastructure.
>
> After the first Python 2.7 maintenance release that implements this PEP, it
> would also be appropriate to refer to Python 2.7.6 and earlier releases as
> "Python 2.7 with Legacy SSL".
>
>
> Rejected variant: synchronise particular modules entirely with Python 3
> -----------------------------------------------------------------------
>
> Earlier versions of this PEP suggested synchronising the ``hmac``,
> ``hashlib`` and ``ssl`` modules entirely with their Python 3 counterparts.
>
> This approach proved too vague to build a compelling case for the
> exception,
> and has thus been replaced by the current more explicit proposal.
>
>
> Rejected variant: open ended backport policy
> --------------------------------------------
>
> Earlier versions of this PEP suggested a general policy change related to
> future Python 3 enhancements that impact the general security of the
> internet.
>
> That approach created unnecessary uncertainty, so it has been simplified to
> propose backport a specific concrete set of changes. Future feature
> backport proposals can refer back to this PEP as precedent, but it will
> still be necessary to make a specific case for each feature addition to
> the Python 2.7 long term support release.
>
>
> Disclosure of Interest
> ======================
>
> The author of this PEP currently works for Red Hat on test automation
> tools.
> If this proposal is accepted, I will be strongly encouraging Red Hat to
> take
> advantage of the resulting opportunity to help improve the overall security
> of the Python ecosystem. However, I do not speak for Red Hat in this
> matter,
> and cannot make any commitments on Red Hat's behalf.
>
>
> Acknowledgements
> ================
>
> Thanks to Christian Heimes and other for their efforts in greatly improving
> Python's SSL support in the Python 3 series, and a variety of members of
> the Python community for helping me to better understand the implications
> of the default settings we provide in our SSL modules, and the impact that
> tolerating the use of SSL infrastructure that was defined in 2010
> (Python 2.7) or even 2008 (Python 2.6) potentially has for the security
> of the web as a whole.
>
> Thanks to Donald Stufft and Alex Gaynor for identifying a more limited set
> of essential security features that allowed the proposal to be made more
> fine-grained than backporting entire modules from Python 3.4 [7,8]_.
>
> Christian and Donald also provided valuable feedback on a preliminary
> draft of this proposal.
>
> Thanks also to participants in the python-dev mailing list threads
> [1,2,5,6]_, as well as the various folks I discussed this issue with at
> PyCon 2014 in Montreal.
>
>
> References
> ==========
>
> .. [1] PEP 466 discussion (round 1)
>    (https://mail.python.org/pipermail/python-dev/2014-March/133334.html)
>
> .. [2] PEP 466 discussion (round 2)
>    (https://mail.python.org/pipermail/python-dev/2014-March/133389.html)
>
> .. [3] Marc-Andre Lemburg's OpenSSL feedback for Windows
>    (https://mail.python.org/pipermail/python-dev/2014-March/133438.html)
>
> .. [4] Ned Deily's OpenSSL feedback for Mac OS X
>    (https://mail.python.org/pipermail/python-dev/2014-March/133347.html)
>
> .. [5] PEP 466 discussion (round 3)
>    (https://mail.python.org/pipermail/python-dev/2014-March/133442.html)
>
> .. [6] PEP 466 discussion (round 4)
>    (https://mail.python.org/pipermail/python-dev/2014-March/133472.html)
>
> .. [7] Donald Stufft's recommended set of backported features
>    (https://mail.python.org/pipermail/python-dev/2014-March/133500.html)
>
> .. [8] Alex Gaynor's recommended set of backported features
>    (https://mail.python.org/pipermail/python-dev/2014-March/133503.html)
>
>
>
> Copyright
> =========
>
> This document has been placed in the public domain.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140418/d12d80e3/attachment-0001.html>

From steve at pearwood.info  Sat Apr 19 02:30:16 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 19 Apr 2014 10:30:16 +1000
Subject: [Python-Dev] dict and required hashing
In-Reply-To: <1397858275.17357.108033801.64457FB7@webmail.messagingengine.com>
References: <CA+OGgf4Asw_VZ-jTtVxVfhygBfc5nZggnKJoS_wyyR1Jm2Wr6g@mail.gmail.com>
 <1397858275.17357.108033801.64457FB7@webmail.messagingengine.com>
Message-ID: <20140419003016.GG28400@ando>

On Fri, Apr 18, 2014 at 02:57:55PM -0700, Benjamin Peterson wrote:
> On Fri, Apr 18, 2014, at 14:46, Jim J. Jewett wrote:
> > (1)  I believe the recent consensus was that the number of comparisons
> > made in a dict lookup is an implementation detail.  (Please correct me
> > if I am wrong.)
> 
> Absolutely.
> 
> > 
> > (2)  Is "the item will be hashed at least once" a language guarantee?
> 
> No. (Would that be useful at all?)

I'm curious under what circumstances a key won't be hashed at all.


-- 
Steven

From benjamin at python.org  Sat Apr 19 02:39:49 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 18 Apr 2014 17:39:49 -0700
Subject: [Python-Dev] dict and required hashing
In-Reply-To: <20140419003016.GG28400@ando>
References: <CA+OGgf4Asw_VZ-jTtVxVfhygBfc5nZggnKJoS_wyyR1Jm2Wr6g@mail.gmail.com>
 <1397858275.17357.108033801.64457FB7@webmail.messagingengine.com>
 <20140419003016.GG28400@ando>
Message-ID: <1397867989.15072.108061465.2C96A6FA@webmail.messagingengine.com>

On Fri, Apr 18, 2014, at 17:30, Steven D'Aprano wrote:
> On Fri, Apr 18, 2014 at 02:57:55PM -0700, Benjamin Peterson wrote:
> > On Fri, Apr 18, 2014, at 14:46, Jim J. Jewett wrote:
> > > (1)  I believe the recent consensus was that the number of comparisons
> > > made in a dict lookup is an implementation detail.  (Please correct me
> > > if I am wrong.)
> > 
> > Absolutely.
> > 
> > > 
> > > (2)  Is "the item will be hashed at least once" a language guarantee?
> > 
> > No. (Would that be useful at all?)
> 
> I'm curious under what circumstances a key won't be hashed at all.

It's probably always hashed now, but as Jim suggested, it might happen
in an optimization. For example, if you try to do a lookup on an empty
dict, you definitely don't need to hash anything.

From bcannon at gmail.com  Sat Apr 19 02:57:31 2014
From: bcannon at gmail.com (Dr. Brett Cannon)
Date: Sat, 19 Apr 2014 00:57:31 +0000
Subject: [Python-Dev] devguide: Add note about Kushal Das' privs
References: <3g72kV00hjz7LjZ@mail.python.org> <20140418202923.46f20653@fsol>
 <1397846097.5285.107984529.11878B72@webmail.messagingengine.com>
 <CAP1=2W4J8fFndzkjAsQzzr59kPF5eu9rL0qGLCBK4CUsaE7xUw@mail.gmail.com>
 <CAKJDb-M2MOXwzX-uyGZ0V8A8hniXat0hp3p5fvVVJVzLAaQQ9Q@mail.gmail.com>
Message-ID: <CAP1=2W7E7+emCQWLmyVHZu7Vf2BP9sogZD0WS6MX2fYmWBVHPA@mail.gmail.com>

On Friday, April 18, 2014 3:10:54 PM, Zachary Ware <
zachary.ware+pydev at gmail.com> wrote:

On Fri, Apr 18, 2014 at 1:54 PM, Brett Cannon <bcannon@ <bcannon at gmail.com>
gmail.com <bcannon at gmail.com>> wrote:
> On Friday, April 18, 2014 2:35:32 PM, Benjamin Peterson
> <benjamin <benjamin at python.org>@ <benjamin at python.org>python.org<benjamin at python.org>>
wrote:
>> On Fri, Apr 18, 2014, at 11:29, Antoine Pitrou wrote:
>> > On Mon, 14 Apr 2014 23:18:42 +0200 (CEST)
>> > brett.cannon <python- <python-checkins at python.org>checkins<python-checkins at python.org>
@ <python-checkins at python.org>python.org <python-checkins at python.org>>
wrote:
>> > > http:// <http://hg.python.org/devguide/rev/c14c8a195fec>hg.python.org<http://hg.python.org/devguide/rev/c14c8a195fec>
/ <http://hg.python.org/devguide/rev/c14c8a195fec>devguide<http://hg.python.org/devguide/rev/c14c8a195fec>
/rev/c14c8a195fec <http://hg.python.org/devguide/rev/c14c8a195fec>
>> > > changeset:   686:c14c8a195fec
>> > > user:        Brett Cannon <brett@ <brett at python.org>python.org<brett at python.org>
>
>> > > date:        Mon Apr 14 17:18:37 2014 -0400
>> > > summary:
>> > >   Add note about Kushal Das' privs
>> >
>> > I have no objection to Kushal getting commit rights, but is there a
>> > public record of the discussion leading to this decision? I can't find
>> > anything on either python-dev or python-committers (perhaps my search
>> > skills are failing me).
>>
>> Probably in person conversion at PyCon sprints
>
> What Benjamin said; discussions at PyCon amongst several of us.

Looking at that page, it doesn't look like I was ever added.  Brett,
was it you that enabled me



I don't remember. It's somewhat annoying keeping the log updated since it's
a separate repo from the ssh keys.

-brett


Also, welcome Kushal!

--
Zach
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140419/0b868d00/attachment.html>

From ncoghlan at gmail.com  Sat Apr 19 02:59:05 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 18 Apr 2014 20:59:05 -0400
Subject: [Python-Dev] PEP 466: Network security enhancements for Python
	2.7.7
In-Reply-To: <CAP7+vJJVvf8-NG5yqEwmMyU-U20SXzXC+B9BJibSHBi3H9znhg@mail.gmail.com>
References: <CADiSq7caOVWvKLP0wErBrfO5Bx1fR9Kg9nbVBruvJTCW-k_msQ@mail.gmail.com>
 <CAP7+vJJVvf8-NG5yqEwmMyU-U20SXzXC+B9BJibSHBi3H9znhg@mail.gmail.com>
Message-ID: <CADiSq7f0B8idrcR8v=SD_3kZFKjEdZrkNDOYazhZ2qNY6dtQmQ@mail.gmail.com>

On 18 April 2014 19:34, Guido van Rossum <guido at python.org> wrote:
> Thanks, Nick. I hereby approve this PEP. You can update the status yourself.
> Congrats!

Thanks! PEP status updated: http://hg.python.org/peps/rev/d905b6f9c6a9

I also tweaked the type, since it's just an ordinary Standards Track
PEP now, rather than an ongoing policy PEP.

I filed separate tracker issues for the four parts of the proposal:

os.urandom: http://bugs.python.org/issue21305
hmac: http://bugs.python.org/issue21306
hashlib: http://bugs.python.org/issue21307
ssl:http://bugs.python.org/issue21308

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Sat Apr 19 04:31:29 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 18 Apr 2014 22:31:29 -0400
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods
Message-ID: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>

After spending some time talking to the folks at the PyCon Twisted
sprints, they persuaded me that adding back the iterkeys/values/items
methods for mapping objects would be a nice way to eliminate a key
porting hassle for them (and likely others), without significantly
increasing the complexity of Python 3.

I personally put this one in the same category as PEP 414 - not
particularly useful from a Python 3 perspective, but not really
harmful either, and helpful enough from a transition perspective to be
worth doing.

Regards,
Nick.

==============================================
PEP: 469
Title: Simplified migration of iterator-based mapping code to Python 3
Version: $Revision$
Last-Modified: $Date$
Author: Nick Coghlan <ncoghlan at gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 2014-04-18
Python-Version: 3.5
Post-History: 2014-04-18


Abstract
========

For Python 3, PEP 3106 changed the design of the ``dict`` builtin and the
mapping API in general to replace the separate list based and iterator based
APIs in Python 2 with a merged, memory efficient set and multiset view
based API.

This means that Python 3 code always requires an additional qualifier to
reliably reproduce classic Python 2 mapping semantics:

    * List based (e.g. ``d.keys()``): ``list(d.keys())``
    * Iterator based (e.g. ``d.iterkeys()``): ``iter(d.keys())``

Some Python 2 code that uses ``d.keys()`` may be migrated to Python 3
(or the common subset of Python 2 and Python 3) without alteration, but
*all* code using the iterator based API requires modification. Code that
is migrating to the common subset of Python 2 and 3 and needs to retain the
memory efficient implementation that avoids creating an unnecessary list
object must switch away from using a method to instead using a helper
function (such as those provided by the ``six`` module)

To simplify the process of migrating Python 2 code that uses the existing
iterator based APIs to Python 3, this PEP proposes the reintroduction
of the Python 2 spelling of the iterator based semantics in Python 3.5, by
restoring the following methods to the builtin ``dict`` API and the
``collections.abc.Mapping`` ABC definition:

    * ``iterkeys()``
    * ``itervalues()``
    * ``iteritems()``


Proposal
========

Methods with the following exact semantics will be added to the builtin
``dict`` type and ``collections.abc.Mapping`` ABC::

    def iterkeys(self):
        return iter(self.keys())

    def itervalues(self):
        return iter(self.values())

    def iteritems(self):
        return iter(self.items())

These semantics ensure that the methods also work as expected for subclasses
of these base types.


Rationale
=========

Similar in spirit to PEP 414 (which restored explicit Unicode literal
support in Python 3.3), this PEP is aimed primarily at helping users
that currently feel punished for making use of a feature that needed to be
requested explicitly in Python 2, but was effectively made the default
behaviour in Python 3.

Users of list-based iteration in Python 2 that aren't actually relying on
those semantics get a free memory efficiency improvement when migrating to
Python 3, and face no additional difficulties when migrating via the common
subset of Python 2 and 3.

By contrast, users that actually want the increased efficiency may have
faced a three phase migration process by the time they have fully migrated
to Python 3:

* original migration to the iterator based APIs after they were added in
  Python 2.2
* migration to a separate function based API in order to run in the common
  subset of Python 2 and 3
* eventual migration back to unprefixed method APIs when finally dropping
  Python 2.7 support at some point in the future

The view based APIs that were added to Python 2.7 don't actually help with
the transition process, as they don't exist in Python 3 and hence aren't
part of the common subset of Python 2 and Python 3, and also aren't supported
by most Python 2 mappings (including the collection ABCs).

This PEP proposes to just eliminate all that annoyance by making the iterator
based APIs work again in Python 3.5+. As with the restoration of Unicode
literals, it does add a bit of additional noise to the definition of Python
3, but it does so while bringing a significant benefit in increasing the size
of the common subset of Python 2 and Python 3 and so simplifying the process
of migrating to Python 3 for affected Python 2 users.


Acknowledgements
================

Thanks to the folks at the Twisted sprint table at PyCon for a very
vigorous discussion of this idea (and several other topics), and especially
to Hynek Schlawack for acting as a moderator when things got a little too
heated :)


Copyright
=========

This document has been placed in the public domain.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From victor.stinner at gmail.com  Sat Apr 19 05:05:15 2014
From: victor.stinner at gmail.com (Victor Stinner)
Date: Fri, 18 Apr 2014 23:05:15 -0400
Subject: [Python-Dev] dict and required hashing
In-Reply-To: <CA+OGgf4Asw_VZ-jTtVxVfhygBfc5nZggnKJoS_wyyR1Jm2Wr6g@mail.gmail.com>
References: <CA+OGgf4Asw_VZ-jTtVxVfhygBfc5nZggnKJoS_wyyR1Jm2Wr6g@mail.gmail.com>
Message-ID: <CAMpsgwbP7VjxZ_rzz7gderjYNCidrbbjb79cdGCuCUcpGy+FeQ@mail.gmail.com>

Does it mean that depending of the number of items, keys can be mutable? It
sounds like a terrible idea.

Victor

Le vendredi 18 avril 2014, Jim J. Jewett <jimjjewett at gmail.com> a ?crit :

> (1)  I believe the recent consensus was that the number of comparisons
> made in a dict lookup is an implementation detail.  (Please correct me
> if I am wrong.)
>
> (2)  Is "the item will be hashed at least once" a language guarantee?
>
> For small mappings, it might well be more efficient to just store the
> 2-3 key/value pairs and skip the bucket calculation.
>
> On the other hand, if a key is not hashable, discovering that long
> after it has already been added to the dict is suboptimal.
>
> Of course, that sort of delayed exception can already happen if it is
> the __eq__ method that is messed up ...
>
> -jJ
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org <javascript:;>
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140418/6e9cb4ac/attachment.html>

From benjamin at python.org  Sat Apr 19 05:08:13 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 18 Apr 2014 20:08:13 -0700
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
Message-ID: <1397876893.11662.108082729.0AF51881@webmail.messagingengine.com>

On Fri, Apr 18, 2014, at 19:31, Nick Coghlan wrote:
> After spending some time talking to the folks at the PyCon Twisted
> sprints, they persuaded me that adding back the iterkeys/values/items
> methods for mapping objects would be a nice way to eliminate a key
> porting hassle for them (and likely others), without significantly
> increasing the complexity of Python 3.
> 
> I personally put this one in the same category as PEP 414 - not
> particularly useful from a Python 3 perspective, but not really
> harmful either, and helpful enough from a transition perspective to be
> worth doing.

It doesn't seem to be widely known that Python 2.7's dict has
viewkeys()/viewvalues()/viewitems() methods which implement Python 3
dictionary views. Thus, an alternate (or concurrent) proposal could be
add these aliases to the Python 3 dictionary type. At any rate, the PEP
should mention these methods' existence.

From benjamin at python.org  Sat Apr 19 05:09:02 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 18 Apr 2014 20:09:02 -0700
Subject: [Python-Dev] dict and required hashing
In-Reply-To: <CAMpsgwbP7VjxZ_rzz7gderjYNCidrbbjb79cdGCuCUcpGy+FeQ@mail.gmail.com>
References: <CA+OGgf4Asw_VZ-jTtVxVfhygBfc5nZggnKJoS_wyyR1Jm2Wr6g@mail.gmail.com>
 <CAMpsgwbP7VjxZ_rzz7gderjYNCidrbbjb79cdGCuCUcpGy+FeQ@mail.gmail.com>
Message-ID: <1397876942.11846.108083013.2D105366@webmail.messagingengine.com>

On Fri, Apr 18, 2014, at 20:05, Victor Stinner wrote:
> Does it mean that depending of the number of items, keys can be mutable?
> It
> sounds like a terrible idea.

I believe Jim is talking about internal implementation.

From steve at pearwood.info  Sat Apr 19 06:25:53 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 19 Apr 2014 14:25:53 +1000
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
Message-ID: <20140419042553.GJ28400@ando>

On Fri, Apr 18, 2014 at 10:31:29PM -0400, Nick Coghlan wrote:
> After spending some time talking to the folks at the PyCon Twisted
> sprints, they persuaded me that adding back the iterkeys/values/items
> methods for mapping objects would be a nice way to eliminate a key
> porting hassle for them (and likely others), without significantly
> increasing the complexity of Python 3.

It would also considerable add to the cruft of Python 3. One motive for 
going through the pain of Python 2 to 3 migration was to remove cruft. 
Adding it back in again is not just an aid to porting but actively 
making Python 3 a worse (well, "less better") experience.

In the case of u'' syntax for strings, that was (somewhat) unavoidable, 
as you get a syntax error in Python 3 otherwise. So you cannot write:

    if sys.version < "3":
        s = u"???"
    else:
        s = "???"

it simply doesn't work. But you can write:

    if sys.version < "3":
        items = mydict.iteritems()  # or viewitems
    else:
        items = mydict.items()

Feature discovery is better than explicit version checks:

    try:
        items = mydict.iteritems()
    except AttributeError:
        items = mydict.items()


In my experience, writing polyglot 2+3 code can be easily handled with a 
few helper functions, which is not the case with unicode string 
literals. (Non-polygot code of course can just use the methods 
directly.) I don't see this is a problem to be solved, or even much of a 
nuisance. Polyglot code is never going to be quite as straightforward or 
idiomic as non-polyglot code, and that's a good thing, as it reminds the 
reader that they are dealing with polyglot code.

So while I'm sympathetic to wanting to ease the 2/3 transition, even at 
the expense of re-introducing some cruft when unavoidable, I don't think 
that the difficulty of dealing with dict items|iteritems|viewitems etc. 
justifies re-adding cruft.


[...] 
> Rationale
> =========
> 
> Similar in spirit to PEP 414 (which restored explicit Unicode literal
> support in Python 3.3), this PEP is aimed primarily at helping users
> that currently feel punished for making use of a feature that needed to be
> requested explicitly in Python 2, but was effectively made the default
> behaviour in Python 3.

"Feel punished"? That's awfully strong language. It may even be true, in 
the sense that some people *feel* that they are being punished, but I 
think the barrier to doing something about that needs to be a bit 
higher, namely that they *actually are* being punished.

I think that if "write a helper function" is punishment, then nearly 
every programmer is being punished *all the time*. That's part of 
programming. And if every 2+3 helper is seen as punishment that needs to 
be reversed, then we'll end up with Python 3.5 or 3.6 being virtually 
the same as Python 2.7 only with a few extra features.


> Users of list-based iteration in Python 2 that aren't actually relying on
> those semantics get a free memory efficiency improvement when migrating to
> Python 3, and face no additional difficulties when migrating via the common
> subset of Python 2 and 3.
> 
> By contrast, users that actually want the increased efficiency may have
> faced a three phase migration process by the time they have fully migrated
> to Python 3:
> 
> * original migration to the iterator based APIs after they were added in
>   Python 2.2

That was a long time ago. Surely we're not counting new features 
introduced a decade ago as part of the cost of migrating to Python 3?


> * migration to a separate function based API in order to run in the common
>   subset of Python 2 and 3
> * eventual migration back to unprefixed method APIs when finally dropping
>   Python 2.7 support at some point in the future

I'm not actually seeing the problem here. Forget 2+3, what you've 
described is surely part of the process of dealing with nearly any 
multi-version code. Back when I was still writing code to support Python 
2.3 through 2.5, I had helper functions like this:

def sort(alist, key=None):
    if key is None:
        alist.sort()
    else:
        try:
            alist.sort(key=key)
        except TypeError:
            # Python too old. Use the DSU idiom.
            ...

When I dropped support for 2.3, I eventually (but not at first) moved 
back from sort(mylist) to mylist.sort(). I don't see that the situation 
with iterkeys etc. is terribly different.


> This PEP proposes to just eliminate all that annoyance by making the iterator
> based APIs work again in Python 3.5+. As with the restoration of Unicode
> literals, it does add a bit of additional noise to the definition of Python
> 3, but it does so while bringing a significant benefit in increasing the size
> of the common subset of Python 2 and Python 3 and so simplifying the process
> of migrating to Python 3 for affected Python 2 users.

To me, this feels more like re-adding cruft for trivial benefit. I would 
want to see justification for why this gives *significant* benefit 
before I could support it. As the PEP stands now, I'm -1. I might be 
persuaded to change to 0 if the dict.iter* methods were added and 
immediately deprecated, rather than sold as a positive feature to be 
added to the Mapping API.



-- 
Steven

From tjreedy at udel.edu  Sat Apr 19 06:35:30 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 19 Apr 2014 00:35:30 -0400
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
Message-ID: <lisufb$h9f$1@ger.gmane.org>

On 4/18/2014 10:31 PM, Nick Coghlan wrote:
> After spending some time talking to the folks at the PyCon Twisted
> sprints, they persuaded me that adding back the iterkeys/values/items
> methods for mapping objects would be a nice way to eliminate a key
> porting hassle for them (and likely others), without significantly
> increasing the complexity of Python 3.

I hate this idea. It strikes me as junking up Python3 with stuff it is 
well rid of. I think anything that can be left to the transition modules 
should be. The u'' syntax had to be in the language itself. This does 
not have to be.

> I personally put this one in the same category as PEP 414 -

When I suggested that PEP 414 might be seen as a precedent for restoring 
more of Py2, I was trashed for saying so. "No, no, u'' is a unique case. 
[it is] This will be the last proposal like this." What will come next?

 > not
> particularly useful from a Python 3 perspective, but not really
> harmful either,

It makes the language a bit harder to learn and remember and slightly 
more confusing.

It will not help inter-operating with Python before 3.5, at the earliest 
and cannot be backported. Most things in an independent module can be 
used with any 3.x.

I would have preferred that you started by presenting the problem on 
python-ideas with possible solutions, rather than as a finished PEP 
pushing my least favorite solution.

-- 
Terry Jan Reedy



From cournape at gmail.com  Sat Apr 19 10:36:23 2014
From: cournape at gmail.com (David Cournapeau)
Date: Sat, 19 Apr 2014 09:36:23 +0100
Subject: [Python-Dev] Software integrators vs end users (was Re:
 Language Summit notes)
In-Reply-To: <54CE2ED8-C2C9-481B-9509-CB5F4A5F52D1@stufft.io>
References: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>
 <CACac1F9k_kDN0UspOUs8BUxkbr=tZnLTorkFf2fJGJUfBR9P+g@mail.gmail.com>
 <CADiSq7d1FWdM2g658dPzt+JptEpDc6B_DFnsVVxHPcDCDUECuw@mail.gmail.com>
 <0334A003-E9FF-4C62-B3AF-2E21488955B8@stufft.io>
 <CADiSq7funpGBCOzuye41sJUxBZWA9ZtmqWv=M=Z-+NSNhCsP9w@mail.gmail.com>
 <8068621A-C864-4E16-BE7A-2A358C478EC5@stufft.io>
 <CADiSq7dH4HAOBEYe7KLyGE5OtfZF+BQ=PRD_23b7BR6npZCpQg@mail.gmail.com>
 <8AF38D6F-0E7A-48EB-AB70-F19666E7B372@stufft.io>
 <CADiSq7fbmMNy=Gp2wLc9g7Uxb6t0wXnm6oAUm2m70Ejpmz2=uw@mail.gmail.com>
 <CACac1F_XwUAwv2AsbKUJg7XvLHjxCnBA4dHWkFyDk-u8OkbuKg@mail.gmail.com>
 <BC567E5D-D19C-4737-AADA-AB2C3DCFEEFF@stufft.io>
 <CACac1F8ohmCDR9SUBef=qVAx52hjoKYOWtA7LRaMxdzixwQpHA@mail.gmail.com>
 <CADiSq7d32BnpKOB=8r465X7Bdv-VzH6UABHo8t9-+sEamPog5Q@mail.gmail.com>
 <54CE2ED8-C2C9-481B-9509-CB5F4A5F52D1@stufft.io>
Message-ID: <CAGY4rcXhFTtAMKZc79+W6q06t6g6um0HUFCQOuvMU0dRxin_8Q@mail.gmail.com>

On Fri, Apr 18, 2014 at 11:28 PM, Donald Stufft <donald at stufft.io> wrote:

>
> On Apr 18, 2014, at 6:24 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
> > On 18 April 2014 18:17, Paul Moore <p.f.moore at gmail.com> wrote:
> >> On 18 April 2014 22:57, Donald Stufft <donald at stufft.io> wrote:
> >>> Maybe Nick meant ``pip install ipython[all]`` but I don?t actually
> know what that
> >>> includes. I?ve never used ipython except for the console.
> >>
> >> The hard bit is the QT Console, but that's because there aren't wheels
> >> for PySide AFAICT.
> >
> > IPython, matplotlib, scikit-learn, NumPy, nltk, etc. The things that
> > let you break programming out of the low level box of controlling the
> > computer, and connect it directly to the more universal high level
> > task of understanding and visualising the world.
> >
> > Regards,
> > Nick.
> >
> >>
> >> Paul
> >
> >
> >
> > --
> > Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>
> FWIW It?s been David Cournapeau?s opinion (on Twitter at least) that
> some/all/most
> (I?m not sure exactly which) of these can be handled by Wheels (they just
> aren?t right now!).
>

Indeed, and the scipy community has been working on making wheels for new
releases. The details of the format does not matter as much as having one
format: at Enthought, we have been using the egg format for years to deploy
python, C/C++ libraries and other assets, but we would have been using
wheels if it existed at that time. Adding features like pre remove/post
install to wheels would be great, but that's a relatively simpler
discussion.

I agree with your sentiment that the main value of sumo distributions like
anaconda, active python or our own canopy is the binary packaging + making
sure it all works together. There will always be some limitations in making
those sumo distributions work seamlessly with 'standard' python, but those
are pretty much the same issues as e.g. linux integrators have.

If the python packaging efforts help the linux distribution integration, it
is very likely to help us too (us == sumo distributions builders) too.

David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140419/3abf1771/attachment-0001.html>

From arigo at tunes.org  Sat Apr 19 10:43:44 2014
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 19 Apr 2014 10:43:44 +0200
Subject: [Python-Dev] dict and required hashing
In-Reply-To: <CA+OGgf4Asw_VZ-jTtVxVfhygBfc5nZggnKJoS_wyyR1Jm2Wr6g@mail.gmail.com>
References: <CA+OGgf4Asw_VZ-jTtVxVfhygBfc5nZggnKJoS_wyyR1Jm2Wr6g@mail.gmail.com>
Message-ID: <CAMSv6X2cYa3jhoTfKaXkdiMNxR8FGHykh4geM8QJibixVpQWrw@mail.gmail.com>

Hi Jim,

On 18 April 2014 23:46, Jim J. Jewett <jimjjewett at gmail.com> wrote:
> (2)  Is "the item will be hashed at least once" a language guarantee?

I think that a reasonable implementation needs to hash at least once
all keys that are added to the dictionary.  Otherwise we end up, as
you said, with a dictionary that contains non-hashable keys: this
crashes when it grows and tries to convert from a list storage to a
hash table.  This is against the expectations and probably against the
language spec...

It's a bit unclear to me if the language spec requires "if [ ] in
mydict:" to raise TypeError, as the key doesn't get added to the dict
in this case.  Similarly for "[ ] in { }" where the dictionary is
empty anyway.  I would say yes, erring on the side of caution...


A bient?t,

Armin.

From solipsis at pitrou.net  Sat Apr 19 11:35:37 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 19 Apr 2014 11:35:37 +0200
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
Message-ID: <20140419113537.436ce991@fsol>

On Fri, 18 Apr 2014 22:31:29 -0400
Nick Coghlan <ncoghlan at gmail.com> wrote:
> After spending some time talking to the folks at the PyCon Twisted
> sprints, they persuaded me that adding back the iterkeys/values/items
> methods for mapping objects would be a nice way to eliminate a key
> porting hassle for them (and likely others), without significantly
> increasing the complexity of Python 3.

I'm -1 on this. This is destroying the simplification effort of the
dict API in Python 3.

Regards

Antoine.



From solipsis at pitrou.net  Sat Apr 19 11:32:52 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 19 Apr 2014 11:32:52 +0200
Subject: [Python-Dev] Software integrators vs end users (was Re:
 Language Summit notes)
References: <CADiSq7fSHSR5VQYwZDS+BoNFf19YtOEP0i6ahEjQNy1ED94taw@mail.gmail.com>
Message-ID: <20140419113252.1a2356a6@fsol>

On Fri, 18 Apr 2014 11:58:59 -0400
Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> Software integrators:
> 
> * Linux distributions and other operating system vendors
> * Sumo redistributions (commercial or otherwise)
> * "Python based environments" (PTVS, Enthought Canopy, wakari.io,
> Python Anywhere, etc)
> * Software-as-a-Service developers
> * Device manufacturers
> * PC OEMs
> * creators of corporate "Standard Operating Environment" definitions
> * System integrators (IBM, Boeing et al)
> * Application developers (from simple CLI tools to OpenStack)

I don't understand this. Why are SaaS developpers or application
developpers (oh, why are they separate, by the way? :-)) "software
integrators" rather than "end users"? While Linux distributions, OS
vendors and the like will build their own Python from source, normal
developers will usually rely on an already existing packaging of Python
(an installer, or a distribution-provided package).

> For end users, Python is likely consumed as *part of something else*.

Well, even for some developers. Consider someone writing a Web
application with Django (*): are they interested in Django because of
Python, or in Python because of Django? How can you reliably make the
difference?

(*) or a video game with Ren'Py

Regards

Antoine.



From g.rodola at gmail.com  Sat Apr 19 14:12:24 2014
From: g.rodola at gmail.com (Giampaolo Rodola')
Date: Sat, 19 Apr 2014 14:12:24 +0200
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <20140419113537.436ce991@fsol>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <20140419113537.436ce991@fsol>
Message-ID: <CAFYqXL-THn4DXZYk3z-b_vH6_u3-Zf_aJYwUvGc5j4_AQJBSUg@mail.gmail.com>

On Sat, Apr 19, 2014 at 4:31 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> After spending some time talking to the folks at the PyCon Twisted
> sprints, they persuaded me that adding back the iterkeys/values/items
> methods for mapping objects would be a nice way to eliminate a key
> porting hassle for them (and likely others), without significantly
> increasing the complexity of Python 3.
>

I don't see this as a key porting hassle *at all* and I don't understand
why they think this would significantly help their porting (it wouldn't).
The only real barrier is the str/bytes conversion, really, and this is even
more true for projects massively centered around IO, such as Twisted and,
I'm sure, the main (only?) reason why Twisted hasn't been ported yet. They
will get much more benefit from additions such as PEP-461, which is of
great help for verbose protocols such as FTP, not this.

-- 
Giampaolo - http://grodola.blogspot.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140419/712c9a62/attachment.html>

From kristjan at ccpgames.com  Sat Apr 19 13:41:35 2014
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Sat, 19 Apr 2014 11:41:35 +0000
Subject: [Python-Dev] PEP 469: Restoring the
	iterkeys/values/items()	methods
In-Reply-To: <20140419113537.436ce991@fsol>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <20140419113537.436ce991@fsol>
Message-ID: <EFE3877620384242A686D52278B7CCD3A52D9701@RKV-IT-EXCH104.ccp.ad.local>

Wouldn't "iterkeys" simply be an alias for "keys" and so on?
I'm +1 on that.
It is a signigificant portion of the incompatibility, and seems like such a minor concession to compatibility to make.
K

-----Original Message-----
From: Python-Dev [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Antoine Pitrou
Sent: 19. apr?l 2014 09:36
To: python-dev at python.org
Subject: Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

On Fri, 18 Apr 2014 22:31:29 -0400
Nick Coghlan <ncoghlan at gmail.com> wrote:
> After spending some time talking to the folks at the PyCon Twisted 
> sprints, they persuaded me that adding back the iterkeys/values/items 
> methods for mapping objects would be a nice way to eliminate a key 
> porting hassle for them (and likely others), without significantly 
> increasing the complexity of Python 3.

I'm -1 on this. This is destroying the simplification effort of the dict API in Python 3.

Regards

Antoine.


_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com

From solipsis at pitrou.net  Sat Apr 19 15:04:14 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 19 Apr 2014 15:04:14 +0200
Subject: [Python-Dev] PEP 469: Restoring the
	iterkeys/values/items()	methods
In-Reply-To: <EFE3877620384242A686D52278B7CCD3A52D9701@RKV-IT-EXCH104.ccp.ad.local>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <20140419113537.436ce991@fsol>
 <EFE3877620384242A686D52278B7CCD3A52D9701@RKV-IT-EXCH104.ccp.ad.local>
Message-ID: <20140419150414.3cfea356@fsol>

On Sat, 19 Apr 2014 11:41:35 +0000
Kristj?n Valur J?nsson <kristjan at ccpgames.com> wrote:
> Wouldn't "iterkeys" simply be an alias for "keys" and so on?

The PEP shows the following semantics:

    def iterkeys(self):
        return iter(self.keys())

    def itervalues(self):
        return iter(self.values())

    def iteritems(self):
        return iter(self.items())

Regards

Antoine.

From kristjan at ccpgames.com  Sat Apr 19 14:51:02 2014
From: kristjan at ccpgames.com (=?utf-8?B?S3Jpc3Rqw6FuIFZhbHVyIErDs25zc29u?=)
Date: Sat, 19 Apr 2014 12:51:02 +0000
Subject: [Python-Dev] Language Summit notes
In-Reply-To: <CADiSq7cnPt4Z1gYybXfezHiDqyPZF7yaBZdYmoiofKiWsw32JQ@mail.gmail.com>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
 <EFE3877620384242A686D52278B7CCD3A52D8EB0@RKV-IT-EXCH104.ccp.ad.local>
 <CADiSq7cnPt4Z1gYybXfezHiDqyPZF7yaBZdYmoiofKiWsw32JQ@mail.gmail.com>
Message-ID: <EFE3877620384242A686D52278B7CCD3A52D9768@RKV-IT-EXCH104.ccp.ad.local>



> -----Original Message-----
> From: Nick Coghlan [mailto:ncoghlan at gmail.com]
> 
> > 2.       Feature enhancement to 2.8.  Take a robust and popular version of
> > python and add some of the language goodies that have been added to
> > 3.x and that don?t have an inherent 3.x aspect.  Yield from.  New exception
> model.
> > Stdlib enhancements such as futures.   The argument goes like this:  We
> have
> > a very popular platform out there with lots of momentum.  People want
> > incremental enhancements to it.  Why not give them what they want?
> > Bread and games and all that?  A Rockband cannot stay cooped up in a
> > studio producing experimental concept albums all the time.  That is death.
> > Sometimes it needs to go on tour and play old hits for the fans!
> 
> Do you know how much work a new Python 2.x release creates for people?
> All the redistributors have to update, books get outdated, a new wrinkle gets
> added to the compatibility matrix for everyone. A new Python 2.x release is
> simple untenable at this point in the transition
The key word here, "transition".  I?m not sure that everyone wants to transit.
This may be the core of the issue.   I know that this has been hashed to death before.  Never the less the pressure is there, I think and it would, I predict, be a crowd-pleaser.

> - it's a *massively* expensive way to achieve things that can be achieved
> more cheaply in other ways.
More cheaply from "our" point of view, perhaps.
> 
> Take yield from, for example. Hy is able to compile *LISP* syntax to Python
> AST structures. PEP 380 includes a semantic expansion of yield from in terms
> of yield. Is it *really* impossible to get "yield from"
> based code running in Python 2.6? Or have people just assumed it's not
> possible and never even tried, because the idea of using import hooks to
> backport syntax to earlier feature releases is too novel?

Some things are better done as language features than as complicated reverse hacks.
You keep saying that this and that can be done with add-on modules.  I think you underestimate the practical and psychological barrier towards those things.  Every new dependency on a package In some package directory is a new complication in a project.  It is something new you have t get, something requiing yet another virtualenv directory, something unblessed.
Another factor is the simply sheer size of Pypi by now.  How do you find things?  How do you even guess that things like "yield from" might be available as a package to pip install?  I know that this can be improved and tat there is work in process in doing that but the PyPi is still not core python and there is a gap that must be bridged for a user to start looking for solutions there.

> > 3.5 features
> >
> > When asked what should we aim for in 3.5, there were mostly some very
> > minor incremental changes suggested, IIRC.  In my opinion, the reason
> > 3.x has not caught on is that there is no real carrot there.  There is
> > no new vision, no killer feature.  Nothing that a programmer sees and
> > makes him say ?Yeah! I want to program my next project using this feature,
> it will be super!?.
> 
> I *really* wish folks from North America, Europe and other regions where 8-
> bit encodings can handle their native language and where Anglicisation of
> terms to fit them into the ASCII identifier restriction poses no barrier to
> communication would stop trotting out this "no killer feature in Python 3"
> canard.
I intentionally didn't mention this because it is like the GIL.  It is a technical feature of the language, a refinement if you will.  But not a new thing in terms of language evolution.  Look, I have no disregard for the importance of this, myself coming from a non-ascii language.  I also worked on the internationalization of EVE, many years ago, and am familiar with the annoyance of implicit Unicode conversions.   I work in a company that uses unicode in all its products, new and old.  The other day a new stand alone service was developed.  I suggested to the developer that he might want to use Python 3 because it is unicode through and through.  He just shrugged, said it wasn't an issue.  He'll store the relevant database tables as unicode, get unicode out of the stuff, encode unicode in json and everything will just work.  While I'm not saying that the new model is not better (I think it is) it does come with some baggage, particularly in how it has been more cumbersome to work with bytes.  But anyway, this is why I didn"t mention unicode and why I don't count it as a killer feature.

 While it is *possible* to write internationalised and localised
> applications in it, Python 2's Unicode support is so broken that some people
> can't even run the interpreter from their home directory because it can't
> cope with their username. 
Years ago we implemented fixes to that for python 2.5.  core dev wasn't interested :).  

> If anyone is *ever* tempted to utter the words "Python 3 has no killer
> feature" without immediately following it up with the "for me"
> qualifier, please go read this post about the creation of a Portuguese version
> of Stack Overflow:
> http://blog.stackoverflow.com/2014/02/cant-we-all-be-reasonable-and-
> speak-english/
Look, unicode is important.  Just as having an automatic choke in a car is important.  Do you remember manual choke levers in cars?  I?m sure the Ford Falcon had one.  Removing the choke is a refinement.  But not a quantum leap.

> 
> Late last year, Alex Gaynor made the insightful observation that we write
> software either for the users we have or for the users we want.
> Python 3 is the software we're writing for the users we want. Python
> 2.7 long term maintenance, PyPy, standard library backports, transition
> libraries and tools, packaging ecosystem improvements, Python 3 features
> designed primarily to easy migration from Python 2, PyPI modules that run
> on both Python 2 & 3 - that's the software we and many other people are
> writing for the users we have.
> 
> >
> > 1.       Code blocks as a core language construct.  Re-implement context
> > managers as block executors.  We shouldn?t let details such as syntax
> > questions distract us.   That?s like saying that we can?t  eat spaghetti
> > because our Italian is so poor.
> 
> When it comes to blocks, the syntax isn't a distraction - it's the fundamental
> framing of what the construct is and how it should be used. How does a block
> get used to define a sorting key function? How does it get broadcast over a
> NumPy array? How does it get distributed to multiple threads, processes or
> greenlets, or passed to an IO scheduler? How does it differ from an ordinary
> closure? How does it relate to loops and other constructs?
> 
> There is a large existing constellation of concepts that blocks need to be
> fitted into, and any successful proposal for adding blocks to the language will
> require giving that process the respect and attention it deserves.

You're getting bogged down in implementation details, letting side issues distract you from the core goal.  We could for example start with a vertical slice.  Enable code blocks for context managers.  See how we solve the syntax (already done) and explore the possibilities that it gives us.  Then look at sorting functions.  One step at a time.  I'm sure we'll find a way, unless we are already convinced that it is impossible. 
But anyway, I mentioned code blocks because it is such an obvious thing to explore and has been an ongoing chestnut on python-dev for eons.


> This won't happen without a full usability study, but I've been speaking to
> Fernando Perez and Greg Wilson about some vague concepts for doing
> exactly that. I hope to be able to explore that possibility further at SciPy in
> July - Greg laments the lack of "evidence based language design" and I think
> the strong scientific presence in the Python community provides us with a
> genuine opportunity to move in that direction when it comes to controversial
> changes like code blocks.
The literature is full of them and there are languages out there that use them.  This isn't even pioneering CS :(

> 
> If people are genuinely interested in building and supporting the Python
> *community*, stop thinking about tinkering with the *language*.
This is python-dev.  It is about the *language*.  CPython is the reference implementation.
It is where most of that are passionate about the language operate. I'm trying to instill a passion for boldness here, to not be happy with letting the evolution of python die some beurocratic death.  I'm sure we can come up with ways of making Python continue to innovate, while still maintaining all this *community*.

> So folks looking to core development as the engine room of Python's growth
> are *looking in the wrong place*. We laid the foundation (and will continue
> to sustain and evolve it), but the real growth drivers have now moved
> further out to things like the work the PyPA is doing to make upstream
> packaging easier to consume, the work Travis Oliphant and others are doing
> to make the scientific Python stack easier to consume, the work Fernando
> Perez is coordinating around IPython and IPython notebook, the work Titus
> Brown et al are doing around reproducable science, the work Greg Wilson
> and Mozilla Science Labs are doing around Software Carpentry, the work Van
> Lindberg and others are putting into opening up the PSF, the work Jesse
> Noller has been driving around opening up python.org to more contributors,
> the work Jessica McKellar and more are doing around getting Python into
> high school curricula in the US, the work the Raspberry Pi Foundation are
> doing in the UK, the work around the core mentorship program, GSoC and
> the Gnome Outreach Program for Women, the work to build the regional
> network of Python conferences (I'm told we're currently running at a rate of
> something like 2 new PyCons per week!), the work folks like me, Guido,
> Steve Dower, Jesse and more are doing to better engage with corporate
> users and inviting them to more directly support the sustainability of the
> upstream Python community (on *our* terms), the work PyLadies is doing
> around education and outreach, the work Steve Holden is now doing around
> improving the ready availability of Python training courses, and on, and on,
> and on.
This is all super work, and laudable.  But I don't understand your point.  Because all this python related work is being done, does that mean that we, as core python developers, should not continue to innovate on the language? 
People have different interests and aspirations and everyone should be able to find a slot in our great community where he feels he can contribute at his best.
I have already mentioned C# above.  This is a language probably as big as Python.  Yet with every revision of it, it seems to adopt previously experimental language features into the language core, making it even more powerful.  At the same time, it is also being used by lots of people.  These need not be mutually exclusive goals, mass marked popularity and language innovation.

It's like Country and Western:  We can have both kinds.  Let's do it.  Let's have fun!

K


From steve at pearwood.info  Sat Apr 19 15:14:56 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 19 Apr 2014 23:14:56 +1000
Subject: [Python-Dev] PEP 469: Restoring the
	iterkeys/values/items()	methods
In-Reply-To: <EFE3877620384242A686D52278B7CCD3A52D9701@RKV-IT-EXCH104.ccp.ad.local>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <20140419113537.436ce991@fsol>
 <EFE3877620384242A686D52278B7CCD3A52D9701@RKV-IT-EXCH104.ccp.ad.local>
Message-ID: <20140419131456.GK28400@ando>

On Sat, Apr 19, 2014 at 11:41:35AM +0000, Kristj?n Valur J?nsson wrote:
> Wouldn't "iterkeys" simply be an alias for "keys" and so on?
> I'm +1 on that.

No.

[steve at ando ~]$ python2.7 -c "it = {}.iterkeys(); print it is iter(it)"
True
[steve at ando ~]$ python3.3 -c "it = {}.keys(); print(it is iter(it))"
False


> It is a signigificant portion of the incompatibility, and seems like 
> such a minor concession to compatibility to make.

I don't think it is a significant portion of incompatibility. Or at 
least, I think that the Twisted folks (or Nick, if he wants to speak for 
them) have to justify why it's significant.



-- 
Steven

From ncoghlan at gmail.com  Sat Apr 19 16:38:39 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 19 Apr 2014 10:38:39 -0400
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <20140419042553.GJ28400@ando>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <20140419042553.GJ28400@ando>
Message-ID: <CADiSq7dQyXMcBY-fhUgB1+6204+wwvBW1=uWvyze9V-ZCbQy-A@mail.gmail.com>

On 19 Apr 2014 00:27, "Steven D'Aprano" <steve at pearwood.info> wrote:
>
> On Fri, Apr 18, 2014 at 10:31:29PM -0400, Nick Coghlan wrote:
> > After spending some time talking to the folks at the PyCon Twisted
> > sprints, they persuaded me that adding back the iterkeys/values/items
> > methods for mapping objects would be a nice way to eliminate a key
> > porting hassle for them (and likely others), without significantly
> > increasing the complexity of Python 3.
>
> It would also considerable add to the cruft of Python 3. One motive for
> going through the pain of Python 2 to 3 migration was to remove cruft.
> Adding it back in again is not just an aid to porting but actively
> making Python 3 a worse (well, "less better") experience.

The cruft is unavoidable in this case. The decision we face is *where the
cruft lives*, and *how much work is involved* in creating that cruft.

Status quo: we preserve the "purity" of the Python 3 mapping API, and
require every developer making the transition from Python 2 to Python 3
replace every occurrence of these methods with a helper function that is
not idiomatic code in either language.

My proposal: we add three trivial helper methods to the Python 3 mapping
API. For the cost of 3 additional methods that are easily explained in
terms of combining a builtin with other existing methods, a whole pile of
work just evaporates for affected segments of the community.

>
> So while I'm sympathetic to wanting to ease the 2/3 transition, even at
> the expense of re-introducing some cruft when unavoidable, I don't think
> that the difficulty of dealing with dict items|iteritems|viewitems etc.
> justifies re-adding cruft.

The cruft will be there regardless, the only question is whether it exists
in the definition of Python 3 or is distributed across all of the single
source projects supporting both 2 & 3.

> [...]
> > Rationale
> > =========
> >
> > Similar in spirit to PEP 414 (which restored explicit Unicode literal
> > support in Python 3.3), this PEP is aimed primarily at helping users
> > that currently feel punished for making use of a feature that needed to
be
> > requested explicitly in Python 2, but was effectively made the default
> > behaviour in Python 3.
>
> "Feel punished"? That's awfully strong language. It may even be true, in
> the sense that some people *feel* that they are being punished, but I
> think the barrier to doing something about that needs to be a bit
> higher, namely that they *actually are* being punished.

Yes, they are *actually* being punished for using the memory efficient APIs
that were being added in Python 2.2. That's why the PEP covers how the
transition to Python 3 is harder for them than if they just hadn't cared
about memory efficiency in the first place.

They're Pythonistas too - they care about readability, and they *don't
like* having to add crufty helper functions to all their mapping
manipulation code. They also legitimately don't want to put up with all the
code churn that results from doing so, requiring additional reviews and
retesting of currently working code.

I spend a fair bit of time talking to users that have put a lot of work
into supporting a language transition that doesn't really help them
personally. A non-trivial number of them are deeply, viscerally angry about
what they see as pointlessly changing the spelling of a core language
feature for no real technical gain.

So let me be clear: this is *not* a proposal driven primarily by technical
considerations. Rather, it is a social one, where we do something simple
and easy and low cost for us to send a clear message to a set of users that
feel justifiably angry about the amount of work we imposed on other members
of the community by putting Python 2 into maintenance mode that we do value
their time and energy, and are willing to show some flexibility in adding
"not harmful" changes to Python 3 that don't significantly increase the
complexity of the language, while making it easier to write single source
code that looks like idiomatic Python code.

> I think that if "write a helper function" is punishment, then nearly
> every programmer is being punished *all the time*.

No, the punishment is "replace every usage of the memory efficient iterator
based APIs that have existed since Python 2.2 with a helper function
because we think that is a reasonable burden to place on you just so we can
avoid adding three trivial helper methods in Python 3".

Regards,
Nick.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140419/d1a21805/attachment.html>

From ncoghlan at gmail.com  Sat Apr 19 16:44:36 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 19 Apr 2014 10:44:36 -0400
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <1397876893.11662.108082729.0AF51881@webmail.messagingengine.com>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <1397876893.11662.108082729.0AF51881@webmail.messagingengine.com>
Message-ID: <CADiSq7dP9bSi+zWR+nXXCX5ySveh3AqyeZpGGGN1_USLB9nt2w@mail.gmail.com>

On 18 Apr 2014 23:08, "Benjamin Peterson" <benjamin at python.org> wrote:
>
> On Fri, Apr 18, 2014, at 19:31, Nick Coghlan wrote:
> > After spending some time talking to the folks at the PyCon Twisted
> > sprints, they persuaded me that adding back the iterkeys/values/items
> > methods for mapping objects would be a nice way to eliminate a key
> > porting hassle for them (and likely others), without significantly
> > increasing the complexity of Python 3.
> >
> > I personally put this one in the same category as PEP 414 - not
> > particularly useful from a Python 3 perspective, but not really
> > harmful either, and helpful enough from a transition perspective to be
> > worth doing.
>
> It doesn't seem to be widely known that Python 2.7's dict has
> viewkeys()/viewvalues()/viewitems() methods which implement Python 3
> dictionary views. Thus, an alternate (or concurrent) proposal could be
> add these aliases to the Python 3 dictionary type. At any rate, the PEP
> should mention these methods' existence.

It does:
=============
The view based APIs that were added to Python 2.7 don't actually help with
the transition process, as they don't exist in Python 3 and hence aren't
part of the common subset of Python 2 and Python 3, and also aren't
supported
by most Python 2 mappings (including the collection ABCs).
=============

I should be more explicit that the other reason they don't really help is
because most potential single source code dates back further than 2.7, so
it's the iterator based APIs that are needed to avoid code churn when
migrating to single source.

Cheers,
Nick.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140419/ce043aa0/attachment-0001.html>

From solipsis at pitrou.net  Sat Apr 19 16:44:57 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 19 Apr 2014 16:44:57 +0200
Subject: [Python-Dev] PEP 466 (round 2): Network security enhancements
	for Python 2.7
References: <CADiSq7eRRN7NXTvgHXh0ADXj0s7YcscDYkPX7w2HiJDZGyGw9Q@mail.gmail.com>
 <532FF67A.3010206@egenix.com>
Message-ID: <20140419164457.08c7b614@fsol>

On Mon, 24 Mar 2014 10:10:18 +0100
"M.-A. Lemburg" <mal at egenix.com> wrote:
> 
> The OpenSSL version used for 2.7.6 is 0.9.8y.
> 
> Upgrading to 1.0.0 or 1.0.1 will likely need a few minor tweaks, but
> not cause general breakage - at least that's my experience with
> the egenix-pyopenssl distribution.

For the record, if we had done that a few months ago, the breakage would
have been called Heartbleed.

Regards

Antoine.



From solipsis at pitrou.net  Sat Apr 19 16:47:10 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 19 Apr 2014 16:47:10 +0200
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <1397876893.11662.108082729.0AF51881@webmail.messagingengine.com>
 <CADiSq7dP9bSi+zWR+nXXCX5ySveh3AqyeZpGGGN1_USLB9nt2w@mail.gmail.com>
Message-ID: <20140419164710.3ae3d386@fsol>

On Sat, 19 Apr 2014 10:44:36 -0400
Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> I should be more explicit that the other reason they don't really help is
> because most potential single source code dates back further than 2.7, so
> it's the iterator based APIs that are needed to avoid code churn when
> migrating to single source.

Why don't you just suggest adding convenience functions to six (or
another porting library)?

Regards

Antoine.



From guido at python.org  Sat Apr 19 16:52:55 2014
From: guido at python.org (Guido van Rossum)
Date: Sat, 19 Apr 2014 07:52:55 -0700
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <CADiSq7dP9bSi+zWR+nXXCX5ySveh3AqyeZpGGGN1_USLB9nt2w@mail.gmail.com>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <1397876893.11662.108082729.0AF51881@webmail.messagingengine.com>
 <CADiSq7dP9bSi+zWR+nXXCX5ySveh3AqyeZpGGGN1_USLB9nt2w@mail.gmail.com>
Message-ID: <CAP7+vJJsxBbXcVSHUvvQZWYgXmotNw+2qpGxs4FcDtHi3WWF4w@mail.gmail.com>

Does everyone involved know that "for x in d.iterkeys()" is equivalent to
"for x in d" and works the same in Python 2 and 3? Similarly, "list(d)" is
a simple, fast way to spell the Python 2 semantics of "d.keys()"  that
works in both versions (but I doubt it is much needed -- usually the actual
code follows up with sorting, so you should use sorted(d)).

This doesn't solve itervalues() and iteritems() but I expect those are less
common, and "for x, y in d.iteritems(): <blah>" is rewritten nicely as

  for x in d:
    y = d[x]
    <blah>

If there is a measurable slowdown in the latter I would be totally okay
with some kind of one-element cache for the most recent lookup.

I get the social aspect of the PEP, but I think it's too high a price to
pay.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140419/dec2082b/attachment.html>

From chrism at plope.com  Sat Apr 19 17:29:38 2014
From: chrism at plope.com (Chris McDonough)
Date: Sat, 19 Apr 2014 11:29:38 -0400
Subject: [Python-Dev] PEP 469: Restoring the
	iterkeys/values/items()	methods
In-Reply-To: <EFE3877620384242A686D52278B7CCD3A52D9701@RKV-IT-EXCH104.ccp.ad.local>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <20140419113537.436ce991@fsol>
 <EFE3877620384242A686D52278B7CCD3A52D9701@RKV-IT-EXCH104.ccp.ad.local>
Message-ID: <53529662.4040106@plope.com>

On 04/19/2014 07:41 AM, Kristj?n Valur J?nsson wrote:
> Wouldn't "iterkeys" simply be an alias for "keys" and so on?
> I'm +1 on that.
> It is a signigificant portion of the incompatibility, and seems like such a minor concession to compatibility to make.
> K

FWIW, I'm +1 on this and other minor changes and concessions to bw 
compat like this improve the subset language that 2/3 straddling 
projects need to use.

- C


From njs at pobox.com  Sat Apr 19 14:36:50 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Sat, 19 Apr 2014 13:36:50 +0100
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
Message-ID: <CAPJVwBmYD5ZDNvZ5bSEgJZ1GFrbqdphwt3GFN2=ut3JD7Ludug@mail.gmail.com>

On Sat, Apr 19, 2014 at 3:31 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Some Python 2 code that uses ``d.keys()`` may be migrated to Python 3
> (or the common subset of Python 2 and Python 3) without alteration, but
> *all* code using the iterator based API requires modification. Code that
> is migrating to the common subset of Python 2 and 3 and needs to retain the
> memory efficient implementation that avoids creating an unnecessary list
> object must switch away from using a method to instead using a helper
> function (such as those provided by the ``six`` module)

I don't know enough about the issues to have an opinion on the
proposal as a whole, but the foo.iterkeys() -> six.iterkeys(foo)
transformation strikes me as exactly the kind of change that can be
easily and accurately automated (as in modernize etc.). I assume Glyph
et al have considered this option -- do you know why it was rejected?

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

From ncoghlan at gmail.com  Sat Apr 19 18:17:14 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 19 Apr 2014 12:17:14 -0400
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <CAP7+vJJsxBbXcVSHUvvQZWYgXmotNw+2qpGxs4FcDtHi3WWF4w@mail.gmail.com>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <1397876893.11662.108082729.0AF51881@webmail.messagingengine.com>
 <CADiSq7dP9bSi+zWR+nXXCX5ySveh3AqyeZpGGGN1_USLB9nt2w@mail.gmail.com>
 <CAP7+vJJsxBbXcVSHUvvQZWYgXmotNw+2qpGxs4FcDtHi3WWF4w@mail.gmail.com>
Message-ID: <CADiSq7d=cQdB=51y-1LqBA6Aaamu=OeQn7xsxeCWV7YYMO5WiQ@mail.gmail.com>

On 19 Apr 2014 10:53, "Guido van Rossum" <guido at python.org> wrote:
>
> Does everyone involved know that "for x in d.iterkeys()" is equivalent to
"for x in d" and works the same in Python 2 and 3? Similarly, "list(d)" is
a simple, fast way to spell the Python 2 semantics of "d.keys()"  that
works in both versions (but I doubt it is much needed -- usually the actual
code follows up with sorting, so you should use sorted(d)).
>
> This doesn't solve itervalues() and iteritems() but I expect those are
less common, and "for x, y in d.iteritems(): <blah>" is rewritten nicely as
>
>   for x in d:
>     y = d[x]
>     <blah>
>
> If there is a measurable slowdown in the latter I would be totally okay
with some kind of one-element cache for the most recent lookup.
>
> I get the social aspect of the PEP, but I think it's too high a price to
pay.

OK, I think the main thing I need to add to the PEP next is a clear
description of the current state of the art in translating these methods to
the common 2/3 subset across the following modes of interaction with the
existing Python 2 API:

* for loops
* comprehensions and generator expressions
* iterator object
* bound methods
* unbound methods

That may help clarify the tricky warts and edge cases that can arise when
moving from the current relatively straightforward and consistent method
based approach to a more complicated combination of dedicated syntax and
helper functions.

I also asked JP Calderone to trawl the Twisted code base for specific cases
where the status quo causes migration problems. Since my main argument is
that we should do this to save collective modification and review effort
for affected projects (i.e. I think it's a "death by 1000 cuts" situation
rather than a single gaping wound anyone can point to), it would be good if
those affected could help with quantifying the scale of the problem so we
can make a more informed trade-off between that work and future users of
Python 3 needing to learn about the existence of the proposed largely
redundant compatibility methods.

In several ways, I see my proposal as similar to what we did when PEP
8'ifying the threading.Thread API - the old camel case functions are still
there, but clearly marked as only existing for legacy compatibility reasons
(they may actually be completely undocumented in Py3 - I don't recall off
the top of my head). While I'm personally a big fan of cleaning up APIs and
strongly encouraging code modernisation, I've also become convinced that it
will be worth our while to start soliciting more quantitative feedback in
relation to various decisions to help make sure we're OK with the full
consequences of those design decisions.

Regards,
Nick.

>
> --
> --Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140419/d0344cb1/attachment.html>

From ezio.melotti at gmail.com  Sat Apr 19 18:28:14 2014
From: ezio.melotti at gmail.com (Ezio Melotti)
Date: Sat, 19 Apr 2014 19:28:14 +0300
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <20140419131456.GK28400@ando>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <20140419113537.436ce991@fsol>
 <EFE3877620384242A686D52278B7CCD3A52D9701@RKV-IT-EXCH104.ccp.ad.local>
 <20140419131456.GK28400@ando>
Message-ID: <CACBhJdFqALY0zdmsGMO5L-WVkvcfumMY2NdG8QM8oEMLrBNcig@mail.gmail.com>

Hi,

On Sat, Apr 19, 2014 at 4:14 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Sat, Apr 19, 2014 at 11:41:35AM +0000, Kristj?n Valur J?nsson wrote:
>> It is a signigificant portion of the incompatibility, and seems like
>> such a minor concession to compatibility to make.
>
> I don't think it is a significant portion of incompatibility. Or at
> least, I think that the Twisted folks (or Nick, if he wants to speak for
> them) have to justify why it's significant.
>

Assuming this gets included in 3.5 (which will be released around the
end of 2015), are they planning to disregard all the previous 3.x
releases and then wait a couple more years (so 2018+) for 3.5 to
become common?
Are they going to support 3.3+ only (with u'...') and have extra cruft
for 3.3/3.4 to deal with the missing iter* methods and then remove the
cruft once 3.5 is the oldest 3.x release that is supported?
What happens if this addition will still not push people to move their
code to 3.x and similar requests are made for 3.6+ (and shift what I
just said for another 18 months)?

Best Regards,
Ezio Melotti

>
>
> --
> Steven

From stephen at xemacs.org  Sat Apr 19 18:30:20 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 20 Apr 2014 01:30:20 +0900
Subject: [Python-Dev] PEP 469: Restoring the
	iterkeys/values/items()	methods
In-Reply-To: <CAP7+vJJsxBbXcVSHUvvQZWYgXmotNw+2qpGxs4FcDtHi3WWF4w@mail.gmail.com>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <1397876893.11662.108082729.0AF51881@webmail.messagingengine.com>
 <CADiSq7dP9bSi+zWR+nXXCX5ySveh3AqyeZpGGGN1_USLB9nt2w@mail.gmail.com>
 <CAP7+vJJsxBbXcVSHUvvQZWYgXmotNw+2qpGxs4FcDtHi3WWF4w@mail.gmail.com>
Message-ID: <874n1ps2n7.fsf@uwakimon.sk.tsukuba.ac.jp>

Guido van Rossum writes:

 > Does everyone involved know that "for x in d.iterkeys()" is
 > equivalent to "for x in d" and works the same in Python 2 and 3? 
[...]

 > This doesn't solve itervalues() and iteritems() but I expect those
 > are less common, and "for x, y in d.iteritems(): <blah>" is
 > rewritten nicely as
 >
 >   for x in d:
 >     y = d[x]
 >    <blah>

I suppose there's no way to get the compiler to both make "for x in d"
work as above, and make "for k, v in d" be equivalent to Python 2's
"for k, v in d.iteritems()"?  It seems totally analogous to getting
both "for x in list" and "for x, y in list_of_couples" to DTRT.  (To
me, anyway.)

You'd still be stuck on itervalues(), but at least you'd have the
option of "for _, v in d" (ie, the usual idiom for a value you're
going to ignore) without creating a list.





From benjamin at python.org  Sat Apr 19 18:33:27 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 19 Apr 2014 09:33:27 -0700
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <874n1ps2n7.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <1397876893.11662.108082729.0AF51881@webmail.messagingengine.com>
 <CADiSq7dP9bSi+zWR+nXXCX5ySveh3AqyeZpGGGN1_USLB9nt2w@mail.gmail.com>
 <CAP7+vJJsxBbXcVSHUvvQZWYgXmotNw+2qpGxs4FcDtHi3WWF4w@mail.gmail.com>
 <874n1ps2n7.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <1397925207.26188.108197637.229A3326@webmail.messagingengine.com>



On Sat, Apr 19, 2014, at 9:30, Stephen J. Turnbull wrote:
> Guido van Rossum writes:
> 
>  > Does everyone involved know that "for x in d.iterkeys()" is
>  > equivalent to "for x in d" and works the same in Python 2 and 3? 
> [...]
> 
>  > This doesn't solve itervalues() and iteritems() but I expect those
>  > are less common, and "for x, y in d.iteritems(): <blah>" is
>  > rewritten nicely as
>  >
>  >   for x in d:
>  >     y = d[x]
>  >    <blah>
> 
> I suppose there's no way to get the compiler to both make "for x in d"
> work as above, and make "for k, v in d" be equivalent to Python 2's
> "for k, v in d.iteritems()"?  It seems totally analogous to getting
> both "for x in list" and "for x, y in list_of_couples" to DTRT.  (To
> me, anyway.)

That doesn't make sense. What if your keys are tuples?

From guido at python.org  Sat Apr 19 18:35:38 2014
From: guido at python.org (Guido van Rossum)
Date: Sat, 19 Apr 2014 09:35:38 -0700
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <CADiSq7d=cQdB=51y-1LqBA6Aaamu=OeQn7xsxeCWV7YYMO5WiQ@mail.gmail.com>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <1397876893.11662.108082729.0AF51881@webmail.messagingengine.com>
 <CADiSq7dP9bSi+zWR+nXXCX5ySveh3AqyeZpGGGN1_USLB9nt2w@mail.gmail.com>
 <CAP7+vJJsxBbXcVSHUvvQZWYgXmotNw+2qpGxs4FcDtHi3WWF4w@mail.gmail.com>
 <CADiSq7d=cQdB=51y-1LqBA6Aaamu=OeQn7xsxeCWV7YYMO5WiQ@mail.gmail.com>
Message-ID: <CAP7+vJLcpp2JXXZyQEhLv97nXTkPGoE0dFbLwOiczhQ5wv_HKg@mail.gmail.com>

Thinking about this more, I expect that an issue might be classes that
emulate dicts without being too formal about it (e.g. no ABCs) and then
porting code that works for both such class instances and dicts. Many
examples of such classes I've seen have a keys() method that returns a
list, and for various reasons these classes won't be rewritten to be more
like Python 3 as part of the port.

And then again, adding iterkeys() back to dict doesn't solve *that*
problem, and presumably such a class still has an __iter__() method, and
presumably the alternatives I proposed would still work.

I am also concerned about the dependency on Python 3.5 that we're building
here. I'd much rather be able to use Twisted sooner, with 3.3 or at least
3.4.


On Sat, Apr 19, 2014 at 9:17 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

>
> On 19 Apr 2014 10:53, "Guido van Rossum" <guido at python.org> wrote:
> >
> > Does everyone involved know that "for x in d.iterkeys()" is equivalent
> to "for x in d" and works the same in Python 2 and 3? Similarly, "list(d)"
> is a simple, fast way to spell the Python 2 semantics of "d.keys()"  that
> works in both versions (but I doubt it is much needed -- usually the actual
> code follows up with sorting, so you should use sorted(d)).
> >
> > This doesn't solve itervalues() and iteritems() but I expect those are
> less common, and "for x, y in d.iteritems(): <blah>" is rewritten nicely as
> >
> >   for x in d:
> >     y = d[x]
> >     <blah>
> >
> > If there is a measurable slowdown in the latter I would be totally okay
> with some kind of one-element cache for the most recent lookup.
> >
> > I get the social aspect of the PEP, but I think it's too high a price to
> pay.
>
> OK, I think the main thing I need to add to the PEP next is a clear
> description of the current state of the art in translating these methods to
> the common 2/3 subset across the following modes of interaction with the
> existing Python 2 API:
>
> * for loops
> * comprehensions and generator expressions
> * iterator object
> * bound methods
> * unbound methods
>
> That may help clarify the tricky warts and edge cases that can arise when
> moving from the current relatively straightforward and consistent method
> based approach to a more complicated combination of dedicated syntax and
> helper functions.
>
> I also asked JP Calderone to trawl the Twisted code base for specific
> cases where the status quo causes migration problems. Since my main
> argument is that we should do this to save collective modification and
> review effort for affected projects (i.e. I think it's a "death by 1000
> cuts" situation rather than a single gaping wound anyone can point to), it
> would be good if those affected could help with quantifying the scale of
> the problem so we can make a more informed trade-off between that work and
> future users of Python 3 needing to learn about the existence of the
> proposed largely redundant compatibility methods.
>
> In several ways, I see my proposal as similar to what we did when PEP
> 8'ifying the threading.Thread API - the old camel case functions are still
> there, but clearly marked as only existing for legacy compatibility reasons
> (they may actually be completely undocumented in Py3 - I don't recall off
> the top of my head). While I'm personally a big fan of cleaning up APIs and
> strongly encouraging code modernisation, I've also become convinced that it
> will be worth our while to start soliciting more quantitative feedback in
> relation to various decisions to help make sure we're OK with the full
> consequences of those design decisions.
>
> Regards,
> Nick.
>
> >
> > --
> > --Guido van Rossum (python.org/~guido)
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140419/1eb88286/attachment-0001.html>

From donald at stufft.io  Sat Apr 19 19:16:54 2014
From: donald at stufft.io (Donald Stufft)
Date: Sat, 19 Apr 2014 13:16:54 -0400
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <CAP7+vJLcpp2JXXZyQEhLv97nXTkPGoE0dFbLwOiczhQ5wv_HKg@mail.gmail.com>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <1397876893.11662.108082729.0AF51881@webmail.messagingengine.com>
 <CADiSq7dP9bSi+zWR+nXXCX5ySveh3AqyeZpGGGN1_USLB9nt2w@mail.gmail.com>
 <CAP7+vJJsxBbXcVSHUvvQZWYgXmotNw+2qpGxs4FcDtHi3WWF4w@mail.gmail.com>
 <CADiSq7d=cQdB=51y-1LqBA6Aaamu=OeQn7xsxeCWV7YYMO5WiQ@mail.gmail.com>
 <CAP7+vJLcpp2JXXZyQEhLv97nXTkPGoE0dFbLwOiczhQ5wv_HKg@mail.gmail.com>
Message-ID: <DFA42453-62C2-4010-87B8-5F6041C0C6ED@stufft.io>


On Apr 19, 2014, at 12:35 PM, Guido van Rossum <guido at python.org> wrote:

> I am also concerned about the dependency on Python 3.5 that we're building here. I'd much rather be able to use Twisted sooner, with 3.3 or at least 3.4.

Anyone who is planning on using the bytes modulo formatting is going to be 3.5+ anyways. It seems like trying to fit as many of these compatibility things as Python is willing to do into 3.5 is the best possible solution since it?s likely that for a lot of these hanger-ons 3.5 is likely to be a minimum target anyways.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140419/b422dae0/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140419/b422dae0/attachment.sig>

From turnbull at sk.tsukuba.ac.jp  Sat Apr 19 19:31:34 2014
From: turnbull at sk.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Sun, 20 Apr 2014 02:31:34 +0900
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
 methods
In-Reply-To: <1397925207.26188.108197637.229A3326@webmail.messagingengine.com>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <1397876893.11662.108082729.0AF51881@webmail.messagingengine.com>
 <CADiSq7dP9bSi+zWR+nXXCX5ySveh3AqyeZpGGGN1_USLB9nt2w@mail.gmail.com>
 <CAP7+vJJsxBbXcVSHUvvQZWYgXmotNw+2qpGxs4FcDtHi3WWF4w@mail.gmail.com>
 <874n1ps2n7.fsf@uwakimon.sk.tsukuba.ac.jp>
 <1397925207.26188.108197637.229A3326@webmail.messagingengine.com>
Message-ID: <8738h9rzt5.fsf@uwakimon.sk.tsukuba.ac.jp>

Benjamin Peterson writes:

 > > I suppose there's no way to get the compiler to both make "for x in d"
 > > work as above, and make "for k, v in d" be equivalent to Python 2's
 > > "for k, v in d.iteritems()"?

 > That doesn't make sense. What if your keys are tuples?

Oh, I still think it makes sense.  Both x and k would be bound to the
key tuples.  For example it would "work" consistently with Common Lisp
multiple values.  And it's not clear to me that unpacking key tuples
would be used anywhere near as often as item unpacking.

But Python doesn't have other objects that behave like Common Lisp
multiple values, and it would change the meaning of currently correct
programs, so it's a non-starter.


From ethan at stoneleaf.us  Sat Apr 19 20:05:27 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Sat, 19 Apr 2014 11:05:27 -0700
Subject: [Python-Dev] static typing of input arguments in signatures
In-Reply-To: <20140413235826.142BF250030@webabinitio.net>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
 <libs4b$qqg$1@ger.gmane.org> <lid4uj$qj1$1@ger.gmane.org>
 <21C31DB4-6F80-43CE-B811-FB61938E9423@langa.pl> <lieqbs$d6s$1@ger.gmane.org>
 <20140413235826.142BF250030@webabinitio.net>
Message-ID: <5352BAE7.1040809@stoneleaf.us>

On 04/13/2014 04:58 PM, R. David Murray wrote:
> On Sun, 13 Apr 2014 15:59:36 -0400, Terry Reedy wrote:
>> On 4/13/2014 4:11 AM, ??ukasz Langa wrote:
>>> On Apr 13, 2014, at 12:48 AM, Stefan Behnel wrote:
>>>>
>>>> So, what I've learned from seven years of Cython is that static typing in
>>>> signatures is actually less interesting than you might think at first
>>>> sight. It might be ok for documentation purposes, although its verboseness
>>>> makes that also a bit questionable.
>>>
>>> You raise a valid point that type hinting a dict instead of a Mapping is
>>> likely to create an overly limiting API. This sort of error is however
>>> quite easy to fix forward.
>>>
>>> To counter, we???ve had multiple data points during the summit suggesting
>>> that duck typing is not really that often used in production code. In
>>> other words, after initial prototyping and testing, an API of a callable
>>> gets settled and is later used with a very limited number of types.
>>
>> For public library code, where the use case is not known, apis should
>> usually be as generic as sensible. For private library code, I can
>> imagine that apis are and possibly even should be limited to classes
>> actually used. There naturally is a bit of a bias here for public code.
>
> The way *I* heard it was that types of the input arguments did not, in
> general, change *during the running of an application*.  That doesn't
> mean that the types wouldn't be different in a different application,
> which means that in a library that duck types, duck typing is indeed used
> in production, it's just that the types don't *change* for a production
> application.  So, yeah, pretty much what Terry said about library code
> versus application code.  That is, after all, what duck typing is about,
> and there is a *reason* we use it.

Personal experience:  I have my own copy of paramiko because it type checks for strings, and I routinely use a str-subclass.

--
~Ethan~

From ncoghlan at gmail.com  Sat Apr 19 20:48:25 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 19 Apr 2014 14:48:25 -0400
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <CACBhJdFqALY0zdmsGMO5L-WVkvcfumMY2NdG8QM8oEMLrBNcig@mail.gmail.com>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <20140419113537.436ce991@fsol>
 <EFE3877620384242A686D52278B7CCD3A52D9701@RKV-IT-EXCH104.ccp.ad.local>
 <20140419131456.GK28400@ando>
 <CACBhJdFqALY0zdmsGMO5L-WVkvcfumMY2NdG8QM8oEMLrBNcig@mail.gmail.com>
Message-ID: <CADiSq7fLKXVkGp4MyPBtP6d6J3f9D17hoUS7M5Wouoggm2SqTQ@mail.gmail.com>

On 19 Apr 2014 12:29, "Ezio Melotti" <ezio.melotti at gmail.com> wrote:
>
> Hi,
>
> On Sat, Apr 19, 2014 at 4:14 PM, Steven D'Aprano <steve at pearwood.info>
wrote:
> > On Sat, Apr 19, 2014 at 11:41:35AM +0000, Kristj?n Valur J?nsson wrote:
> >> It is a signigificant portion of the incompatibility, and seems like
> >> such a minor concession to compatibility to make.
> >
> > I don't think it is a significant portion of incompatibility. Or at
> > least, I think that the Twisted folks (or Nick, if he wants to speak for
> > them) have to justify why it's significant.
> >
>
> Assuming this gets included in 3.5 (which will be released around the
> end of 2015), are they planning to disregard all the previous 3.x
> releases and then wait a couple more years (so 2018+) for 3.5 to
> become common?
> Are they going to support 3.3+ only (with u'...') and have extra cruft
> for 3.3/3.4 to deal with the missing iter* methods and then remove the
> cruft once 3.5 is the oldest 3.x release that is supported?
> What happens if this addition will still not push people to move their
> code to 3.x and similar requests are made for 3.6+ (and shift what I
> just said for another 18 months)?

This isn't really about porting major libraries or frameworks (except
perhaps Twisted, which was already blocked by the binary interpolation
issue anyway), but about chipping away at the barriers to migration for the
large custom in-house code bases, and the wide array of Python
*applications* that are part of the reason that people pay companies like
Red Hat to provide long term support.

Cheers,
Nick.

>
> Best Regards,
> Ezio Melotti
>
> >
> >
> > --
> > Steven
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140419/a75b738d/attachment.html>

From barry at python.org  Sun Apr 20 00:56:52 2014
From: barry at python.org (Barry Warsaw)
Date: Sat, 19 Apr 2014 18:56:52 -0400
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
 methods
In-Reply-To: <CAFYqXL-THn4DXZYk3z-b_vH6_u3-Zf_aJYwUvGc5j4_AQJBSUg@mail.gmail.com>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <20140419113537.436ce991@fsol>
 <CAFYqXL-THn4DXZYk3z-b_vH6_u3-Zf_aJYwUvGc5j4_AQJBSUg@mail.gmail.com>
Message-ID: <20140419185652.3d62d4ab@anarchist.wooz.org>

On Apr 19, 2014, at 02:12 PM, Giampaolo Rodola' wrote:

>I don't see this as a key porting hassle *at all* and I don't understand
>why they think this would significantly help their porting (it wouldn't).
>The only real barrier is the str/bytes conversion, really, and this is even
>more true for projects massively centered around IO, such as Twisted and,
>I'm sure, the main (only?) reason why Twisted hasn't been ported yet.

I agree.  I've been trying to get rid of iter*() when porting because most of
the time, there is no significant memory savings to be achieved anyway.

-Barry

From ericsnowcurrently at gmail.com  Sun Apr 20 01:14:42 2014
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Sat, 19 Apr 2014 17:14:42 -0600
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <20140419185652.3d62d4ab@anarchist.wooz.org>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <20140419113537.436ce991@fsol>
 <CAFYqXL-THn4DXZYk3z-b_vH6_u3-Zf_aJYwUvGc5j4_AQJBSUg@mail.gmail.com>
 <20140419185652.3d62d4ab@anarchist.wooz.org>
Message-ID: <CALFfu7C+eFLVQPQ6vsXPMFx3G_6c_viRgcSmpnooPakKv9=Zjw@mail.gmail.com>

On Sat, Apr 19, 2014 at 4:56 PM, Barry Warsaw <barry at python.org> wrote:
> On Apr 19, 2014, at 02:12 PM, Giampaolo Rodola' wrote:
>
>>I don't see this as a key porting hassle *at all* and I don't understand
>>why they think this would significantly help their porting (it wouldn't).
>>The only real barrier is the str/bytes conversion, really, and this is even
>>more true for projects massively centered around IO, such as Twisted and,
>>I'm sure, the main (only?) reason why Twisted hasn't been ported yet.
>
> I agree.  I've been trying to get rid of iter*() when porting because most of
> the time, there is no significant memory savings to be achieved anyway.

+1

-eric

From raymond.hettinger at gmail.com  Sun Apr 20 01:14:14 2014
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Sat, 19 Apr 2014 16:14:14 -0700
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
Message-ID: <A136BB3D-091F-4F23-9CC2-2521729832CD@gmail.com>


On Apr 18, 2014, at 7:31 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> After spending some time talking to the folks at the PyCon Twisted
> sprints, they persuaded me that adding back the iterkeys/values/items
> methods for mapping objects would be a nice way to eliminate a key
> porting hassle for them (and likely others), without significantly
> increasing the complexity of Python 3.

I'm not keen on letting Python 2 leak into Python 3. 
That defeats one of the goals of Python 3 (simplification
and leaving legacy APIs behind a in fresh start).

As a Python instructor and coach, I can report that we
already have too many methods on dictionaries and
that it creates a usability obstacle when deciding which
methods to use.

In Python 2.7, a dir(dict) or help(dict) presents too many
ways to do it.   In Python 3.4, we finally have a clean
mapping API and it would be a pitty to clutter it up
in perpetuity.


Raymond




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140419/064e9b61/attachment.html>

From greg.ewing at canterbury.ac.nz  Sun Apr 20 01:46:08 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 20 Apr 2014 11:46:08 +1200
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <8738h9rzt5.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <1397876893.11662.108082729.0AF51881@webmail.messagingengine.com>
 <CADiSq7dP9bSi+zWR+nXXCX5ySveh3AqyeZpGGGN1_USLB9nt2w@mail.gmail.com>
 <CAP7+vJJsxBbXcVSHUvvQZWYgXmotNw+2qpGxs4FcDtHi3WWF4w@mail.gmail.com>
 <874n1ps2n7.fsf@uwakimon.sk.tsukuba.ac.jp>
 <1397925207.26188.108197637.229A3326@webmail.messagingengine.com>
 <8738h9rzt5.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <53530AC0.2010005@canterbury.ac.nz>

Stephen J. Turnbull wrote:
> Benjamin Peterson writes:
> 
>  > > I suppose there's no way to get the compiler to both make "for x in d"
>  > > work as above, and make "for k, v in d" be equivalent to Python 2's
>  > > "for k, v in d.iteritems()"?
> 
> it would change the meaning of currently correct
> programs, so it's a non-starter.

Maybe what's wanted is a function analogous to enumerate() for
mappings instead of sequences. Picking a semi-arbitrary name
for now:

    for k, v in tabulate(d):
       ...

It could be special-cased to recognise dicts and do the appropriate
thing for the Python version concerned. If it doesn't recognise the
type, it would fall back to a generic implementation like

    for k in d:
       v = d[k]:
          ...

-- 
Greg

From stephen at xemacs.org  Sun Apr 20 02:56:24 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 20 Apr 2014 09:56:24 +0900
Subject: [Python-Dev] PEP 469: Restoring the
	iterkeys/values/items()	methods
In-Reply-To: <53530AC0.2010005@canterbury.ac.nz>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <1397876893.11662.108082729.0AF51881@webmail.messagingengine.com>
 <CADiSq7dP9bSi+zWR+nXXCX5ySveh3AqyeZpGGGN1_USLB9nt2w@mail.gmail.com>
 <CAP7+vJJsxBbXcVSHUvvQZWYgXmotNw+2qpGxs4FcDtHi3WWF4w@mail.gmail.com>
 <874n1ps2n7.fsf@uwakimon.sk.tsukuba.ac.jp>
 <1397925207.26188.108197637.229A3326@webmail.messagingengine.com>
 <8738h9rzt5.fsf@uwakimon.sk.tsukuba.ac.jp>
 <53530AC0.2010005@canterbury.ac.nz>
Message-ID: <87zjjgrf7r.fsf@uwakimon.sk.tsukuba.ac.jp>

Greg Ewing writes:

 > Maybe what's wanted is a function analogous to enumerate() for
 > mappings instead of sequences. Picking a semi-arbitrary name
 > for now:
 > 
 >     for k, v in tabulate(d):

I thought this already existed in six, though, with a name that is
familiar to Python 2 programmers and not requiring a mental gear
change to recall semantics:

    for k,v in six.iteritems(d):


From davvid at gmail.com  Sun Apr 20 04:02:42 2014
From: davvid at gmail.com (David Aguilar)
Date: Sat, 19 Apr 2014 19:02:42 -0700
Subject: [Python-Dev] subprocess.Popen and win32
Message-ID: <CAJDDKr7ouektX9J6ZMjfm_58_GP7DYa4bcqdL1BeAwZ_NkwAvQ@mail.gmail.com>

Hi,

I just joined python-dev because I found the need to add some code to
paper over python3's subprocess API, and I'm wondering whether I'm
missing something.

On python2 and python3, the (only?) way to get utf-8 arguments to
subprocess was to ensure that all unicode strings are encoded into
bytes before subprocess sees them. This has worked for a long time
(currently compatible across python2 and 3).

On python3, this still works for normal platforms, but on windows we
can't pass a list of byte strings. We have to pass a list of unicode
strings.

This means that the application code ends up needing to do this:
https://github.com/git-cola/git-cola/commit/1109aeb4354c49931d9b0435d2b7cfdc2d5d6966

basically,

def start_command(cmd):
    if sys.platform == 'win32':
        # Python on windows always goes through list2cmdline() internally inside
        # of subprocess.py so we must provide unicode strings here otherwise
        # Python3 breaks when bytes are provided.
        cmd = [decode(c) for c in cmd]
    else:
        cmd = [encode(c) for c in cmd]
     return subprocess.Popen(cmd)

That seems broken to me, so I wonder if this is a bug in the way
python3 is handling Popen with list-of-bytestring on win32?

I'm not a windows user, but I was able to install python3 under wine
and the same traceback happens without the paper bag fix. This is what
the traceback looks like; it dies in list2cmdline (which I am not
calling directly, Popen does it under the covers):

    File "E:\Program Files
(E)\git-cola\share\git-cola\lib\cola\core.py", line 109, in
start_command
    universal_newlines=universal_newlines)
    File "C:\Python32\lib\subprocess.py", line 744, in __init__
    restore_signals, start_new_session)
    File "C:\Python32\lib\subprocess.py", line 936, in _execute_child
    args = list2cmdline(args)
    File "C:\Python32\lib\subprocess.py", line 564, in list2cmdline
    needquote = (" " in arg) or ("\t" in arg) or not arg
    TypeError: Type str doesn't support the buffer API

This is an issue for folks that use python to write cross-platform
code. The unix code paths expect list-of-bytes, but win32 only expects
list-of-unicode, which pushes the burden onto the application
programmer.

It's my opinion that the win32 code path on python3 is the odd man
out. If it allowed list-of-bytes like python2/win32 and python2+3/unix
then this wouldn't be an issue.

Is this an actual problem, or is it something that should be handled
by application-level code as I've done?

Thanks,
-- 
David

From steve at pearwood.info  Sun Apr 20 04:49:55 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 20 Apr 2014 12:49:55 +1000
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <CADiSq7dQyXMcBY-fhUgB1+6204+wwvBW1=uWvyze9V-ZCbQy-A@mail.gmail.com>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <20140419042553.GJ28400@ando>
 <CADiSq7dQyXMcBY-fhUgB1+6204+wwvBW1=uWvyze9V-ZCbQy-A@mail.gmail.com>
Message-ID: <20140420024955.GL28400@ando>

On Sat, Apr 19, 2014 at 10:38:39AM -0400, Nick Coghlan wrote:
> On 19 Apr 2014 00:27, "Steven D'Aprano" <steve at pearwood.info> wrote:
> >
> > On Fri, Apr 18, 2014 at 10:31:29PM -0400, Nick Coghlan wrote:
> > > After spending some time talking to the folks at the PyCon Twisted
> > > sprints, they persuaded me that adding back the iterkeys/values/items
> > > methods for mapping objects would be a nice way to eliminate a key
> > > porting hassle for them (and likely others), without significantly
> > > increasing the complexity of Python 3.
> >
> > It would also considerable add to the cruft of Python 3. One motive for
> > going through the pain of Python 2 to 3 migration was to remove cruft.
> > Adding it back in again is not just an aid to porting but actively
> > making Python 3 a worse (well, "less better") experience.
> 
> The cruft is unavoidable in this case. The decision we face is *where the
> cruft lives*, and *how much work is involved* in creating that cruft.

"How much work" is "very little". This problem is not of the same 
magnitude as trying to deal with unicode literals in a polyglot module, 
or the other string/bytes issues. A lot of the time, we don't even care 
whether iterating over dict.foo() gives a list, an iterator or a view. 
For the times we do care, it isn't hard to use a helper.

As for "where the cruft lives", that's the crux of the matter. In my 
opinion, the decision hinges on this question:

   Are iterkeys(), iteritems() and itervalues() the new, preferred 
   APIs for keys(), items() and values(), with the old APIs being 
   kept only for backward compatibility?

If so, then adding them to the language is certainly the right place. 
The old APIs could be deprecated, or even just left with a note in the 
docs that they aren't yet formerly deprecated but will be some day, in 
the meantime the new iterator-based APIs are preferred.

But if the keys(), items() and values() view-based APIs remain the 
preferred API, then I don't believe the backwards-compatibility layer 
belongs in the language. I believe it belongs as an external library, or 
even just a few helpers on an ad hoc basis.

If this is being driven by Twisted, I think that the onus needs to be on 
them to demonstrate how it will help them. Their official plans are to 
support Python 3.3, and they seem to have made a lot of progress towards 
it:

http://twistedmatrix.com/trac/wiki/Plan/Python3

so unless they drop 3.3 and 3.4 (do we want to encourage that?) 
this change won't even help them.


> Status quo: we preserve the "purity" of the Python 3 mapping API, and
> require every developer making the transition from Python 2 to Python 3
> replace every occurrence of these methods with a helper function that is
> not idiomatic code in either language.

That is part of the cost of writing polyglot code. It's messy. But 
that's not new, and it's not unique to 2+3 polyglot code, it happens 
every time a new feature is added or removed in a point release.


> My proposal: we add three trivial helper methods to the Python 3 mapping
> API. For the cost of 3 additional methods that are easily explained in
> terms of combining a builtin with other existing methods, a whole pile of
> work just evaporates for affected segments of the community.

And a whole lot of additional work suddenly appears for *different* 
affected segments of the community, e.g. educators, writers.

This is where the standard objections to any new language feature come 
out. Just because this feature is being added for the benefit of 
transitional 2+3 polyglot code doesn't render these objections 
irrelevant. I trust I don't need to go through the usual list.

By the way, I earlier suggested adding iter* and immediately deprecating 
them, but I don't believe you commented on that. Even if they are never 
removed, the deprecation warning would be a very strong signal that they 
really are only added as an aid to writing 2+3 code, and are not 
intended as long-term language features. Writers can then ignore the 
iter* API, or at least relegate it to an appendix, and educators might 
not be quite able to ignore it but they can at least say "don't use 
them" in good conscience. The dict iteration issue for polygot code now 
becomes easy: just silence the warning, and you're good to go for at 
least three more years, and likely longer.

I think immediate deprecation would be a reasonable compromise position, 
and would like to hear your thoughts on that.


> > So while I'm sympathetic to wanting to ease the 2/3 transition, even at
> > the expense of re-introducing some cruft when unavoidable, I don't think
> > that the difficulty of dealing with dict items|iteritems|viewitems etc.
> > justifies re-adding cruft.
> 
> The cruft will be there regardless, the only question is whether it exists
> in the definition of Python 3 or is distributed across all of the single
> source projects supporting both 2 & 3.

The same reasoning applies to every change between 2 and 3. If 
dict.iter* methods get language support, why not all the others?

If somebody is feeling particularly brave, and wants to argue that 
dropping (almost all) 2.x features was a mistake and they should be 
readded in bulk, I would much rather have that debate now rather than 
spending the next eighteen months arguing about it feature by feature.


[...]
> I spend a fair bit of time talking to users that have put a lot of work
> into supporting a language transition that doesn't really help them
> personally. A non-trivial number of them are deeply, viscerally angry about
> what they see as pointlessly changing the spelling of a core language
> feature for no real technical gain.
> 
> So let me be clear: this is *not* a proposal driven primarily by technical
> considerations. Rather, it is a social one, where we do something simple
> and easy and low cost for us to send a clear message to a set of users that
> feel justifiably angry about the amount of work we imposed on other members
> of the community by putting Python 2 into maintenance mode that we do value
> their time and energy, and are willing to show some flexibility in adding
> "not harmful" changes to Python 3 that don't significantly increase the
> complexity of the language, while making it easier to write single source
> code that looks like idiomatic Python code.

Thank you for raising this explicitly, I think it is important to 
acknowledge people's feelings. But it's also important to acknowledge 
that just because people are angry, doesn't mean we have to do more than 
acknowledge that they are angry. Being angry doesn't give one better 
insight to the issue, or make one's preferred solution the right 
solution. (If anything, the opposite: angry people's solutions to 
problems tend to make things worse rather than better.)

I think it is also important to acknowledge that while some people may 
feel angry about Python 3 changes, some people are *extremely happy* 
that Python 3 has cleaned up some language cruft, and will be equally 
justifiably, viscerally angry if it is put back. If this is a permanent 
language feature, the *entire* Python community will be dealing with 
this long after Twisted has dropped support for Python 3.5, and maybe 
long after Twisted has been abandoned and is just a memory.

Let me be frank, and call a spade a bloody shovel as we say in 
Australia: from the evidence presented so far (very little), I don't 
think that the Twisted folks are justified in their anger about 
dict.iter* methods or the difficulty in handling them in polygot code. 
It seems to me that they are being awfully precious about a few trivial 
helpers. (I do not speak of other 2+3 issues, which I know can be more 
significant.) I think they are using their privileged position as a 
major Python library to try to impose an unnecessary and harmful (albeit 
minor) regression on Python 3 with no technical justification, for 
"social" (i.e. personal) reasons which I won't speculate about.

If it had been some minor third party library requesting this change, 
the answer would be obvious:

    Not every two or three line function needs to be a built-in.


I do not accept that using helper functions is unpythonic. On the 
evidence presented so far, I don't believe that there is any technical 
justification for adding iter* methods back to the dict API, and if we 
care about social issues, how about the social issues that doing this 
can and will be seen by some as a de facto acknowledgement that Python 3 
was a mistake?

It's one thing to add big, complicated features which are hard for 
individual projects to deal with as an aid to porting, e.g. the whole 
bytes/str issue, but when something as simple as the proposed iter* 
methods is treated as if it were a major imposition on third-parties 
that needs to be fixed in the core language, that's a big signal that 
the whole Python 3 exercise was a mistake. I do not believe it is a 
mistake, and I don't want to send a signal that it was.

I think Terry Reedy's comment earlier is significant:

    [quote]
    When I suggested that PEP 414 might be seen as a precedent for 
    restoring more of Py2, I was trashed for saying so. "No, no, u'' is 
    a unique case. [it is] This will be the last proposal like this." 
    What will come next?
    [end quote]

What's the next Python 2 feature that will be re-added after this one? A 
lot of people are angry that the cmp argument to sorted() and 
list.sort() was removed, and just don't like functools.cmp_to_key, never 
mind that it actually solves their technical problem. Should that be 
added back, "to aid in porting (but really just so people will feel good 
about the fact that we're listening to them)"?

With no good technical justification for this regression, I fear that 
this sets a precedence that will not end until we have:

    from __future__ import just_give_me_back_my_python2_dammit

in another release or three.

I don't believe that will happen, the line *will* be drawn somewhere, 
before Python 3 dies a death of a thousand cuts. I think that the right 
place to draw the line is *here*, not the next time, or the time after 
that. I think that the decision should be made on technical reasons, not 
so people feel that they are being listened to.

(That will only end with resentment, "Why do you listen to *them* and 
not *me*? Am I not important enough?".)

2+3 compatibility code should only live in the language rather than in 
the polygot code when there are compelling technical reasons for it to 
do so, not just to avoid a couple of two or three line helpers.



-- 
Steven

From tjreedy at udel.edu  Sun Apr 20 06:03:58 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 20 Apr 2014 00:03:58 -0400
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <CAP7+vJJsxBbXcVSHUvvQZWYgXmotNw+2qpGxs4FcDtHi3WWF4w@mail.gmail.com>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <1397876893.11662.108082729.0AF51881@webmail.messagingengine.com>
 <CADiSq7dP9bSi+zWR+nXXCX5ySveh3AqyeZpGGGN1_USLB9nt2w@mail.gmail.com>
 <CAP7+vJJsxBbXcVSHUvvQZWYgXmotNw+2qpGxs4FcDtHi3WWF4w@mail.gmail.com>
Message-ID: <livh09$5cu$1@ger.gmane.org>

On 4/19/2014 10:52 AM, Guido van Rossum wrote:
> Does everyone involved know that "for x in d.iterkeys()" is equivalent
> to "for x in d"

Looking at uses I found by searching code.ohloh.net, the answer is 
either 'No, people sometimes add a redundant .iterkeys()' or 'people are 
writing non-dict mapping classes for which it is not redundant (perhaps 
because their custom class iterates by items rather than keys by 
default)'. I could not tell from the quoted snippet.

> and works the same in Python 2 and 3? Similarly,
> "list(d)" is a simple, fast way to spell the Python 2 semantics of
> "d.keys()"  that works in both versions (but I doubt it is much needed
> -- usually the actual code follows up with sorting, so you should use
> sorted(d)).
>
> This doesn't solve itervalues() and iteritems() but I expect those are
> less common,

ohloh gives about 77,000 python hits for iteritems and 16,000 for 
itervalues. A large fraction of itervalue hits are definitions rather 
than uses, often from a compat.py (is this from six?)

if sys.version_info[0] >= 3:
     text_type = str
     string_types = str,
     iteritems = lambda o: o.items()
     itervalues = lambda o: o.values()
     izip = zip

else:
     text_type = unicode
     string_types = basestring,
     iteritems = lambda o: o.iteritems()
     itervalues = lambda o: o.itervalues()
     from itertools import izip

This is three hits for iteritems and three for itervalues and none for 
the unneeded iterkeys. My guess is that there are 5000 itervalue uses 
and 70000 iteritem uses.

There are 1,500,000 python hits for 'for', some unknown fraction of 
which are 'for key in somedict' or 'for key in somedict.keys()'.  There 
are 13000 for iterkeys. As noted above, this is *not* inflated by 3 hits 
for each use of compat.py. I think 10% or 150000 iterations by key might 
be a reasonable guess.

There are other definition sets that include iterkeys or that define 
functions that wrap all three bound methods for a particular dict.

iterkeys = lambda: d.iterkeys() # py2
iterkeys = lambda: d.keys()  # py3

> and "for x, y in d.iteritems(): <blah>" is rewritten nicely as
>
>    for x in d:
>      y = d[x]
>      <blah>
>
> If there is a measurable slowdown in the latter I would be totally okay
> with some kind of one-element cache for the most recent lookup.

About three weeks ago, Raymond opened http://bugs.python.org/issue21101 
with this claim: "It is reasonably common to make two successive 
dictionary accesses with the same key."  I proposed a specialized 
caching as an alternative to adding new C API functions.

Using the iteritems function, there is one simple, extra function call 
for the entire loop. If the body of the loop takes at lease as long as 
that one call, the extra time is a non-issue if the dict has more than, 
say, 20 items.

-- 
Terry Jan Reedy


From ethan at stoneleaf.us  Sun Apr 20 07:31:18 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Sat, 19 Apr 2014 22:31:18 -0700
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
Message-ID: <53535BA6.2030305@stoneleaf.us>

Thank you for taking the time to write this up, Nick.

However, I am -1 on it.  One of the allures of Python 3 is the increase in simplicity and elegance.  Restoring cruft 
does not help with that.  Python 2 idioms that get restored to Python 3 must have real value: unicode literals, 
wire-protocol interpolations -- I don't feel that this comes any where close to meeting that bar.

--
~Ethan~

From victor.stinner at gmail.com  Sun Apr 20 09:43:50 2014
From: victor.stinner at gmail.com (Victor Stinner)
Date: Sun, 20 Apr 2014 03:43:50 -0400
Subject: [Python-Dev] subprocess.Popen and win32
In-Reply-To: <CAJDDKr7ouektX9J6ZMjfm_58_GP7DYa4bcqdL1BeAwZ_NkwAvQ@mail.gmail.com>
References: <CAJDDKr7ouektX9J6ZMjfm_58_GP7DYa4bcqdL1BeAwZ_NkwAvQ@mail.gmail.com>
Message-ID: <CAMpsgwb=gmke2kkUn8V-84ye-ZDuTiJF+Ka4kuTTZxEbcJ=3QA@mail.gmail.com>

On Python 3, you can use Unicode on all platforms. On UNIX, there is no
need to encode explicitly.

Victor
Le 20 avr. 2014 04:17, "David Aguilar" <davvid at gmail.com> a ?crit :

> Hi,
>
> I just joined python-dev because I found the need to add some code to
> paper over python3's subprocess API, and I'm wondering whether I'm
> missing something.
>
> On python2 and python3, the (only?) way to get utf-8 arguments to
> subprocess was to ensure that all unicode strings are encoded into
> bytes before subprocess sees them. This has worked for a long time
> (currently compatible across python2 and 3).
>
> On python3, this still works for normal platforms, but on windows we
> can't pass a list of byte strings. We have to pass a list of unicode
> strings.
>
> This means that the application code ends up needing to do this:
>
> https://github.com/git-cola/git-cola/commit/1109aeb4354c49931d9b0435d2b7cfdc2d5d6966
>
> basically,
>
> def start_command(cmd):
>     if sys.platform == 'win32':
>         # Python on windows always goes through list2cmdline() internally
> inside
>         # of subprocess.py so we must provide unicode strings here
> otherwise
>         # Python3 breaks when bytes are provided.
>         cmd = [decode(c) for c in cmd]
>     else:
>         cmd = [encode(c) for c in cmd]
>      return subprocess.Popen(cmd)
>
> That seems broken to me, so I wonder if this is a bug in the way
> python3 is handling Popen with list-of-bytestring on win32?
>
> I'm not a windows user, but I was able to install python3 under wine
> and the same traceback happens without the paper bag fix. This is what
> the traceback looks like; it dies in list2cmdline (which I am not
> calling directly, Popen does it under the covers):
>
>     File "E:\Program Files
> (E)\git-cola\share\git-cola\lib\cola\core.py", line 109, in
> start_command
>     universal_newlines=universal_newlines)
>     File "C:\Python32\lib\subprocess.py", line 744, in __init__
>     restore_signals, start_new_session)
>     File "C:\Python32\lib\subprocess.py", line 936, in _execute_child
>     args = list2cmdline(args)
>     File "C:\Python32\lib\subprocess.py", line 564, in list2cmdline
>     needquote = (" " in arg) or ("\t" in arg) or not arg
>     TypeError: Type str doesn't support the buffer API
>
> This is an issue for folks that use python to write cross-platform
> code. The unix code paths expect list-of-bytes, but win32 only expects
> list-of-unicode, which pushes the burden onto the application
> programmer.
>
> It's my opinion that the win32 code path on python3 is the odd man
> out. If it allowed list-of-bytes like python2/win32 and python2+3/unix
> then this wouldn't be an issue.
>
> Is this an actual problem, or is it something that should be handled
> by application-level code as I've done?
>
> Thanks,
> --
> David
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140420/7efe679e/attachment.html>

From solipsis at pitrou.net  Sun Apr 20 11:42:07 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 20 Apr 2014 11:42:07 +0200
Subject: [Python-Dev] subprocess.Popen and win32
References: <CAJDDKr7ouektX9J6ZMjfm_58_GP7DYa4bcqdL1BeAwZ_NkwAvQ@mail.gmail.com>
Message-ID: <20140420114207.39674d1d@fsol>

On Sat, 19 Apr 2014 19:02:42 -0700
David Aguilar <davvid at gmail.com> wrote:
> 
> On python3, this still works for normal platforms, but on windows we
> can't pass a list of byte strings. We have to pass a list of unicode
> strings.

Windows native APIs are unicode-based. It is actually necessary to pass
*unicode* strings, not byte strings, if you want your code to be
correct in the face of non-ASCII characters.

Under other platforms, when unicode strings are passed, Python will
encode them using the platform's detected encoding. So, unless your
platform is somehow misconfigured, passing unicode strings will also
work correctly there.

(note this is under Python 3)

Regards

Antoine.



From p.f.moore at gmail.com  Sun Apr 20 13:37:26 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 20 Apr 2014 12:37:26 +0100
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <20140420024955.GL28400@ando>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <20140419042553.GJ28400@ando>
 <CADiSq7dQyXMcBY-fhUgB1+6204+wwvBW1=uWvyze9V-ZCbQy-A@mail.gmail.com>
 <20140420024955.GL28400@ando>
Message-ID: <CACac1F9k5Nvtqs0MizmxoGfArgh7Z19VJP86raDx-xp0SpAY7A@mail.gmail.com>

On 20 April 2014 03:49, Steven D'Aprano <steve at pearwood.info> wrote:
> I don't believe that will happen, the line *will* be drawn somewhere,
> before Python 3 dies a death of a thousand cuts. I think that the right
> place to draw the line is *here*, not the next time, or the time after
> that. I think that the decision should be made on technical reasons, not
> so people feel that they are being listened to.

I agree. I think that it's right that we should listen to users'
frustrations with the (deliberately) backward incompatible changes.
But listening does not imply not questioning. It's perfectly fair and
reasonable to say "We understand that you feel there is an overhead
here - we tried very hard to design Python 3 with future benefits in
mind, and we assessed the costs as best we could given the data we had
available. Could you please provide some quantitative data on how the
loss of iterXXX affects you, with specific code details, so that we
can review what you believe you have to do and consider whether you
have missed an approach we were expecting projects to take, or whether
your figures imply that we seriously underestimated the cost. We can
then consider how best to address the implications of the information
you provide."

Ultimately, every time we add *any* sort of compatibility feature to
Python 3 (Unicode literals, bytes interpolation, this) we are sending
the message that we made a mistake in the design of Python 3. It's
certainly possible that's the case (we didn't have a lot of hard data
to go on) but I do think we should have a little more confidence in
our judgement here.

As Steven said, there are a *lot* of people happy with Python 3. They
don't say much, precisely because they are happy - that's the point.
Let's not fall foul of the mistake of only listening to people who
complain.

Paul

From markus at unterwaditzer.net  Sun Apr 20 14:06:29 2014
From: markus at unterwaditzer.net (Markus Unterwaditzer)
Date: Sun, 20 Apr 2014 14:06:29 +0200
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
 methods
In-Reply-To: <20140419042553.GJ28400@ando>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <20140419042553.GJ28400@ando>
Message-ID: <20140420120629.GA15835@chromebot.lan>

On Sat, Apr 19, 2014 at 02:25:53PM +1000, Steven D'Aprano wrote:
> In my experience, writing polyglot 2+3 code can be easily handled with a 
> few helper functions, which is not the case with unicode string 
> literals. (Non-polygot code of course can just use the methods 
> directly.) I don't see this is a problem to be solved, or even much of a 
> nuisance. Polyglot code is never going to be quite as straightforward or 
> idiomic as non-polyglot code, and that's a good thing, as it reminds the 
> reader that they are dealing with polyglot code.

The problem i currently see is that most polyglot-projects are reimplementing
the same helper functions (for dict access, literals wrapping etc), not
depending on six because the authors don't feel it's "worth the extra
dependency", as they never need the full functionality of six.

So how about including a subset of six' more popular functionality into the
standard libraries of Python 2 and 3? I think it would solve the problem i
described (i.e. remove boilerplate for polyglot code), while it would also
avoid compromising the builtins of Python 3 (and keep polyglot code "explicitly
ugly")

Also, that's why people demanded a Python 2.8... so that you don't have to
pollute Python 3 instead.

-- Markus

From breamoreboy at yahoo.co.uk  Sun Apr 20 15:32:16 2014
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 20 Apr 2014 14:32:16 +0100
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <53535BA6.2030305@stoneleaf.us>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <53535BA6.2030305@stoneleaf.us>
Message-ID: <lj0i90$a43$1@ger.gmane.org>

On 20/04/2014 06:31, Ethan Furman wrote:
> Thank you for taking the time to write this up, Nick.
>
> However, I am -1 on it.  One of the allures of Python 3 is the increase
> in simplicity and elegance.  Restoring cruft does not help with that.
> Python 2 idioms that get restored to Python 3 must have real value:
> unicode literals, wire-protocol interpolations -- I don't feel that this
> comes any where close to meeting that bar.
>
> --
> ~Ethan~

+1 for this summary which echoes my sentiments entirely.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.

Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com



From ncoghlan at gmail.com  Sun Apr 20 15:55:26 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 20 Apr 2014 09:55:26 -0400
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <20140420120629.GA15835@chromebot.lan>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <20140419042553.GJ28400@ando>
 <20140420120629.GA15835@chromebot.lan>
Message-ID: <CADiSq7dxqBXP3WgQQL9jg+n73gFsZ6B0UR-8s_M6+JwMUNcbrQ@mail.gmail.com>

On 20 Apr 2014 08:14, "Markus Unterwaditzer" <markus at unterwaditzer.net>
wrote:
>
> Also, that's why people demanded a Python 2.8... so that you don't have to
> pollute Python 3 instead.

It doesn't actually solve the problem in the library and framework cases
though - most folks that are straddling 2/3 want to keep compatibility back
to at least 2.6.

That said, I think we've covered most of what can be discussed based on
this initial iteration of the PEP. To advance the discussion further I need
to do a new draft that:

- quantifies the scale of the change to the dict API and covers ideas for
mitigating that impact (like immediate deprecation and potentially even
hiding them from dir(), or adding proper function and method deprecation
support to pydoc)

- quantifies the number of lines of Python 2 code potentially saved near
term modification, or, if they have added Python 3 support already, the
amount of work that *could* have been saved (I'm not a big enough fan of
the proposal to do that work myself, though - if the folks requesting the
change aren't sufficiently interested to consider it worth their while to
quantify the direct benefits even deprecated methods could have saved them,
I consider that a data point in its own right)

- explicitly covers all the different ways methods that produce iterators
can be operated on, and how those need to be handled differently when
migrating to the common subset of Python 2 & 3 rather than directly to 3.

Regards,
Nick.

>
> -- Markus
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140420/17fca51e/attachment.html>

From tseaver at palladion.com  Sun Apr 20 17:04:46 2014
From: tseaver at palladion.com (Tres Seaver)
Date: Sun, 20 Apr 2014 11:04:46 -0400
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <CACac1F9k5Nvtqs0MizmxoGfArgh7Z19VJP86raDx-xp0SpAY7A@mail.gmail.com>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <20140419042553.GJ28400@ando>
 <CADiSq7dQyXMcBY-fhUgB1+6204+wwvBW1=uWvyze9V-ZCbQy-A@mail.gmail.com>
 <20140420024955.GL28400@ando>
 <CACac1F9k5Nvtqs0MizmxoGfArgh7Z19VJP86raDx-xp0SpAY7A@mail.gmail.com>
Message-ID: <lj0nl8$bne$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 04/20/2014 07:37 AM, Paul Moore wrote:

> Ultimately, every time we add *any* sort of compatibility feature to 
> Python 3 (Unicode literals, bytes interpolation, this) we are sending 
> the message that we made a mistake in the design of Python 3. It's 
> certainly possible that's the case (we didn't have a lot of hard data 
> to go on) but I do think we should have a little more confidence in 
> our judgement here.

We clearly made mistakes, especially in how we thought migration would
occur:  nobody expected that we would see straddling / compatible subset
as the dominant porting strategy.  Re-adding features to make the
strategy that works less painful is just acknowledging that fact.

Mark such features as BBB-only / deprecated-but-never-to-be-removed, and
move on:  "practicality beats purity".


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlNT4gcACgkQ+gerLs4ltQ4a3wCfcKZWldlrPzNn6byYJrCxm1XG
ttUAniKTQ6ma0n7XNIMf0lP4A1zexT6j
=AkQ+
-----END PGP SIGNATURE-----


From steve at pearwood.info  Sun Apr 20 19:04:21 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 21 Apr 2014 03:04:21 +1000
Subject: [Python-Dev] PEP 469: Restoring the	iterkeys/values/items()
	methods
In-Reply-To: <EFE3877620384242A686D52278B7CCD3A52D9D10@RKV-IT-EXCH104.ccp.ad.local>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <20140419113537.436ce991@fsol>
 <EFE3877620384242A686D52278B7CCD3A52D9701@RKV-IT-EXCH104.ccp.ad.local>
 <20140419131456.GK28400@ando>
 <EFE3877620384242A686D52278B7CCD3A52D9D10@RKV-IT-EXCH104.ccp.ad.local>
Message-ID: <20140420170419.GM28400@ando>

On Sun, Apr 20, 2014 at 03:07:39PM +0000, Kristj?n Valur J?nsson wrote:

> Does one ever use iteritems() et al without first invoking iter() on 
> it?  

I can't speak for others, but I never invoke iteritems *with* iter(). 
What would be the point? iteritems is documented as returning an 
interator.

# never this
for key, value in iter(mydict.iteritems()): ...

# but this
for key, value in mydict.iteritems(): ...


> I.e. is it important that it is an iterator, rather than an 
> iterable? I think we could easily relax that requirement in the pep 
> and solve 99% of the use cases.

And the other 1% of cases would be a land-mine waiting to blow the 
user's code up.

Would it actually solve 99% of the use cases? Or only 90%? Or 50%? How 
do you know?

In Python 2.7 iteritems() etc is documented as returning an iterator. 
That's a promise of the language, and people will rely on it. But they 
won't be able to rely on that promise in polygot 2+3 code -- exactly the 
use-case this PEP is trying to satisfy -- because the promise to return 
an iterator will be broken in 3.

It would be actively misleading, since Python 3's iteritems() would 
return a view, not an iter, and it would fail at solving the backwards 
compatibility issue since views and iterators are not interchangeable 
except for the most basic use of iteration.


From kristjan at ccpgames.com  Sun Apr 20 17:34:22 2014
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Sun, 20 Apr 2014 15:34:22 +0000
Subject: [Python-Dev] PEP 469: Restoring the
	iterkeys/values/items()	methods
In-Reply-To: <CALFfu7C+eFLVQPQ6vsXPMFx3G_6c_viRgcSmpnooPakKv9=Zjw@mail.gmail.com>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <20140419113537.436ce991@fsol>
 <CAFYqXL-THn4DXZYk3z-b_vH6_u3-Zf_aJYwUvGc5j4_AQJBSUg@mail.gmail.com>
 <20140419185652.3d62d4ab@anarchist.wooz.org>
 <CALFfu7C+eFLVQPQ6vsXPMFx3G_6c_viRgcSmpnooPakKv9=Zjw@mail.gmail.com>
Message-ID: <EFE3877620384242A686D52278B7CCD3A52D9D36@RKV-IT-EXCH104.ccp.ad.local>



> -----Original Message-----
> From: Python-Dev [mailto:python-dev-
> bounces+kristjan=ccpgames.com at python.org] On Behalf Of Eric Snow
> Sent: 19. apr?l 2014 23:15
> To: Barry Warsaw
> Cc: Python-Dev
> Subject: Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
> methods
> > I agree.  I've been trying to get rid of iter*() when porting because
> > most of the time, there is no significant memory savings to be achieved
> anyway.
> 

While I also have the gut feeling that we have been placing too much value in the memory savings of iteritems() vs iter(), the approach of just using "iter()" has the problem that it is semantically different.
Compatible source code would have to use "list(mydict.items())" to have the same meaning in 2 and 3.  And then we are starting to feel pain again.

K

From kristjan at ccpgames.com  Sun Apr 20 21:27:08 2014
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Sun, 20 Apr 2014 19:27:08 +0000
Subject: [Python-Dev] PEP 469: Restoring
	the	iterkeys/values/items()	methods
In-Reply-To: <20140420170419.GM28400@ando>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <20140419113537.436ce991@fsol>
 <EFE3877620384242A686D52278B7CCD3A52D9701@RKV-IT-EXCH104.ccp.ad.local>
 <20140419131456.GK28400@ando>
 <EFE3877620384242A686D52278B7CCD3A52D9D10@RKV-IT-EXCH104.ccp.ad.local>,
 <20140420170419.GM28400@ando>
Message-ID: <wyx17cmjwluplxodq7t7wcsj.1398022023996@email.android.com>

Well, "for i in x" and other iteration constructs already call "iter ()" on their iterable. That's the point. Unless you want to manually iterate using "next ()" then the distinction between an iterable and an iterator is academic.


Sent from the ?ther.

-------- Original message --------
From: Steven D'Aprano
Date:20/04/2014 17:05 (GMT+00:00)
To: python-dev at python.org
Subject: Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

On Sun, Apr 20, 2014 at 03:07:39PM +0000, Kristj?n Valur J?nsson wrote:

> Does one ever use iteritems() et al without first invoking iter() on
> it?

I can't speak for others, but I never invoke iteritems *with* iter().
What would be the point? iteritems is documented as returning an
interator.

# never this
for key, value in iter(mydict.iteritems()): ...

# but this
for key, value in mydict.iteritems(): ...


> I.e. is it important that it is an iterator, rather than an
> iterable? I think we could easily relax that requirement in the pep
> and solve 99% of the use cases.

And the other 1% of cases would be a land-mine waiting to blow the
user's code up.

Would it actually solve 99% of the use cases? Or only 90%? Or 50%? How
do you know?

In Python 2.7 iteritems() etc is documented as returning an iterator.
That's a promise of the language, and people will rely on it. But they
won't be able to rely on that promise in polygot 2+3 code -- exactly the
use-case this PEP is trying to satisfy -- because the promise to return
an iterator will be broken in 3.

It would be actively misleading, since Python 3's iteritems() would
return a view, not an iter, and it would fail at solving the backwards
compatibility issue since views and iterators are not interchangeable
except for the most basic use of iteration.

_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140420/fefa8beb/attachment.html>

From kristjan at ccpgames.com  Sun Apr 20 17:07:39 2014
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Sun, 20 Apr 2014 15:07:39 +0000
Subject: [Python-Dev] PEP 469: Restoring
	the	iterkeys/values/items()	methods
In-Reply-To: <20140419131456.GK28400@ando>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <20140419113537.436ce991@fsol>
 <EFE3877620384242A686D52278B7CCD3A52D9701@RKV-IT-EXCH104.ccp.ad.local>
 <20140419131456.GK28400@ando>
Message-ID: <EFE3877620384242A686D52278B7CCD3A52D9D10@RKV-IT-EXCH104.ccp.ad.local>



> -----Original Message-----
> From: Python-Dev [mailto:python-dev-
> bounces+kristjan=ccpgames.com at python.org] On Behalf Of Steven
> On Sat, Apr 19, 2014 at 11:41:35AM +0000, Kristj?n Valur J?nsson wrote:
> > Wouldn't "iterkeys" simply be an alias for "keys" and so on?
> > I'm +1 on that.
> 
> No.
> 
> [steve at ando ~]$ python2.7 -c "it = {}.iterkeys(); print it is iter(it)"
> True
> [steve at ando ~]$ python3.3 -c "it = {}.keys(); print(it is iter(it))"
> False
> 
Does one ever use iteritems() et al without first invoking iter() on it?  I.e. is it important that it is an iterator, rather than an iterable?
I think we could easily relax that requirement in the pep and solve 99% of the use cases.

K

From jeanpierreda at gmail.com  Sun Apr 20 22:03:03 2014
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Sun, 20 Apr 2014 13:03:03 -0700
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <wyx17cmjwluplxodq7t7wcsj.1398022023996@email.android.com>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <20140419113537.436ce991@fsol>
 <EFE3877620384242A686D52278B7CCD3A52D9701@RKV-IT-EXCH104.ccp.ad.local>
 <20140419131456.GK28400@ando>
 <EFE3877620384242A686D52278B7CCD3A52D9D10@RKV-IT-EXCH104.ccp.ad.local>
 <20140420170419.GM28400@ando>
 <wyx17cmjwluplxodq7t7wcsj.1398022023996@email.android.com>
Message-ID: <CABicbJJsbQ5TSivKWnVUAUOU=oSqqceVHo4WsCGTJYAehBwG+A@mail.gmail.com>

On Sun, Apr 20, 2014 at 12:27 PM, Kristj?n Valur J?nsson
<kristjan at ccpgames.com> wrote:
> Well, "for i in x" and other iteration constructs already call "iter ()" on
> their iterable. That's the point. Unless you want to manually iterate using
> "next ()" then the distinction between an iterable and an iterator is
> academic.

Or unless you iterate over the same thing multiple times, which can happen.

P.S.: If Python had intended to support 2.x/3.x polyglots in the first
place, would iteritems etc. have been removed? My feeling is "no", in
which case they should be re-added, since this is the main supported
porting mechanism going forward.

-- Devin


> -------- Original message --------
> From: Steven D'Aprano
> Date:20/04/2014 17:05 (GMT+00:00)
> To: python-dev at python.org
> Subject: Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
> methods
>
> On Sun, Apr 20, 2014 at 03:07:39PM +0000, Kristj?n Valur J?nsson wrote:
>
>> Does one ever use iteritems() et al without first invoking iter() on
>> it?
>
> I can't speak for others, but I never invoke iteritems *with* iter().
> What would be the point? iteritems is documented as returning an
> interator.
>
> # never this
> for key, value in iter(mydict.iteritems()): ...
>
> # but this
> for key, value in mydict.iteritems(): ...
>
>
>> I.e. is it important that it is an iterator, rather than an
>> iterable? I think we could easily relax that requirement in the pep
>> and solve 99% of the use cases.
>
> And the other 1% of cases would be a land-mine waiting to blow the
> user's code up.
>
> Would it actually solve 99% of the use cases? Or only 90%? Or 50%? How
> do you know?
>
> In Python 2.7 iteritems() etc is documented as returning an iterator.
> That's a promise of the language, and people will rely on it. But they
> won't be able to rely on that promise in polygot 2+3 code -- exactly the
> use-case this PEP is trying to satisfy -- because the promise to return
> an iterator will be broken in 3.
>
> It would be actively misleading, since Python 3's iteritems() would
> return a view, not an iter, and it would fail at solving the backwards
> compatibility issue since views and iterators are not interchangeable
> except for the most basic use of iteration.
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/jeanpierreda%40gmail.com
>

From techtonik at gmail.com  Sun Apr 20 15:49:16 2014
From: techtonik at gmail.com (anatoly techtonik)
Date: Sun, 20 Apr 2014 16:49:16 +0300
Subject: [Python-Dev] Python usability study (Was:  Language Summit notes)
Message-ID: <CAPkN8xKZaQgiayK7Sk7=xVrrCD+zPToi4nWtLs__KR-R04Hyeg@mail.gmail.com>

On Thu, Apr 10, 2014 at 2:24 PM, Kushal Das <kushaldas at gmail.com> wrote:
> Glyph wants a PSF fund to a usability study on Python. There were a
> few other suggestion on PSF support for tooling development.

+2 on initiative

-- 
anatoly t.

From davvid at gmail.com  Sun Apr 20 22:28:08 2014
From: davvid at gmail.com (David Aguilar)
Date: Sun, 20 Apr 2014 13:28:08 -0700
Subject: [Python-Dev] subprocess.Popen and win32
In-Reply-To: <20140420114207.39674d1d@fsol>
References: <CAJDDKr7ouektX9J6ZMjfm_58_GP7DYa4bcqdL1BeAwZ_NkwAvQ@mail.gmail.com>
 <20140420114207.39674d1d@fsol>
Message-ID: <CAJDDKr52jjOHOF3FV57HAXoEp-erqRWRuPg-sLngFYaKiLRw0w@mail.gmail.com>

On Sun, Apr 20, 2014 at 2:42 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Sat, 19 Apr 2014 19:02:42 -0700
> David Aguilar <davvid at gmail.com> wrote:
>>
>> On python3, this still works for normal platforms, but on windows we
>> can't pass a list of byte strings. We have to pass a list of unicode
>> strings.
>
> Windows native APIs are unicode-based. It is actually necessary to pass
> *unicode* strings, not byte strings, if you want your code to be
> correct in the face of non-ASCII characters.
>
> Under other platforms, when unicode strings are passed, Python will
> encode them using the platform's detected encoding. So, unless your
> platform is somehow misconfigured, passing unicode strings will also
> work correctly there.
>
> (note this is under Python 3)

Curious.. what if I don't want the default encoding? On UNIX, I can
control what encoding is used because I can encoding unicode strings
into bytes and the programmer is in full control. I was mainly
surprised that this is valid code on unix, but not windows, and which
seems like a portability concern.

If I use unicode strings that means I'm beholden to the default
encoding. I do agree that utf-8 (python3) is the only sane encoding
(for filesystems, etc) which is why it's just a curious question, and
for my use case, the default encoding on python3 (utf-8) is good
enough.

The projects I work on (including at work, where there is a huge
python2 code base) are python2+3 compatible by necessity, so it seems
like the best solution would be to check the python version, rather
than the platform, before deciding whether or not to encode or decode
inputs before calling into subprocess. That works for me :-)

Thanks for the explanation.

ciao,
-- 
David

From mal at egenix.com  Sun Apr 20 23:11:00 2014
From: mal at egenix.com (M.-A. Lemburg)
Date: Sun, 20 Apr 2014 23:11:00 +0200
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
In-Reply-To: <CACBhJdGw5K2+ZzhMW16QS5t49eSm+o3GRxsmRe=haWWVWGtbdA@mail.gmail.com>
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>	<lip39g$jnr$1@ger.gmane.org>	<CAP1=2W4hHLn6rPS36bokKXgKnECDsFgKnp=TGJS_2MgvvcLmhA@mail.gmail.com>
 <CACBhJdGw5K2+ZzhMW16QS5t49eSm+o3GRxsmRe=haWWVWGtbdA@mail.gmail.com>
Message-ID: <535437E4.2070803@egenix.com>

On 18.04.2014 23:03, Ezio Melotti wrote:
> Hi,
> 
> On Thu, Apr 17, 2014 at 9:09 PM, Brett Cannon <bcannon at gmail.com> wrote:
>>
>>
>> On Thu Apr 17 2014 at 1:34:23 PM, Jurko Gospodneti?
>> <jurko.gospodnetic at pke.hr> wrote:
>>>
>>>    Hi.
>>>
>>> On 14.4.2014. 23:51, Brett Cannon wrote:
>>>> Now the question is whether the maintenance cost of having to rebuild
>>>> Python for a select number of stdlib modules is enough to warrant
>>>> putting in the effort to make this work.
>>>
>>>    I would really love to have better startup times in production, but I
>>> would also really hate to lose the ability to hack around in stdlib
>>> sources during development just to get better startup performance.
>>>
>>>    In general, what I really like about using Python for software
>>> development is the ability to open any stdlib file and easily go poking
>>> around using stuff like 'import pdb;pdb.set_trace()' or simple print
>>> statements. Researching mysterious behaviour is generally much much
>>> MUCH! easier (read: takes less hours/days/weeks) if it ends up leading
>>> into a stdlib Python module than if it takes you down into the bowels of
>>> some C module (think zipimport.c *grin*). Not to mention the effect that
>>> being able to quickly resolve a mystery by hacking on some Python
>>> internals leaves you feeling very satisfied, while having to entrench
>>> yourself in those internals for a long time just to find out you've made
>>> something foolish on your end leaves you feeling exhausted at best.
>>
>>
>> Freezing modules does not affect the ability to use gdb. And as long as you
>> set the appropriate __file__ values then tracebacks will contain even the
>> file line and location.
>>
> 
> Will the tracebacks only contain the line number or the line of code too?
> 
> I've seen tracebacks involving importlib,_bootstrap that didn't
> include the code, and I'm wondering if we will get something similar
> for all the other modules you are freezing:
> 
> Traceback (most recent call last):
>   File "/tmp/foo.py", line 7, in <module>
>     import email
>   File "<frozen importlib._bootstrap>", line 1561, in _find_and_load
>   File "<frozen importlib._bootstrap>", line 1519, in _find_and_load_unlocked
>   File "<frozen importlib._bootstrap>", line 1473, in _find_module
>   File "<frozen importlib._bootstrap>", line 1308, in find_module
>   File "<frozen importlib._bootstrap>", line 1284, in _get_loader
>   File "<frozen importlib._bootstrap>", line 1273, in _path_importer_cache
>   File "<frozen importlib._bootstrap>", line 1254, in _path_hooks
> TypeError: 'NoneType' object is not iterable

Yes, this is what you get for frozen modules.

If you'd like to play around with a frozen stdlib this, you can
have a look at PyRun (http://pyrun.org), which does this for Python 2
and will hopefully work for Python 3.4 soonish too.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From kwchan at tsingma.com.hk  Mon Apr 21 02:25:28 2014
From: kwchan at tsingma.com.hk (Ken Chan)
Date: Mon, 21 Apr 2014 08:25:28 +0800
Subject: [Python-Dev] Fw: fail to fetch python-dev for raspian with details
	of error message
Message-ID: <778B321CB94649898D4DDA676A51959F@ADB.local>

Dear sir,

I am working on WebIOPi where library of python-dev is needed, 
so I use "sudo apt-get install python-dev" to install it. 


Then returned message of:     fail to fetch http://mirrordirector.raspbian.org/raspbian/ wheezy......



  details of error message:
  Found Python 2.7.3rc2...
  Trying to install python-dev using apt-get
  Reading package lists... Done
  Building dependency tree
  Reading state information... Done
  The following extra packages will be installed:
    libexpat1-dev libssl-dev libssl-doc python2.7-dev
  The following NEW packages will be installed:
    libexpat1-dev libssl-dev libssl-doc python-dev python2.7-dev
  0 upgraded, 5 newly installed, 0 to remove and 3 not upgraded.
  Need to get 31.3 MB of archives.
  After this operation, 42.2 MB of additional disk space will be used.
  Err http://mirrordirector.raspbian.org/raspbian/ wheezy/main python2.7-dev armhf 2.7.3~rc2-2.1
    404  Not Found
  Get:1 http://mirrordirector.raspbian.org/raspbian/ wheezy/main python-dev all 2.7.3~rc2-1 [912 B]
  Get:2 http://mirrordirector.raspbian.org/raspbian/ wheezy/main libexpat1-dev armhf 2.1.0-1 [210 kB]
  Get:3 http://mirrordirector.raspbian.org/raspbian/ wheezy/main libssl-dev armhf 1.0.1c-4+rpi1 [1,494 kB]
  Get:4 http://mirrordirector.raspbian.org/raspbian/ wheezy/main libssl-doc all 1.0.1c-4+rpi1 [1,204 kB]
  Fetched 2,909 kB in 21s (136 kB/s)
  Failed to fetch http://mirrordirector.raspbian.org/raspbian/pool/main/p/python2.7/python2.7-dev_2.7.3~rc2-2.1_armhf.deb  404  Not Found
  E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
  Cannot install for Python 2.7.3rc2 : missing development headers

  Found Python 3.2.3...
  Trying to install python3-dev using apt-get
  Reading package lists... Done
  Building dependency tree
  Reading state information... Done
  The following extra packages will be installed:
    libexpat1-dev libpython3.2 libssl-dev libssl-doc python3.2-dev
  The following NEW packages will be installed:
    libexpat1-dev libpython3.2 libssl-dev libssl-doc python3-dev python3.2-dev
  0 upgraded, 6 newly installed, 0 to remove and 3 not upgraded.
  Need to get 31.0 MB/33.9 MB of archives.
  After this operation, 46.8 MB of additional disk space will be used.
  Err http://mirrordirector.raspbian.org/raspbian/ wheezy/main libpython3.2 armhf 3.2.3-2
    404  Not Found
  Err http://mirrordirector.raspbian.org/raspbian/ wheezy/main python3.2-dev armhf 3.2.3-2
    404  Not Found
  Get:1 http://mirrordirector.raspbian.org/raspbian/ wheezy/main python3-dev all 3.2.3-5 [1,060 B]
  Fetched 1,060 B in 0s (5,711 B/s)
  Failed to fetch http://mirrordirector.raspbian.org/raspbian/pool/main/p/python3.2/libpython3.2_3.2.3-2_armhf.deb  404  Not Found
  Failed to fetch http://mirrordirector.raspbian.org/raspbian/pool/main/p/python3.2/python3.2-dev_3.2.3-2_armhf.deb  404  Not Found
  E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
  Cannot install for Python 3.2.3 : missing development headers

  ERROR: WebIOPi cannot be installed - please check errors above






  Would you please to advise a solution.

  Many many thanks!

  Ken





          TIML MOM Limited
       
        ????????????
        
        P Please think of the environment before printing this email 


------------------------------------------------------------------------
        Disclaimer:
        This e-mail message (together with any attachments) is confidential to the addressee and may also be privileged. If you are not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this message is strictly prohibited. Please also notify the sender immediately by return e-mail and delete it from your system. Internet communications cannot be guaranteed to be secure or error-free. The sender and the entity through which this message is sent therefore  
     
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140421/574f6dbb/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 1089 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140421/574f6dbb/attachment.png>

From steve at pearwood.info  Mon Apr 21 03:38:57 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 21 Apr 2014 11:38:57 +1000
Subject: [Python-Dev] PEP 469: Restoring the	iterkeys/values/items()
	methods
In-Reply-To: <wyx17cmjwluplxodq7t7wcsj.1398022023996@email.android.com>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <20140419113537.436ce991@fsol>
 <EFE3877620384242A686D52278B7CCD3A52D9701@RKV-IT-EXCH104.ccp.ad.local>
 <20140419131456.GK28400@ando>
 <wyx17cmjwluplxodq7t7wcsj.1398022023996@email.android.com>
Message-ID: <20140421013857.GN28400@ando>

On Sun, Apr 20, 2014 at 07:27:08PM +0000, Kristj?n Valur J?nsson wrote:

> Well, "for i in x" and other iteration constructs already call "iter 
> ()" on their iterable. That's the point. Unless you want to manually 
> iterate using "next ()" then the distinction between an iterable and 
> an iterator is academic.

Do you think that for loops are the only way that iteritems and 
friends get used? You can't know that. I know from my own code that it 
isn't true. Iterators are first class objects like any other object, and 
they get put into tuples, passed around as arguments to functions, 
stored in variables, and so on. They get tested to be iterators, e.g.:

    assert the_items is iter(the_items)

and, yes, sometimes people call next() on them directly.

There is a reason that viewitems() etc. were added to Python 2.7 rather 
than just changing iteritems() to return a view. It would break backward 
compatibility, and break people's code. How much code? I don't know, but 
*any* breakage defeats the purpose of this suggestion. The whole point 
of this PEP is for the iter* methods to work the same way in Python 2 
and Python 3, not "nearly the same, sorry if it breaks your code".

If we wanted "nearly the same", we already have that: dict.items() in 
Python 2 and 3 is already nearly the same, they are both iterable. 
Nearly the same is not good enough.

If this is a cunning plan to make Nick's suggestion sound better by 
suggesting an even worse alternative, it's working :-)



-- 
Steven

From tjreedy at udel.edu  Mon Apr 21 02:54:19 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 20 Apr 2014 20:54:19 -0400
Subject: [Python-Dev] Fw: fail to fetch python-dev for raspian with
 details of error message
In-Reply-To: <778B321CB94649898D4DDA676A51959F@ADB.local>
References: <778B321CB94649898D4DDA676A51959F@ADB.local>
Message-ID: <53546C3B.4040300@udel.edu>

On 4/20/2014 8:25 PM, Ken Chan wrote:

Please send this instead to python-list, where you might find other 
raspian users. Pydev is for development of future version of python, not 
use of current versions.

-- 
Terry Jan Reedy

From cs at zip.com.au  Mon Apr 21 05:01:35 2014
From: cs at zip.com.au (Cameron Simpson)
Date: Mon, 21 Apr 2014 13:01:35 +1000
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
 methods
In-Reply-To: <lj0i90$a43$1@ger.gmane.org>
References: <lj0i90$a43$1@ger.gmane.org>
Message-ID: <20140421030135.GA51021@cskk.homeip.net>

On 20Apr2014 14:32, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
>On 20/04/2014 06:31, Ethan Furman wrote:
>>Thank you for taking the time to write this up, Nick.
>>
>>However, I am -1 on it.  One of the allures of Python 3 is the increase
>>in simplicity and elegance.  Restoring cruft does not help with that.
>>Python 2 idioms that get restored to Python 3 must have real value:
>>unicode literals, wire-protocol interpolations -- I don't feel that this
>>comes any where close to meeting that bar.
>>~Ethan~
>
>+1 for this summary which echoes my sentiments entirely.

Me too. I'm against iteritems and friends coming back.

I've been burned in the past with the burden of writing a mapping class with 
the many methods such a thing must support; both items() and iteritems() and so 
forth. For the example I have in mind I eventually abandoned the objective of 
being a full mapping and am going back to just a few methods to support easy 
element access such as __getitem__ and __contains__.

I have a small python module of my own to aid my python 2+3 efforts, and am of 
the opinion that it is better to add iteritems elper functions to a popular 
module like six than to left the noise back into the python 3 mapping 
interface.

Cheers,
Cameron Simpson <cs at zip.com.au>

From jeanpierreda at gmail.com  Mon Apr 21 05:12:54 2014
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Sun, 20 Apr 2014 20:12:54 -0700
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <20140421030135.GA51021@cskk.homeip.net>
References: <lj0i90$a43$1@ger.gmane.org>
 <20140421030135.GA51021@cskk.homeip.net>
Message-ID: <CABicbJK9OVNmKdm1cae7iZCGtVwkhLmuv=i16-frc8aoJFe_-Q@mail.gmail.com>

On Sun, Apr 20, 2014 at 8:01 PM, Cameron Simpson <cs at zip.com.au> wrote:
> On 20Apr2014 14:32, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
>>
>> On 20/04/2014 06:31, Ethan Furman wrote:
>>>
>>> Thank you for taking the time to write this up, Nick.
>>>
>>> However, I am -1 on it.  One of the allures of Python 3 is the increase
>>> in simplicity and elegance.  Restoring cruft does not help with that.
>>> Python 2 idioms that get restored to Python 3 must have real value:
>>> unicode literals, wire-protocol interpolations -- I don't feel that this
>>> comes any where close to meeting that bar.
>>> ~Ethan~
>>
>>
>> +1 for this summary which echoes my sentiments entirely.
>
>
> Me too. I'm against iteritems and friends coming back.
>
> I've been burned in the past with the burden of writing a mapping class with
> the many methods such a thing must support; both items() and iteritems() and
> so forth. For the example I have in mind I eventually abandoned the
> objective of being a full mapping and am going back to just a few methods to
> support easy element access such as __getitem__ and __contains__.

As far as I know, all you have to implement yourself, to support the
dict-like interface, are:

 - __getitem__
 - __setitem__
 - __delitem__
 - __iter__
 - __len__

MutableMapping can implement the rest. This wouldn't change with the
re-addition of the iter* methods.

You really have to use MutableMapping now that keys/items/values are
complicated objects: it's much harder to implement dict-like objects
from scratch in 3.x than 2.x.

-- Devin

From cs at zip.com.au  Mon Apr 21 05:50:28 2014
From: cs at zip.com.au (Cameron Simpson)
Date: Mon, 21 Apr 2014 13:50:28 +1000
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
 methods
In-Reply-To: <CABicbJK9OVNmKdm1cae7iZCGtVwkhLmuv=i16-frc8aoJFe_-Q@mail.gmail.com>
References: <CABicbJK9OVNmKdm1cae7iZCGtVwkhLmuv=i16-frc8aoJFe_-Q@mail.gmail.com>
Message-ID: <20140421035028.GA23279@cskk.homeip.net>

On 20Apr2014 20:12, Devin Jeanpierre <jeanpierreda at gmail.com> wrote:
>On Sun, Apr 20, 2014 at 8:01 PM, Cameron Simpson <cs at zip.com.au> wrote:
>> Me too. I'm against iteritems and friends coming back.
>> I've been burned in the past with the burden of writing a mapping class with
>> the many methods such a thing must support; both items() and iteritems() and
>> so forth. [...]
>
>As far as I know, all you have to implement yourself, to support the
>dict-like interface, are:
>
> - __getitem__
> - __setitem__
> - __delitem__
> - __iter__
> - __len__
>
>MutableMapping can implement the rest. This wouldn't change with the
>re-addition of the iter* methods.
>
>You really have to use MutableMapping now that keys/items/values are
>complicated objects: it's much harder to implement dict-like objects
>from scratch in 3.x than 2.x.

Fair point.

Thank you,
Cameron Simpson <cs at zip.com.au>

Draw little boxes with arrows.  It helps.       - Michael J. Eager

From stephen at xemacs.org  Mon Apr 21 07:21:49 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 21 Apr 2014 14:21:49 +0900
Subject: [Python-Dev] PEP 469: Restoring the
	iterkeys/values/items()	methods
In-Reply-To: <lj0nl8$bne$1@ger.gmane.org>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <20140419042553.GJ28400@ando>
 <CADiSq7dQyXMcBY-fhUgB1+6204+wwvBW1=uWvyze9V-ZCbQy-A@mail.gmail.com>
 <20140420024955.GL28400@ando>
 <CACac1F9k5Nvtqs0MizmxoGfArgh7Z19VJP86raDx-xp0SpAY7A@mail.gmail.com>
 <lj0nl8$bne$1@ger.gmane.org>
Message-ID: <87wqejqmtu.fsf@uwakimon.sk.tsukuba.ac.jp>

Tres Seaver writes:

 > Re-adding features to make the strategy that works less painful is
 > just acknowledging that fact.

Whether the strategy that works was anticipated is irrelevant, and the
fact that pain *would* be involved was acknowledged all the way back
to the days when "Python 3000" was still a python-dev in joke rather
than something that downstream needed to think about for the future.

The question is "how much pain", and I'm sorry, but I don't see that
the .iterthingee methods involve so much pain.  The request for
explanation and quantification seems eminently reasonable to me.

 > Mark such features as BBB-only / deprecated-but-never-to-be-removed, and
 > move on:  "practicality beats purity".

Since your statement is a first-order sentence, it's implicitly
universally quantified.  "All" is a *lot* of cruft, Tres!  Where do
*you* propose finally saying "the cruft stops here"?

Also, whatever cruft ends up being included *will* be propagated
forward in code that does *not* need it, including new code.  Most
"new" code is plagiarized to some degree, and people plagiarize not
with a critic's eye, but with an eye to "does the API have the
semantics I need when it calls that code?"

Nor do they enable deprecation notices, or read documentation if the
reused code Just Works....



From ncoghlan at gmail.com  Mon Apr 21 07:39:42 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 21 Apr 2014 01:39:42 -0400
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <CADiSq7d=cQdB=51y-1LqBA6Aaamu=OeQn7xsxeCWV7YYMO5WiQ@mail.gmail.com>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <1397876893.11662.108082729.0AF51881@webmail.messagingengine.com>
 <CADiSq7dP9bSi+zWR+nXXCX5ySveh3AqyeZpGGGN1_USLB9nt2w@mail.gmail.com>
 <CAP7+vJJsxBbXcVSHUvvQZWYgXmotNw+2qpGxs4FcDtHi3WWF4w@mail.gmail.com>
 <CADiSq7d=cQdB=51y-1LqBA6Aaamu=OeQn7xsxeCWV7YYMO5WiQ@mail.gmail.com>
Message-ID: <CADiSq7ezB6+6zAL71r1yC4aza-R6B3i6y-U7Sh20OC9vy7RN8w@mail.gmail.com>

On 19 April 2014 12:17, Nick Coghlan <ncoghlan at gmail.com> wrote:
> That may help clarify the tricky warts and edge cases that can arise when
> moving from the current relatively straightforward and consistent method
> based approach to a more complicated combination of dedicated syntax and
> helper functions.

OK, I've now updated the PEP to better described the *problem* (rather
than skipping ahead to proposing a specific solution - exactly what I
was asking people *not* to do at the language summit!), and I now
think this is better handled by tweaking the strategies people use
when writing or converting to hybrid code, rather than by making any
changes to Python 3. The use of more appropriate helper functions
should actually make the hybrid code *cleaner* than it would be with
any Python 3 API restorations. Accordingly, the PEP itself is now
marked as Withdrawn, but it also goes into a lot more detail on the
Python 2 mapping iteration APIs, how 2to3 translates them to Python 3,
and how I now suggest people translate them to the common subset of
Python 2 and 3.

Full text of the revised PEP is below, and it has also been updated at
http://www.python.org/dev/peps/pep-0469/

Cheers,
Nick.

=========================================
PEP: 469
Title: Migration of dict iteration code to Python 3
Version: $Revision$
Last-Modified: $Date$
Author: Nick Coghlan <ncoghlan at gmail.com>
Status: Withdrawn
Type: Standards Track
Content-Type: text/x-rst
Created: 2014-04-18
Python-Version: 3.5
Post-History: 2014-04-18, 2014-04-21


Abstract
========

For Python 3, PEP 3106 changed the design of the ``dict`` builtin and the
mapping API in general to replace the separate list based and iterator based
APIs in Python 2 with a merged, memory efficient set and multiset view
based API. This new style of dict iteration was also added to the Python 2.7
``dict`` type as a new set of iteration methods.

This means that there are now 3 different kinds of dict iteration that may
need to be migrated to Python 3 when an application makes the transition:

* Lists as mutable snapshots: ``d.items()`` -> ``list(d.items())``
* Iterator objects: ``d.iteritems()`` -> ``iter(d.items())``
* Set based dynamic views: ``d.viewitems()`` -> ``d.items()``

There is currently no widely agreed best practice on how to reliably convert
all Python 2 dict iteration code to the common subset of Python 2 and 3,
especially when test coverage of the ported code is limited. This PEP
reviews the various ways the Python 2 iteration APIs may be accessed, and
looks at the available options for migrating that code to Python 3 by way of
the common subset of Python 2.6+ and Python 3.0+.

The PEP also considers the question of whether or not there are any
additions that may be worth making to Python 3.5 that may ease the
transition process for application code that doesn't need to worry about
supporting earlier versions when eventually making the leap to Python 3.


PEP Withdrawal
==============

In writing the second draft of this PEP, I came to the conclusion that
the readability of hybrid Python 2/3 mapping code can actually be best
enhanced by better helper functions rather than by making changes to
Python 3.5+. The main value I now see in this PEP is as a clear record
of the recommended approaches to migrating mapping iteration code from
Python 2 to Python 3, as well as suggesting ways to keep things readable
and maintainable when writing hybrid code that supports both versions.

Notably, I recommend that hybrid code avoid calling mapping iteration
methods directly, and instead rely on builtin functions where possible,
and some additional helper functions for cases that would be a simple
combination of a builtin and a mapping method in pure Python 3 code, but
need to be handled slightly differently to get the exact same semantics in
Python 2.

Static code checkers like pylint could potentially be extended with an
optional warning regarding direct use of the mapping iteration methods in
a hybrid code base.


Mapping iteration models
========================

Python 2.7 provides three different sets of methods to extract the keys,
values and items from a ``dict`` instance, accounting for 9 out of the
18 public methods of the ``dict`` type.

In Python 3, this has been rationalised to just 3 out of 11 public methods
(as the ``has_key`` method has also been removed).


Lists as mutable snapshots
--------------------------

This is the oldest of the three styles of dict iteration, and hence the
one implemented by the ``d.keys()``, ``d.values()`` and ``d.items()``
methods in Python 2.

These methods all return lists that are snapshots of the state of the
mapping at the time the method was called. This has a few consequences:

* the original object can be mutated freely without affecting iteration
  over the snapshot
* the snapshot can be modified independently of the original object
* the snapshot consumes memory proportional to the size of the original
  mapping

The semantic equivalent of these operations in Python 3 are
``list(d.keys())``, ``list(d.values())`` and ``list(d.iteritems())``.


Iterator objects
----------------

In Python 2.2, ``dict`` objects gained support for the then-new iterator
protocol, allowing direct iteration over the keys stored in the dictionary,
thus avoiding the need to build a list just to iterate over the dictionary
contents one entry at a time. ``iter(d)`` provides direct access to the
iterator object for the keys.

Python 2 also provides a ``d.iterkeys()`` method that is essentially
synonymous with ``iter(d)``, along with ``d.itervalues()`` and
``d.iteritems()`` methods.

These iterators provide live views of the underlying object, and hence may
fail if the set of keys in the underlying object is changed during
iteration::

    >>> d = dict(a=1)
    >>> for k in d:
    ...     del d[k]
    ...
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    RuntimeError: dictionary changed size during iteration

As iterators, iteration over these objects is also a one-time operation:
once the iterator is exhausted, you have to go back to the original mapping
in order to iterate again.

In Python 3, direct iteration over mappings works the same way as it does
in Python 2. There are no method based equivalents - the semantic equivalents
of ``d.itervalues()`` and ``d.iteritems()`` in Python 3 are
``iter(d.values())`` and ``iter(d.iteritems())``.

The ``six`` and ``future.utils`` compatibility modules also both provide
``iterkeys()``, ``itervalues()`` and ``iteritems()`` helper functions that
provide efficient iterator semantics in both Python 2 and 3.


Set based dynamic views
-----------------------

The model that is provided in Python 3 as a method based API is that of set
based dynamic views (technically multisets in the case of the ``values()``
view).

In Python 3, the objects returned by ``d.keys()``, ``d.values()`` and
``d. items()`` provide a live view of the current state of
the underlying object, rather than taking a full snapshot of the current
state as they did in Python 2. This change is safe in many circumstances,
but does mean that, as with the direct iteration API, it is necessary to
avoid adding or removing keys during iteration, in order to avoid
encountering the following error::

    >>> d = dict(a=1)
    >>> for k, v in d.items():
    ...     del d[k]
    ...
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    RuntimeError: dictionary changed size during iteration

Unlike the iteration API, these objects are iterables, rather than iterators:
you can iterate over them multiple times, and each time they will iterate
over the entire underlying mapping.

These semantics are also available in Python 2.7 as the ``d.viewkeys()``,
``d.viewvalues()`` and ```d.viewitems()`` methods.

The ``future.utils`` compatibility module also provides
``viewkeys()``, ``viewvalues()`` and ``viewitems()`` helper functions
when running on Python 2.7 or Python 3.x.


Migrating directly to Python 3
==============================

The ``2to3`` migration tool handles direct migrations to Python 3 in
accordance with the semantic equivalents described above:

* ``d.keys()`` -> ``list(d.keys())``
* ``d.values()`` -> ``list(d.values())``
* ``d.items()`` -> ``list(d.items())``
* ``d.iterkeys()`` -> ``iter(d.keys())``
* ``d.itervalues()`` -> ``iter(d.values())``
* ``d.iteritems()`` -> ``iter(d.items())``
* ``d.viewkeys()`` -> ``d.keys()``
* ``d.viewvalues()`` -> ``d.values()``
* ``d.viewitems()`` -> ``d.items()``

Rather than 9 distinct mapping methods for iteration, there are now only the
3 view methods, which combine in straightforward ways with the two relevant
builtin functions to cover all of the behaviours that are available as
``dict`` methods in Python 2.7.

Note that in many cases ``d.keys()`` can be replaced by just ``d``, but the
``2to3`` migration tool doesn't attempt that replacement.

The ``2to3`` migration tool also *does not* provide any automatic assistance
for migrating references to these objects as bound or unbound methods - it
only automates conversions where the API is called immediately.


Migrating to the common subset of Python 2 and 3
================================================

When migrating to the common subset of Python 2 and 3, the above
transformations are not generally appropriate, as they all either result in
the creation of a redundant list in Python 2, have unexpectedly different
semantics in at least some cases, or both.

Since most code running in the common subset of Python 2 and 3 supports
at least as far back as Python 2.6, the currently recommended approach to
conversion of mapping iteration operation depends on two helper functions
for efficient iteration over mapping values and mapping item tuples:

* ``d.keys()`` -> ``list(d)``
* ``d.values()`` -> ``list(itervalues(d))``
* ``d.items()`` -> ``list(iteritems(d))``
* ``d.iterkeys()`` -> ``iter(d)``
* ``d.itervalues()`` -> ``itervalues(d)``
* ``d.iteritems()`` -> ``iteritems(d)``

Both ``six`` and ``future.utils`` provide appropriate definitions of
``itervalues()`` and ``iteritems()`` (along with essentially redundant
definitions of ``iterkeys()``). Creating your own definitions of these
functions in a custom compatibility module is also relatively
straightforward::

    try:
        dict.iteritems
    except AttributeError:
        # Python 3
        def itervalues(d):
            return iter(d.values())
        def iteritems(d):
            return iter(d.items())
    else:
        # Python 2
        def itervalues(d):
            return d.itervalues()
        def iteritems(d):
            return d.iteritems()

The greatest loss of readability currently arises when converting code that
actually *needs* the list based snapshots that were the default in Python
2. This readability loss could likely be mitigated by also providing
``listvalues`` and ``listitems`` helper functions, allowing the affected
conversions to be simplified to:

* ``d.values()`` -> ``listvalues(d)``
* ``d.items()`` -> ``listitems(d)``

The corresponding compatibility function definitions are as straightforward
as their iterator counterparts::

    try:
        dict.iteritems
    except AttributeError:
        # Python 3
        def listvalues(d):
            return list(d.values())
        def listitems(d):
            return list(d.items())
    else:
        # Python 2
        def listvalues(d):
            return d.values()
        def listitems(d):
            return d.items()

With that expanded set of compatibility functions, Python 2 code would
then be converted to "idiomatic" hybrid 2/3 code as:

* ``d.keys()`` -> ``list(d)``
* ``d.values()`` -> ``listvalues(d)``
* ``d.items()`` -> ``listitems(d)``
* ``d.iterkeys()`` -> ``iter(d)``
* ``d.itervalues()`` -> ``itervalues(d)``
* ``d.iteritems()`` -> ``iteritems(d)``

This compares well for readability with the idiomatic pure Python 3
code that uses the mapping methods and builtins directly:

* ``d.keys()`` -> ``list(d)``
* ``d.values()`` -> ``list(d.values())``
* ``d.items()`` -> ``list(d.items())``
* ``d.iterkeys()`` -> ``iter(d)``
* ``d.itervalues()`` -> ``iter(d.values())``
* ``d.iteritems()`` -> ``iter(d.items())``

It's also notable that when using this approach, hybrid code would *never*
invoke the mapping methods directly: it would always invoke either a
builtin or helper function instead, in order to ensure the exact same
semantics on both Python 2 and 3.


Possible changes to Python 3.5+
===============================

The main proposal put forward to potentially aid migration of existing
Python 2 code to Python 3 is the restoration of some or all of the
alternate iteration APIs to the Python 3 mapping API. In particular,
the initial draft of this PEP proposed making the following conversions
possible when migrating to the common subset of Python 2 and Python 3.5+:

* ``d.keys()`` -> ``list(d)``
* ``d.values()`` -> ``list(d.itervalues())``
* ``d.items()`` -> ``list(d.iteritems())``
* ``d.iterkeys()`` -> ``d.iterkeys()``
* ``d.itervalues()`` -> ``d.itervalues()``
* ``d.iteritems()`` -> ``d.iteritems()``

Possible mitigations of the additional language complexity in Python 3
created by restoring these methods included immediately deprecating them,
as well as potentially hiding them from the ``dir()`` function (or perhaps
even defining a way to make ``pydoc`` aware of function deprecations).

However, in the case where the list output is actually desired, the end
result of that proposal is actually less readable than an appropriately
defined helper function, and the function and method forms of the iterator
versions are pretty much equivalent from a readability perspective.

So unless I've missed something critical, readily available ``listvalues()``
and ``listitems()`` helper functions look like they will improve the
readability of hybrid code more than anything we could add back to the
Python 3.5+ mapping API, and won't have any long term impact on the
complexity of Python 3 itself.


Discussion
==========

The fact that 5 years in to the Python 3 migration we still have users
considering the dict API changes a significant barrier to migration suggests
that there are problems with previously recommended approaches. This PEP
attempts to explore those issues and tries to isolate those cases where
previous advice (such as it was) could prove problematic.

My assessment (largely based on feedback from Twisted devs) is that
problems are most likely to arise when attempting to use ``d.keys()``,
``d.values()``, and ``d.items()`` in hybrid code. While superficially it
seems as though there should be cases where it is safe to ignore the
semantic differences, in practice, the change from "mutable snapshot" to
"dynamic view" is significant enough that it is likely better
to just force the use of either list or iterator semantics for hybrid code,
and leave the use of the view semantics to pure Python 3 code.

This approach also creates rules that are simple enough and safe enough that
it should be possible to automate them in code modernisation scripts that
target the common subset of Python 2 and Python 3, just as ``2to3`` converts
them automatically when targeting pure Python 3 code.


Acknowledgements
================

Thanks to the folks at the Twisted sprint table at PyCon for a very
vigorous discussion of this idea (and several other topics), and especially
to Hynek Schlawack for acting as a moderator when things got a little too
heated :)

Thanks also to JP Calderone and Itamar Turner-Trauring for their email
feedback, as well to the participants in the `python-dev review
<https://mail.python.org/pipermail/python-dev/2014-April/134168.html>`__ of
the initial version of the PEP.


Copyright
=========

This document has been placed in the public domain.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From tjreedy at udel.edu  Mon Apr 21 09:34:15 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 21 Apr 2014 03:34:15 -0400
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <CADiSq7ezB6+6zAL71r1yC4aza-R6B3i6y-U7Sh20OC9vy7RN8w@mail.gmail.com>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <1397876893.11662.108082729.0AF51881@webmail.messagingengine.com>
 <CADiSq7dP9bSi+zWR+nXXCX5ySveh3AqyeZpGGGN1_USLB9nt2w@mail.gmail.com>
 <CAP7+vJJsxBbXcVSHUvvQZWYgXmotNw+2qpGxs4FcDtHi3WWF4w@mail.gmail.com>
 <CADiSq7d=cQdB=51y-1LqBA6Aaamu=OeQn7xsxeCWV7YYMO5WiQ@mail.gmail.com>
 <CADiSq7ezB6+6zAL71r1yC4aza-R6B3i6y-U7Sh20OC9vy7RN8w@mail.gmail.com>
Message-ID: <lj2hmk$s3s$1@ger.gmane.org>

On 4/21/2014 1:39 AM, Nick Coghlan wrote:

> OK, I've now updated the PEP to better described the *problem* (rather
> than skipping ahead to proposing a specific solution - exactly what I
> was asking people *not* to do at the language summit!),

Looks great. I think the analysis should be part of a dict section of 
the porting how-to.

-- 
Terry Jan Reedy


From arigo at tunes.org  Mon Apr 21 09:41:50 2014
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 21 Apr 2014 09:41:50 +0200
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <CADiSq7ezB6+6zAL71r1yC4aza-R6B3i6y-U7Sh20OC9vy7RN8w@mail.gmail.com>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <1397876893.11662.108082729.0AF51881@webmail.messagingengine.com>
 <CADiSq7dP9bSi+zWR+nXXCX5ySveh3AqyeZpGGGN1_USLB9nt2w@mail.gmail.com>
 <CAP7+vJJsxBbXcVSHUvvQZWYgXmotNw+2qpGxs4FcDtHi3WWF4w@mail.gmail.com>
 <CADiSq7d=cQdB=51y-1LqBA6Aaamu=OeQn7xsxeCWV7YYMO5WiQ@mail.gmail.com>
 <CADiSq7ezB6+6zAL71r1yC4aza-R6B3i6y-U7Sh20OC9vy7RN8w@mail.gmail.com>
Message-ID: <CAMSv6X3nEEuKK7YQiuzu==sjhwdKK_nm5Q-xXo_KejjXhma8gQ@mail.gmail.com>

Hi Nick,

On 21 April 2014 07:39, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Notably, I recommend that hybrid code avoid calling mapping iteration
> methods directly, and instead rely on builtin functions where possible,
> and some additional helper functions for cases that would be a simple
> combination of a builtin and a mapping method in pure Python 3 code, but
> need to be handled slightly differently to get the exact same semantics in
> Python 2.

How about explicitly noting that in Python 2, a large fraction of
usages of the iterkeys(), itervalues() and iteritems() methods (that's
more than 99% in my experience, but I might be biased) should just be
written as keys(), values() and items() in the first place, with no
measurable difference of performance or memory usage?  I would
recommend to anyone with a large Python 2 code base to simply do a
textual search-and-replace and see if any test breaks.  If not,
problem solved.


A bient?t,

Armin.

From ethan at stoneleaf.us  Mon Apr 21 17:23:42 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 21 Apr 2014 08:23:42 -0700
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <CADiSq7ezB6+6zAL71r1yC4aza-R6B3i6y-U7Sh20OC9vy7RN8w@mail.gmail.com>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <1397876893.11662.108082729.0AF51881@webmail.messagingengine.com>
 <CADiSq7dP9bSi+zWR+nXXCX5ySveh3AqyeZpGGGN1_USLB9nt2w@mail.gmail.com>
 <CAP7+vJJsxBbXcVSHUvvQZWYgXmotNw+2qpGxs4FcDtHi3WWF4w@mail.gmail.com>
 <CADiSq7d=cQdB=51y-1LqBA6Aaamu=OeQn7xsxeCWV7YYMO5WiQ@mail.gmail.com>
 <CADiSq7ezB6+6zAL71r1yC4aza-R6B3i6y-U7Sh20OC9vy7RN8w@mail.gmail.com>
Message-ID: <535537FE.80108@stoneleaf.us>

On 04/20/2014 10:39 PM, Nick Coghlan wrote:
> Lists as mutable snapshots
> --------------------------
>
[...]
>
> The semantic equivalent of these operations in Python 3 are
> ``list(d.keys())``, ``list(d.values())`` and ``list(d.iteritems())``.

Last item should be ``list(d.items())``.


> Iterator objects
> ----------------
>
[...]
>
> In Python 3, direct iteration over mappings works the same way as it does
> in Python 2. There are no method based equivalents - the semantic equivalents
> of ``d.itervalues()`` and ``d.iteritems()`` in Python 3 are
> ``iter(d.values())`` and ``iter(d.iteritems())``.


``iter(d.items())``.


Thanks again, Nick.

--
~Ethan~

From kristjan at ccpgames.com  Mon Apr 21 16:02:12 2014
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Mon, 21 Apr 2014 14:02:12 +0000
Subject: [Python-Dev] PEP 469: Restoring
	the	iterkeys/values/items()	methods
In-Reply-To: <20140421013857.GN28400@ando>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <20140419113537.436ce991@fsol>
 <EFE3877620384242A686D52278B7CCD3A52D9701@RKV-IT-EXCH104.ccp.ad.local>
 <20140419131456.GK28400@ando>
 <wyx17cmjwluplxodq7t7wcsj.1398022023996@email.android.com>
 <20140421013857.GN28400@ando>
Message-ID: <EFE3877620384242A686D52278B7CCD3A52DA11E@RKV-IT-EXCH104.ccp.ad.local>



> -----Original Message-----
> From: Python-Dev [mailto:python-dev-
> bounces+kristjan=ccpgames.com at python.org] On Behalf Of Steven
> D'Aprano
> If this is a cunning plan to make Nick's suggestion sound better by suggesting
> an even worse alternative, it's working :-)

You got me.
However, I also admit to having learned something today from the PEP.  That 2to3 actually replaces d.iteritems() with iter(d.items()).
In all my porting experience this conservative approach is redundant for my use cases which is usally just immediate iteration, so I have successfully replaced d.iteritems() with d.items() without issue.  For polyglot code, with rare exceptions simply using d.items() in both places is good enough, since IMHO the ill effects of creating temporary list objects is somewhat overstated.

The PEP however explicitly wants to do it "correctly" because testing is often limited.  So I withdraw my suggestion :)

K

From luca.sbardella at gmail.com  Mon Apr 21 22:02:47 2014
From: luca.sbardella at gmail.com (Luca Sbardella)
Date: Mon, 21 Apr 2014 21:02:47 +0100
Subject: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
	methods
In-Reply-To: <CAMSv6X3nEEuKK7YQiuzu==sjhwdKK_nm5Q-xXo_KejjXhma8gQ@mail.gmail.com>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <1397876893.11662.108082729.0AF51881@webmail.messagingengine.com>
 <CADiSq7dP9bSi+zWR+nXXCX5ySveh3AqyeZpGGGN1_USLB9nt2w@mail.gmail.com>
 <CAP7+vJJsxBbXcVSHUvvQZWYgXmotNw+2qpGxs4FcDtHi3WWF4w@mail.gmail.com>
 <CADiSq7d=cQdB=51y-1LqBA6Aaamu=OeQn7xsxeCWV7YYMO5WiQ@mail.gmail.com>
 <CADiSq7ezB6+6zAL71r1yC4aza-R6B3i6y-U7Sh20OC9vy7RN8w@mail.gmail.com>
 <CAMSv6X3nEEuKK7YQiuzu==sjhwdKK_nm5Q-xXo_KejjXhma8gQ@mail.gmail.com>
Message-ID: <CAF94-jYph7a5XcFLjGSNohAtf1QaSg1-Yygsn4aKhmC6c7jFPA@mail.gmail.com>

Hi Guys,
please don't do it!
We don't need this legacy really.


On 21 April 2014 08:41, Armin Rigo <arigo at tunes.org> wrote:

> Hi Nick,
>
> On 21 April 2014 07:39, Nick Coghlan <ncoghlan at gmail.com> wrote:
> > Notably, I recommend that hybrid code avoid calling mapping iteration
> > methods directly, and instead rely on builtin functions where possible,
> > and some additional helper functions for cases that would be a simple
> > combination of a builtin and a mapping method in pure Python 3 code, but
> > need to be handled slightly differently to get the exact same semantics
> in
> > Python 2.
>
> How about explicitly noting that in Python 2, a large fraction of
> usages of the iterkeys(), itervalues() and iteritems() methods (that's
> more than 99% in my experience, but I might be biased) should just be
> written as keys(), values() and items() in the first place, with no
> measurable difference of performance or memory usage?  I would
> recommend to anyone with a large Python 2 code base to simply do a
> textual search-and-replace and see if any test breaks.  If not,
> problem solved.
>
>
Couldn't agree more with Armin.


-- 
http://lucasbardella.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140421/83dea6fe/attachment.html>

From kristjan at ccpgames.com  Mon Apr 21 16:04:20 2014
From: kristjan at ccpgames.com (=?utf-8?B?S3Jpc3Rqw6FuIFZhbHVyIErDs25zc29u?=)
Date: Mon, 21 Apr 2014 14:04:20 +0000
Subject: [Python-Dev] PEP 469: Restoring the
	iterkeys/values/items()	methods
In-Reply-To: <CAMSv6X3nEEuKK7YQiuzu==sjhwdKK_nm5Q-xXo_KejjXhma8gQ@mail.gmail.com>
References: <CADiSq7cKMKFKeRiT10jpu55gbtBZ416PNtwv9PGO+L_Bo6JnhA@mail.gmail.com>
 <1397876893.11662.108082729.0AF51881@webmail.messagingengine.com>
 <CADiSq7dP9bSi+zWR+nXXCX5ySveh3AqyeZpGGGN1_USLB9nt2w@mail.gmail.com>
 <CAP7+vJJsxBbXcVSHUvvQZWYgXmotNw+2qpGxs4FcDtHi3WWF4w@mail.gmail.com>
 <CADiSq7d=cQdB=51y-1LqBA6Aaamu=OeQn7xsxeCWV7YYMO5WiQ@mail.gmail.com>
 <CADiSq7ezB6+6zAL71r1yC4aza-R6B3i6y-U7Sh20OC9vy7RN8w@mail.gmail.com>
 <CAMSv6X3nEEuKK7YQiuzu==sjhwdKK_nm5Q-xXo_KejjXhma8gQ@mail.gmail.com>
Message-ID: <EFE3877620384242A686D52278B7CCD3A52DA132@RKV-IT-EXCH104.ccp.ad.local>



> -----Original Message-----
> From: Python-Dev [mailto:python-dev-
> bounces+kristjan=ccpgames.com at python.org] On Behalf Of Armin Rigo
> Sent: 21. apr?l 2014 07:42
> To: Nick Coghlan
> Cc: Python-Dev
> Subject: Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items()
> methods
> 
> How about explicitly noting that in Python 2, a large fraction of usages of the
> iterkeys(), itervalues() and iteritems() methods (that's more than 99% in my
> experience, but I might be biased) should just be written as keys(), values()
> and items() in the first place, with no measurable difference of performance
> or memory usage?  I would recommend to anyone with a large Python 2
> code base to simply do a textual search-and-replace and see if any test
> breaks.  If not, problem solved.

+1

K

From Marius.Storm-Olsen at student.bi.no  Tue Apr 22 04:31:08 2014
From: Marius.Storm-Olsen at student.bi.no (Marius Storm-Olsen)
Date: Tue, 22 Apr 2014 02:31:08 +0000
Subject: [Python-Dev] Open Source Organizational Culture
Message-ID: <df50822389ef4e25ae0168f72cd691a7@DBXPR07MB110.eurprd07.prod.outlook.com>

Hi,

I would like to request your participation in a survey on
    Open Source Organizational Culture, 
which will provide valuable insight into how Open Source projects are run, how their participants act, how they might change going forward, and how particular Open Source projects compare with one another and with traditional business cultures. The survey will take 10-15 minutes to complete.

    http://bit.ly/osocas2014


Why?
----
The survey will be used as part of my thesis on Open Source Organizational Culture at BI Norwegian Business School (www.bi.no/en, or www.bi.edu), but in true Open Source spirit the raw - but anonymized - results will be open for all. So, your Open Source project will be able to massage and dissect the results any way you wish, and see how you compare with other projects out there.

Up until now, most research in Open Source culture has been based on mining mailing lists to find out how people act, who they interact with, and how projects organize themselves.

In this research we would rather ask the participants directly about how a project is managed and what should change for the project to be spectacularly successful.

When?
-----

The survey is open now through May 1st.

Where?
------

The bit.ly address brings you to the following survey

     https://osocas.2014.sgizmo.com/s3/

Remember that you can save your progress at any time and come back to the survey at a later point when you have time to finish it.

Who are you?
------------
My name is Marius Storm-Olsen, and I am currently working on a thesis on Open Source Organizational Culture. I've been an active part of Open Source for years, most notably on the Qt and Git projects. Although I have my own experiences to draw on in the thesis, they do not qualify for the Open Source community at large, hence the survey.

How to help?
------------
If you want to help, feel free to forward this email to any Open Source project you would want to participate the survey. Once you have send the invitation, please either send me an email with the name of the project, or update the table shown on

    https://github.com/mstormo/OSOCAS/wiki


I do hope you can participate, and thank you for your consideration!


Best regards,
Marius Storm-Olsen

From Marius.Storm-Olsen at student.bi.no  Tue Apr 22 06:13:04 2014
From: Marius.Storm-Olsen at student.bi.no (Marius Storm-Olsen)
Date: Tue, 22 Apr 2014 04:13:04 +0000
Subject: [Python-Dev] Open Source Organizational Culture
In-Reply-To: <df50822389ef4e25ae0168f72cd691a7@DBXPR07MB110.eurprd07.prod.outlook.com>
References: <df50822389ef4e25ae0168f72cd691a7@DBXPR07MB110.eurprd07.prod.outlook.com>
Message-ID: <5355EC42.5030303@student.bi.no>

On 4/21/2014 9:31 PM, Marius Storm-Olsen wrote:
> Hi,
>
> I would like to request your participation in a survey on
>      Open Source Organizational Culture,
> which will provide valuable insight into how Open Source projects are
> run, how their participants act, how they might change going forward,
> and how particular Open Source projects compare with one another and
> with traditional business cultures. The survey will take 10-15
> minutes to complete.
>
>      http://bit.ly/osocas2014
...
> The bit.ly address brings you to the following survey
>
>       https://osocas.2014.sgizmo.com/s3/

Turns out the osocas.2014.sgizmo.com survey subdomain gives an SSL 
warning for the *.sgizmo.com certificate. *face palm*

Feel free to use
      http://bit.ly/OSOCAS2014
instead, which will redirect to the non-subdomain version
      https://www.surveygizmo.com/s3/1587798/osocas-2014

Sorry for the inconvenience, and thanks to ChrisA for pointing out the 
issue!

Sincerely,
Marius Storm-Olsen



--

From raymond.hettinger at gmail.com  Tue Apr 22 08:39:13 2014
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Mon, 21 Apr 2014 23:39:13 -0700
Subject: [Python-Dev] List vs Tuple / Homogeneous vs Heterogeneous /
	Mutable vs Immutable
In-Reply-To: <5350E095.3070705@farowl.co.uk>
References: <CAEaFV_iasbNn0_jbN6XaR8+oNq5m+9vi1TfwTZcuHp7OnbqLSg@mail.gmail.com>
 <CAP1=2W6NMJdtnVr22=gqx8bOwoY7JM7uhk78V6D4iU_WBVF=cA@mail.gmail.com>
 <CAP7+vJKMyJ8sm3a3Hce3zngcmDz0SG+V4e31K6qFP38_-2LpYg@mail.gmail.com>
 <CAEaFV_jWbG+u6O5Zzy0+TvYc6otJ11dXPqYD7EXb0VhV+mdPFg@mail.gmail.com>
 <5350E095.3070705@farowl.co.uk>
Message-ID: <3F7F8B70-4558-42CA-BC12-9A241746B2D0@gmail.com>


On Apr 18, 2014, at 1:21 AM, Jeff Allen <ja.py at farowl.co.uk> wrote:

> The "think of tuples like a struct in C" explanation immediately reminded me that ...
> 
> On 16/04/2014 21:42, Taavi Burns wrote (in his excellent notes from the language summit):
>>  The demographics have changed. How do
>> we change the docs and ecosystem to avoid the assumption that Python
>> programmers already know how to program in C? 

In teaching Python, I find that analogs to other languages are helpful
in explaining Python even if a person doesn't know the other language.

    sorted(set(open(somefile)))

is like:

    cat somefile | sort | uniq       # different algorithm, same outcome

or:

   SELECT DISTINCT line FROM somelines ORDER BY line;

In particular, it is effective to say that tuples are used like structs in other languages
or like records in database tables.

I don't think we should go down the road of removing all the similes and metaphors
from the docs.   While children don't usually benefit from them, adults face different
learning challenges.  Usually, their biggest learning hurdle is figuring-out what is
and what is not a valid comparison to other things they already know.

One problem I have seen with the usual list vs tuple explanations is that
they seem to suggest that the list-are-for-looping and tuples-are-like-records
notions are baked into the implementation of lists and tuples.

In fact, the distinction is extrinsic to their implementations.  It is only important
because the rest of the language tends to treat them differently.  For example,
you could store ['raymond', 'red'] as a list or as a tuple ('raymond', 'red'), but you
wouldn't be punished until later when you tried:

     'I think %s likes %s' % container     # str.__mod__ treats lists and tuples differently

Likewise, there seems to be wide-spread confusion about make makes an
object immutable.  People seem to miss that ints, tuples, None and str are
immutable only because they lack any mutating methods,


Raymond


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140421/a83dbf26/attachment.html>

From leandropls at cpti.cetuc.puc-rio.br  Tue Apr 22 17:52:55 2014
From: leandropls at cpti.cetuc.puc-rio.br (Leandro Pereira de Lima e Silva)
Date: Tue, 22 Apr 2014 12:52:55 -0300
Subject: [Python-Dev] List vs Tuple / Homogeneous vs Heterogeneous /
 Mutable vs Immutable
In-Reply-To: <3F7F8B70-4558-42CA-BC12-9A241746B2D0@gmail.com>
References: <CAEaFV_iasbNn0_jbN6XaR8+oNq5m+9vi1TfwTZcuHp7OnbqLSg@mail.gmail.com>
 <CAP1=2W6NMJdtnVr22=gqx8bOwoY7JM7uhk78V6D4iU_WBVF=cA@mail.gmail.com>
 <CAP7+vJKMyJ8sm3a3Hce3zngcmDz0SG+V4e31K6qFP38_-2LpYg@mail.gmail.com>
 <CAEaFV_jWbG+u6O5Zzy0+TvYc6otJ11dXPqYD7EXb0VhV+mdPFg@mail.gmail.com>
 <5350E095.3070705@farowl.co.uk>
 <3F7F8B70-4558-42CA-BC12-9A241746B2D0@gmail.com>
Message-ID: <CAEaFV_iRMGwDDxpBh1fmkX=PTWBOgwD+cRvxv3RW=ERs=JJbxQ@mail.gmail.com>

>
> In teaching Python, I find that analogs to other languages are helpful
> in explaining Python even if a person doesn't know the other language.
>     sorted(set(open(somefile)))
> is like:
>     cat somefile | sort | uniq       # different algorithm, same outcome
> or:
>    SELECT DISTINCT line FROM somelines ORDER BY line;


I think that it helps by making the learning curve less steep, although it
also has the potential to produce poor code because different languages may
have different idioms to solving different problems. Much like if try to
think in English and translate word by word to Spanish, for example (you'll
probably get by, but it won't really be like how a native in that language
does).

Leandro


2014-04-22 3:39 GMT-03:00 Raymond Hettinger <raymond.hettinger at gmail.com>:

>
> On Apr 18, 2014, at 1:21 AM, Jeff Allen <ja.py at farowl.co.uk> wrote:
>
> The "think of tuples like a struct in C" explanation immediately reminded
> me that ...
>
>  On 16/04/2014 21:42, Taavi Burns wrote (in his excellent notes from the
> language summit):
>
>  The demographics have changed. How do
> we change the docs and ecosystem to avoid the assumption that Python
> programmers already know how to program in C?
>
>
> In teaching Python, I find that analogs to other languages are helpful
> in explaining Python even if a person doesn't know the other language.
>
>     sorted(set(open(somefile)))
>
> is like:
>
>     cat somefile | sort | uniq       # different algorithm, same outcome
>
> or:
>
>    SELECT DISTINCT line FROM somelines ORDER BY line;
>
> In particular, it is effective to say that tuples are used like structs in
> other languages
> or like records in database tables.
>
> I don't think we should go down the road of removing all the similes and
> metaphors
> from the docs.   While children don't usually benefit from them, adults
> face different
> learning challenges.  Usually, their biggest learning hurdle is
> figuring-out what is
> and what is not a valid comparison to other things they already know.
>
> One problem I have seen with the usual list vs tuple explanations is that
> they seem to suggest that the list-are-for-looping and
> tuples-are-like-records
> notions are baked into the implementation of lists and tuples.
>
> In fact, the distinction is extrinsic to their implementations.  It is
> only important
> because the rest of the language tends to treat them differently.  For
> example,
> you could store ['raymond', 'red'] as a list or as a tuple ('raymond',
> 'red'), but you
> wouldn't be punished until later when you tried:
>
>      'I think %s likes %s' % container     # str.__mod__ treats lists and
> tuples differently
>
> Likewise, there seems to be wide-spread confusion about make makes an
> object immutable.  People seem to miss that ints, tuples, None and str are
> immutable only because they lack any mutating methods,
>
>
> Raymond
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/leandropls%40cpti.cetuc.puc-rio.br
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140422/4f3afdb4/attachment.html>

From leandropls at cpti.cetuc.puc-rio.br  Tue Apr 22 17:46:17 2014
From: leandropls at cpti.cetuc.puc-rio.br (Leandro Pereira de Lima e Silva)
Date: Tue, 22 Apr 2014 12:46:17 -0300
Subject: [Python-Dev] List vs Tuple / Homogeneous vs Heterogeneous /
 Mutable vs Immutable
In-Reply-To: <CACBhJdGPBPOaScoLAgWuGvfg24c-6eEzpjFQCSOLo05yiHAuNg@mail.gmail.com>
References: <CAEaFV_iasbNn0_jbN6XaR8+oNq5m+9vi1TfwTZcuHp7OnbqLSg@mail.gmail.com>
 <CAP1=2W6NMJdtnVr22=gqx8bOwoY7JM7uhk78V6D4iU_WBVF=cA@mail.gmail.com>
 <CAP7+vJKMyJ8sm3a3Hce3zngcmDz0SG+V4e31K6qFP38_-2LpYg@mail.gmail.com>
 <CACBhJdGPBPOaScoLAgWuGvfg24c-6eEzpjFQCSOLo05yiHAuNg@mail.gmail.com>
Message-ID: <CAEaFV_hGayMobEfe-athVR8zkm8c1XqSXZJuQjY5N2-1XYz4DA@mail.gmail.com>

That's rather vague, isn't it? "Usually contains" isn't nearly as
prescriptive as "should be used for".

A co-worker with whom I discussed the matter these days also argued that a
language shouldn't prescribe as one uses a data structure, although I do
think conventions in semantics helps maintainability.

Leandro


2014-04-18 16:59 GMT-03:00 Ezio Melotti <ezio.melotti at gmail.com>:

> Hi,
>
> On Thu, Apr 17, 2014 at 10:23 PM, Guido van Rossum <guido at python.org>
> wrote:
> > It's definitely something that should be put in some documentation,
>
> see http://bugs.python.org/issue14840 and
> https://docs.python.org/3/tutorial/datastructures.html#tuples-and-sequences
> :
> """
> Though tuples may seem similar to lists, they are often used in
> different situations and for different purposes. Tuples are immutable,
> and usually contain an heterogeneous sequence of elements that are
> accessed via unpacking (see later in this section) or indexing (or
> even by attribute in the case of namedtuples). Lists are mutable, and
> their elements are usually homogeneous and are accessed by iterating
> over the list.
> """
>
> Best Regards,
> Ezio Melotti
>
> > probably
> > at the point when people have learned enough to be designing their own
> > programs where this issue comes up -- before they're wizards but well
> after
> > they have learned the semantic differences between lists and tuples.
> >
> >
> > On Thu, Apr 17, 2014 at 11:49 AM, Brett Cannon <bcannon at gmail.com>
> wrote:
> >>
> >>
> >>
> >> On Thu Apr 17 2014 at 2:43:35 PM, Leandro Pereira de Lima e Silva
> >> <leandropls at cpti.cetuc.puc-rio.br> wrote:
> >>>
> >>> Hello there!
> >>>
> >>> I've stumbled upon this discussion on python-dev about what the choice
> >>> between using a list or a tuple is all about in 2003:
> >>> 1. https://mail.python.org/pipermail/python-dev/2003-March/033962.html
> >>> 2. https://mail.python.org/pipermail/python-dev/2003-March/034029.html
> >>>
> >>> There's a vague comment about it on python documentation but afaik
> there
> >>> the discussion hasn't made into any PEPs. Is there an understanding
> about
> >>> it?
> >>
> >>
> >> Think of tuples like a struct in C, lists like an array. That's just out
> >> of Guido's head so I don't think we have ever bothered to write it down
> >> somewhere as an important distinction of the initial design that should
> be
> >> emphasized.
> >>
> >> _______________________________________________
> >> Python-Dev mailing list
> >> Python-Dev at python.org
> >> https://mail.python.org/mailman/listinfo/python-dev
> >> Unsubscribe:
> >> https://mail.python.org/mailman/options/python-dev/guido%40python.org
> >>
> >
> >
> >
> > --
> > --Guido van Rossum (python.org/~guido)
> >
> > _______________________________________________
> > Python-Dev mailing list
> > Python-Dev at python.org
> > https://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe:
> >
> https://mail.python.org/mailman/options/python-dev/ezio.melotti%40gmail.com
> >
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/leandropls%40cpti.cetuc.puc-rio.br
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140422/0aa1f17f/attachment.html>

From stephen at xemacs.org  Tue Apr 22 19:41:17 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 23 Apr 2014 02:41:17 +0900
Subject: [Python-Dev] List vs Tuple / Homogeneous vs Heterogeneous /
 Mutable vs Immutable
In-Reply-To: <CAEaFV_iRMGwDDxpBh1fmkX=PTWBOgwD+cRvxv3RW=ERs=JJbxQ@mail.gmail.com>
References: <CAEaFV_iasbNn0_jbN6XaR8+oNq5m+9vi1TfwTZcuHp7OnbqLSg@mail.gmail.com>
 <CAP1=2W6NMJdtnVr22=gqx8bOwoY7JM7uhk78V6D4iU_WBVF=cA@mail.gmail.com>
 <CAP7+vJKMyJ8sm3a3Hce3zngcmDz0SG+V4e31K6qFP38_-2LpYg@mail.gmail.com>
 <CAEaFV_jWbG+u6O5Zzy0+TvYc6otJ11dXPqYD7EXb0VhV+mdPFg@mail.gmail.com>
 <5350E095.3070705@farowl.co.uk>
 <3F7F8B70-4558-42CA-BC12-9A241746B2D0@gmail.com>
 <CAEaFV_iRMGwDDxpBh1fmkX=PTWBOgwD+cRvxv3RW=ERs=JJbxQ@mail.gmail.com>
Message-ID: <87d2g9qn2a.fsf@uwakimon.sk.tsukuba.ac.jp>

Leandro Pereira de Lima e Silva writes:

 >> In teaching Python, I find that analogs to other languages are
 >> helpful in explaining Python even if a person doesn't know the other
 >> language.
 >>?  ? sorted(set(open(somefile)))
 >> is like:
 >>?  ? cat somefile | sort | uniq ? ? ? # different algorithm, same outcome
 >> or:
 >>  ? ?SELECT DISTINCT line FROM somelines ORDER BY line;

 > I think that it helps by making the learning curve less steep,
 > although it also has the potential to produce poor code because
 > different languages may have different idioms to solving different
 > problems.

True, of course.  I think what Raymond is saying is that these
other-language examples can help smooth over those "?Qu? pasa?"
moments, so the teacher can focus on style and idiomatic coding in the
language at hand.  Some teachers will use them more than others do.

On the other hand, there's no need to purge them from the docs and
tutorials.  If someone notices that some example/analogy seems likely
to get in the way of understanding, they can and should write a patch
to improve that example.

From bcannon at gmail.com  Wed Apr 23 17:03:38 2014
From: bcannon at gmail.com (Brett Cannon)
Date: Wed, 23 Apr 2014 15:03:38 +0000
Subject: [Python-Dev] this is what happens if you freeze all the modules
 required for startup
References: <CAP1=2W4ovtgXuVLz3zN+85DOhc4s4LWQuFa+3vSqnhqbyqj66w@mail.gmail.com>
 <lip39g$jnr$1@ger.gmane.org>
 <CAP1=2W4hHLn6rPS36bokKXgKnECDsFgKnp=TGJS_2MgvvcLmhA@mail.gmail.com>
 <CACBhJdGw5K2+ZzhMW16QS5t49eSm+o3GRxsmRe=haWWVWGtbdA@mail.gmail.com>
Message-ID: <CAP1=2W5CwDttqrHXQ-uoPeWyPRa49TvdEx5yQGnNf2GX_3JkVg@mail.gmail.com>

On Fri Apr 18 2014 at 5:03:33 PM, Ezio Melotti <ezio.melotti at gmail.com>
wrote:

> Hi,
>
> On Thu, Apr 17, 2014 at 9:09 PM, Brett Cannon <bcannon at gmail.com> wrote:
> >
> >
> > On Thu Apr 17 2014 at 1:34:23 PM, Jurko Gospodneti?
> > <jurko.gospodnetic at pke.hr> wrote:
> >>
> >>    Hi.
> >>
> >> On 14.4.2014. 23:51, Brett Cannon wrote:
> >> > Now the question is whether the maintenance cost of having to rebuild
> >> > Python for a select number of stdlib modules is enough to warrant
> >> > putting in the effort to make this work.
> >>
> >>    I would really love to have better startup times in production, but I
> >> would also really hate to lose the ability to hack around in stdlib
> >> sources during development just to get better startup performance.
> >>
> >>    In general, what I really like about using Python for software
> >> development is the ability to open any stdlib file and easily go poking
> >> around using stuff like 'import pdb;pdb.set_trace()' or simple print
> >> statements. Researching mysterious behaviour is generally much much
> >> MUCH! easier (read: takes less hours/days/weeks) if it ends up leading
> >> into a stdlib Python module than if it takes you down into the bowels of
> >> some C module (think zipimport.c *grin*). Not to mention the effect that
> >> being able to quickly resolve a mystery by hacking on some Python
> >> internals leaves you feeling very satisfied, while having to entrench
> >> yourself in those internals for a long time just to find out you've made
> >> something foolish on your end leaves you feeling exhausted at best.
> >
> >
> > Freezing modules does not affect the ability to use gdb. And as long as
> you
> > set the appropriate __file__ values then tracebacks will contain even the
> > file line and location.
> >
>
> Will the tracebacks only contain the line number or the line of code too?
>

Yes.


>
> I've seen tracebacks involving importlib,_bootstrap that didn't
> include the code, and I'm wondering if we will get something similar
> for all the other modules you are freezing:
>
> Traceback (most recent call last):
>   File "/tmp/foo.py", line 7, in <module>
>     import email
>   File "<frozen importlib._bootstrap>", line 1561, in _find_and_load
>   File "<frozen importlib._bootstrap>", line 1519, in
> _find_and_load_unlocked
>   File "<frozen importlib._bootstrap>", line 1473, in _find_module
>   File "<frozen importlib._bootstrap>", line 1308, in find_module
>   File "<frozen importlib._bootstrap>", line 1284, in _get_loader
>   File "<frozen importlib._bootstrap>", line 1273, in _path_importer_cache
>   File "<frozen importlib._bootstrap>", line 1254, in _path_hooks
> TypeError: 'NoneType' object is not iterable
>
> Best Regards,
> Ezio Melotti
>

That's because the frozen importer doesn't define get_source(). But since
we have the source in this instance the __loader__ can be updated to be
SourceFileLoader so that get_source() is available:
http://bugs.python.org/issue21335 .
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140423/1e06d016/attachment.html>

From pcmanticore at gmail.com  Wed Apr 23 21:27:27 2014
From: pcmanticore at gmail.com (Claudiu Popa)
Date: Wed, 23 Apr 2014 22:27:27 +0300
Subject: [Python-Dev] Patches in need of review
Message-ID: <CAMy=CLpETk36fsRJaSv74KeWGo5y_F5qehbaSSBg2UO--Um5aw@mail.gmail.com>

Hello!

This approach worked for Nikolaus and I hope that it could work for me.
I have a couple of languishing patches, waiting for a core dev
to review, reject or commit them. I consider them ready to go,
but another pair of eyes could unveil unknown problems to me.
I pinged some issues from this list on core-mentorship a month ago,
without luck though. Here they are:

* http://bugs.python.org/issue19385

`dbm.dumb should be consistent when the database is closed`

This patch adds a check for operations with closed
dbm.dumb databases. The problem is that currently this
fails with AttributeError:

>>> import dbm.dumb as d
>>> db = d.open('test.dat', 'c')
>>> db.close()
>>> db.keys()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/tank/libs/cpython/Lib/dbm/dumb.py", line 212, in keys
    return list(self._index.keys())
AttributeError: 'NoneType' object has no attribute 'keys'

With this patch, the error is consistent with the other
dbm flavours.


* http://bugs.python.org/issue16104

`Use multiprocessing in compileall script`

This patch adds a new command line argument to `compileall`, also
a new argument to `compileall.compile_dir`, which controls the number
of worker processes used to compile a given directory.
The title of the issue is actually misleading a little,
because the patch uses `concurrent.futures` instead.


* http://bugs.python.org/issue19714

`Add tests for importlib.machinery.WindowsRegistryFinder.`

This is Windows specific and adds tests for the finder
in the title.

* http://bugs.python.org/issue17442

`code.InteractiveInterpreter doesn't display the exception cause`

This patch adds the ability for `code.InteractiveInterpreter` instances
to display the exception cause as seen below:

- currently

>>> try:
...    1 / 0
... except ZeroDivisionError as exc:
...    raise IOError from exc
...
Traceback (most recent call last):
  File "<console>", line 4, in <module>
OSError


- with the patch

>>> try:
...    1 / 0
... except ZeroDivisionError as exc:
...    raise IOError from exc
...
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
ZeroDivisionError: division by zero

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 4, in <module>
OSError

Thank you for your time!


Claudiu

From tjreedy at udel.edu  Thu Apr 24 01:07:24 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 23 Apr 2014 19:07:24 -0400
Subject: [Python-Dev] Patches in need of review
In-Reply-To: <CAMy=CLpETk36fsRJaSv74KeWGo5y_F5qehbaSSBg2UO--Um5aw@mail.gmail.com>
References: <CAMy=CLpETk36fsRJaSv74KeWGo5y_F5qehbaSSBg2UO--Um5aw@mail.gmail.com>
Message-ID: <lj9h4c$cr8$1@ger.gmane.org>

On 4/23/2014 3:27 PM, Claudiu Popa wrote:

> * http://bugs.python.org/issue16104
>
> `Use multiprocessing in compileall script`
>
> This patch adds a new command line argument to `compileall`, also
> a new argument to `compileall.compile_dir`, which controls the number
> of worker processes used to compile a given directory.
> The title of the issue is actually misleading a little,
> because the patch uses `concurrent.futures` instead.

I think titles should give goal, not intended or expected method.
I changed the title accordingly.
Compileall script: add option to use multiple cores

-- 
Terry Jan Reedy


From zachary.ware+pydev at gmail.com  Thu Apr 24 03:30:09 2014
From: zachary.ware+pydev at gmail.com (Zachary Ware)
Date: Wed, 23 Apr 2014 20:30:09 -0500
Subject: [Python-Dev] [Python-checkins] cpython (3.4): Prevent Sphinx
	error on whatsnew/changelog
In-Reply-To: <53583DB5.8070809@udel.edu>
References: <3gDQ1J1DHsz7Lkd@mail.python.org> <53583DB5.8070809@udel.edu>
Message-ID: <e4704e73-398d-41cb-acb5-e2321309a719@email.android.com>



On April 23, 2014 5:24:53 PM CDT, Terry Reedy <tjreedy at udel.edu> wrote:
>On 4/23/2014 11:05 AM, zach.ware wrote:
>> http://hg.python.org/cpython/rev/75419257fec3
>> changeset:   90440:75419257fec3
>> branch:      3.4
>> parent:      90437:5d745d97b7da
>> user:        Zachary Ware <zachary.ware at gmail.com>
>> date:        Wed Apr 23 10:04:20 2014 -0500
>> summary:
>>    Prevent Sphinx error on whatsnew/changelog
>>
>> files:
>>    Misc/NEWS |  2 +-
>>    1 files changed, 1 insertions(+), 1 deletions(-)
>>
>>
>> diff --git a/Misc/NEWS b/Misc/NEWS
>> --- a/Misc/NEWS
>> +++ b/Misc/NEWS
>> @@ -44,7 +44,7 @@
>>     at the same time without losing the Popen.returncode value.
>>
>>   - Issue #21127: Path objects can now be instantiated from str
>subclass
>> -  instances (such as numpy.str_).
>> +  instances (such as ``numpy.str_``).
>
>Could you explain this? I thought NEWS was a plain text file without 
>.rst markup.

Doc/whatsnew/changelog.rst includes the full text of Misc/NEWS, which means Sphinx doesn't like markup issues in NEWS.

--
Zach (on a phone)

From ethan at stoneleaf.us  Thu Apr 24 04:14:04 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Wed, 23 Apr 2014 19:14:04 -0700
Subject: [Python-Dev] ref leaks
Message-ID: <5358736C.3060703@stoneleaf.us>

Command line:

   ./python -m test.regrtest -v -R3:3 test_tools

Results:

   Ran 44 tests in 7.628s

   OK (skipped=1)
   .
   test_tools leaked [0, 2, 2] references, sum=4
   1 test failed:
       test_tools

Any words of wisdom for tracking those leaks?

--
~Ethan~

From lukasz at langa.pl  Thu Apr 24 05:59:09 2014
From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=)
Date: Wed, 23 Apr 2014 20:59:09 -0700
Subject: [Python-Dev] List vs Tuple / Homogeneous vs Heterogeneous /
	Mutable vs Immutable
In-Reply-To: <CAP1=2W6NMJdtnVr22=gqx8bOwoY7JM7uhk78V6D4iU_WBVF=cA@mail.gmail.com>
References: <CAEaFV_iasbNn0_jbN6XaR8+oNq5m+9vi1TfwTZcuHp7OnbqLSg@mail.gmail.com>
 <CAP1=2W6NMJdtnVr22=gqx8bOwoY7JM7uhk78V6D4iU_WBVF=cA@mail.gmail.com>
Message-ID: <889A5C6D-8A3F-4200-A10E-F68B15CEE21B@langa.pl>

On Apr 17, 2014, at 11:49 AM, Brett Cannon <bcannon at gmail.com> wrote:

> Think of tuples like a struct in C, lists like an array.

I generally agree but it?s a bit more complex, for instance when you have a homogenous sequence but want it to be hashable. I just hit that today and felt a little bad using tuple because of that ?struct? mindset :)

-- 
Best regards,
?ukasz Langa

WWW: http://lukasz.langa.pl/
Twitter: @llanga
IRC: ambv on #python-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140423/c0639ca8/attachment.html>

From benjamin at python.org  Thu Apr 24 06:06:02 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 23 Apr 2014 21:06:02 -0700
Subject: [Python-Dev] ref leaks
In-Reply-To: <5358736C.3060703@stoneleaf.us>
References: <5358736C.3060703@stoneleaf.us>
Message-ID: <1398312362.821.109788265.1EC35C4C@webmail.messagingengine.com>

On Wed, Apr 23, 2014, at 19:14, Ethan Furman wrote:
> Command line:
> 
>    ./python -m test.regrtest -v -R3:3 test_tools
> 
> Results:
> 
>    Ran 44 tests in 7.628s
> 
>    OK (skipped=1)
>    .
>    test_tools leaked [0, 2, 2] references, sum=4
>    1 test failed:
>        test_tools
> 
> Any words of wisdom for tracking those leaks?

Unless it's consistent, that sort of behavior usually just gets
dismissed as intermittent.

From rosuav at gmail.com  Thu Apr 24 08:43:50 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 24 Apr 2014 16:43:50 +1000
Subject: [Python-Dev] List vs Tuple / Homogeneous vs Heterogeneous /
 Mutable vs Immutable
In-Reply-To: <889A5C6D-8A3F-4200-A10E-F68B15CEE21B@langa.pl>
References: <CAEaFV_iasbNn0_jbN6XaR8+oNq5m+9vi1TfwTZcuHp7OnbqLSg@mail.gmail.com>
 <CAP1=2W6NMJdtnVr22=gqx8bOwoY7JM7uhk78V6D4iU_WBVF=cA@mail.gmail.com>
 <889A5C6D-8A3F-4200-A10E-F68B15CEE21B@langa.pl>
Message-ID: <CAPTjJmoiUEgSFjtg5vVbqVZtBFnpTjxwR-gRnNAZWJ282ma5+w@mail.gmail.com>

On Thu, Apr 24, 2014 at 1:59 PM, ?ukasz Langa <lukasz at langa.pl> wrote:
> On Apr 17, 2014, at 11:49 AM, Brett Cannon <bcannon at gmail.com> wrote:
>
>> Think of tuples like a struct in C, lists like an array.
>
>
> I generally agree but it?s a bit more complex, for instance when you have a
> homogenous sequence but want it to be hashable. I just hit that today and
> felt a little bad using tuple because of that ?struct? mindset :)

All you need is a "frozenlist" type, parallel to "frozenset":

frozenlist = tuple # Magic!

some_dict[frozenlist([...])] = some_value

ChrisA

From chris at simplistix.co.uk  Thu Apr 24 09:18:22 2014
From: chris at simplistix.co.uk (Chris Withers)
Date: Thu, 24 Apr 2014 08:18:22 +0100
Subject: [Python-Dev] pep8 reasoning
Message-ID: <5358BABE.6030300@simplistix.co.uk>

Hi All,

Apologies if this is considered off topic, but I'm keen to get the 
"language designers" point of view and short of emailing Barry, Guido 
and Nick directly, this seemed like the best place.

I'm having a tough time persuading some people of the benefits of pep8, 
particularly when it comes to applying to an existing large code base.

The biggest sticking point is naming, particularly as it's the one thing 
that can't necessarily be changed module by module. What were the 
compelling reasons to go from mixedCase to underscore_separated? What's 
considered the best approach for migrating from the former to the latter?

A couple of others that have raised some consternation; what are the 
technical reasons for this pattern being bad:

if len(seq)
if not len(seq)

...or, for me, the rather ugly:

if 0 != len(seq)

Likewise, these:

if greeting == True:
if greeting is True:

Please don't misunderstand me: I dislike the above intensely, but it's 
an emotional response based on 10-15 years of doing things the other 
way. I'm interested in arguments that don't include things like "it's 
pythonic" (which some people consider a derogatory term ;-)), or "just 
because", I trust that the stuff in pep8 was done with specific 
reasoning in mind, but I'm feeling rather useless at giving that 
reasoning and I'm hoping you can help :-)

cheers,

Chris

From skip at pobox.com  Thu Apr 24 13:06:40 2014
From: skip at pobox.com (Skip Montanaro)
Date: Thu, 24 Apr 2014 06:06:40 -0500
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <5358BABE.6030300@simplistix.co.uk>
References: <5358BABE.6030300@simplistix.co.uk>
Message-ID: <CANc-5UxQP+Oy9=EKqn=hY65vvuoQ4YC21XG=r0C0-5OR_QvATA@mail.gmail.com>

On Thu, Apr 24, 2014 at 2:18 AM, Chris Withers <chris at simplistix.co.uk> wrote:
> What were the compelling reasons to go from mixedCase to
> underscore_separated? What's considered the best approach for migrating from
> the former to the latter?

I never recall Python "going from" camelCase to separate_words. The
descriptions of best practice you see in the PEP were always that way,
as I recall. If you have a code base that does it some other way, I
would leave it be. The primary hunk of code I work with is full of
Boost.Python-generated bindings for C++ libraries, so leaks C++ naming
style out of all its pores. A lot of the other pure Python code in
this code base was written by people who were mostly C++ programmers,
and didn't know PEP 8 from a hole in the ground. Consequently, the
whole thing is riddled with all sorts of non-pep8-ness. I used to want
to change everything, but just let that sleeping dog lie. I have
better things to do with my life. New stuff I write tends to be much
more pep8-ish.

Skip

From taleinat at gmail.com  Thu Apr 24 13:59:47 2014
From: taleinat at gmail.com (Tal Einat)
Date: Thu, 24 Apr 2014 14:59:47 +0300
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <5358BABE.6030300@simplistix.co.uk>
References: <5358BABE.6030300@simplistix.co.uk>
Message-ID: <CALWZvp5g8TxbvvegdKLgw-iArF7RZc7BS-MWS5FJpCfFhd+T1g@mail.gmail.com>

On Thu, Apr 24, 2014 at 10:18 AM, Chris Withers <chris at simplistix.co.uk> wrote:
> A couple of others that have raised some consternation; what are the
> technical reasons for this pattern being bad:
>
> if len(seq)
> if not len(seq)
>
> ...or, for me, the rather ugly:
>
> if 0 != len(seq)
>
> Likewise, these:
>
> if greeting == True:
> if greeting is True:

As far as I know that reason for these examples being frowned upon is
that they are needlessly redundant. This is not just a matter of
minimalism or style; it's a matter of readability.

If you're relying on the "len()" call to fail if "seq" isn't a
container (for example, if it's None), the code for that should be
separate and more explicit.

Regarding "if something == True", that's only the same as "if
greeting" if greeting is assumed to be a boolean value. If so, it is
better (for readability) to use a descriptive variable name: "if
should_greet:". Otherwise (e.g. if greeting could also be None) it is
reasonable to check if it is equal to a specific value (with a short
comment explaining this point!).

Also, there's no end to how much comparison to True one can do:

if (greeting == True) == True
if ((greeting == True) == True) == True

Considering that, IMO it makes sense to just avoid such comparisons altogether.

- Tal Einat

From mail at timgolden.me.uk  Thu Apr 24 14:57:38 2014
From: mail at timgolden.me.uk (Tim Golden)
Date: Thu, 24 Apr 2014 13:57:38 +0100
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <CALWZvp5g8TxbvvegdKLgw-iArF7RZc7BS-MWS5FJpCfFhd+T1g@mail.gmail.com>
References: <5358BABE.6030300@simplistix.co.uk>
 <CALWZvp5g8TxbvvegdKLgw-iArF7RZc7BS-MWS5FJpCfFhd+T1g@mail.gmail.com>
Message-ID: <53590A42.4070700@timgolden.me.uk>

On 24/04/2014 12:59, Tal Einat wrote:
> As far as I know that reason for these examples being frowned upon is
> that they are needlessly redundant. 

Oh, the irony! (Sorry, couldn't resist)

TJG

From barry at python.org  Thu Apr 24 15:59:31 2014
From: barry at python.org (Barry Warsaw)
Date: Thu, 24 Apr 2014 09:59:31 -0400
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <5358BABE.6030300@simplistix.co.uk>
References: <5358BABE.6030300@simplistix.co.uk>
Message-ID: <20140424095931.2fe5d048@anarchist.wooz.org>

On Apr 24, 2014, at 08:18 AM, Chris Withers wrote:

>I'm having a tough time persuading some people of the benefits of pep8,
>particularly when it comes to applying to an existing large code base.

First of all, the purposes of PEP 8 is not to impose mandates of style on your
colleagues. :)  Two important quotes are useful to keep in mind:

Introduction

    This document gives coding conventions for the Python code comprising the
    *standard library* in the main Python distribution.

(emphasis added)

A Foolish Consistency is the Hobgoblin of Little Minds

    A style guide is about consistency. Consistency with this style guide is
    important. Consistency within a project is more important. Consistency
    within one module or function is most important.
    [...]

    Some other good reasons to ignore a particular guideline:

    To be consistent with surrounding code that also breaks it (maybe for
    historic reasons) -- although this is also an opportunity to clean up
    someone else's mess (in true XP style).

While I personally think PEP 8 is good enough to adopt for my own projects,
and I encourage other projects to also adopt the guidelines where appropriate,
doing so is *not* always appropriate in existing source code outside the
standard library (and sometimes even within the stdlib, e.g. unittest).

>The biggest sticking point is naming, particularly as it's the one thing that
>can't necessarily be changed module by module. What were the compelling
>reasons to go from mixedCase to underscore_separated? What's considered the
>best approach for migrating from the former to the latter?

If your existing code base is already mixedCase, then there's no compelling
reason for a wholesale transition, IMO.  I work with plenty of existing
mixedCase code and it's much more jarring to see a mix of styles than it is to
see consistent mixedCase naming.  OTOH, some projects do choose to adopt PEP 8
style names over time, and they have to manage all the expected API guarantees
involved in that.

I will say this: the original preference for underscore_names in PEP 8 was
spurred by user studies some of our early non-native English speaking users
conducted many years ago.  We learned that it was more difficult for many of
them to parse mixedCase names than underscore_names.  I'm afraid I probably no
longer have references to those studies, but the difference was pronounced,
IIRC, and I think it's easy to see why.  Underscores can be scanned by the eye
as spaces, while I'd hypothesize that the brain has to do more work to read
mixedCase names.

>A couple of others that have raised some consternation; what are the
>technical reasons for this pattern being bad:
>
>if len(seq)
>if not len(seq)
>
>...or, for me, the rather ugly:
>
>if 0 != len(seq)

Well, I'd write that as `len(seq) != 0` but still, I get your point.

The rationale for this is that generic truthiness is a wider check then an
explicit value check, and EIBTI.  This really comes down to the difference
between

    if not thing:

vs

    if thing != other:

In the first case, any falsey value makes it through, while in the second
case, you're being more explicit about what you think the value of `thing` can
be.  So when semantically `thing` can be 0 or None or '', then the first test
is preferred.  When `thing` can't just be any truthy or falsey value, then a
more explicit check is preferred, otherwise some unexpected falsey value might
slip through.

It might seem less applicable to the len(seq) case, but I'd offer that that's
just an application of a more general principle.

>Likewise, these:
>
>if greeting == True:
>if greeting is True:

This one is based on the preference for identity checks when singletons are
involved, rather than equality tests.  Being composed of English words, the
latter is also more readable.  It's the same reason why you would do identity
checks against None or enum values.

>Please don't misunderstand me: I dislike the above intensely, but it's an
>emotional response based on 10-15 years of doing things the other way. I'm
>interested in arguments that don't include things like "it's pythonic" (which
>some people consider a derogatory term ;-)), or "just because", I trust that
>the stuff in pep8 was done with specific reasoning in mind, but I'm feeling
>rather useless at giving that reasoning and I'm hoping you can help :-)

I hope those explanations give you the basis for why the choices were made.
They aren't arbitrary, although folks can and do disagree.  It's much more
important that your project comes up with its own guidelines and applies them
self-consistently.  For example, I have my own small set of refinements on top
of PEP 8.  I'm not providing a link because they're a bit out of date
w.r.t. Python 3. ;)

Cheers,
-Barry

From skip at pobox.com  Thu Apr 24 16:11:24 2014
From: skip at pobox.com (Skip Montanaro)
Date: Thu, 24 Apr 2014 09:11:24 -0500
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <20140424095931.2fe5d048@anarchist.wooz.org>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
Message-ID: <CANc-5UxED8wRa+B9CMOQcywL8WeXTg+be6aLLPAM4VDO3q6t0A@mail.gmail.com>

On Thu, Apr 24, 2014 at 8:59 AM, Barry Warsaw <barry at python.org> wrote:
> I will say this: the original preference for underscore_names in PEP 8 was
> spurred by user studies some of our early non-native English speaking users
> conducted many years ago.  We learned that it was more difficult for many of
> them to parse mixedCase names than underscore_names.  I'm afraid I probably no
> longer have references to those studies, but the difference was pronounced,
> IIRC, and I think it's easy to see why.  Underscores can be scanned by the eye
> as spaces, while I'd hypothesize that the brain has to do more work to read
> mixedCase names.

Given Guido's background, I suspect these studies might have been done
at CWI in the context of the ABC language.

Skip

From ethan at stoneleaf.us  Thu Apr 24 16:31:22 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 24 Apr 2014 07:31:22 -0700
Subject: [Python-Dev] ref leaks
In-Reply-To: <1398312362.821.109788265.1EC35C4C@webmail.messagingengine.com>
References: <5358736C.3060703@stoneleaf.us>
 <1398312362.821.109788265.1EC35C4C@webmail.messagingengine.com>
Message-ID: <5359203A.3040109@stoneleaf.us>

On 04/23/2014 09:06 PM, Benjamin Peterson wrote:
> On Wed, Apr 23, 2014, at 19:14, Ethan Furman wrote:
>> Command line:
>>
>>     ./python -m test.regrtest -v -R3:3 test_tools
>>
>> Results:
>>
>>     Ran 44 tests in 7.628s
>>
>>     OK (skipped=1)
>>     .
>>     test_tools leaked [0, 2, 2] references, sum=4
>>     1 test failed:
>>         test_tools
>>
>> Any words of wisdom for tracking those leaks?
>
> Unless it's consistent, that sort of behavior usually just gets
> dismissed as intermittent.

test_tools leaked [331, 0, 0] references, sum=331
test_tools leaked [108, 1, 0] memory blocks, sum=109

test_tools leaked [2, 0, 0] references, sum=2

test_tools leaked [0, 0, 4] references, sum=4
test_tools leaked [0, 0, 3] memory blocks, sum=3

Consistently the same, or consistently happening?  ;)

--
~Ethan~

From robert.kern at gmail.com  Thu Apr 24 17:02:06 2014
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 24 Apr 2014 16:02:06 +0100
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <20140424095931.2fe5d048@anarchist.wooz.org>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
Message-ID: <ljb91f$k9v$1@ger.gmane.org>

On 2014-04-24 14:59, Barry Warsaw wrote:

> I will say this: the original preference for underscore_names in PEP 8 was
> spurred by user studies some of our early non-native English speaking users
> conducted many years ago.  We learned that it was more difficult for many of
> them to parse mixedCase names than underscore_names.  I'm afraid I probably no
> longer have references to those studies, but the difference was pronounced,
> IIRC, and I think it's easy to see why.  Underscores can be scanned by the eye
> as spaces, while I'd hypothesize that the brain has to do more work to read
> mixedCase names.

A more recent set of studies show some mixedResults (ha ha). On a low-level 
reading task, the studies agree with yours in that mixedCase takes more time and 
effort; however, it appears to improve accuracy as well. On a higher-level 
comprehension task, mixedCase took less or the same time and still improved 
accuracy. Experienced programmers don't see too much of a difference either way, 
but inexperienced programmers see a more marked benefit to mixedCase.

   http://www.cs.loyola.edu/~binkley/papers/tr-loy110720.pdf?

That said, I can't vouch for the experiments or the analysis, and it isn't 
really germane to Chris' historical question. I mention it only because I had 
just run across this paper last night, so it was fresh in my mind when you 
mentioned studies on the subject.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From chris at withers.org  Thu Apr 24 08:48:38 2014
From: chris at withers.org (Chris Withers)
Date: Thu, 24 Apr 2014 07:48:38 +0100
Subject: [Python-Dev] pep8 reasoning
Message-ID: <5358B3C6.4070407@withers.org>

Hi All,

Apologies if this is considered off topic, but I'm keen to get the 
"language designers" point of view and short of emailing Barry, Guido 
and Nick directly, this seemed like the best place.

I'm having a tough time persuading some people of the benefits of pep8, 
particularly when it comes to applying to an existing large code base.

The biggest sticking point is naming, particularly as it's the one thing 
that can't necessarily be changed module by module. What were the 
compelling reasons to go from mixedCase to underscore_separated? What's 
considered the best approach for migrating from the former to the latter?

A couple of others that have raised some consternation; what are the 
technical reasons for this pattern being bad:

if len(seq)
if not len(seq)

...or, for me, the rather ugly:

if 0 != len(seq)

Likewise, these:

if greeting == True:
if greeting is True:

Please don't misunderstand me: I dislike the above intensely, but it's 
an emotional response based on 10-15 years of doing things the other 
way. I'm interested in arguments that don't include things like "it's 
pythonic" (which some people consider a derogatory term ;-)), or "just 
because", I trust that the stuff in pep8 was done with specific 
reasoning in mind, but I'm feeling rather useless at giving that 
reasoning and I'm hoping you can help :-)

cheers,

Chris

From stefan at bytereef.org  Thu Apr 24 17:17:41 2014
From: stefan at bytereef.org (Stefan Krah)
Date: Thu, 24 Apr 2014 17:17:41 +0200
Subject: [Python-Dev] ref leaks
In-Reply-To: <5359203A.3040109@stoneleaf.us>
References: <5358736C.3060703@stoneleaf.us>
 <1398312362.821.109788265.1EC35C4C@webmail.messagingengine.com>
 <5359203A.3040109@stoneleaf.us>
Message-ID: <20140424151741.GA26075@sleipnir.bytereef.org>

Ethan Furman <ethan at stoneleaf.us> wrote:
> >>Any words of wisdom for tracking those leaks?

Often the easiest way is to compile --with-valgrind and run the test
under Valgrind (obviously).

In the Valgrind output, search for "definitely lost" and ignore the rest.


If that does not turn up anything, consider a bug in regrtest.py:

It happens that certain caches are initialized incrementally in each
repetition of the test, so the reported leaks are spurious. That is
why some caches like small integers are "warmed up" in regrtest.py.

Perhaps there is some recently added cache that needs attention.


Stefan Krah



From thomas at python.org  Thu Apr 24 17:35:56 2014
From: thomas at python.org (Thomas Wouters)
Date: Thu, 24 Apr 2014 08:35:56 -0700
Subject: [Python-Dev] ref leaks
In-Reply-To: <5359203A.3040109@stoneleaf.us>
References: <5358736C.3060703@stoneleaf.us>
 <1398312362.821.109788265.1EC35C4C@webmail.messagingengine.com>
 <5359203A.3040109@stoneleaf.us>
Message-ID: <CAPdQG2ruG_X9sLmUh_ko3Oq1xSO-L6nBFnKrTZEBK5U27NUBvQ@mail.gmail.com>

On Thu, Apr 24, 2014 at 7:31 AM, Ethan Furman <ethan at stoneleaf.us> wrote:

> On 04/23/2014 09:06 PM, Benjamin Peterson wrote:
>
>> On Wed, Apr 23, 2014, at 19:14, Ethan Furman wrote:
>>
>>> Command line:
>>>
>>>     ./python -m test.regrtest -v -R3:3 test_tools
>>>
>>> Results:
>>>
>>>     Ran 44 tests in 7.628s
>>>
>>>     OK (skipped=1)
>>>     .
>>>     test_tools leaked [0, 2, 2] references, sum=4
>>>     1 test failed:
>>>         test_tools
>>>
>>> Any words of wisdom for tracking those leaks?
>>>
>>
>> Unless it's consistent, that sort of behavior usually just gets
>> dismissed as intermittent.
>>
>
> test_tools leaked [331, 0, 0] references, sum=331
> test_tools leaked [108, 1, 0] memory blocks, sum=109
>
> test_tools leaked [2, 0, 0] references, sum=2
>
> test_tools leaked [0, 0, 4] references, sum=4
> test_tools leaked [0, 0, 3] memory blocks, sum=3
>
> Consistently the same, or consistently happening?  ;)
>

Consistently a leak. If it's an occasional reported leak (especially in the
first or last of the reported runs) or a mixture of positive and negative
numbers, it's more likely it's an effect of delayed cleanup, for whatever
reason. regrtest's -R takes two integers, 'stab' and 'run', for the number
of times to run the same test without tracking reference counts (to
stabilize caching effects) and the number of times to run the test while
tracking reference counts. Some tests don't stabilize as easily as others,
for example because the actual workload or timings rely on external
sources. '2, 0, 0' and '0, 0, 4' are probably not leaks, but if you're
worried you can run them with larger 'stab' and 'run' to see if they
stabilize, or a smaller 'stab' to see if they just have unreliable refcount
effects.

The easiest way to debug these things is to reduce the test until it has no
net refcount effect, then look at the last thing you removed :)

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm an email virus! Think twice before sending your email to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140424/ec17bfa4/attachment.html>

From taleinat at gmail.com  Thu Apr 24 17:51:03 2014
From: taleinat at gmail.com (Tal Einat)
Date: Thu, 24 Apr 2014 18:51:03 +0300
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <20140424095931.2fe5d048@anarchist.wooz.org>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
Message-ID: <CALWZvp6Ouu58HvqPEwFLYY=r9e35dp2UB2+P-f3oYGvBMbMG6w@mail.gmail.com>

On Thu, Apr 24, 2014 at 4:59 PM, Barry Warsaw <barry at python.org> wrote:
> I will say this: the original preference for underscore_names in PEP 8 was
> spurred by user studies some of our early non-native English speaking users
> conducted many years ago.  We learned that it was more difficult for many of
> them to parse mixedCase names than underscore_names.  I'm afraid I probably no
> longer have references to those studies, but the difference was pronounced,
> IIRC, and I think it's easy to see why.  Underscores can be scanned by the eye
> as spaces, while I'd hypothesize that the brain has to do more work to read
> mixedCase names.

I speak two languages as mother tongues - English and Hebrew. Hebrew
has no capital letters (and is also right-to-left) and is the spoken
and written language in the parts of Israel where I've lived most of
my life. Perhaps because of this, I do find that capital letters don't
quite "jump out" for me and therefore I find mixedCase and CamelCase
harder to grok at a quick glance than under_score_names.

- Tal

From guido at python.org  Thu Apr 24 18:05:40 2014
From: guido at python.org (Guido van Rossum)
Date: Thu, 24 Apr 2014 09:05:40 -0700
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <5358B3C6.4070407@withers.org>
References: <5358B3C6.4070407@withers.org>
Message-ID: <CAP7+vJKSGxi0RhSQoFHT+Kdn0LvO_seB_E+HQU3jKSoAdU8c4g@mail.gmail.com>

On Wed, Apr 23, 2014 at 11:48 PM, Chris Withers <chris at withers.org> wrote:

> Apologies if this is considered off topic, but I'm keen to get the
> "language designers" point of view and short of emailing Barry, Guido and
> Nick directly, this seemed like the best place.
>
> I'm having a tough time persuading some people of the benefits of pep8,
> particularly when it comes to applying to an existing large code base.
>

I guess you're not in a position of power or authority there? :-)


> The biggest sticking point is naming, particularly as it's the one thing
> that can't necessarily be changed module by module. What were the
> compelling reasons to go from mixedCase to underscore_separated? What's
> considered the best approach for migrating from the former to the latter?
>

Changing method names is always very painful. E.g. it took us forever to
make threading.py compliant with the PEP. What we did there is to keep all
the old method names (as aliases). E.g. search threading.py for notify_all
and notifyAll. Then compare how this is done in Python 2.7 vs. 3.4.
Eventually the old names will be summarily removed.

Why we did this? Before I wrote PEP 8 people were using different
conventions and I (perhaps foolishly) didn't want to declare half of the
modules out of style, so the original PEP 8 (and my style guide that
predated it) allowed either. But eventually we got agreement on what style
we preferred, so we took out the "you can name modules and methods either
way."


> A couple of others that have raised some consternation; what are the
> technical reasons for this pattern being bad:
>
> if len(seq)
> if not len(seq)
>
> Some user-defined data types have an expensive __len__() (e.g. when you
have to count the items one by one) while __bool__() can be more efficient.


> ...or, for me, the rather ugly:
>
> if 0 != len(seq)
>

I know several engineering teams whose style prefers this, but personally I
hate it with a vengeance -- it always makes me jump when trying to
understand code written like that. In math on a blackboard you would never
write this (unless as the result of a simplification of a more complex
equation). Also, try to read it out loud -- it sounds awful. (The reason
people write this doesn't really apply in Python anyway -- it comes from a
desire to avoid the bug in C where you write

if (x = 42) { ... }

instead of

if (x == 42) { ... }

But the Python equivalent of the former is invalid syntactically in Python
anyway.


> Likewise, these:
>
> if greeting == True:
> if greeting is True:
>

Both these consider as falsey many values that are actually considered
truthy by convention (e.g. non-zero numbers, non-empty
sequences/containers).


> Please don't misunderstand me: I dislike the above intensely, but it's an
> emotional response based on 10-15 years of doing things the other way. I'm
> interested in arguments that don't include things like "it's pythonic"
> (which some people consider a derogatory term ;-)), or "just because", I
> trust that the stuff in pep8 was done with specific reasoning in mind, but
> I'm feeling rather useless at giving that reasoning and I'm hoping you can
> help :-)
>
> cheers,
>
> Chris
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140424/5990aa4d/attachment-0001.html>

From guido at python.org  Thu Apr 24 18:09:33 2014
From: guido at python.org (Guido van Rossum)
Date: Thu, 24 Apr 2014 09:09:33 -0700
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <CANc-5UxED8wRa+B9CMOQcywL8WeXTg+be6aLLPAM4VDO3q6t0A@mail.gmail.com>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CANc-5UxED8wRa+B9CMOQcywL8WeXTg+be6aLLPAM4VDO3q6t0A@mail.gmail.com>
Message-ID: <CAP7+vJK=+o2_1j4VR9ZFq_t+xXXnEDn5PqNA-GAun=2wE5neBA@mail.gmail.com>

On Thu, Apr 24, 2014 at 7:11 AM, Skip Montanaro <skip at pobox.com> wrote:

> On Thu, Apr 24, 2014 at 8:59 AM, Barry Warsaw <barry at python.org> wrote:
> > I will say this: the original preference for underscore_names in PEP 8
> was
> > spurred by user studies some of our early non-native English speaking
> users
> > conducted many years ago.  We learned that it was more difficult for
> many of
> > them to parse mixedCase names than underscore_names.  I'm afraid I
> probably no
> > longer have references to those studies, but the difference was
> pronounced,
> > IIRC, and I think it's easy to see why.  Underscores can be scanned by
> the eye
> > as spaces, while I'd hypothesize that the brain has to do more work to
> read
> > mixedCase names.
>
> Given Guido's background, I suspect these studies might have been done
> at CWI in the context of the ABC language.


I'm sorry, I have no recollection of such studies. (ABC used case
differently anyway, so camelcase wasn't possible there -- and neither were
underscores).

Barry maybe referring to a more informal survey of feedback from Python
users at the time. But I have no recollection of that either.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140424/904f2b7c/attachment.html>

From chris.barker at noaa.gov  Thu Apr 24 17:07:16 2014
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Thu, 24 Apr 2014 08:07:16 -0700
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <20140424095931.2fe5d048@anarchist.wooz.org>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
Message-ID: <-3139544165801430113@unknownmsgid>

One added note:

>> if greeting == True:
>> if greeting is True:
>
> This one is based on the preference for identity checks when singletons are
> involved, rather than equality tests.  Being composed of English words, the
> latter is also more readable.  It's the same reason why you would do identity
> checks against None or enum values.

There is more to it that that -- == can be overridden for a particular
class. So doing:

X == None

Could yield surprising results if __eq__ has been defined for the
class X is bound to at the moment. Numpy arrays are a common case for
me.

As None is often used to mean undefined, X could be any type. Granted,
in your code, you likely know what type(s) X is likely to be, but it's
a good habit to get in to.

-Chris


>
>> Please don't misunderstand me: I dislike the above intensely, but it's an
>> emotional response based on 10-15 years of doing things the other way. I'm
>> interested in arguments that don't include things like "it's pythonic" (which
>> some people consider a derogatory term ;-)), or "just because", I trust that
>> the stuff in pep8 was done with specific reasoning in mind, but I'm feeling
>> rather useless at giving that reasoning and I'm hoping you can help :-)
>
> I hope those explanations give you the basis for why the choices were made.
> They aren't arbitrary, although folks can and do disagree.  It's much more
> important that your project comes up with its own guidelines and applies them
> self-consistently.  For example, I have my own small set of refinements on top
> of PEP 8.  I'm not providing a link because they're a bit out of date
> w.r.t. Python 3. ;)
>
> Cheers,
> -Barry
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/chris.barker%40noaa.gov

From rosuav at gmail.com  Thu Apr 24 18:25:38 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 25 Apr 2014 02:25:38 +1000
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <20140424095931.2fe5d048@anarchist.wooz.org>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
Message-ID: <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>

On Thu, Apr 24, 2014 at 11:59 PM, Barry Warsaw <barry at python.org> wrote:
> I will say this: the original preference for underscore_names in PEP 8 was
> spurred by user studies some of our early non-native English speaking users
> conducted many years ago.  We learned that it was more difficult for many of
> them to parse mixedCase names than underscore_names.  I'm afraid I probably no
> longer have references to those studies, but the difference was pronounced,
> IIRC, and I think it's easy to see why.  Underscores can be scanned by the eye
> as spaces, while I'd hypothesize that the brain has to do more work to read
> mixedCase names.

Underscores also play much more nicely with initialisms. How would you
spell a function named "Add HTTP Header"?

add_HTTP_header
add_http_header

addHTTPHeader
addHttpHeader

Four options to choose from. The first two clearly separate the
initialism from the other words; take your pick whether you want it
uppercased or not, because it's separated either way. In mixedCase,
the first one merges the H of Header in with HTTP; with something less
well known, that can be a nasty readability problem. The second one is
probably more readable, but looks weird. Or, here's another one:
converting one thing into another, where both are named by their
initials:

convert_XML_to_JSON
convert_xml_to_json

convertXMLToJSON
convertXmlToJson

Same four options. Which is the more readable?

ChrisA

From tim.peters at gmail.com  Thu Apr 24 18:36:03 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Thu, 24 Apr 2014 11:36:03 -0500
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
Message-ID: <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>

There's been a bit of serious study on this.  The results are still
open to interpretation, though ;-)  Here's a nice summary:

http://whathecode.wordpress.com/2011/02/10/camelcase-vs-underscores-scientific-showdown/

of-course-dashes-are-most-natural-ly y'rs  - tim

On Thu, Apr 24, 2014 at 11:25 AM, Chris Angelico <rosuav at gmail.com> wrote:
> On Thu, Apr 24, 2014 at 11:59 PM, Barry Warsaw <barry at python.org> wrote:
>> I will say this: the original preference for underscore_names in PEP 8 was
>> spurred by user studies some of our early non-native English speaking users
>> conducted many years ago.  We learned that it was more difficult for many of
>> them to parse mixedCase names than underscore_names.  I'm afraid I probably no
>> longer have references to those studies, but the difference was pronounced,
>> IIRC, and I think it's easy to see why.  Underscores can be scanned by the eye
>> as spaces, while I'd hypothesize that the brain has to do more work to read
>> mixedCase names.
>
> Underscores also play much more nicely with initialisms. How would you
> spell a function named "Add HTTP Header"?
>
> add_HTTP_header
> add_http_header
>
> addHTTPHeader
> addHttpHeader
>
> Four options to choose from. The first two clearly separate the
> initialism from the other words; take your pick whether you want it
> uppercased or not, because it's separated either way. In mixedCase,
> the first one merges the H of Header in with HTTP; with something less
> well known, that can be a nasty readability problem. The second one is
> probably more readable, but looks weird. Or, here's another one:
> converting one thing into another, where both are named by their
> initials:
>
> convert_XML_to_JSON
> convert_xml_to_json
>
> convertXMLToJSON
> convertXmlToJson
>
> Same four options. Which is the more readable?
>
> ChrisA
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/tim.peters%40gmail.com

From chris.barker at noaa.gov  Thu Apr 24 18:53:52 2014
From: chris.barker at noaa.gov (Chris Barker)
Date: Thu, 24 Apr 2014 09:53:52 -0700
Subject: [Python-Dev] List vs Tuple / Homogeneous vs Heterogeneous /
 Mutable vs Immutable
In-Reply-To: <CALGmxEKYaMg8Mqgpwis9pivVapCbm6ok1tUkrBE2m3VFJOUYBQ@mail.gmail.com>
References: <CAEaFV_iasbNn0_jbN6XaR8+oNq5m+9vi1TfwTZcuHp7OnbqLSg@mail.gmail.com>
 <CAP1=2W6NMJdtnVr22=gqx8bOwoY7JM7uhk78V6D4iU_WBVF=cA@mail.gmail.com>
 <CAP7+vJKMyJ8sm3a3Hce3zngcmDz0SG+V4e31K6qFP38_-2LpYg@mail.gmail.com>
 <CAEaFV_jWbG+u6O5Zzy0+TvYc6otJ11dXPqYD7EXb0VhV+mdPFg@mail.gmail.com>
 <5350E095.3070705@farowl.co.uk>
 <3F7F8B70-4558-42CA-BC12-9A241746B2D0@gmail.com>
 <CALGmxEKYaMg8Mqgpwis9pivVapCbm6ok1tUkrBE2m3VFJOUYBQ@mail.gmail.com>
Message-ID: <CALGmxEKCCXkM0TKk6k-hc-TLVzup6HK_f54J=4VwWdDbccA=CA@mail.gmail.com>

Ooops,

Forgot "reply all" last time -- here it is again.

On Wed, Apr 23, 2014 at 3:17 PM, Chris Barker <chris.barker at noaa.gov> wrote:

> On Mon, Apr 21, 2014 at 11:39 PM, Raymond Hettinger <
> raymond.hettinger at gmail.com> wrote:
>
>> In fact, the distinction is extrinsic to their implementations.  It is
>> only important
>> because the rest of the language tends to treat them differently.  For
>> example,
>> you could store ['raymond', 'red'] as a list or as a tuple ('raymond',
>> 'red'), but you
>> wouldn't be punished until later when you tried:
>>
>>      'I think %s likes %s' % container     # str.__mod__ treats lists and
>> tuples differently
>>
>
> I've been bitten by that a lot -- I always wondered why that didn't work
> with any sequence. like "tuple unpacking", which is really sequence
> unpacking:
>
> x, y = [3,4]
>
> But anyway, when I teach Python, I do struggle with this issue -- I tend
> to give the explanation of "structs" vs. "homogenous sequences", but I feel
> like I am repeating a party line, rather than anything useful.  What I do
> is:
>
> If it needs to be immutable (dict key), then use a tuple
> If it needs to be mutable, then use a list
>
> Otherwise, you can use either, but I tend to use a tuple for small things
> I don't need to mutate, regardless of whether they are homogenous or not --
> it makes me feel better to get the perception of a *tiny* bit more
> efficiency.
>
> And sometimes you want to mutate small collections of inhomogeneous values
> (C sructs are mutable, after all).
>
> So I don't think this needs to be codified this in the docs anywhere.
>
>
>>  Likewise, there seems to be wide-spread confusion about make makes an
>> object immutable.  People seem to miss that ints, tuples, None and str are
>> immutable only because they lack any mutating methods,
>>
>
> not sure your point here -- that under the hood one could mutate them with
> C code?
>



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.go <Chris.Barker at noaa.gov>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140424/f46441cf/attachment.html>

From kristjan at ccpgames.com  Thu Apr 24 12:30:29 2014
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Thu, 24 Apr 2014 10:30:29 +0000
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <5358BABE.6030300@simplistix.co.uk>
References: <5358BABE.6030300@simplistix.co.uk>
Message-ID: <EFE3877620384242A686D52278B7CCD3A52DF4E7@RKV-IT-EXCH104.ccp.ad.local>



> -----Original Message-----
> From: Python-Dev [mailto:python-dev-
> bounces+kristjan=ccpgames.com at python.org] On Behalf Of Chris Withers
> Sent: 24. apr?l 2014 07:18
> To: Python-Dev
> Subject: [Python-Dev] pep8 reasoning
> The biggest sticking point is naming, particularly as it's the one thing that can't
> necessarily be changed module by module. What were the compelling
> reasons to go from mixedCase to underscore_separated? What's considered
> the best approach for migrating from the former to the latter?

I doubt that it was the original motivation, but there have been evidence of late suggesting that snake-case is in fact _better_ than CamelCase.  See for instance http://www.cs.kent.edu/~jmaletic/papers/ICPC2010-CamelCaseUnderScoreClouds.pdf
Science!

K

From babu.saidu at gmail.com  Thu Apr 24 18:42:50 2014
From: babu.saidu at gmail.com (babu)
Date: Thu, 24 Apr 2014 09:42:50 -0700 (PDT)
Subject: [Python-Dev] Import java classes from python
Message-ID: <1398357770814-5054707.post@n6.nabble.com>

Hi, I am new to python. I have a requirement to import java class from
python. Is there any possibility to import java class from python.



--
View this message in context: http://python.6.x6.nabble.com/Import-java-classes-from-python-tp5054707.html
Sent from the Python - python-dev mailing list archive at Nabble.com.

From bcannon at gmail.com  Thu Apr 24 19:15:01 2014
From: bcannon at gmail.com (Brett Cannon)
Date: Thu, 24 Apr 2014 17:15:01 +0000
Subject: [Python-Dev]  Import java classes from python
References: <1398357770814-5054707.post@n6.nabble.com>
Message-ID: <CAP1=2W4x7-yazt01wjo-tvSVgUTZEqp4ZvP4-PRFy4Sp3b9yZQ@mail.gmail.com>

On Thu Apr 24 2014 at 1:10:39 PM, babu <babu.saidu at gmail.com> wrote:

> Hi, I am new to python. I have a requirement to import java class from
> python. Is there any possibility to import java class from python.
>

This list is for the development *of* Python, not the *use* of Python. But
to quickly answer  your question you want to look at Jython.

-Brett


>
>
>
> --
> View this message in context: http://python.6.x6.nabble.com/
> Import-java-classes-from-python-tp5054707.html
> Sent from the Python - python-dev mailing list archive at Nabble.com.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140424/2538f5a4/attachment.html>

From barry at python.org  Thu Apr 24 19:31:13 2014
From: barry at python.org (Barry Warsaw)
Date: Thu, 24 Apr 2014 13:31:13 -0400
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <CALWZvp6Ouu58HvqPEwFLYY=r9e35dp2UB2+P-f3oYGvBMbMG6w@mail.gmail.com>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CALWZvp6Ouu58HvqPEwFLYY=r9e35dp2UB2+P-f3oYGvBMbMG6w@mail.gmail.com>
Message-ID: <20140424133113.4553688c@anarchist.wooz.org>

On Apr 24, 2014, at 06:51 PM, Tal Einat wrote:

>I speak two languages as mother tongues - English and Hebrew. Hebrew
>has no capital letters (and is also right-to-left) and is the spoken
>and written language in the parts of Israel where I've lived most of
>my life. Perhaps because of this, I do find that capital letters don't
>quite "jump out" for me and therefore I find mixedCase and CamelCase
>harder to grok at a quick glance than under_score_names.

I was musing at Pycon with others about how Go would handle identifiers in
languages that don't have capital letters, such as Hebrew, given the fact that
they use the case of the first letter to control identifier visibility.  Their
FAQ essentially punts on the issue (start your identifier with a letter that
can be capitalized, e.g. X???) but says that this feature will likely stay.
I guess it's Go's flavor of Python's whitespace rule. :)

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140424/549a3252/attachment.sig>

From dholth at gmail.com  Thu Apr 24 20:01:50 2014
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 24 Apr 2014 14:01:50 -0400
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
Message-ID: <CAG8k2+4BxweLOFOH4yZ9fbxKVOADFZaOTsDC4Vdrav3QXm7zWg@mail.gmail.com>

Fortunately, Unicode provides us with the COMBINING LOW LINE
character, combining the horizontal space-savings of camelCase with
the underscore-indicates-separation properties of _. And it's a valid
Python identifier.

convertx?mlt?oj?son



On Thu, Apr 24, 2014 at 12:25 PM, Chris Angelico <rosuav at gmail.com> wrote:
> On Thu, Apr 24, 2014 at 11:59 PM, Barry Warsaw <barry at python.org> wrote:
>> I will say this: the original preference for underscore_names in PEP 8 was
>> spurred by user studies some of our early non-native English speaking users
>> conducted many years ago.  We learned that it was more difficult for many of
>> them to parse mixedCase names than underscore_names.  I'm afraid I probably no
>> longer have references to those studies, but the difference was pronounced,
>> IIRC, and I think it's easy to see why.  Underscores can be scanned by the eye
>> as spaces, while I'd hypothesize that the brain has to do more work to read
>> mixedCase names.
>
> Underscores also play much more nicely with initialisms. How would you
> spell a function named "Add HTTP Header"?
>
> add_HTTP_header
> add_http_header
>
> addHTTPHeader
> addHttpHeader
>
> Four options to choose from. The first two clearly separate the
> initialism from the other words; take your pick whether you want it
> uppercased or not, because it's separated either way. In mixedCase,
> the first one merges the H of Header in with HTTP; with something less
> well known, that can be a nasty readability problem. The second one is
> probably more readable, but looks weird. Or, here's another one:
> converting one thing into another, where both are named by their
> initials:
>
> convert_XML_to_JSON
> convert_xml_to_json
>
> convertXMLToJSON
> convertXmlToJson
>
> Same four options. Which is the more readable?
>
> ChrisA
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/dholth%40gmail.com

From lukasz at langa.pl  Thu Apr 24 20:02:35 2014
From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=)
Date: Thu, 24 Apr 2014 11:02:35 -0700
Subject: [Python-Dev] static typing of input arguments in signatures
In-Reply-To: <5352BAE7.1040809@stoneleaf.us>
References: <CAP7+vJJbQ9hchijqDmpEdF_BU=WPsYiL1WHFzDfqk5vszFJ1GQ@mail.gmail.com>
 <libs4b$qqg$1@ger.gmane.org> <lid4uj$qj1$1@ger.gmane.org>
 <21C31DB4-6F80-43CE-B811-FB61938E9423@langa.pl> <lieqbs$d6s$1@ger.gmane.org>
 <20140413235826.142BF250030@webabinitio.net> <5352BAE7.1040809@stoneleaf.us>
Message-ID: <928F2080-1199-4963-8A42-685711CA0066@langa.pl>

On Apr 19, 2014, at 11:05 AM, Ethan Furman <ethan at stoneleaf.us> wrote:

> Personal experience:  I have my own copy of paramiko because it type checks for strings, and I routinely use a str-subclass.

I have had that kind of problem myself in the past. Most of the time the core issue wasn?t type checking, it was how (badly) it was implemented. `assert obj.__class__ == list` being my favorite one (especially when any iterable would work just fine).

-- 
Best regards,
?ukasz Langa

WWW: http://lukasz.langa.pl/
Twitter: @llanga
IRC: ambv on #python-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140424/da6526e2/attachment.html>

From rdmurray at bitdance.com  Thu Apr 24 20:32:16 2014
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 24 Apr 2014 14:32:16 -0400
Subject: [Python-Dev] ref leaks
In-Reply-To: <20140424151741.GA26075@sleipnir.bytereef.org>
References: <5358736C.3060703@stoneleaf.us>
 <1398312362.821.109788265.1EC35C4C@webmail.messagingengine.com>
 <5359203A.3040109@stoneleaf.us>
 <20140424151741.GA26075@sleipnir.bytereef.org>
Message-ID: <20140424183217.303BD250D6A@webabinitio.net>

On Thu, 24 Apr 2014 17:17:41 +0200, Stefan Krah <stefan at bytereef.org> wrote:
> Ethan Furman <ethan at stoneleaf.us> wrote:
> > >>Any words of wisdom for tracking those leaks?
> 
> Often the easiest way is to compile --with-valgrind and run the test
> under Valgrind (obviously).
> 
> In the Valgrind output, search for "definitely lost" and ignore the rest.
> 
> If that does not turn up anything, consider a bug in regrtest.py:
> 
> It happens that certain caches are initialized incrementally in each
> repetition of the test, so the reported leaks are spurious. That is
> why some caches like small integers are "warmed up" in regrtest.py.
> 
> Perhaps there is some recently added cache that needs attention.

Note that if you do find a "real" leak, I would start with the assumption
that any leak is a leak at the python level (eg: a python cache or an
possibly-accidentally-persistent data structure) and only move to the
C level once I'd ruled out a python source for the "leak".

Unless you are dealing with a wholly or primarily C module, of course :)

--David

From tjreedy at udel.edu  Thu Apr 24 20:51:57 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 24 Apr 2014 14:51:57 -0400
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
Message-ID: <ljbmhc$t8h$1@ger.gmane.org>

On 4/24/2014 12:36 PM, Tim Peters wrote:
> There's been a bit of serious study on this.  The results are still
> open to interpretation, though ;-)  Here's a nice summary:
>
> http://whathecode.wordpress.com/2011/02/10/camelcase-vs-underscores-scientific-showdown/

The linked poll is almost evenly split, 52% to 48% for camel case with 
over 2000 votes each.

-- 
Terry Jan Reedy


From tim.peters at gmail.com  Thu Apr 24 21:05:41 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Thu, 24 Apr 2014 14:05:41 -0500
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <ljbmhc$t8h$1@ger.gmane.org>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <ljbmhc$t8h$1@ger.gmane.org>
Message-ID: <CAExdVNmyxyFWhL6RV5JWzEdzmsVDe8MQ6ux0sJxoRmWTzB+T8Q@mail.gmail.com>

[Tim]
>> There's been a bit of serious study on this.  The results are still
>> open to interpretation, though ;-)  Here's a nice summary:
>>
>>
>> http://whathecode.wordpress.com/2011/02/10/camelcase-vs-underscores-scientific-showdown/

[Terry Reedy]
> The linked poll is almost evenly split, 52% to 48% for camel case with over
> 2000 votes each.

There are two polls in the post.  You're referring to the first, which
the author asked viewers to fill out before reading any of the
"evidence".

There's another poll at the end, which the author asked viewers to
fill out after reading the whole post.  In that poll, preference
changed (56% to 44% in favor of underscore).

A followup post summarized later research, confirming the earlier
result that CamelCase'd names are harder to read (as measured by eye
tracking studies):

    http://whathecode.wordpress.com/2013/02/16/camelcase-vs-underscores-revisited/

but-all-obvious-a-priori-to-the-most-casual-observer<wink>-ly y'rs  - tim

From v+python at g.nevcal.com  Thu Apr 24 23:24:51 2014
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Thu, 24 Apr 2014 14:24:51 -0700
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <CAG8k2+4BxweLOFOH4yZ9fbxKVOADFZaOTsDC4Vdrav3QXm7zWg@mail.gmail.com>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAG8k2+4BxweLOFOH4yZ9fbxKVOADFZaOTsDC4Vdrav3QXm7zWg@mail.gmail.com>
Message-ID: <53598123.9070700@g.nevcal.com>

On 4/24/2014 11:01 AM, Daniel Holth wrote:
> Fortunately, Unicode provides us with the COMBINING LOW LINE
> character, combining the horizontal space-savings of camelCase with
> the underscore-indicates-separation properties of _. And it's a valid
> Python identifier.
>
> convertx?mlt?oj?son

That should be   convertX?MLt?oJ?SON  of course :) :)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140424/b111151a/attachment.html>

From cyberdupo56 at gmail.com  Fri Apr 25 03:40:01 2014
From: cyberdupo56 at gmail.com (Allen Li)
Date: Thu, 24 Apr 2014 21:40:01 -0400
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
Message-ID: <20140425014001.GA31627@gensokyo.amherst.edu>

On Thu, Apr 24, 2014 at 11:36:03AM -0500, Tim Peters wrote:
> There's been a bit of serious study on this.  The results are still
> open to interpretation, though ;-)  Here's a nice summary:
> 
> http://whathecode.wordpress.com/2011/02/10/camelcase-vs-underscores-scientific-showdown/

To summarize the key points of the two papers referenced in this thread:

Blinky 2009:  Participants were trained in camelCase.  camelCase was
13.5% slower, 51.5% more accurate on an identifying task.

Sharif 2010:  Participants were trained in underscores.  Same accuracy,
camelCase was 20% slower.

It seems like, generalizing, camelCase is slower and more accurate,
while training affects both speed and accuracy.

But, really, there is no compelling scientific argument here.  It really
boils down to CONSISTENCY:

1) If the existing code uses one or the other, follow the original code
   to preserve CONSISTENCY.
2) If you're starting a new project, follow PEP8 (or the standards for
   the language you're using) to preserve CONSISTENCY.

From rosuav at gmail.com  Fri Apr 25 04:00:17 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 25 Apr 2014 12:00:17 +1000
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <20140425014001.GA31627@gensokyo.amherst.edu>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
Message-ID: <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>

On Fri, Apr 25, 2014 at 11:40 AM, Allen Li <cyberdupo56 at gmail.com> wrote:
> 2) If you're starting a new project, follow PEP8 (or the standards for
>    the language you're using) to preserve CONSISTENCY.

Don't forget that PEP 8 is not the standard for the Python language,
only the Python stdlib. Particularly, there's no strong reason to
follow some of its lesser advices (eg spaces rather than tabs, the
exact maximum line length) for new projects; if all your developers
have good quality screens, there's no point wrapping all your code to
79 just because the Python standard library is wrapped to 79. That
particular example is touched on in the PEP itself (suggesting 20 more
characters); you could easily set a maximum of 89, 99, or 159
characters if it makes sense for your project. Consistency with the
standard library in names is slightly more useful, but the standard
library isn't perfectly consistent itself (eg not all class names
start with a capital), and if you're spending too much time arguing
about style instead of getting code written, your style guide isn't
doing its job :)

ChrisA

From barry at python.org  Fri Apr 25 05:03:38 2014
From: barry at python.org (Barry Warsaw)
Date: Thu, 24 Apr 2014 23:03:38 -0400
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
Message-ID: <20140424230338.1d11c6c9@limelight.wooz.org>

On Apr 25, 2014, at 12:00 PM, Chris Angelico wrote:

>Don't forget that PEP 8 is not the standard for the Python language,
>only the Python stdlib. Particularly, there's no strong reason to
>follow some of its lesser advices (eg spaces rather than tabs, the
>exact maximum line length) for new projects;

I'd say it depends.  If the code is going to be shared with people outside of
your organization (e.g. open source libraries), then there's a strong
motivation to be consistent throughout the community, which means PEP 8.

-Barry

From greg.ewing at canterbury.ac.nz  Fri Apr 25 03:00:31 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 25 Apr 2014 13:00:31 +1200
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
Message-ID: <5359B3AF.5080805@canterbury.ac.nz>

Chris Angelico wrote:

> add_HTTP_header
> add_http_header
> 
> addHTTPHeader
> addHttpHeader

Five... there are FIVE options...

convertXMLtoJSON

i.e. don't capitalise a part that follows capitalised
initials.

-- 
Greg

From guido at python.org  Fri Apr 25 05:28:27 2014
From: guido at python.org (Guido van Rossum)
Date: Thu, 24 Apr 2014 20:28:27 -0700
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
Message-ID: <CAP7+vJK6qoq3vU=zR081bt8cL6=42Re3O2pbn1r+aq+OE5bE_g@mail.gmail.com>

On Apr 24, 2014 7:01 PM, "Chris Angelico" <rosuav at gmail.com> wrote:
>
> On Fri, Apr 25, 2014 at 11:40 AM, Allen Li <cyberdupo56 at gmail.com> wrote:
> > 2) If you're starting a new project, follow PEP8 (or the standards for
> >    the language you're using) to preserve CONSISTENCY.
>
> Don't forget that PEP 8 is not the standard for the Python language,
> only the Python stdlib.

That is not true. It is mandatory for the stdlib, for other projects it is
optional,  but still recommend.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140424/ec108d0f/attachment.html>

From steve at pearwood.info  Fri Apr 25 05:55:49 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 25 Apr 2014 13:55:49 +1000
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <53590A42.4070700@timgolden.me.uk>
References: <5358BABE.6030300@simplistix.co.uk>
 <CALWZvp5g8TxbvvegdKLgw-iArF7RZc7BS-MWS5FJpCfFhd+T1g@mail.gmail.com>
 <53590A42.4070700@timgolden.me.uk>
Message-ID: <20140425035548.GB4273@ando>

On Thu, Apr 24, 2014 at 01:57:38PM +0100, Tim Golden wrote:
> On 24/04/2014 12:59, Tal Einat wrote:
> > As far as I know that reason for these examples being frowned upon is
> > that they are needlessly redundant. 
> 
> Oh, the irony! (Sorry, couldn't resist)

Not really ironic, since not all redundancy is needless.

Useful redundancy: an emergency backup chute; if your main parachute 
fails to release, you have a backup. RAID. Backups in general. There's a 
lot of redundancy in human language in general, which makes it possible 
to understand speech even in noisy enviroments.

Needless redundancy: a double-headed spoon, in case the first bowl 
disintegrates in your soup, just turn the spoon around and use the 
spare. Or the abuse of Hungarian Notation perpetrated by the Windows 
development team.

http://www.joelonsoftware.com/articles/Wrong.html

One mild but legitimate criticism of Python's significant indentation 
feature is that it loses some useful redundancy: you can reconstruct the 
indentation from the braces, or the braces from the indentation, but if 
you only have one, and it gets lost in transmission, you're screwed.

(The lesson of this, as far as I am concerned, is "Don't transmit source 
code through noisy channels that delete parts of your code.")



-- 
Steven

From chris at simplistix.co.uk  Fri Apr 25 14:08:05 2014
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 25 Apr 2014 13:08:05 +0100
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
Message-ID: <535A5025.3040700@simplistix.co.uk>

On 25/04/2014 03:00, Chris Angelico wrote:
> On Fri, Apr 25, 2014 at 11:40 AM, Allen Li <cyberdupo56 at gmail.com> wrote:
>> 2) If you're starting a new project, follow PEP8 (or the standards for
>>     the language you're using) to preserve CONSISTENCY.
>
> Don't forget that PEP 8 is not the standard for the Python language,
> only the Python stdlib.

I'm not sure I agree with that. PEP 8 *is* the standard for the 
language, in that there isn't anything else.

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
             - http://www.simplistix.co.uk

From chris at simplistix.co.uk  Fri Apr 25 14:09:36 2014
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 25 Apr 2014 13:09:36 +0100
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <20140424230338.1d11c6c9@limelight.wooz.org>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
Message-ID: <535A5080.6030406@simplistix.co.uk>

On 25/04/2014 04:03, Barry Warsaw wrote:
> On Apr 25, 2014, at 12:00 PM, Chris Angelico wrote:
>
>> Don't forget that PEP 8 is not the standard for the Python language,
>> only the Python stdlib. Particularly, there's no strong reason to
>> follow some of its lesser advices (eg spaces rather than tabs, the
>> exact maximum line length) for new projects;
>
> I'd say it depends.  If the code is going to be shared with people outside of
> your organization (e.g. open source libraries), then there's a strong
> motivation to be consistent throughout the community, which means PEP 8.

Right, so for me this means even in a private code base, there are big 
benefits to using PEP 8; everything looks the same, whether it's a third 
party library, python core or your own code...

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
             - http://www.simplistix.co.uk

From mail at timgolden.me.uk  Fri Apr 25 14:53:24 2014
From: mail at timgolden.me.uk (Tim Golden)
Date: Fri, 25 Apr 2014 13:53:24 +0100
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <535A5080.6030406@simplistix.co.uk>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
Message-ID: <535A5AC4.6000600@timgolden.me.uk>

On 25/04/2014 13:09, Chris Withers wrote:
> On 25/04/2014 04:03, Barry Warsaw wrote:
>> On Apr 25, 2014, at 12:00 PM, Chris Angelico wrote:
>>
>>> Don't forget that PEP 8 is not the standard for the Python language,
>>> only the Python stdlib. Particularly, there's no strong reason to
>>> follow some of its lesser advices (eg spaces rather than tabs, the
>>> exact maximum line length) for new projects;
>>
>> I'd say it depends.  If the code is going to be shared with people
>> outside of
>> your organization (e.g. open source libraries), then there's a strong
>> motivation to be consistent throughout the community, which means PEP 8.
> 
> Right, so for me this means even in a private code base, there are big
> benefits to using PEP 8; everything looks the same, whether it's a third
> party library, python core or your own code...

I essentially had this conversation via my blog a couple of years ago:

  http://ramblings.timgolden.me.uk/2012/03/27/pep8-or-not/

  http://ramblings.timgolden.me.uk/2012/03/29/more-on-pep8/

  http://ramblings.timgolden.me.uk/2012/04/09/pep8-it-is-then/

and was quite surprised at the strength of feeling evoked. As you can
even tell from the titles of the posts, I ended up by accepting that,
even though I'm at liberty to apply my own format to my own code, it
would be more of a community-friendly idea to use [near-enough] PEP8
regardless.

As you can also see from the comments throughout, YMMV.

:)

TJG

From rosuav at gmail.com  Fri Apr 25 15:23:50 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 25 Apr 2014 23:23:50 +1000
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <CAP7+vJK6qoq3vU=zR081bt8cL6=42Re3O2pbn1r+aq+OE5bE_g@mail.gmail.com>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <CAP7+vJK6qoq3vU=zR081bt8cL6=42Re3O2pbn1r+aq+OE5bE_g@mail.gmail.com>
Message-ID: <CAPTjJmpCpuWdAMAwXHCG1GESFKOPHjtFmC_oS3qH=C6SQ+dmsA@mail.gmail.com>

On Fri, Apr 25, 2014 at 1:28 PM, Guido van Rossum <guido at python.org> wrote:
> On Apr 24, 2014 7:01 PM, "Chris Angelico" <rosuav at gmail.com> wrote:
>>
>> On Fri, Apr 25, 2014 at 11:40 AM, Allen Li <cyberdupo56 at gmail.com> wrote:
>> > 2) If you're starting a new project, follow PEP8 (or the standards for
>> >    the language you're using) to preserve CONSISTENCY.
>>
>> Don't forget that PEP 8 is not the standard for the Python language,
>> only the Python stdlib.
>
> That is not true. It is mandatory for the stdlib, for other projects it is
> optional,  but still recommend.

Hmm. There've been a good few conversations on python-list where
people have been reminded that PEP 8 is *not* all that important for
other projects (beyond that it's an already-written style guide - when
you're starting a C project you can argue style guides for weeks, but
when you start a Python project you can just say "PEP 8" and skip the
bikeshedding), which I took to mean that it's on par with any other
style guide; but looking at it the other way, it can be seen as a
strong recommendation. I'm just not sure about the word "standard"
there - to me a standard is something that you MUST follow, like RFCs
for internet protocols.

But that's just a matter of terminology, I guess.

ChrisA

From stephen at xemacs.org  Fri Apr 25 16:06:35 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 25 Apr 2014 23:06:35 +0900
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <535A5080.6030406@simplistix.co.uk>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
Message-ID: <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>

Chris Withers writes:
 > On 25/04/2014 04:03, Barry Warsaw wrote:

 > > I'd say it depends.  If the code is going to be shared with
 > > people outside of your organization (e.g. open source libraries),
 > > then there's a strong motivation to be consistent throughout the
 > > community, which means PEP 8.
 > 
 > Right, so for me this means even in a private code base, there are big 
 > benefits to using PEP 8; everything looks the same, whether it's a third 
 > party library, python core or your own code...

Indeed.  But channeling the FLUFL here, I would say that in your
situation where people have already developed a large code base that
apparently isn't shared much (if at all) outside the organization,
they've already paid the price of nonconformity for the existing code
base.  If there's a formal standard, and you don't expect the
"insiders only" policy to change soon (assuming it actually exists), I
don't see why you would push for a change to PEP 8 at all.

If you *don't* have a formal standard and existing code is a mixture
of styles, I'd be very tempted to say "PEP 8 for new modules" (even if
some of the mix is consistent -- for example all the ex-C++
programmers like CamelCase).  Ditto if you're going to be developing
modules to publish as OSS.

And from *outside* of your organization, it's a no-brainer.  PEP 8 is
what Python itself and most 3rd party OSS modules use.  Getting your
people to use PEP 8 will make it a lot easier for them to learn to
read Python core and stdlib code, and once they start reading --- why,
the obvious next step is *contributing*.  Yay! ;-)

From barry at python.org  Fri Apr 25 16:36:47 2014
From: barry at python.org (Barry Warsaw)
Date: Fri, 25 Apr 2014 10:36:47 -0400
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20140425103647.1eafca9f@anarchist.wooz.org>

On Apr 25, 2014, at 11:06 PM, Stephen J. Turnbull wrote:

>And from *outside* of your organization, it's a no-brainer.  PEP 8 is
>what Python itself and most 3rd party OSS modules use.  Getting your
>people to use PEP 8 will make it a lot easier for them to learn to
>read Python core and stdlib code, and once they start reading --- why,
>the obvious next step is *contributing*.  Yay! ;-)

And if you do deviate from PEP 8, I think it's quite helpful to publish a
style guide outlining the deviations.  It helps others contribute to your
project.  Of course, the first line of that style guide should be:

    `PEP 8`_ is the basis for this style guide so its recommendations
    should be followed except for the differences outlined here.

:)

cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140425/3fe87b5d/attachment.sig>

From status at bugs.python.org  Fri Apr 25 18:07:37 2014
From: status at bugs.python.org (Python tracker)
Date: Fri, 25 Apr 2014 18:07:37 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20140425160737.3AA5C56A2A@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2014-04-18 - 2014-04-25)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    4600 (+21)
  closed 28521 (+28)
  total  33121 (+49)

Open issues with patches: 2126 


Issues opened (41)
==================

#21301: pathlib missing Path.expandvars(env=os.environ)
http://bugs.python.org/issue21301  opened by Alain.Mellan

#21302: time.sleep (floatsleep()) should use clock_nanosleep() on Linu
http://bugs.python.org/issue21302  opened by shankarunni

#21304: PEP 466: Backport hashlib.pbkdf2_hmac to Python 2.7
http://bugs.python.org/issue21304  opened by alex

#21305: PEP 466: update os.urandom
http://bugs.python.org/issue21305  opened by ncoghlan

#21306: PEP 466: backport hmac.compare_digest
http://bugs.python.org/issue21306  opened by ncoghlan

#21307: PEP 466: backport hashlib changes
http://bugs.python.org/issue21307  opened by ncoghlan

#21308: PEP 466: backport ssl changes
http://bugs.python.org/issue21308  opened by ncoghlan

#21309: Confusing "see also" for generic C-level __init__ methods in h
http://bugs.python.org/issue21309  opened by r.david.murray

#21310: ResourceWarning when open() fails with io.UnsupportedOperation
http://bugs.python.org/issue21310  opened by vadmium

#21312: Update thread_foobar.h to include timed locking and TLS suppor
http://bugs.python.org/issue21312  opened by Jack.McCracken

#21313: Py_GetVersion() is broken when using mqueue and a long patch n
http://bugs.python.org/issue21313  opened by eric.snow

#21314: Bizarre help
http://bugs.python.org/issue21314  opened by Vedran.??a??i??

#21315: email._header_value_parser does not recognise in-line encoding
http://bugs.python.org/issue21315  opened by valhallasw

#21316: mark test_devpoll to be meaningfull only for Solaris
http://bugs.python.org/issue21316  opened by AapoRantalainen

#21318: sdist fails with symbolic links do non-existing files
http://bugs.python.org/issue21318  opened by jgosmann

#21319: WindowsRegistryFinder never added to sys.meta_path
http://bugs.python.org/issue21319  opened by eric.snow

#21320: dict() allows keyword expansion with integer keys, e.g. dict(*
http://bugs.python.org/issue21320  opened by Cyphase

#21321: itertools.islice() doesn't release reference to the source ite
http://bugs.python.org/issue21321  opened by Anton.Afanasyev

#21322: Pathlib .owner() and .group() methods fail on broken links
http://bugs.python.org/issue21322  opened by inigoserna

#21323: CGI HTTP server not running scripts from subdirectories
http://bugs.python.org/issue21323  opened by k.s.solnushkin

#21324: dbhash/bsddb leaks random memory fragments to a database
http://bugs.python.org/issue21324  opened by wodny

#21325: Missing Generic EXIF library for images in the standard librar
http://bugs.python.org/issue21325  opened by karlcow

#21326: asyncio: request clearer error message when event loop closed
http://bugs.python.org/issue21326  opened by mark.dickinson

#21327: socket.type value changes after using settimeout()
http://bugs.python.org/issue21327  opened by giampaolo.rodola

#21328: Resize doesn't change reported length on create_string_buffer(
http://bugs.python.org/issue21328  opened by Dustin.Oprea

#21329: configparser can't parse MySQL style config
http://bugs.python.org/issue21329  opened by dveeden

#21331: Reversing an encoding with unicode-escape returns a different 
http://bugs.python.org/issue21331  opened by Sworddragon

#21332: subprocess bufsize=1 docs are misleading
http://bugs.python.org/issue21332  opened by raylu

#21333: Document recommended exception for objects that shouldn't be p
http://bugs.python.org/issue21333  opened by sschwarzer

#21335: Update importlib.__init__ to reset _frozen_importlib's loader 
http://bugs.python.org/issue21335  opened by brett.cannon

#21337: Add tests for Tix
http://bugs.python.org/issue21337  opened by zach.ware

#21338: Silent mode for compileall
http://bugs.python.org/issue21338  opened by takluyver

#21339: IDLE crash on OS X 1.9 upon shut-down with many windows open
http://bugs.python.org/issue21339  opened by rhettinger

#21340: Possible concurrency bug in asyncio, AttributeError in tasks.p
http://bugs.python.org/issue21340  opened by Jack.Murray

#21341: Configuring 'font' with ttk.Style for 'TEntry' does not change
http://bugs.python.org/issue21341  opened by barron

#21342: multiprocessing RLock and Lock raise incorrect exceptions when
http://bugs.python.org/issue21342  opened by steinn

#21343: os.path.relpath returns inconsistent types
http://bugs.python.org/issue21343  opened by Matt.Bachmann

#21344: save scores or ratios in difflib get_close_matches
http://bugs.python.org/issue21344  opened by russellballestrini

#21345: multiprocessing.Pool._handle_workers sleeps too long
http://bugs.python.org/issue21345  opened by Johannes.Baiter

#21347: Don't use a list argument together with shell=True in subproce
http://bugs.python.org/issue21347  opened by akira

#21349: crash in winreg SetValueEx with memoryview
http://bugs.python.org/issue21349  opened by bdkearns



Most recent 15 issues with no replies (15)
==========================================

#21349: crash in winreg SetValueEx with memoryview
http://bugs.python.org/issue21349

#21347: Don't use a list argument together with shell=True in subproce
http://bugs.python.org/issue21347

#21345: multiprocessing.Pool._handle_workers sleeps too long
http://bugs.python.org/issue21345

#21343: os.path.relpath returns inconsistent types
http://bugs.python.org/issue21343

#21342: multiprocessing RLock and Lock raise incorrect exceptions when
http://bugs.python.org/issue21342

#21341: Configuring 'font' with ttk.Style for 'TEntry' does not change
http://bugs.python.org/issue21341

#21339: IDLE crash on OS X 1.9 upon shut-down with many windows open
http://bugs.python.org/issue21339

#21337: Add tests for Tix
http://bugs.python.org/issue21337

#21335: Update importlib.__init__ to reset _frozen_importlib's loader 
http://bugs.python.org/issue21335

#21333: Document recommended exception for objects that shouldn't be p
http://bugs.python.org/issue21333

#21332: subprocess bufsize=1 docs are misleading
http://bugs.python.org/issue21332

#21329: configparser can't parse MySQL style config
http://bugs.python.org/issue21329

#21328: Resize doesn't change reported length on create_string_buffer(
http://bugs.python.org/issue21328

#21323: CGI HTTP server not running scripts from subdirectories
http://bugs.python.org/issue21323

#21319: WindowsRegistryFinder never added to sys.meta_path
http://bugs.python.org/issue21319



Most recent 15 issues waiting for review (15)
=============================================

#21349: crash in winreg SetValueEx with memoryview
http://bugs.python.org/issue21349

#21347: Don't use a list argument together with shell=True in subproce
http://bugs.python.org/issue21347

#21344: save scores or ratios in difflib get_close_matches
http://bugs.python.org/issue21344

#21343: os.path.relpath returns inconsistent types
http://bugs.python.org/issue21343

#21342: multiprocessing RLock and Lock raise incorrect exceptions when
http://bugs.python.org/issue21342

#21338: Silent mode for compileall
http://bugs.python.org/issue21338

#21322: Pathlib .owner() and .group() methods fail on broken links
http://bugs.python.org/issue21322

#21321: itertools.islice() doesn't release reference to the source ite
http://bugs.python.org/issue21321

#21315: email._header_value_parser does not recognise in-line encoding
http://bugs.python.org/issue21315

#21313: Py_GetVersion() is broken when using mqueue and a long patch n
http://bugs.python.org/issue21313

#21312: Update thread_foobar.h to include timed locking and TLS suppor
http://bugs.python.org/issue21312

#21306: PEP 466: backport hmac.compare_digest
http://bugs.python.org/issue21306

#21304: PEP 466: Backport hashlib.pbkdf2_hmac to Python 2.7
http://bugs.python.org/issue21304

#21300: Docs (incorrectly) suggest email.policy.default is the default
http://bugs.python.org/issue21300

#21293: Remove "capsule hack" from object.c?
http://bugs.python.org/issue21293



Top 10 most discussed issues (10)
=================================

#18967: Find a less conflict prone approach to Misc/NEWS
http://bugs.python.org/issue18967  22 msgs

#17552: socket.sendfile()
http://bugs.python.org/issue17552  19 msgs

#21344: save scores or ratios in difflib get_close_matches
http://bugs.python.org/issue21344  10 msgs

#21207: urandom persistent fd - not re-openned after fd close
http://bugs.python.org/issue21207   7 msgs

#21306: PEP 466: backport hmac.compare_digest
http://bugs.python.org/issue21306   7 msgs

#8387: use universal newline mode in csv module examples
http://bugs.python.org/issue8387   6 msgs

#21305: PEP 466: update os.urandom
http://bugs.python.org/issue21305   6 msgs

#21312: Update thread_foobar.h to include timed locking and TLS suppor
http://bugs.python.org/issue21312   6 msgs

#21331: Reversing an encoding with unicode-escape returns a different 
http://bugs.python.org/issue21331   6 msgs

#21338: Silent mode for compileall
http://bugs.python.org/issue21338   6 msgs



Issues closed (27)
==================

#7221: DispatcherWithSendTests_UsePoll with test_asyncore does nothin
http://bugs.python.org/issue7221  closed by pitrou

#8297: AttributeError message text should include module name
http://bugs.python.org/issue8297  closed by benjamin.peterson

#9364: some problems with the documentation of pydoc
http://bugs.python.org/issue9364  closed by r.david.murray

#9764: Tools/buildbot/external.bat should download and built tix
http://bugs.python.org/issue9764  closed by zach.ware

#9765: tcl-8 vs tcl8
http://bugs.python.org/issue9765  closed by zach.ware

#10291: Clean-up turtledemo in-package documentation
http://bugs.python.org/issue10291  closed by ned.deily

#10362: AttributeError: addinfourl instance has no attribute 'tell'
http://bugs.python.org/issue10362  closed by orsenthil

#11748: test_ftplib failure in test for source_address
http://bugs.python.org/issue11748  closed by jesstess

#12220: minidom xmlns not handling spaces in xmlns attribute value fie
http://bugs.python.org/issue12220  closed by r.david.murray

#20565: Update Tcl/Tk version for buildbots
http://bugs.python.org/issue20565  closed by zach.ware

#21068: Make ssl.PROTOCOL_* an enum
http://bugs.python.org/issue21068  closed by pitrou

#21127: Path objects cannot be constructed from str subclasses
http://bugs.python.org/issue21127  closed by pitrou

#21139: Idle: change default reformat width from 70 to 72
http://bugs.python.org/issue21139  closed by terry.reedy

#21200: pkgutil.get_loader() fails on "__main__"
http://bugs.python.org/issue21200  closed by eric.snow

#21211: pkgutil.find_loader() raises ImportError instead of returning 
http://bugs.python.org/issue21211  closed by eric.snow

#21232: Use of '1' instead of 'True' as 'splitlines' argument in diffl
http://bugs.python.org/issue21232  closed by python-dev

#21284: IDLE reformat tests fail in presence of non-default FormatPara
http://bugs.python.org/issue21284  closed by terry.reedy

#21289: make.bat not building documentation
http://bugs.python.org/issue21289  closed by zach.ware

#21291: subprocess Popen objects are not thread safe w.r.t. wait() and
http://bugs.python.org/issue21291  closed by gregory.p.smith

#21303: Python 2.7 Spinbox Format Behaves Differently On Windows Versu
http://bugs.python.org/issue21303  closed by zach.ware

#21311: Fix compiler detection when brew's ccache is installed on Mac 
http://bugs.python.org/issue21311  closed by ned.deily

#21317: Home page certificate troubles
http://bugs.python.org/issue21317  closed by alex

#21330: Typo in "Unicode HOWTO" documentation
http://bugs.python.org/issue21330  closed by benjamin.peterson

#21334: nntplib throws exceptions making sinntp unusable
http://bugs.python.org/issue21334  closed by randomcoder1

#21336: ntpath.splitdrive fails on None argument
http://bugs.python.org/issue21336  closed by eric.smith

#21346: typos in test_itertools.py
http://bugs.python.org/issue21346  closed by zach.ware

#21348: File "C:\Python27\lib\distutils\msvc9compiler.py", line 295, i
http://bugs.python.org/issue21348  closed by skrah

From ncoghlan at gmail.com  Fri Apr 25 18:10:20 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 25 Apr 2014 12:10:20 -0400
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <20140425103647.1eafca9f@anarchist.wooz.org>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
Message-ID: <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>

On 25 April 2014 10:36, Barry Warsaw <barry at python.org> wrote:
> On Apr 25, 2014, at 11:06 PM, Stephen J. Turnbull wrote:
>
>>And from *outside* of your organization, it's a no-brainer.  PEP 8 is
>>what Python itself and most 3rd party OSS modules use.  Getting your
>>people to use PEP 8 will make it a lot easier for them to learn to
>>read Python core and stdlib code, and once they start reading --- why,
>>the obvious next step is *contributing*.  Yay! ;-)
>
> And if you do deviate from PEP 8, I think it's quite helpful to publish a
> style guide outlining the deviations.  It helps others contribute to your
> project.  Of course, the first line of that style guide should be:
>
>     `PEP 8`_ is the basis for this style guide so its recommendations
>     should be followed except for the differences outlined here.

And if you're going to publish a tool to enforce your *personal* style
guide and include your own custom rules that the "this is OK" examples
in PEP 8 fail to satisfy, don't call it "pep8". Especially don't do
that if you're then going to ignore a core developer that calls you on
misappropriating python-dev's authority :(

Regards,
Nick.

P.S. https://github.com/jcrocholl/pep8/issues/256

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From me at the-compiler.org  Fri Apr 25 16:22:06 2014
From: me at the-compiler.org (Florian Bruhin)
Date: Fri, 25 Apr 2014 16:22:06 +0200
Subject: [Python-Dev] ConfigParser mangles keys with special chars
Message-ID: <20140425142206.GD6735@lupin>

Hi,

I noticed configparser does behave in a surprising way when a key
has a special meaning in ini-format.

Consider this example:

    >>> cp = configparser.ConfigParser()
    >>> cp.read_dict({'DEFAULT': {';foo': 'bar'}})
    >>> cp.write(sys.stdout)
    [DEFAULT]
    ;foo = bar

Now when reading this again, ";foo = bar" will be a comment and
ignored. There's probably similiar behaviour in other corner cases
(like if you'd use "[foo]" as key for example).

While it seems ConfigParser doesn't do any escaping as all, I'm
thinking it should at least raise some exception when such a value is
trying to be set.

I'd expect writing something and then reading it back via the same
configparser to *always* result in the same data, as long as writing
worked without error.

Thoughts? Should I submit a bug report?

Florian

-- 
() ascii ribbon campaign - stop html mail    www.asciiribbon.org
/\ www.the-compiler.org  | I love long mails http://email.is-not-s.ms/
To give happiness is to deserve happiness. 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140425/2c50c918/attachment.sig>

From fred at fdrake.net  Fri Apr 25 18:46:42 2014
From: fred at fdrake.net (Fred Drake)
Date: Fri, 25 Apr 2014 12:46:42 -0400
Subject: [Python-Dev] ConfigParser mangles keys with special chars
In-Reply-To: <20140425142206.GD6735@lupin>
References: <20140425142206.GD6735@lupin>
Message-ID: <CAFT4OTHjHdEhEzpG=SE_=spNaG7kmjZHwgxBfTKpEnw=iaLTjg@mail.gmail.com>

On Fri, Apr 25, 2014 at 10:22 AM, Florian Bruhin <me at the-compiler.org> wrote:
> While it seems ConfigParser doesn't do any escaping as all, I'm
> thinking it should at least raise some exception when such a value is
> trying to be set.
>
> I'd expect writing something and then reading it back via the same
> configparser to *always* result in the same data, as long as writing
> worked without error.
>
> Thoughts? Should I submit a bug report?

I believe you should, if only to provide a place to record why no
change gets made.

Had ConfigParser been more careful from the beginning, that would have
been really good.

At this point, it would be a backward-incompatible change, so it's
unlikely such a change could be allowed to affect existing code.


  -Fred

-- 
Fred L. Drake, Jr.    <fred at fdrake.net>
"A storm broke loose in my mind."  --Albert Einstein

From tjreedy at udel.edu  Fri Apr 25 19:30:26 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 25 Apr 2014 13:30:26 -0400
Subject: [Python-Dev] ConfigParser mangles keys with special chars
In-Reply-To: <CAFT4OTHjHdEhEzpG=SE_=spNaG7kmjZHwgxBfTKpEnw=iaLTjg@mail.gmail.com>
References: <20140425142206.GD6735@lupin>
 <CAFT4OTHjHdEhEzpG=SE_=spNaG7kmjZHwgxBfTKpEnw=iaLTjg@mail.gmail.com>
Message-ID: <lje64j$csb$1@ger.gmane.org>

On 4/25/2014 12:46 PM, Fred Drake wrote:
> On Fri, Apr 25, 2014 at 10:22 AM, Florian Bruhin <me at the-compiler.org> wrote:
>> While it seems ConfigParser doesn't do any escaping as all, I'm
>> thinking it should at least raise some exception when such a value is
>> trying to be set.
>>
>> I'd expect writing something and then reading it back via the same
>> configparser to *always* result in the same data, as long as writing
>> worked without error.
>>
>> Thoughts? Should I submit a bug report?
>
> I believe you should, if only to provide a place to record why no
> change gets made.

When you do, add lukasz.langa as nosy
https://docs.python.org/devguide/experts.html#experts

You might also take a look in test/test_configparser.py to see if any 
edge cases are tested for.

> Had ConfigParser been more careful from the beginning, that would have
> been really good.
>
> At this point, it would be a backward-incompatible change, so it's
> unlikely such a change could be allowed to affect existing code.
>
>
>    -Fred
>


-- 
Terry Jan Reedy


From ethan at stoneleaf.us  Fri Apr 25 19:41:56 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 25 Apr 2014 10:41:56 -0700
Subject: [Python-Dev] ConfigParser mangles keys with special chars
In-Reply-To: <CAFT4OTHjHdEhEzpG=SE_=spNaG7kmjZHwgxBfTKpEnw=iaLTjg@mail.gmail.com>
References: <20140425142206.GD6735@lupin>
 <CAFT4OTHjHdEhEzpG=SE_=spNaG7kmjZHwgxBfTKpEnw=iaLTjg@mail.gmail.com>
Message-ID: <535A9E64.1010107@stoneleaf.us>

On 04/25/2014 09:46 AM, Fred Drake wrote:
>
> At this point, it would be a backward-incompatible change, so it's
> unlikely such a change could be allowed to affect existing code.

All bug-fixes are backwards-incompatible, yet we fix them anyway.  ;)

It seems to me the real question is do we fix it in 3.5 only, or can we fix it in 3.4 and previous?  And the answer 
depends on whether this behavior can be reasonably relied on.

--
~Ethan~

From tjreedy at udel.edu  Fri Apr 25 20:45:15 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 25 Apr 2014 14:45:15 -0400
Subject: [Python-Dev] ConfigParser mangles keys with special chars
In-Reply-To: <535A9E64.1010107@stoneleaf.us>
References: <20140425142206.GD6735@lupin>
 <CAFT4OTHjHdEhEzpG=SE_=spNaG7kmjZHwgxBfTKpEnw=iaLTjg@mail.gmail.com>
 <535A9E64.1010107@stoneleaf.us>
Message-ID: <ljeags$l3t$1@ger.gmane.org>

On 4/25/2014 1:41 PM, Ethan Furman wrote:
> On 04/25/2014 09:46 AM, Fred Drake wrote:
>>
>> At this point, it would be a backward-incompatible change, so it's
>> unlikely such a change could be allowed to affect existing code.
>
> All bug-fixes are backwards-incompatible, yet we fix them anyway.  ;)
>
> It seems to me the real question is do we fix it in 3.5 only, or can we
> fix it in 3.4 and previous?  And the answer depends on whether this
> behavior can be reasonably relied on.

And that depends on whether the current thought-to-be-buggy behavior is 
specified by the doc, compatible with an ambiguous or under-specified 
doc, or prohibited by the doc (because the doc specified something else).

I leave it to someone to carefully read the doc, but a brief glance 
indicates "There are nearly as many INI format variants as there are 
applications using it. configparser goes a long way to provide support 
for the largest sensible set of INI styles available." So I wonder 
whether the thought-to-be-buggy behavior is actually buggy with respect 
to *all* the supported styles or just some of them.

-- 
Terry Jan Reedy


From fred at fdrake.net  Fri Apr 25 20:54:56 2014
From: fred at fdrake.net (Fred Drake)
Date: Fri, 25 Apr 2014 14:54:56 -0400
Subject: [Python-Dev] ConfigParser mangles keys with special chars
In-Reply-To: <ljeags$l3t$1@ger.gmane.org>
References: <20140425142206.GD6735@lupin>
 <CAFT4OTHjHdEhEzpG=SE_=spNaG7kmjZHwgxBfTKpEnw=iaLTjg@mail.gmail.com>
 <535A9E64.1010107@stoneleaf.us> <ljeags$l3t$1@ger.gmane.org>
Message-ID: <CAFT4OTE4Sf1_zs7RrSOPnC_vq5-8w91=OLK=cXcaMR51PXAO9A@mail.gmail.com>

On Fri, Apr 25, 2014 at 2:45 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> I leave it to someone to carefully read the doc, but a brief glance
> indicates "There are nearly as many INI format variants as there are
> applications using it. configparser goes a long way to provide support for
> the largest sensible set of INI styles available." So I wonder whether the
> thought-to-be-buggy behavior is actually buggy with respect to *all* the
> supported styles or just some of them.

Given that most variants are completely undocumented, answering this
is sufficiently intractable for me to call it intractable.

We've also run into compatibility issues because some applications
treated the "parser" object as an in-memory configuration database
throughout the process lifetime, updating values with non-string
values of various sorts.  Given the age of the module, the existing
number of uses of undocumented accidents of implementation is very
large.  Fixing "bugs" like this has an excellent track record of
uncovering these uses, so... we've grown a bit wary.  It's not unheard
of for projects to fork their own copies of configparser because of
changes to the stdlib version.  (No, I haven't done a census of such
cases, but I do know of at least one in a popular package.)


  -Fred

-- 
Fred L. Drake, Jr.    <fred at fdrake.net>
"A storm broke loose in my mind."  --Albert Einstein

From ethan at stoneleaf.us  Fri Apr 25 21:12:13 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 25 Apr 2014 12:12:13 -0700
Subject: [Python-Dev] ConfigParser mangles keys with special chars
In-Reply-To: <CAFT4OTE4Sf1_zs7RrSOPnC_vq5-8w91=OLK=cXcaMR51PXAO9A@mail.gmail.com>
References: <20140425142206.GD6735@lupin>
 <CAFT4OTHjHdEhEzpG=SE_=spNaG7kmjZHwgxBfTKpEnw=iaLTjg@mail.gmail.com>
 <535A9E64.1010107@stoneleaf.us> <ljeags$l3t$1@ger.gmane.org>
 <CAFT4OTE4Sf1_zs7RrSOPnC_vq5-8w91=OLK=cXcaMR51PXAO9A@mail.gmail.com>
Message-ID: <535AB38D.5090207@stoneleaf.us>

On 04/25/2014 11:54 AM, Fred Drake wrote:
> On Fri, Apr 25, 2014 at 2:45 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>> I leave it to someone to carefully read the doc, but a brief glance
>> indicates "There are nearly as many INI format variants as there are
>> applications using it. configparser goes a long way to provide support for
>> the largest sensible set of INI styles available." So I wonder whether the
>> thought-to-be-buggy behavior is actually buggy with respect to *all* the
>> supported styles or just some of them.
>
> Given that most variants are completely undocumented, answering this
> is sufficiently intractable for me to call it intractable.
>
> We've also run into compatibility issues because some applications
> treated the "parser" object as an in-memory configuration database
> throughout the process lifetime, updating values with non-string
> values of various sorts.  Given the age of the module, the existing
> number of uses of undocumented accidents of implementation is very
> large.  Fixing "bugs" like this has an excellent track record of
> uncovering these uses, so... we've grown a bit wary.  It's not unheard
> of for projects to fork their own copies of configparser because of
> changes to the stdlib version.  (No, I haven't done a census of such
> cases, but I do know of at least one in a popular package.)

Seems reasonable.

Perhaps an enhancement request then:  a way to provide a regex that keys must match, with an exception raised when a key 
doesn't.  That way the safety belt could be used when desired.

--
~Ethan~

From florent.xicluna at gmail.com  Fri Apr 25 21:45:09 2014
From: florent.xicluna at gmail.com (Florent)
Date: Fri, 25 Apr 2014 21:45:09 +0200
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
Message-ID: <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>

2014-04-25 18:10 GMT+02:00 Nick Coghlan <ncoghlan at gmail.com>:
>
> And if you're going to publish a tool to enforce your *personal* style
> guide and include your own custom rules that the "this is OK" examples
> in PEP 8 fail to satisfy, don't call it "pep8". Especially don't do
> that if you're then going to ignore a core developer that calls you on
> misappropriating python-dev's authority :(
>
> Regards,
> Nick.
>
> P.S. https://github.com/jcrocholl/pep8/issues/256
>

I am the current maintainer of this 'pep8' tool which was mentionned
in a previous message.
The tool was first published in 2006 by its original author.

I take the feedbacks of the users in consideration, as much as I can.
When something is unambiguously identified as a bug, I try to merge a
patch in a timely manner.
If it is an enhancement or a change of behavior, it is not in the
priority list, of course.

Two cases where signaled in the issue #256 last month, where the tool
is arguably not compliant with some of the current conventions of the
PEP.
I've highlighted the reasons behind these checkers in the issue
tracker directly.
I disagree that they are in direct opposition with the PEP 8 coding
conventions, though.

The first concern is about 4 spaces indentation for continuation lines.
For this point, the text of the PEP says "Use 4 spaces per indentation
level", which lets room for interpretation (like some other parts of
the PEP).
However I changed some stuff in v1.5 to better isolate this case, so
the user can add E121 safely to the ignore list:
https://github.com/jcrocholl/pep8/blob/master/CHANGES.txt#L96-L105

The second concern is about compound statements, which are "generally
discouraged" in the words of the PEP. Technically, there's no obvious
way to implement "generally discouraged" in code.
The compromise is to disable the check E701 and "use your own
judgment" instead, if you feel it does not match you project's style
guide.
By the way, the example reported in #256 was only added last year to the PEP:
    def f(x): return 2*x

And if we look closer to the PEP 8 history for the last two years
(since March 2012), there were lots of changes and I've already
updated the tool to catch up with some of them.
However the tool has a large base of users, and I try to preserve some
stability instead of removing features every time someone has an
objection.
In such case, what I recommend to the users is to configure the tool
to ignore the checks
which conflict with their own habits or conventions.

I wrote some words in the documentation, one year ago, to explain what
is the purpose of the tool and its limitations. There's no claim of
any endorsement implicit or explicit by the PSF, the PSU or any other
python developer :-)
http://pep8.readthedocs.org/en/latest/intro.html#disclaimer

I'm sorry if you felt that my previous answer was an offense to some
python-dev authority, this is not intentional.

Kind regards,

-- 
Florent

From fred at fdrake.net  Fri Apr 25 21:58:26 2014
From: fred at fdrake.net (Fred Drake)
Date: Fri, 25 Apr 2014 15:58:26 -0400
Subject: [Python-Dev] ConfigParser mangles keys with special chars
In-Reply-To: <535AB38D.5090207@stoneleaf.us>
References: <20140425142206.GD6735@lupin>
 <CAFT4OTHjHdEhEzpG=SE_=spNaG7kmjZHwgxBfTKpEnw=iaLTjg@mail.gmail.com>
 <535A9E64.1010107@stoneleaf.us> <ljeags$l3t$1@ger.gmane.org>
 <CAFT4OTE4Sf1_zs7RrSOPnC_vq5-8w91=OLK=cXcaMR51PXAO9A@mail.gmail.com>
 <535AB38D.5090207@stoneleaf.us>
Message-ID: <CAFT4OTGaDEmhEHUh8W-q2cFpY8dpAJjLeN5iaD3JZ4LCrj4x7w@mail.gmail.com>

On Fri, Apr 25, 2014 at 3:12 PM, Ethan Furman <ethan at stoneleaf.us> wrote:
> Perhaps an enhancement request then:  a way to provide a regex that keys
> must match, with an exception raised when a key doesn't.  That way the
> safety belt could be used when desired.

You can subclass the parser class you're using and override the
``optionxform`` method to perform the checks you want for option
names.  I don't know if current versions provide a similar hook to
validate section names; providing *that* would be an excellent
enhancement.


  -Fred

-- 
Fred L. Drake, Jr.    <fred at fdrake.net>
"A storm broke loose in my mind."  --Albert Einstein

From ethan at stoneleaf.us  Fri Apr 25 21:55:07 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 25 Apr 2014 12:55:07 -0700
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
Message-ID: <535ABD9B.8040807@stoneleaf.us>

On 04/25/2014 12:45 PM, Florent wrote:
> 2014-04-25 18:10 GMT+02:00 Nick Coghlan:
>>
>> And if you're going to publish a tool to enforce your *personal* style
>> guide and include your own custom rules that the "this is OK" examples
>> in PEP 8 fail to satisfy, don't call it "pep8".
>
> Two cases where signaled in the issue #256 last month, where the tool
> is arguably not compliant with some of the current conventions of the
> PEP.
> I've highlighted the reasons behind these checkers in the issue
> tracker directly.
> I disagree that they are in direct opposition with the PEP 8 coding
> conventions, though.

The problem is that you've named it pep8.  To me, that means I can run it and get PEP 8 results.  If I have to add a 
manual configuration to get actual PEP 8 semantics, it's a buggy tool.

For a similar example, I am the author/maintainer of enum34, which purports to be the backport of Python's 3.4 enum 
class.  While I could go and make changes to it to better match my style, or even the styles of other users, calling it 
enum34 after that would be misleading as some one couldn't then switch from using enum34 in Python 3.2 to using the 
default enum in Python 3.4.

If you had extra switches to turn on extra behavior, that would be fine.  Leaving it as it is, and calling it something 
else would be fine.  But as it stands now, with the name of pep8 and the behavior of failing on the PEP 8 document... 
well, that's false advertising.

--
~Ethan~

From carl at oddbird.net  Fri Apr 25 23:52:00 2014
From: carl at oddbird.net (Carl Meyer)
Date: Fri, 25 Apr 2014 15:52:00 -0600
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <535ABD9B.8040807@stoneleaf.us>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us>
Message-ID: <535AD900.6090606@oddbird.net>

On 04/25/2014 01:55 PM, Ethan Furman wrote:
> On 04/25/2014 12:45 PM, Florent wrote:
>> 2014-04-25 18:10 GMT+02:00 Nick Coghlan:
>>>
>>> And if you're going to publish a tool to enforce your *personal* style
>>> guide and include your own custom rules that the "this is OK" examples
>>> in PEP 8 fail to satisfy, don't call it "pep8".
>>
>> Two cases where signaled in the issue #256 last month, where the tool
>> is arguably not compliant with some of the current conventions of the
>> PEP.
>> I've highlighted the reasons behind these checkers in the issue
>> tracker directly.
>> I disagree that they are in direct opposition with the PEP 8 coding
>> conventions, though.
> 
> The problem is that you've named it pep8.  To me, that means I can run
> it and get PEP 8 results.  If I have to add a manual configuration to
> get actual PEP 8 semantics, it's a buggy tool.
> 
> For a similar example, I am the author/maintainer of enum34, which
> purports to be the backport of Python's 3.4 enum class.  While I could
> go and make changes to it to better match my style, or even the styles
> of other users, calling it enum34 after that would be misleading as some
> one couldn't then switch from using enum34 in Python 3.2 to using the
> default enum in Python 3.4.
> 
> If you had extra switches to turn on extra behavior, that would be
> fine.  Leaving it as it is, and calling it something else would be
> fine.  But as it stands now, with the name of pep8 and the behavior of
> failing on the PEP 8 document... well, that's false advertising.

I think this fuss is unreasonable and unwarranted.

I'd like to thank Florent for taking the time to maintain an
extremely-useful tool that makes it feasible to keep to a consistent PEP
8 style throughout a large codebase with many contributors, and I think
he should have the leeway as maintainer to make the necessary judgment
calls about precisely which PEP 8 recommendations are reported precisely
how by the tool, given that:

- we aren't talking about real variance from the spirit or
recommendations of PEP 8 (though you wouldn't know it from some of the
rhetoric here about "personal preferences")

- the tool makes it very easy to turn off whichever errors you don't
prefer to enforce.

- PEP 8 changes regularly (as Florent noted, the offending code example
was only added recently), and there is value in the automated tool
maintaining some stability for its users.

Personally, I would much rather see pep8.py err on the side of being too
strict with PEP 8's recommendations than too loose. Again, it's not hard
to turn off the ones you don't want.

If python-dev wants to control the precise behavior of pep8.py, bring it
into the standard library and adopt maintenance of it. Otherwise, please
give Florent some grace.

Carl

From donald at stufft.io  Sat Apr 26 00:26:27 2014
From: donald at stufft.io (Donald Stufft)
Date: Fri, 25 Apr 2014 18:26:27 -0400
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <535AD900.6090606@oddbird.net>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
Message-ID: <F8E67462-9CBD-44D0-B136-FF8ABDB24259@stufft.io>


On Apr 25, 2014, at 5:52 PM, Carl Meyer <carl at oddbird.net> wrote:

> On 04/25/2014 01:55 PM, Ethan Furman wrote:
>> On 04/25/2014 12:45 PM, Florent wrote:
>>> 2014-04-25 18:10 GMT+02:00 Nick Coghlan:
>>>> 
>>>> And if you're going to publish a tool to enforce your *personal* style
>>>> guide and include your own custom rules that the "this is OK" examples
>>>> in PEP 8 fail to satisfy, don't call it "pep8".
>>> 
>>> Two cases where signaled in the issue #256 last month, where the tool
>>> is arguably not compliant with some of the current conventions of the
>>> PEP.
>>> I've highlighted the reasons behind these checkers in the issue
>>> tracker directly.
>>> I disagree that they are in direct opposition with the PEP 8 coding
>>> conventions, though.
>> 
>> The problem is that you've named it pep8.  To me, that means I can run
>> it and get PEP 8 results.  If I have to add a manual configuration to
>> get actual PEP 8 semantics, it's a buggy tool.
>> 
>> For a similar example, I am the author/maintainer of enum34, which
>> purports to be the backport of Python's 3.4 enum class.  While I could
>> go and make changes to it to better match my style, or even the styles
>> of other users, calling it enum34 after that would be misleading as some
>> one couldn't then switch from using enum34 in Python 3.2 to using the
>> default enum in Python 3.4.
>> 
>> If you had extra switches to turn on extra behavior, that would be
>> fine.  Leaving it as it is, and calling it something else would be
>> fine.  But as it stands now, with the name of pep8 and the behavior of
>> failing on the PEP 8 document... well, that's false advertising.
> 
> I think this fuss is unreasonable and unwarranted.
> 
> I'd like to thank Florent for taking the time to maintain an
> extremely-useful tool that makes it feasible to keep to a consistent PEP
> 8 style throughout a large codebase with many contributors, and I think
> he should have the leeway as maintainer to make the necessary judgment
> calls about precisely which PEP 8 recommendations are reported precisely
> how by the tool, given that:
> 
> - we aren't talking about real variance from the spirit or
> recommendations of PEP 8 (though you wouldn't know it from some of the
> rhetoric here about "personal preferences")
> 
> - the tool makes it very easy to turn off whichever errors you don't
> prefer to enforce.
> 
> - PEP 8 changes regularly (as Florent noted, the offending code example
> was only added recently), and there is value in the automated tool
> maintaining some stability for its users.
> 
> Personally, I would much rather see pep8.py err on the side of being too
> strict with PEP 8's recommendations than too loose. Again, it's not hard
> to turn off the ones you don't want.
> 
> If python-dev wants to control the precise behavior of pep8.py, bring it
> into the standard library and adopt maintenance of it. Otherwise, please
> give Florent some grace.
> 
> Carl

Carl?s post mirrors my own thoughts and it?s said much better than I could have.

pep8.py doesn?t violate PEP8, it just takes a stricter view of it.


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140425/9bcab52d/attachment-0001.sig>

From ncoghlan at gmail.com  Sat Apr 26 00:33:36 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 25 Apr 2014 18:33:36 -0400
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <535AD900.6090606@oddbird.net>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
Message-ID: <CADiSq7c-RZnYp_cfARMv27V7TqQvhq00_pLJtLL3ofTXZQUfVg@mail.gmail.com>

On 25 April 2014 17:52, Carl Meyer <carl at oddbird.net> wrote:
> On 04/25/2014 01:55 PM, Ethan Furman wrote:
>> On 04/25/2014 12:45 PM, Florent wrote:
>>> 2014-04-25 18:10 GMT+02:00 Nick Coghlan:
> I think this fuss is unreasonable and unwarranted.
>
> I'd like to thank Florent for taking the time to maintain an
> extremely-useful tool that makes it feasible to keep to a consistent PEP
> 8 style throughout a large codebase with many contributors, and I think
> he should have the leeway as maintainer to make the necessary judgment
> calls about precisely which PEP 8 recommendations are reported precisely
> how by the tool, given that:
>
> - we aren't talking about real variance from the spirit or
> recommendations of PEP 8 (though you wouldn't know it from some of the
> rhetoric here about "personal preferences")

Yes we are. My name is on PEP 8 as one of the three co-authors, and my
request to downgrade anything in the "pep8" tools that is not
*explicitly* disallowed by the PEP to be a warning at most has been
ignored. If you read the GitHub issue, you can see I *want* to be able
to recommend this tool universally. But at the moment, I cannot, as
its name is a lie: it enforces rules I don't personally agree with,
yet claims to be based on a PEP I helped write.

There is a way to get changes made to the common guidelines in PEP 8:
you bring your case to python-dev and argue for the adoption of those
rules in the standard library.

If a tool doesn't claim to be speaking in my name, I don't care what
rules it enforces. If a tool adds extra warnings, I also don't care.
But "pep8" currently claims as errors code that is listed *in PEP 8
itself* as acceptable. I am *not* OK with that.

> - the tool makes it very easy to turn off whichever errors you don't
> prefer to enforce.

This is about the default behaviour, and claiming as errors things
that are explicitly listed in the PEP as OK.

> - PEP 8 changes regularly (as Florent noted, the offending code example
> was only added recently), and there is value in the automated tool
> maintaining some stability for its users.
>
> Personally, I would much rather see pep8.py err on the side of being too
> strict with PEP 8's recommendations than too loose. Again, it's not hard
> to turn off the ones you don't want.

No, this is exactly the *wrong* way to approach it. A tool laying
claim to PEP 8's authority should err on the side of *not* enforcing
rules that are not clearly rules - if more clarity is desired, then
ask for clarity from python-dev.

> If python-dev wants to control the precise behavior of pep8.py, bring it
> into the standard library and adopt maintenance of it. Otherwise, please
> give Florent some grace.

Note that I don't complain about the default behaviour of pylint,
pychecker, or any other linter tools. But by continuing to call the
tool "pep8", Florent is claiming the endorsement of the PEP 8 authors
and the consensus of python-dev for the tool's default behaviour (as
noted above, this makes it personal for me, as I am a co-author of PEP
8).

There is a very, very simple guideline that can be followed here:
anything which is not clearly and unambiguously disallowed in the PEP
itself should never be more than a warning in a tool that is called
"pep8".

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Sat Apr 26 00:46:38 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 25 Apr 2014 18:46:38 -0400
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <F8E67462-9CBD-44D0-B136-FF8ABDB24259@stufft.io>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
 <F8E67462-9CBD-44D0-B136-FF8ABDB24259@stufft.io>
Message-ID: <CADiSq7dgKhJ0xd6a_bq8F_8Wm6pU2LBrpo5p1K9dcihncbSdhQ@mail.gmail.com>

On 25 April 2014 18:26, Donald Stufft <donald at stufft.io> wrote:
>> If python-dev wants to control the precise behavior of pep8.py, bring it
>> into the standard library and adopt maintenance of it. Otherwise, please
>> give Florent some grace.
>>
>> Carl
>
> Carl?s post mirrors my own thoughts and it?s said much better than I could have.
>
> pep8.py doesn?t violate PEP8, it just takes a stricter view of it.

In other words, it adds rules that have not been discussed on
python-dev, and effectively claims that those rules are part of PEP 8
when they're not. Note that my problem is *specifically* with errors,
and especially with errors that are on by default. Even with the name
"pep8", warnings and below are clearly at the discretion of the tool
authors and maintainers, and while "off-by-default" errors strike me
as nonsensical (why not just make them warnings instead?), I also
don't have a problem with adding more of them.

However, keeping E121 (which requires all indentation of continued
lines in expressions to also be a multiple of 4, rather than only
semantically significant indentations) and E701 (which completely
disallows the use of single line compound statements, even through the
PEP text only discourages them) as errors rather than warnings is
apparently more important than my wholehearted endorsement of the tool
as a co-author of PEP 8. So be it - but while that is the case, pep8
is *not* a PEP 8 linter, it remains "PEP 8 plus extra rules python-dev
has never discussed".

Regards,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ethan at stoneleaf.us  Sat Apr 26 01:20:58 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 25 Apr 2014 16:20:58 -0700
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <F8E67462-9CBD-44D0-B136-FF8ABDB24259@stufft.io>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
 <F8E67462-9CBD-44D0-B136-FF8ABDB24259@stufft.io>
Message-ID: <535AEDDA.5000506@stoneleaf.us>

On 04/25/2014 03:26 PM, Donald Stufft wrote:
>
> pep8.py doesn?t violate PEP8, it just takes a stricter view of it.

If pep8 reports errors on things that PEP 8 says are okay, that's a violation.

--
~Ethan~

From guido at python.org  Sat Apr 26 01:46:16 2014
From: guido at python.org (Guido van Rossum)
Date: Fri, 25 Apr 2014 16:46:16 -0700
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <535AEDDA.5000506@stoneleaf.us>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
 <F8E67462-9CBD-44D0-B136-FF8ABDB24259@stufft.io>
 <535AEDDA.5000506@stoneleaf.us>
Message-ID: <CAP7+vJJ9phE15YK4XTWaNZJD1Q5GJ_HGB8jPmzvGqXhTET-iSA@mail.gmail.com>

I think the tool's name is unfortunate. The first time I heard about it I
was having an in-person discussion with a developer who (I thought) said
that "PEP 8" was okay with his code (which I knew couldn't be the case) but
in fact he meant to say that (some configuration of) "pep8" didn't mind it.
This took some time to sort out and it would have avoided if the tool had
had a better name.

That said I think it's a great tool.


On Fri, Apr 25, 2014 at 4:20 PM, Ethan Furman <ethan at stoneleaf.us> wrote:

> On 04/25/2014 03:26 PM, Donald Stufft wrote:
>
>>
>> pep8.py doesn?t violate PEP8, it just takes a stricter view of it.
>>
>
> If pep8 reports errors on things that PEP 8 says are okay, that's a
> violation.
>
> --
> ~Ethan~
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140425/52b8710c/attachment.html>

From florent.xicluna at gmail.com  Sat Apr 26 01:56:52 2014
From: florent.xicluna at gmail.com (Florent)
Date: Sat, 26 Apr 2014 01:56:52 +0200
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <CADiSq7dgKhJ0xd6a_bq8F_8Wm6pU2LBrpo5p1K9dcihncbSdhQ@mail.gmail.com>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
 <F8E67462-9CBD-44D0-B136-FF8ABDB24259@stufft.io>
 <CADiSq7dgKhJ0xd6a_bq8F_8Wm6pU2LBrpo5p1K9dcihncbSdhQ@mail.gmail.com>
Message-ID: <CAPFzasOYJi2Sr+HuCtkQNAoewXUp7f5wfVHXvMOJLySptMnukw@mail.gmail.com>

2014-04-26 0:46 GMT+02:00 Nick Coghlan <ncoghlan at gmail.com>:
> Florent is claiming the endorsement of the PEP 8 authors
> and the consensus of python-dev for the tool's default behaviour
> (as noted above, this makes it personal for me, as I am a
> co-author of PEP 8).

You're a co-author of PEP 8 since less than a year.
I'm the maintainer of the pep8 tool since 2010.

You should probably read the LICENSE file which is shipped with the
pep8 too, and the disclaimer that I've posted previously.  Never I
engage the responsibility of the authors of the PEP 8 document, and I
don't give any guarantee of being a *strict* PEP 8 compliance tool.

However, you should notice that your ticket in the tracker is opened
for 2 months only, and I did not flagged it as being resolved.  As
I've stated in my previous mail, I give priority to bugs over other
requests.  And even if you think it is a critical bug for yourself, it
did not appear like that for the thousands of people which used the
library for the last few years.

But if you read the documentation carefully, you can see that I've
already excluded some checks from the default behavior in the previous
releases, when there was an argument which was backed by the PEP 8
itself:
"In the default configuration, the checks E123, E133, E226, E241 and
E242 are ignored
because they are not rules unanimously accepted, and PEP 8 does not
enforce them."
http://pep8.readthedocs.org/en/latest/intro.html#error-codes
The question stay opened for issue #256 and codes E121 and E701.
As I said before, they are not against PEP 8, they interpret some words.
If you're nitpicking, you can probably reject half the checks of the
pep8 tool with similar allegations.

> I *want* to be able to recommend this tool universally. But at the moment,
> I cannot, as its name is a lie: it enforces rules I don't personally agree with.

Maybe I prefer you don't recommend it in the PEP 8 documentation if it
means I'll be tied to your "personal preferences" and that I'll be
forced to patch it every now and then when you change a line, or a
punctuation in a PEP 8 example.

At the end, I find you're a bit rude when you come to this project
which is not endorsed by the PSF or any Python-Dev related entity and
you say "this project is wrong, because the developer did not obey to
me when I order to remove that feature".
You're more sympathetic and less in a hurry when it comes to some
languishing bug on b.p.o :-)

If you're so impatient, let's fork it and put in in cpython/Tools/
I would not fight against it.

Sincerely,

-- 
Florent

From donald at stufft.io  Sat Apr 26 02:13:35 2014
From: donald at stufft.io (Donald Stufft)
Date: Fri, 25 Apr 2014 20:13:35 -0400
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <CAPFzasOYJi2Sr+HuCtkQNAoewXUp7f5wfVHXvMOJLySptMnukw@mail.gmail.com>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
 <F8E67462-9CBD-44D0-B136-FF8ABDB24259@stufft.io>
 <CADiSq7dgKhJ0xd6a_bq8F_8Wm6pU2LBrpo5p1K9dcihncbSdhQ@mail.gmail.com>
 <CAPFzasOYJi2Sr+HuCtkQNAoewXUp7f5wfVHXvMOJLySptMnukw@mail.gmail.com>
Message-ID: <19D3889A-CA82-465B-93BD-8682B2E38F4B@stufft.io>


On Apr 25, 2014, at 7:56 PM, Florent <florent.xicluna at gmail.com> wrote:

> 2014-04-26 0:46 GMT+02:00 Nick Coghlan <ncoghlan at gmail.com>:
>> Florent is claiming the endorsement of the PEP 8 authors
>> and the consensus of python-dev for the tool's default behaviour
>> (as noted above, this makes it personal for me, as I am a
>> co-author of PEP 8).
> 
> You're a co-author of PEP 8 since less than a year.
> I'm the maintainer of the pep8 tool since 2010.
> 
> You should probably read the LICENSE file which is shipped with the
> pep8 too, and the disclaimer that I've posted previously.  Never I
> engage the responsibility of the authors of the PEP 8 document, and I
> don't give any guarantee of being a *strict* PEP 8 compliance tool.

I agree that I?ve never taken the name to mean that you?re claiming any
sort of endorsement. There are a *vast* number of projects that implement
something that was defined somewhere else and I don?t think any reasonable
person can assume that all of those tools are endorsed by the authors
of what they are implementing.

> 
> However, you should notice that your ticket in the tracker is opened
> for 2 months only, and I did not flagged it as being resolved.  As
> I've stated in my previous mail, I give priority to bugs over other
> requests.  And even if you think it is a critical bug for yourself, it
> did not appear like that for the thousands of people which used the
> library for the last few years.
> 
> But if you read the documentation carefully, you can see that I've
> already excluded some checks from the default behavior in the previous
> releases, when there was an argument which was backed by the PEP 8
> itself:
> "In the default configuration, the checks E123, E133, E226, E241 and
> E242 are ignored
> because they are not rules unanimously accepted, and PEP 8 does not
> enforce them."
> http://pep8.readthedocs.org/en/latest/intro.html#error-codes
> The question stay opened for issue #256 and codes E121 and E701.
> As I said before, they are not against PEP 8, they interpret some words.
> If you're nitpicking, you can probably reject half the checks of the
> pep8 tool with similar allegations.
> 
>> I *want* to be able to recommend this tool universally. But at the moment,
>> I cannot, as its name is a lie: it enforces rules I don't personally agree with.
> 
> Maybe I prefer you don't recommend it in the PEP 8 documentation if it
> means I'll be tied to your "personal preferences" and that I'll be
> forced to patch it every now and then when you change a line, or a
> punctuation in a PEP 8 example.
> 
> At the end, I find you're a bit rude when you come to this project
> which is not endorsed by the PSF or any Python-Dev related entity and
> you say "this project is wrong, because the developer did not obey to
> me when I order to remove that feature".
> You're more sympathetic and less in a hurry when it comes to some
> languishing bug on b.p.o :-)
> 
> If you're so impatient, let's fork it and put in in cpython/Tools/
> I would not fight against it.
> 
> Sincerely,
> 
> -- 
> Florent
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/donald%40stufft.io


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140425/15190127/attachment.sig>

From florent.xicluna at gmail.com  Sat Apr 26 02:22:57 2014
From: florent.xicluna at gmail.com (Florent)
Date: Sat, 26 Apr 2014 02:22:57 +0200
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <CAP7+vJJ9phE15YK4XTWaNZJD1Q5GJ_HGB8jPmzvGqXhTET-iSA@mail.gmail.com>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
 <F8E67462-9CBD-44D0-B136-FF8ABDB24259@stufft.io>
 <535AEDDA.5000506@stoneleaf.us>
 <CAP7+vJJ9phE15YK4XTWaNZJD1Q5GJ_HGB8jPmzvGqXhTET-iSA@mail.gmail.com>
Message-ID: <CAPFzasPevAT6U9Ro4VTREnz8PsoWJgyW=P4H4it2HRA-=No3FQ@mail.gmail.com>

2014-04-26 1:46 GMT+02:00 Guido van Rossum <guido at python.org>:
> I think the tool's name is unfortunate. The first time I heard about it I
> was having an in-person discussion with a developer who (I thought) said
> that "PEP 8" was okay with his code (which I knew couldn't be the case) but
> in fact he meant to say that (some configuration of) "pep8" didn't mind it.
> This took some time to sort out and it would have avoided if the tool had
> had a better name.
>
> That said I think it's a great tool.

Thank you,

Actually I did not choose the name. I just took over the project 4
years ago by accident because I had some pull requests languishing and
the author of the project was no longer interested in it.
I agree that the name is a source of confusion in some cases.
I've mitigated some conflicts with PEP 8 by incitating contributors to
write "flake8" extensions instead of pushing more features into "pep8"
itself, and also I tried to define a default configuration which is
"good enough" to support the PEP 8 recommendations.


-- 
Florent

From donald at stufft.io  Sat Apr 26 02:42:02 2014
From: donald at stufft.io (Donald Stufft)
Date: Fri, 25 Apr 2014 20:42:02 -0400
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <535AEDDA.5000506@stoneleaf.us>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
 <F8E67462-9CBD-44D0-B136-FF8ABDB24259@stufft.io>
 <535AEDDA.5000506@stoneleaf.us>
Message-ID: <EE49991B-172C-4F51-B0E3-993D18A06791@stufft.io>


On Apr 25, 2014, at 7:20 PM, Ethan Furman <ethan at stoneleaf.us> wrote:

> On 04/25/2014 03:26 PM, Donald Stufft wrote:
>> 
>> pep8.py doesn?t violate PEP8, it just takes a stricter view of it.
> 
> If pep8 reports errors on things that PEP 8 says are okay, that's a violation.
> 
> --
> ~Ethan~
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/donald%40stufft.io

Not really, any code that passes the pep8.py check is perfectly valid in the eyes of PEP8,
if a check was implemented to say, require camelCase method names, then that would
be a violation of a check. Being stricter is not a violation, it?s being stricter.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140425/dd7f5f2c/attachment.sig>

From ncoghlan at gmail.com  Sat Apr 26 03:29:35 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 25 Apr 2014 21:29:35 -0400
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <CAPFzasOYJi2Sr+HuCtkQNAoewXUp7f5wfVHXvMOJLySptMnukw@mail.gmail.com>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
 <F8E67462-9CBD-44D0-B136-FF8ABDB24259@stufft.io>
 <CADiSq7dgKhJ0xd6a_bq8F_8Wm6pU2LBrpo5p1K9dcihncbSdhQ@mail.gmail.com>
 <CAPFzasOYJi2Sr+HuCtkQNAoewXUp7f5wfVHXvMOJLySptMnukw@mail.gmail.com>
Message-ID: <CADiSq7dGUORg6zz3MmTkYLJKvqTPseJap4fu+pPLSfuEJK9s6w@mail.gmail.com>

On 25 April 2014 19:56, Florent <florent.xicluna at gmail.com> wrote:
> 2014-04-26 0:46 GMT+02:00 Nick Coghlan <ncoghlan at gmail.com>:
>> Florent is claiming the endorsement of the PEP 8 authors
>> and the consensus of python-dev for the tool's default behaviour
>> (as noted above, this makes it personal for me, as I am a
>> co-author of PEP 8).
>
> You're a co-author of PEP 8 since less than a year.
> I'm the maintainer of the pep8 tool since 2010.
>
> You should probably read the LICENSE file which is shipped with the
> pep8 too, and the disclaimer that I've posted previously.  Never I
> engage the responsibility of the authors of the PEP 8 document, and I
> don't give any guarantee of being a *strict* PEP 8 compliance tool.
>
> However, you should notice that your ticket in the tracker is opened
> for 2 months only, and I did not flagged it as being resolved.  As
> I've stated in my previous mail, I give priority to bugs over other
> requests.  And even if you think it is a critical bug for yourself, it
> did not appear like that for the thousands of people which used the
> library for the last few years.

I apologise for my impatience. As I stated in that issue, I really
*want* to be able to uncritically endorse pep8 for use on new Python
projects, as while I like pylint personally, I think it's too
complicated (and noisy by default) to recommend its use outside large
multi-developer projects where taking the time to set up a custom
config file is likely to pay off.

Hence it's frustrating to me that I can't currently recommend pep8 for
this use case, as the default behaviour includes rules I disagree with
(in particular, the ones that the examples in PEP 8 itself fail). I
believe the combination of the implied endorsement from the name of
the tool and an explicit endorsement from me saying "use this on your
projects" would imply that I agree with *all* the default behaviour,
and that simply isn't the case (E121 in particular is outright wrong,
since it violates the approach of aligning indented lines with an
opening parenthesis).

It seemed simple enough to me to say "yes, that makes sense, we can
make those warnings eventually, but it's not a high priority to do
so". That would put it back on me to craft a pull request to make the
change, and would be an entirely appropriate response when I'm asking
other people to do extra work.

That hasn't been my impression of the response I received to date
though - my impression of the response has been "we don't care about
your feelings". Defending PEP 8 against people who think it's overly
prescriptive is irritating enough, without having to also figure out
whether people are actually complaining about the PEP or about a tool
I didn't help write :(

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Sat Apr 26 04:05:56 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 25 Apr 2014 22:05:56 -0400
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <CADiSq7dGUORg6zz3MmTkYLJKvqTPseJap4fu+pPLSfuEJK9s6w@mail.gmail.com>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
 <F8E67462-9CBD-44D0-B136-FF8ABDB24259@stufft.io>
 <CADiSq7dgKhJ0xd6a_bq8F_8Wm6pU2LBrpo5p1K9dcihncbSdhQ@mail.gmail.com>
 <CAPFzasOYJi2Sr+HuCtkQNAoewXUp7f5wfVHXvMOJLySptMnukw@mail.gmail.com>
 <CADiSq7dGUORg6zz3MmTkYLJKvqTPseJap4fu+pPLSfuEJK9s6w@mail.gmail.com>
Message-ID: <CADiSq7dbXrfN1kwacDiVWTVJrANMYn2R6EJMYGjKPJkCsvRFUQ@mail.gmail.com>

On 25 April 2014 21:29, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 25 April 2014 19:56, Florent <florent.xicluna at gmail.com> wrote:
>> 2014-04-26 0:46 GMT+02:00 Nick Coghlan <ncoghlan at gmail.com>:
>>> Florent is claiming the endorsement of the PEP 8 authors
>>> and the consensus of python-dev for the tool's default behaviour
>>> (as noted above, this makes it personal for me, as I am a
>>> co-author of PEP 8).
>>
>> You're a co-author of PEP 8 since less than a year.
>> I'm the maintainer of the pep8 tool since 2010.
>>
>> You should probably read the LICENSE file which is shipped with the
>> pep8 too, and the disclaimer that I've posted previously.  Never I
>> engage the responsibility of the authors of the PEP 8 document, and I
>> don't give any guarantee of being a *strict* PEP 8 compliance tool.
>>
>> However, you should notice that your ticket in the tracker is opened
>> for 2 months only, and I did not flagged it as being resolved.  As
>> I've stated in my previous mail, I give priority to bugs over other
>> requests.  And even if you think it is a critical bug for yourself, it
>> did not appear like that for the thousands of people which used the
>> library for the last few years.
>
> I apologise for my impatience. As I stated in that issue, I really
> *want* to be able to uncritically endorse pep8 for use on new Python
> projects, as while I like pylint personally, I think it's too
> complicated (and noisy by default) to recommend its use outside large
> multi-developer projects where taking the time to set up a custom
> config file is likely to pay off.

Note that I've now resumed this discussion on the pep8 issue tracker
(including splitting out a separate RFE specifically relating to the
new'ish guideline for handling of single line function definitions).

My apologies to Florent specifically, and the list in general, for
letting frustrations with unrelated issues boil over into impatience
and unconstructive criticism of a fellow open source contributor's
efforts.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ethan at stoneleaf.us  Sat Apr 26 03:53:44 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 25 Apr 2014 18:53:44 -0700
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <EE49991B-172C-4F51-B0E3-993D18A06791@stufft.io>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
 <F8E67462-9CBD-44D0-B136-FF8ABDB24259@stufft.io>
 <535AEDDA.5000506@stoneleaf.us>
 <EE49991B-172C-4F51-B0E3-993D18A06791@stufft.io>
Message-ID: <535B11A8.9050105@stoneleaf.us>

On 04/25/2014 05:42 PM, Donald Stufft wrote:
> On Apr 25, 2014, at 7:20 PM, Ethan Furman wrote:
>> On 04/25/2014 03:26 PM, Donald Stufft wrote:
>>>
>>> pep8.py doesn?t violate PEP8, it just takes a stricter view of it.
>>
>> If pep8 reports errors on things that PEP 8 says are okay, that's a violation.
>
> Being stricter is not a violation, it?s being stricter.

Then it should be called stricterThanPep8.  ;)

I think we're going to have to agree to disagree on this point.  A tool that implements PEP 8 [1], but tells me that 
something PEP 8 allows is an error -- well, that's not PEP 8 then, is it?

At any rate, this part of the thread is pretty off-topic, so this is my last post about it.

--
~Ethan~


[1] Yes, I checked the site for pep8, and saw the escape clause of "some of the style conventions" -- the problem is the 
cognitive dissonance between the name of the tool and the actions of the tool.

From akonsta at icloud.com  Sat Apr 26 05:11:58 2014
From: akonsta at icloud.com (Andrew Konstantaras)
Date: Sat, 26 Apr 2014 03:11:58 +0000 (GMT)
Subject: [Python-Dev] Help with changes in stack from 2.7 to 3.x
Message-ID: <8d805463-00b2-46a3-abae-34dabe51b516@me.com>

I wrote the following code that works in Python 2.7 that takes the variables passed to the function into a dictionary.  The following call:

     strA = 'a'
     intA = 1
     dctA = makeDict(strA, intA)

produces the following dictionary:

      {'strA':'a', 'intA':1}

To access the names passed into the function, I had to find the information by parsing through the stack.  The code that used to work is:

from traceback import extract_stack

def makeDict(*args):
    strAllStack = str(extract_stack())
    intNumLevels = len(extract_stack())
    intLevel = 0
    blnFinished = False
    while not blnFinished:
        strStack = str(extract_stack()[intLevel])
        if strStack.find("makeDict(")>0:
            blnFinished = True
        intLevel += 1
        if intLevel >= intNumLevels:
            blnFinished = True
    strStartText = "= makeDict("
    intLen = len(strStartText)
    intOpenParenLoc = strStack.find(strStartText)
    intCloseParenLoc = strStack.find(")", intOpenParenLoc)
    strArgs = strStack[ intOpenParenLoc+intLen : intCloseParenLoc ].strip()
    lstVarNames = strArgs.split(",")
    lstVarNames = [ s.strip() for s in lstVarNames ]  
    if len(lstVarNames) == len(args):
        tplArgs = map(None, lstVarNames, args)
        newDict = dict(tplArgs)
        return newDict
    else:
        return "Error, argument name-value mismatch in function 'makeDict'. lstVarNames: " + str(lstVarNames) + "\n args: " + str(args), strAllStack

The same code does not work in Python 3.3.4.  I have tried parsing through the stack information and frames and I can't find any reference to the names of the arguments passed to the function.  I have tried inspecting the function and other functions in the standard modules, but I can't seem to find anything that will provide this information.

Can anyone point me in the direction to find this information?  Any help is appreciated.

---Andrew?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140426/7594d8e3/attachment.html>

From stephen at xemacs.org  Sat Apr 26 06:31:57 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 26 Apr 2014 13:31:57 +0900
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
Message-ID: <87vbtwogn6.fsf@uwakimon.sk.tsukuba.ac.jp>

Florent writes:

 > I wrote some words in the documentation, one year ago, to explain what
 > is the purpose of the tool and its limitations. There's no claim of
 > any endorsement implicit or explicit by the PSF, the PSU or any other
 > python developer :-)

Of course there is an implicit claim of endorsement: the name.  Read
the TeX license -- pretty much the only restriction is *you can't call
it "TeX" or any variation including that trademark if it doesn't pass
TRIP*.  When Don Knuth cares that much that names denote conformance
to standard, Nick is in really good company.

I agree 100% with Nick: in a program named "pep8", the examples in PEP
8 should *all* pass in the sense of not being labelled errors.  Of
course if the PEP changes that doesn't mean you should withdraw or
rename the program, or even that you are required to address the issue
within any time span.  But you should consider it a bug.

That said, issuing *warnings* for discouraged-but-sometimes-allowed
practices is a great idea.  A stricter warnings-are-errors *mode* is a
good idea, especially for automated checkers, but it should be
configurable for different policies.

 > http://pep8.readthedocs.org/en/latest/intro.html#disclaimer

Disclaimers are for lawyers.  Other people assume names mean what they
say.



From janzert at janzert.com  Sat Apr 26 06:33:52 2014
From: janzert at janzert.com (Janzert)
Date: Sat, 26 Apr 2014 00:33:52 -0400
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <535AD900.6090606@oddbird.net>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
Message-ID: <ljfcvm$ek7$1@ger.gmane.org>

On 4/25/2014 5:52 PM, Carl Meyer wrote:
> - we aren't talking about real variance from the spirit or
> recommendations of PEP 8
>

So the one example under discussion is:
foo = long_function_name(
   var_one, var_two,
   var_three, var_four)

and comes from http://legacy.python.org/dev/peps/pep-0008/#indentation

Specifically the third example with a heading of "Optional".

 From my reading of the text, plus all the other examples around it, I 
would have assumed the 2 space indent was simply a typo and should have 
indeed been 4 spaces. If this is really meant to show that indents other 
than 4 spaces are allowed in this situation maybe verbiage to that 
effect could be added.

Janzert


From rosuav at gmail.com  Sat Apr 26 06:35:41 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 26 Apr 2014 14:35:41 +1000
Subject: [Python-Dev] Help with changes in stack from 2.7 to 3.x
In-Reply-To: <8d805463-00b2-46a3-abae-34dabe51b516@me.com>
References: <8d805463-00b2-46a3-abae-34dabe51b516@me.com>
Message-ID: <CAPTjJmoJc47dtfEmzbZRHoGcNaMtwrtHJ6DZVeZWQtLZx8fz=g@mail.gmail.com>

On Sat, Apr 26, 2014 at 1:11 PM, Andrew Konstantaras <akonsta at icloud.com> wrote:
> Can anyone point me in the direction to find this information?  Any help is
> appreciated.

I'd recommend python-list rather than python-dev (the latter is for
the development *of* Python, rather than development *with* Python).
But to be quite honest, I don't think you should be doing this at all.
It's extremely odd for this function to even work, and most users will
not expect that renaming a local variable will materially change
anything. Fiddling around with stack backtraces in string form and
using those to find the source code line and fetching parameter names?
Definitely weird, definitely not recommended, and not at all
surprising that it broke between versions.

ChrisA

From donald at stufft.io  Sat Apr 26 06:50:07 2014
From: donald at stufft.io (Donald Stufft)
Date: Sat, 26 Apr 2014 00:50:07 -0400
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <87vbtwogn6.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <87vbtwogn6.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <44A0DB47-BF08-467B-8C52-B91E5C48FB9F@stufft.io>


On Apr 26, 2014, at 12:31 AM, Stephen J. Turnbull <stephen at xemacs.org> wrote:

> Florent writes:
> 
>> I wrote some words in the documentation, one year ago, to explain what
>> is the purpose of the tool and its limitations. There's no claim of
>> any endorsement implicit or explicit by the PSF, the PSU or any other
>> python developer :-)
> 
> Of course there is an implicit claim of endorsement: the name.  Read
> the TeX license -- pretty much the only restriction is *you can't call
> it "TeX" or any variation including that trademark if it doesn't pass
> TRIP*.  When Don Knuth cares that much that names denote conformance
> to standard, Nick is in really good company.

No there isn?t.

For instance there are things on PyPI named after websites, like github,
twitter, Facebook, etc. These things are not written by the companies
behind those websites and are merely written to interface with those
websites. Should we assume that those companies all endorse every
single one of those projects simply because they chose a descriptive
name for their project?

The name indicates what it attempts to do, create a PEP8 linter, it makes
no claim of endorsement by the authors of the PEP.

You can say it?s confusing if it?s named pep8 but doesn?t actually make it
?PEP8?, but claiming it?s endorsement isn?t correct.

> 
> I agree 100% with Nick: in a program named "pep8", the examples in PEP
> 8 should *all* pass in the sense of not being labelled errors.  Of
> course if the PEP changes that doesn't mean you should withdraw or
> rename the program, or even that you are required to address the issue
> within any time span.  But you should consider it a bug.

Or required to address it at all if you don?t wish to.

Since the issue is still open I assume that means that the author
hasn?t decided what the correct remediation is yet.

> 
> That said, issuing *warnings* for discouraged-but-sometimes-allowed
> practices is a great idea.  A stricter warnings-are-errors *mode* is a
> good idea, especially for automated checkers, but it should be
> configurable for different policies.
> 
>> http://pep8.readthedocs.org/en/latest/intro.html#disclaimer
> 
> Disclaimers are for lawyers.  Other people assume names mean what they
> say.
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/donald%40stufft.io


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140426/bd00649a/attachment.sig>

From steve at pearwood.info  Sat Apr 26 07:34:04 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 26 Apr 2014 15:34:04 +1000
Subject: [Python-Dev] ConfigParser mangles keys with special chars
In-Reply-To: <CAFT4OTHjHdEhEzpG=SE_=spNaG7kmjZHwgxBfTKpEnw=iaLTjg@mail.gmail.com>
References: <20140425142206.GD6735@lupin>
 <CAFT4OTHjHdEhEzpG=SE_=spNaG7kmjZHwgxBfTKpEnw=iaLTjg@mail.gmail.com>
Message-ID: <20140426053404.GE4273@ando>

On Fri, Apr 25, 2014 at 12:46:42PM -0400, Fred Drake wrote:
> On Fri, Apr 25, 2014 at 10:22 AM, Florian Bruhin <me at the-compiler.org> wrote:
> > While it seems ConfigParser doesn't do any escaping as all, I'm
> > thinking it should at least raise some exception when such a value is
> > trying to be set.
> >
> > I'd expect writing something and then reading it back via the same
> > configparser to *always* result in the same data, as long as writing
> > worked without error.
> >
> > Thoughts? Should I submit a bug report?
> 
> I believe you should, if only to provide a place to record why no
> change gets made.
> 
> Had ConfigParser been more careful from the beginning, that would have
> been really good.
> 
> At this point, it would be a backward-incompatible change, so it's
> unlikely such a change could be allowed to affect existing code.


It seems to me that the current behaviour is a bug and should be fixed. 
I think the relevant part of the docs is the part following the 
"comment_prefixes" and "inline_comment_prefixes" section:

   Please note that config parsers don?t support escaping of comment 
   prefixes so using inline_comment_prefixes may prevent users from 
   specifying option values with characters used as comment prefixes.
   When in doubt, avoid setting inline_comment_prefixes. In any 
   circumstances, the only way of storing comment prefix characters 
   at the beginning of a line in multiline values is to interpolate 
   the prefix, for example: ...


This is *seriously* underspecified, which is a bug in itself: option 
*values* don't support including inline_comment_prefixes, but do option 
*keys* support them? How about keys beginning with comment_prefixes? The 
specification doesn't say, but nor does it say that it's undefined 
behaviour.

I think that a line beginning with "#spam" is ambiguous, it isn't clear 
if it is intended as a comment "spam" or a key starting with #, so by 
the Zen, configparser should refuse to guess.


-- 
Steven

From stephen at xemacs.org  Sat Apr 26 07:40:28 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 26 Apr 2014 14:40:28 +0900
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <535AD900.6090606@oddbird.net>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
Message-ID: <87tx9godgz.fsf@uwakimon.sk.tsukuba.ac.jp>

Carl Meyer writes:

 > I'd like to thank Florent for taking the time to maintain an
 > extremely-useful tool that makes it feasible to keep to a
 > consistent PEP 8 style throughout a large codebase with many
 > contributors,

I would too.

N.B. Nick's complaints are a sort of left-handed compliment, too.

 > and I think he should have the leeway as maintainer to make the
 > necessary judgment calls about precisely which PEP 8
 > recommendations are reported precisely how by the tool, given that:

Of course.  But whether an example in the PEP passes or not is *not* a
judgment call at all, IMO.  Do you think that it is?

 > - we aren't talking about real variance from the spirit or
 > recommendations of PEP 8 (though you wouldn't know it from some of
 > the rhetoric here about "personal preferences")

That parenthetical remark was unnecessary, and at least in my case, I
understood very well that there is no "real" variance.  I disagree
with the implied requirements, however.  A program *intended to test
standard conformance* MUST (in the sense of RFC 2119) aspire to a
higher standard than "in the spirit", don't you think?  Anyway, that's
what I think.

 > - the tool makes it very easy to turn off whichever errors you
 > don't prefer to enforce.

This is precisely backward.  *I* have no errors I prefer to enforce or
not to enforce.  I want "PEP 8 conformance", I don't want to think
or make judgments about individual recommendations in PEP 8.  I want
to leave that up to the PEP 8 maintainers.

I suspect that is a common use case.

 > - PEP 8 changes regularly (as Florent noted, the offending code
 > example was only added recently), and there is value in the
 > automated tool maintaining some stability for its users.

Achieving "some" stability would not be difficult: version the error
sets of the tool, and provide a switch to invoke specific versions.
Each version could be dated, as well, to allow such stability without
knowing the precise content of the version of PEP 8 or pep8.py.

I personally would not use such a switch, however.  I would either
change my code to conform to current PEP 8, or not. ;-)

 > Personally, I would much rather see pep8.py err on the side of
 > being too strict with PEP 8's recommendations than too
 > loose. Again, it's not hard to turn off the ones you don't want.

Again, for many users of the tool, that is precisely *not* why they
want to use the tool.  They want to delegate the decision of which
rules to enforce to the PEP 8 authors.

Since *you* have preferences, I repeat your words back to you: it's
not hard to turn on the ones you want.  (Or it needn't be.)

 > If python-dev wants to control the precise behavior of pep8.py,
 > bring it into the standard library and adopt maintenance of it.
 > Otherwise, please give Florent some grace.

Note that what Nick is complaining about is not that pep8.py varies
from PEP 8 -- that's inevitable -- but rather that the variation is
not *acknowledged* as a bug.

So python-dev (but you really mean Nick) doesn't want to control the
precise behavior of pep8.py, as code.  What Nick wants is for code
*bearing the name* to conform to the PEP it was named for.  For users
who want PEP 8 conformance merely because it is the standard, that's
exactly right.  For those of you who want to pick and choose which
rules to follow and how strictly, the name doesn't much matter, does
it?  I think that in naming we should consider the use cases of those
who depend on the name to be meaningful as higher priority than those
who don't, but make their own judgments anyway.

Regards,


From fred at fdrake.net  Sat Apr 26 07:59:27 2014
From: fred at fdrake.net (Fred Drake)
Date: Sat, 26 Apr 2014 01:59:27 -0400
Subject: [Python-Dev] ConfigParser mangles keys with special chars
In-Reply-To: <20140426053404.GE4273@ando>
References: <20140425142206.GD6735@lupin>
 <CAFT4OTHjHdEhEzpG=SE_=spNaG7kmjZHwgxBfTKpEnw=iaLTjg@mail.gmail.com>
 <20140426053404.GE4273@ando>
Message-ID: <CAFT4OTHNYF2NOK7_ubocmM4nR3C=EGOHz8DwJKrNkXMc5nnayQ@mail.gmail.com>

On Sat, Apr 26, 2014 at 1:34 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> I think that a line beginning with "#spam" is ambiguous, it isn't clear
> if it is intended as a comment "spam" or a key starting with #, so by
> the Zen, configparser should refuse to guess.

Seriously?

Perhaps the second paragraph here could be strengthened a little:
https://docs.python.org/3/library/configparser.html#supported-ini-file-structure

But that seems clear to me.  Lines starting with the comment prefix
are comments.


  -Fred

-- 
Fred L. Drake, Jr.    <fred at fdrake.net>
"A storm broke loose in my mind."  --Albert Einstein

From steve at pearwood.info  Sat Apr 26 08:46:39 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 26 Apr 2014 16:46:39 +1000
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <19D3889A-CA82-465B-93BD-8682B2E38F4B@stufft.io>
References: <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
 <F8E67462-9CBD-44D0-B136-FF8ABDB24259@stufft.io>
 <CADiSq7dgKhJ0xd6a_bq8F_8Wm6pU2LBrpo5p1K9dcihncbSdhQ@mail.gmail.com>
 <CAPFzasOYJi2Sr+HuCtkQNAoewXUp7f5wfVHXvMOJLySptMnukw@mail.gmail.com>
 <19D3889A-CA82-465B-93BD-8682B2E38F4B@stufft.io>
Message-ID: <20140426064639.GF4273@ando>

On Fri, Apr 25, 2014 at 08:13:35PM -0400, Donald Stufft wrote:

> I agree that I?ve never taken the name to mean that you?re claiming any
> sort of endorsement. There are a *vast* number of projects that implement
> something that was defined somewhere else and I don?t think any reasonable
> person can assume that all of those tools are endorsed by the authors
> of what they are implementing.

I think that is wrong. "Endorsed" is the wrong word -- what's 
important is that if package PEP8 "checks for PEP 8 compliance", then if 
package PEP8 checks for foo, then people assume that foo-checking is 
specified by PEP 8 -- whether it does or not.

If something claims to be "JSON", say, then most 
people would expect that by default it would implement the JSON standard 
rather than some tweaked version of almost JSON. If their JSON library 
fails to match the standard, a significant number of people will assume 
that the library's non-standard version *is* the standard.

It may even become the de-facto standard, regardless of what the 
standard actually says, and so introduce ambiguity where there was none. 
E.g. consider browser's treatment of ISO-8859-1 as Windows-1252.

People and corporations often take this sort of stuff really seriously, 
and I'm not surprised that Nick feels as strongly as he does. I would 
too. Try releasing a package using the name "iPhone" without Apple's 
approval:

https://developer.apple.com/softwarelicensing/index.php

In this case, I too would have assumed that a package calling itself 
"PEP8" checks for compliance with PEP 8, and that any differences 
between PEP 8 and the package is a bug in the package. Providing the 
ability to turn on or off checks, or make them more or less strict, is 
okay, but I would expect that by default it should check for PEP 8 
compliance, exactly PEP 8 compliance, and nothing but PEP 8 compliance.



-- 
Steven

From steve at pearwood.info  Sat Apr 26 09:07:40 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 26 Apr 2014 17:07:40 +1000
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <EE49991B-172C-4F51-B0E3-993D18A06791@stufft.io>
References: <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
 <F8E67462-9CBD-44D0-B136-FF8ABDB24259@stufft.io>
 <535AEDDA.5000506@stoneleaf.us>
 <EE49991B-172C-4F51-B0E3-993D18A06791@stufft.io>
Message-ID: <20140426070740.GG4273@ando>

On Fri, Apr 25, 2014 at 08:42:02PM -0400, Donald Stufft wrote:
> 
> On Apr 25, 2014, at 7:20 PM, Ethan Furman <ethan at stoneleaf.us> wrote:
> 
> > On 04/25/2014 03:26 PM, Donald Stufft wrote:
> >> 
> >> pep8.py doesn?t violate PEP8, it just takes a stricter view of it.
> > 
> > If pep8 reports errors on things that PEP 8 says are okay, that's a violation.

> Not really, any code that passes the pep8.py check is perfectly valid in the eyes of PEP8,
> if a check was implemented to say, require camelCase method names, then that would
> be a violation of a check. Being stricter is not a violation, it?s being stricter.

Suppose that some package claiming to check for PEP 8 compliance decided 
to flag variable names like "foo" and "bar" as errors. Perhaps it was 
inspired by Tim Peters:

    Indeed, when I design my killer language, the identifiers 
    "foo" and "bar" will be reserved words, never used, and not 
    even mentioned in the reference manual. Any program using 
    one will simply dump core without comment. Multitudes will 
    rejoice.  --  Tim Peters, 29 Apr 1998

Or perhaps it decides that one should never, ever have more than a 
single dot in a row, a treat that as an error:

    obj.method()  # okay
    obj.method().attribute  # Law Of Demeter violation

This hypothetical checker is stricter than PEP 8, in that it allows 
nothing that PEP 8 forbids. But it also forbids things which are allowed 
by PEP 8, which is a false negative error: things which should be 
allowed (according to PEP 8) are forbidden by the checker.

(The other kind of error is a false positive, where things which are 
forbidden by PEP 8 are wrongly allowed by the checker.)

I stress that they're only errors if the checker is running in PEP 8 
compatibility mode, where it is supposed to check for PEP 8 compliance. 
That's the crux of Nick's, and Ethan's, argument: with a name like 
"PEP8", it is reasonable to expect that by default the checker operates 
in PEP 8 compatibility mode. I agree with them.



-- 
Steven

From stephen at xemacs.org  Sat Apr 26 09:42:18 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 26 Apr 2014 16:42:18 +0900
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <44A0DB47-BF08-467B-8C52-B91E5C48FB9F@stufft.io>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <87vbtwogn6.fsf@uwakimon.sk.tsukuba.ac.jp>
 <44A0DB47-BF08-467B-8C52-B91E5C48FB9F@stufft.io>
Message-ID: <87r44ko7tx.fsf@uwakimon.sk.tsukuba.ac.jp>

Donald Stufft writes:

 > For instance there are things on PyPI named after websites, like
 > github, twitter, Facebook, etc. These things are not written by the
 > companies behind those websites and are merely written to interface
 > with those websites. Should we assume that those companies all
 > endorse every single one of those projects simply because they
 > chose a descriptive name for their project?

OK, "endorse" is an inappropriate word to describe the problem
inherent in naming a script "pep8.py" when the maintainers of PEP 8
say it doesn't implement the PEP.

(Aside: In fact, in the cases you cite, you *may* assume implicit
approval (or that the company in question has not yet discovered and
complained about the usage), if not active endorsement.  That is the
implication of a "trade name", and I suspect many of the names you
mention are registered trademarks as well.)

However, be that as it may, your examples are irrelevant to the
"spirit" of Nick's issue: PEP 8 is a standard, not a corporate
reputation.  Even if there is no implicit "endorsement", the
maintainer of the program is using the name of the document, while he
and others are arguing that it's up to him to decide about conforming
to it.

That is unfair to those users who do not care about the *content* of
the standard, but only about the interoperability implied by a
certification of conformance to the standard.  And if you think
"interoperability" is a strong word to apply to a style guide like PEP
8, maybe so -- but RFC 2119 is also just a style guide, and I would
hope that you would not argue that misuse of terms defined in that
document has no interoperability consequences.

 > > I agree 100% with Nick: in a program named "pep8", the examples
 > > in PEP 8 should *all* pass in the sense of not being labelled
 > > errors.  Of course if the PEP changes that doesn't mean you
 > > should withdraw or rename the program, or even that you are
 > > required to address the issue within any time span.  But you
 > > should consider it a bug.
 > 
 > Or required to address it at all if you don?t wish to.

In the case of a explicit decision to WONTFIX just because you don't
want to, it becomes a deliberately sustained false-y, and renaming is
the appropriate action.

There is a huge namespace of strings starting with "py".  Although
some of the more attractive ones are already taken (pylint, pychecker,
pyflakes), I'm sure there's plenty of room for creative naming left if
100% conformance to the standard is a non-goal for the maintainer of
the script.

 > Since the issue is still open I assume that means that the author
 > hasn?t decided what the correct remediation is yet.

In the case of a program intended to check conformance to a standard,
it's not a matter for the script maintainer's decision.  It's obvious
what the specification of the correct remediation is.  (The author may
of course choose the implementation.  Or, having tried implementation
he may decide to acknowledge the bug but WONTFIX it as too difficult,
but too difficult is obviously not true here.)

It's always possible that the document is buggy (as suggested by
Janzert the 2- or 3- space indentation may have been a typo), but
again, that's for the authors of the standard, not of the script, to
decide.


From steve at pearwood.info  Sat Apr 26 13:23:11 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 26 Apr 2014 21:23:11 +1000
Subject: [Python-Dev] ConfigParser mangles keys with special chars
In-Reply-To: <CAFT4OTHNYF2NOK7_ubocmM4nR3C=EGOHz8DwJKrNkXMc5nnayQ@mail.gmail.com>
References: <20140425142206.GD6735@lupin>
 <CAFT4OTHjHdEhEzpG=SE_=spNaG7kmjZHwgxBfTKpEnw=iaLTjg@mail.gmail.com>
 <20140426053404.GE4273@ando>
 <CAFT4OTHNYF2NOK7_ubocmM4nR3C=EGOHz8DwJKrNkXMc5nnayQ@mail.gmail.com>
Message-ID: <20140426112311.GH4273@ando>

On Sat, Apr 26, 2014 at 01:59:27AM -0400, Fred Drake wrote:
> On Sat, Apr 26, 2014 at 1:34 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> > I think that a line beginning with "#spam" is ambiguous, it isn't clear
> > if it is intended as a comment "spam" or a key starting with #, so by
> > the Zen, configparser should refuse to guess.
> 
> Seriously?

Perhaps I could have worded my post more clearly, but yes.

In context, we were talking about a situation where the user creates a 
key value pair. Copying from the OP's initial post:

    >>> cp = configparser.ConfigParser()
    >>> cp.read_dict({'DEFAULT': {';foo': 'bar'}})
    >>> cp.write(sys.stdout)
    [DEFAULT]
    ;foo = bar


The intent of the creator of the ConfigParser was for a key ";foo" with 
value "bar", and that is the way it is treated while still in memory. 
It's only when reading it back from a file does it become treated as a 
comment. There's no way to tell whether the line ";foo = bar" *in a 
file* came from a key ";foo" with value "bar", or from a genuine comment 
that merely happens to look like key=value.

The obvious way to avoid the ambiguity is to disallow keys beginning 
with a comment prefix. Then ";foo = bar" cannot represent the key 
";foo", because such a key is illegal.


> Perhaps the second paragraph here could be strengthened a little:
> https://docs.python.org/3/library/configparser.html#supported-ini-file-structure
> 
> But that seems clear to me.  Lines starting with the comment prefix
> are comments.

But the entry in question wasn't a line, it was a key=value pair in a 
dict. Here's that line again:

cp.read_dict({'DEFAULT': {';foo': 'bar'}})

or it could have been:

cp['DEFAULT'][';foo'] = 'bar'

Either way, if there's no way to escape comment characters, I think it 
is a bug that you can insert illegal keys into the cp object in the 
first place.


-- 
Steven

From akonsta at me.com  Sat Apr 26 07:09:29 2014
From: akonsta at me.com (Andrew Konstantaras)
Date: Fri, 25 Apr 2014 22:09:29 -0700
Subject: [Python-Dev] Apologies for last post
Message-ID: <41660FBB-E45B-4C6A-8A82-E0E531443057@me.com>

I accidentally posted a question to here.  I am returning to Python after a couple years away and I am trying to brush up my skills and make the switch to Python 3.x. 

---Andrew



From fred at fdrake.net  Sat Apr 26 17:32:18 2014
From: fred at fdrake.net (Fred Drake)
Date: Sat, 26 Apr 2014 11:32:18 -0400
Subject: [Python-Dev] ConfigParser mangles keys with special chars
In-Reply-To: <20140426112311.GH4273@ando>
References: <20140425142206.GD6735@lupin>
 <CAFT4OTHjHdEhEzpG=SE_=spNaG7kmjZHwgxBfTKpEnw=iaLTjg@mail.gmail.com>
 <20140426053404.GE4273@ando>
 <CAFT4OTHNYF2NOK7_ubocmM4nR3C=EGOHz8DwJKrNkXMc5nnayQ@mail.gmail.com>
 <20140426112311.GH4273@ando>
Message-ID: <CAFT4OTGLH=upL4nteqdZ5NPvuEaqB+UjJBe0WauP00b46hPYeA@mail.gmail.com>

On Sat, Apr 26, 2014 at 7:23 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> But the entry in question wasn't a line, it was a key=value pair in a
> dict. Here's that line again:
>
> cp.read_dict({'DEFAULT': {';foo': 'bar'}})
>
> or it could have been:
>
> cp['DEFAULT'][';foo'] = 'bar'
>
> Either way, if there's no way to escape comment characters, I think it
> is a bug that you can insert illegal keys into the cp object in the
> first place.

Fair enough.

I'm not trying to argue that it isn't a bug, but that risk of breaking
currently-working applications with data they consider acceptable is
high.  Many use configparser for input only, and make no use of the
serialization feature.  Those applications can be broken by adding a
constraint on the data that's allowed, regardless of what we think of
their keys.

Chaning this in an application for which it's safe (easier to know at
the application level) is easy enough:

    import configparser
    import re

    class ProtectionistParser(configparser.RawConfigParser):

        _rx = re.compile(r"[a-z]([-a-z]*[a-z])?$")  # or whatever
makes sense for your app

        def optionxform(self, opt):
            if self._rx.match(opt) is None:
                raise ValueError("don't like this: %r" % opt)
            return opt

Maybe the "strict" mode is considered new enough that this isn't as
significant a risk, and it could be allowed when that's enabled; I'm
sure ?ukasz will have a carefully considered opinion (and I'll defer
to him in the end).


  -Fred

-- 
Fred L. Drake, Jr.    <fred at fdrake.net>
"A storm broke loose in my mind."  --Albert Einstein

From chris.barker at noaa.gov  Sun Apr 27 03:19:57 2014
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Sat, 26 Apr 2014 18:19:57 -0700
Subject: [Python-Dev] ConfigParser mangles keys with special chars
In-Reply-To: <ljeags$l3t$1@ger.gmane.org>
References: <20140425142206.GD6735@lupin>
 <CAFT4OTHjHdEhEzpG=SE_=spNaG7kmjZHwgxBfTKpEnw=iaLTjg@mail.gmail.com>
 <535A9E64.1010107@stoneleaf.us> <ljeags$l3t$1@ger.gmane.org>
Message-ID: <9072605469979100843@unknownmsgid>

On Apr 25, 2014, at 11:47 AM, Terry Reedy <tjreedy at udel.edu> wrote:
>So I wonder whether the thought-to-be-buggy behavior is
>actually buggy with respect to *all* the supported styles or just some of them.

I can't how being able to write a file that isn't read back with the
same meaning could be anything but a bug (or missing feature).
Regardless of what ( lack of ) standard it is or isn't conforming to.

Also, if it is often used just to read ini files, then those uses
wouldn't be effected by more strictness on writing ...

But the way, last I checked elmentree had a similar problem -- it
would quite happily write invalid XML that it wouldn't read back in...

-Chris


>
> --
> Terry Jan Reedy
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/chris.barker%40noaa.gov

From barry at python.org  Sun Apr 27 18:40:39 2014
From: barry at python.org (Barry Warsaw)
Date: Sun, 27 Apr 2014 12:40:39 -0400
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <ljfcvm$ek7$1@ger.gmane.org>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
 <ljfcvm$ek7$1@ger.gmane.org>
Message-ID: <20140427124039.27125a72@limelight.wooz.org>

On Apr 26, 2014, at 12:33 AM, Janzert wrote:

>So the one example under discussion is:
>foo = long_function_name(
>   var_one, var_two,
>   var_three, var_four)
>
>and comes from http://legacy.python.org/dev/peps/pep-0008/#indentation
>
>Specifically the third example with a heading of "Optional".
>
>From my reading of the text, plus all the other examples around it, I would
>have assumed the 2 space indent was simply a typo and should have indeed been
>4 spaces. If this is really meant to show that indents other than 4 spaces
>are allowed in this situation maybe verbiage to that effect could be added.

No, I think it's a typo and should be 4 spaces.  I'll fix that.

-Barry

From Nikolaus at rath.org  Sun Apr 27 21:10:46 2014
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Sun, 27 Apr 2014 12:10:46 -0700
Subject: [Python-Dev] Commit-ready patches needing review
Message-ID: <877g6a1tc9.fsf@vostro.rath.org>

Hello,

While my last appeal resulted in quite some commits (thanks!), I still
have some more commit-ready patches waiting for review.  It'd be great
if some people could find time to take a look:

* http://bugs.python.org/issue1738 (filecmp.dircmp does exact match
  only)

* http://bugs.python.org/issue20951 (SSLSocket.send() returns 0 for
  non-blocking socket)

  In this case someone just needs to decide if we want to (a) document
  the current behavior, (b) deprecate the current behavior or (c) change
  the current behavior. I have attached patches for (a) and (b), and if
  (c) is the desired route I'll be happy to create a patch on short
  notice.

* http://bugs.python.org/issue20177 (Derby #8: Convert 28 sites to
  Argument Clinic across 2 files)

* http://bugs.python.org/issue19414 (iter(ordered_dict) yields keys
  not in dict in some circumstances)

* http://bugs.python.org/issue15955 (gzip, bz2, lzma: add option to
  limit output size)

  Nadeem has kindly reviewed the first iteration of this patch, but the
  second iteration has been waiting for attention for quite some time
  now.

* http://bugs.python.org/issue20578 (BufferedIOBase.readinto1 is
  missing)

* http://bugs.python.org/issue21057 (TextIOWrapper does not support
  reading bytearrays or memoryviews)


Best,
Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             ?Time flies like an arrow, fruit flies like a Banana.?

From solipsis at pitrou.net  Sun Apr 27 21:20:10 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 27 Apr 2014 21:20:10 +0200
Subject: [Python-Dev] Commit-ready patches needing review
References: <877g6a1tc9.fsf@vostro.rath.org>
Message-ID: <20140427212010.39d69c67@fsol>

On Sun, 27 Apr 2014 12:10:46 -0700
Nikolaus Rath <Nikolaus at rath.org> wrote:
> 
> * http://bugs.python.org/issue20951 (SSLSocket.send() returns 0 for
>   non-blocking socket)
> 
>   In this case someone just needs to decide if we want to (a) document
>   the current behavior, (b) deprecate the current behavior or (c) change
>   the current behavior. I have attached patches for (a) and (b), and if
>   (c) is the desired route I'll be happy to create a patch on short
>   notice.

In this case I'd be inclined to follow Ben Darnell's advice and change
the current behaviour (i.e., let the exception bubble up rather than
catch it). This is what your initial patch does. However, it would need
a documentation addition to explain the change (and perhaps a test,
though that doesn't seem terribly necessary here).

Regards

Antoine.



From chris.barker at noaa.gov  Sun Apr 27 21:34:33 2014
From: chris.barker at noaa.gov (Chris Barker)
Date: Sun, 27 Apr 2014 12:34:33 -0700
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <20140427124039.27125a72@limelight.wooz.org>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
 <ljfcvm$ek7$1@ger.gmane.org> <20140427124039.27125a72@limelight.wooz.org>
Message-ID: <CALGmxEKDoc5_Xqc1h4vhQdFOVRFsXFnHajbShCv_z2dR2EngLQ@mail.gmail.com>

On Sun, Apr 27, 2014 at 9:40 AM, Barry Warsaw <barry at python.org> wrote:

> On Apr 26, 2014, at 12:33 AM, Janzert wrote:
>
> >So the one example under discussion is:
> >foo = long_function_name(
> >   var_one, var_two,
> >   var_three, var_four)
> >
> >and comes from http://legacy.python.org/dev/peps/pep-0008/#indentation
>

wow! just looked at that part of the PEP again, and that is a LOT of
options. Is it impossible to come to any consensus on this? And as it
happens, my favorite is not in there, though as far as I can tell not
forbidden:

foo = long_function_name(var_one,
                                          var_two,
                                          var_three,
                                          var_four)

That is, I find that if the argument list is too long for one line, then
splitting it out to only one argument per line is much more readable to me.
This becomes more important with default parameters:

foo = long_function_name(var_one,
                                          var_two=a_value,
                                          var_three=some_other_value,
                                          var_four=(a, tuple, of, values)
                                          )
as with more information in each argument, it's a lot more clear where one
starts and the other begins. And it provides a nice place for comments:

foo = long_function_name(var_one,
                                          var_two=a_value, # because
default doesn't frobnicate in this case
                                          var_three=some_other_value,
                                          var_four=(a, tuple, of, values)
                                          )


Anyway -- is there a point in trying to standardize this a bit more in
PEP8, or has that battle long since been fought and conceded ?

-Chris

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140427/f610e7f4/attachment.html>

From guido at python.org  Sun Apr 27 21:35:38 2014
From: guido at python.org (Guido van Rossum)
Date: Sun, 27 Apr 2014 12:35:38 -0700
Subject: [Python-Dev] Commit-ready patches needing review
In-Reply-To: <20140427212010.39d69c67@fsol>
References: <877g6a1tc9.fsf@vostro.rath.org>
	<20140427212010.39d69c67@fsol>
Message-ID: <CAP7+vJKddOih3k0K-Punr+K9i466L_FSna9xHkx_HK+JJsXsrQ@mail.gmail.com>

Agreed.

On Sunday, April 27, 2014, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Sun, 27 Apr 2014 12:10:46 -0700
> Nikolaus Rath <Nikolaus at rath.org <javascript:;>> wrote:
> >
> > * http://bugs.python.org/issue20951 (SSLSocket.send() returns 0 for
> >   non-blocking socket)
> >
> >   In this case someone just needs to decide if we want to (a) document
> >   the current behavior, (b) deprecate the current behavior or (c) change
> >   the current behavior. I have attached patches for (a) and (b), and if
> >   (c) is the desired route I'll be happy to create a patch on short
> >   notice.
>
> In this case I'd be inclined to follow Ben Darnell's advice and change
> the current behaviour (i.e., let the exception bubble up rather than
> catch it). This is what your initial patch does. However, it would need
> a documentation addition to explain the change (and perhaps a test,
> though that doesn't seem terribly necessary here).
>
> Regards
>
> Antoine.
>
>

-- 
--Guido van Rossum (on iPad)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140427/29b1ba9d/attachment.html>

From tjreedy at udel.edu  Sun Apr 27 22:28:20 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 27 Apr 2014 16:28:20 -0400
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <CALGmxEKDoc5_Xqc1h4vhQdFOVRFsXFnHajbShCv_z2dR2EngLQ@mail.gmail.com>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
 <ljfcvm$ek7$1@ger.gmane.org> <20140427124039.27125a72@limelight.wooz.org>
 <CALGmxEKDoc5_Xqc1h4vhQdFOVRFsXFnHajbShCv_z2dR2EngLQ@mail.gmail.com>
Message-ID: <ljjpa7$2g0$1@ger.gmane.org>

On 4/27/2014 3:34 PM, Chris Barker wrote:
> On Sun, Apr 27, 2014 at 9:40 AM, Barry Warsaw <barry at python.org
> <mailto:barry at python.org>> wrote:
>
>     On Apr 26, 2014, at 12:33 AM, Janzert wrote:
>
>      >So the one example under discussion is:
>      >foo = long_function_name(
>      >   var_one, var_two,
>      >   var_three, var_four)
>      >
>      >and comes from http://legacy.python.org/dev/peps/pep-0008/#indentation
>
>
> wow! just looked at that part of the PEP again, and that is a LOT of
> options. Is it impossible to come to any consensus on this? And as it
> happens, my favorite is not in there, though as far as I can tell not
> forbidden:

One arg per line is definitely permitted either when lining up with the 
first arg on the first line or with hanging indents.

> foo = long_function_name(var_one,
>                                            var_two,
>                                            var_three,
>                                            var_four)

In a fixed-pitch font, you have too many spaces for lines after the 
first. One advantage of hanging indent is that all arg lines have the 
same indent and thus the args line up regardless of the font. Hanging 
indents also give more space for default or named args and comments.

I would agree with having at least one example done with one arg per line.

-- 
Terry Jan Reedy


From florent.xicluna at gmail.com  Sun Apr 27 22:32:44 2014
From: florent.xicluna at gmail.com (Florent)
Date: Sun, 27 Apr 2014 22:32:44 +0200
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <CALGmxEKDoc5_Xqc1h4vhQdFOVRFsXFnHajbShCv_z2dR2EngLQ@mail.gmail.com>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
 <ljfcvm$ek7$1@ger.gmane.org> <20140427124039.27125a72@limelight.wooz.org>
 <CALGmxEKDoc5_Xqc1h4vhQdFOVRFsXFnHajbShCv_z2dR2EngLQ@mail.gmail.com>
Message-ID: <CAPFzasPbBjyvdiU68RcAmyf0Pu_2wq4_VieD1xrDDE7ceUaXwg@mail.gmail.com>

2014-04-27 21:34 GMT+02:00 Chris Barker <chris.barker at noaa.gov>:
>
> wow! just looked at that part of the PEP again, and that is a LOT of
> options. Is it impossible to come to any consensus on this? And as it
> happens, my favorite is not in there, though as far as I can tell not
> forbidden:
>
> foo = long_function_name(var_one,
>                                            var_two,
>                                            var_three,
>                                            var_four)
>
>
> Anyway -- is there a point in trying to standardize this a bit more in PEP8,
> or has that battle long since been fought and conceded ?
>

FWIW, the tool pep8 has introduced continuation line indentation
checks in 2012, based on the PEP 8 recommendations which were active
at this date, and doing few extrapolations on cases which were not
directly covered by the document (pep8.py v1.2, issue #64).

Then we had a lot of iterations with the users of the library, and
we've added more flexibility to cover the various styles which were in
the spirit of PEP 8, and which give some added value for code
readability.

The end result of this work can be seen in the test suite :
 * more than 300 lines of rejected indentations:
    https://github.com/jcrocholl/pep8/blob/master/testsuite/E12.py
* and more than 600 lines of acceptable cases:
    https://github.com/jcrocholl/pep8/blob/master/testsuite/E12not.py

I don't think that the PEP 8 document should go in so much details, though.
However these examples might help to have pragmatic discussions about
which rules we want to recommend.

Do not hesitate to give feedback here, or on the issue tracker.
There's also a code-quality mailing-list which covers a little more
than pep8.py:
https://mail.python.org/mailman/listinfo/code-quality

-- 
Florent

From barry at python.org  Sun Apr 27 22:40:39 2014
From: barry at python.org (Barry Warsaw)
Date: Sun, 27 Apr 2014 16:40:39 -0400
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <CALGmxEKDoc5_Xqc1h4vhQdFOVRFsXFnHajbShCv_z2dR2EngLQ@mail.gmail.com>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
 <ljfcvm$ek7$1@ger.gmane.org>
 <20140427124039.27125a72@limelight.wooz.org>
 <CALGmxEKDoc5_Xqc1h4vhQdFOVRFsXFnHajbShCv_z2dR2EngLQ@mail.gmail.com>
Message-ID: <20140427164039.75e0f041@limelight.wooz.org>

On Apr 27, 2014, at 12:34 PM, Chris Barker wrote:

>wow! just looked at that part of the PEP again, and that is a LOT of
>options. Is it impossible to come to any consensus on this? And as it
>happens, my favorite is not in there, though as far as I can tell not
>forbidden:
>
>foo = long_function_name(var_one,
>                                          var_two,
>                                          var_three,
>                                          var_four)

Wow, do you really indent those 42 columns?

This actually is forbidden because you're not using "vertical alignment" when
there is an argument on the first line.  You would have to line up `var_two`
right under `var_one` to be compliant.

>That is, I find that if the argument list is too long for one line, then
>splitting it out to only one argument per line is much more readable to me.

Sure.  The PEP outlines ways to do that.

>This becomes more important with default parameters:
>
>foo = long_function_name(var_one,
>                                          var_two=a_value,
>                                          var_three=some_other_value,
>                                          var_four=(a, tuple, of, values)
>                                          )
>as with more information in each argument, it's a lot more clear where one
>starts and the other begins. And it provides a nice place for comments:
>
>foo = long_function_name(var_one,
>                                          var_two=a_value, # because
>default doesn't frobnicate in this case
>                                          var_three=some_other_value,
>                                          var_four=(a, tuple, of, values)
>                                          )
>
>
>Anyway -- is there a point in trying to standardize this a bit more in
>PEP8, or has that battle long since been fought and conceded ?

The only thing I see wrong (from a PEP 8 standpoint) with your example, is
that your arguments are indented too far to the right.  But e.g. this would be
fine:

foo = long_function_name(var_one,
                         var_two=a_value,
                         var_three=some_other_value,
                         var_four=(a, tuple, of, values)
                         )

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140427/cdc027fa/attachment.sig>

From janzert at janzert.com  Sun Apr 27 23:55:11 2014
From: janzert at janzert.com (Janzert)
Date: Sun, 27 Apr 2014 17:55:11 -0400
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <20140427124039.27125a72@limelight.wooz.org>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
 <ljfcvm$ek7$1@ger.gmane.org> <20140427124039.27125a72@limelight.wooz.org>
Message-ID: <ljjuc0$3ot$1@ger.gmane.org>

On 4/27/2014 12:40 PM, Barry Warsaw wrote:
> On Apr 26, 2014, at 12:33 AM, Janzert wrote:
>
>> So the one example under discussion is:
>> foo = long_function_name(
>>    var_one, var_two,
>>    var_three, var_four)
>>
>> and comes from http://legacy.python.org/dev/peps/pep-0008/#indentation
>>
>> Specifically the third example with a heading of "Optional".
>>
>>From my reading of the text, plus all the other examples around it, I would
>> have assumed the 2 space indent was simply a typo and should have indeed been
>> 4 spaces. If this is really meant to show that indents other than 4 spaces
>> are allowed in this situation maybe verbiage to that effect could be added.
>
> No, I think it's a typo and should be 4 spaces.  I'll fix that.
>
> -Barry
>

Given the commits to pep 8 after this, I take it you decided the example 
really is correct.

I'm glad the clarifying sentence got added and comment in the example 
changed, that should make it clear that the 2 space indent isn't a typo.

Janzert


From Nikolaus at rath.org  Mon Apr 28 00:42:10 2014
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Sun, 27 Apr 2014 15:42:10 -0700
Subject: [Python-Dev] Commit-ready patches needing review
In-Reply-To: <20140427212010.39d69c67@fsol> (Antoine Pitrou's message of "Sun, 
 27 Apr 2014 21:20:10 +0200")
References: <877g6a1tc9.fsf@vostro.rath.org> <20140427212010.39d69c67@fsol>
Message-ID: <871twi1jjx.fsf@vostro.rath.org>

Antoine Pitrou <solipsis at pitrou.net> writes:
> On Sun, 27 Apr 2014 12:10:46 -0700
> Nikolaus Rath <Nikolaus at rath.org> wrote:
>> 
>> * http://bugs.python.org/issue20951 (SSLSocket.send() returns 0 for
>>   non-blocking socket)
>> 
>>   In this case someone just needs to decide if we want to (a) document
>>   the current behavior, (b) deprecate the current behavior or (c) change
>>   the current behavior. I have attached patches for (a) and (b), and if
>>   (c) is the desired route I'll be happy to create a patch on short
>>   notice.
>
> In this case I'd be inclined to follow Ben Darnell's advice and change
> the current behaviour (i.e., let the exception bubble up rather than
> catch it). This is what your initial patch does. However, it would need
> a documentation addition to explain the change (and perhaps a test,
> though that doesn't seem terribly necessary here).

Sounds good to me. I just attached an updated patch to the issue.


Thanks for looking at this!


Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             ?Time flies like an arrow, fruit flies like a Banana.?

From steve at pearwood.info  Mon Apr 28 03:34:57 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 28 Apr 2014 11:34:57 +1000
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <ljjpa7$2g0$1@ger.gmane.org>
References: <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
 <ljfcvm$ek7$1@ger.gmane.org> <20140427124039.27125a72@limelight.wooz.org>
 <CALGmxEKDoc5_Xqc1h4vhQdFOVRFsXFnHajbShCv_z2dR2EngLQ@mail.gmail.com>
 <ljjpa7$2g0$1@ger.gmane.org>
Message-ID: <20140428013457.GJ4273@ando>

On Sun, Apr 27, 2014 at 04:28:20PM -0400, Terry Reedy wrote:
> On 4/27/2014 3:34 PM, Chris Barker wrote:
> >On Sun, Apr 27, 2014 at 9:40 AM, Barry Warsaw <barry at python.org
> ><mailto:barry at python.org>> wrote:
> >
> >    On Apr 26, 2014, at 12:33 AM, Janzert wrote:
> >
> >     >So the one example under discussion is:
> >     >foo = long_function_name(
> >     >   var_one, var_two,
> >     >   var_three, var_four)
> >     >
> >     >and comes from 
> >     http://legacy.python.org/dev/peps/pep-0008/#indentation
> >
> >
> >wow! just looked at that part of the PEP again, and that is a LOT of
> >options. Is it impossible to come to any consensus on this? And as it
> >happens, my favorite is not in there, though as far as I can tell not
> >forbidden:
> 
> One arg per line is definitely permitted either when lining up with the 
> first arg on the first line or with hanging indents.
[...]
> I would agree with having at least one example done with one arg per line.

Is it really necessary? I think that one-arg-per-line is an obvious 
variation of the existing example.

If the only example given was one-arg-per-line, then the reader might 
wrongly assume that *only* one arg was allowed. But since the example 
shows more than one arg per line (two in the above example), I expect 
that people will read it as "some arbitrary number" rather than "exactly 
two". So I don't think there's any need to show yet another example, 
especially since it's just a variation on the existing examples.



-- 
Steven

From pmiscml at gmail.com  Mon Apr 28 04:59:45 2014
From: pmiscml at gmail.com (Paul Sokolovsky)
Date: Mon, 28 Apr 2014 05:59:45 +0300
Subject: [Python-Dev] Clarification on MRO when inheriting from builtin type.
Message-ID: <20140428055945.311dea37@x34f>

Hello,

I used
http://python-history.blogspot.com/2010/06/method-resolution-order.html
and https://www.python.org/download/releases/2.3/mro/ as the reference,
but it doesn't explain MRO in the following example (python3.4):

~~~~
class User:
    def __str__(self):
        return "User.__str__"

    def append(self, a):
        print("User", a)

class C(list, User):
    pass

t = C([1, 2, 3])
print(t)
t.append(10)
print(t)
print(t[-1])
~~~~

The output is:

=====
User.__str__
User.__str__
10
=====

From the output, "User" class as expected does not override
list.append(), but does override list.__str__(). Is this behavior
documented somewhere (complete arrangement)? What's the rationale
behind it?

I need this info to implement MRO in an alternative Python
implementation (MicroPython).


Thanks,
 Paul                          mailto:pmiscml at gmail.com

From rosuav at gmail.com  Mon Apr 28 05:13:53 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 28 Apr 2014 13:13:53 +1000
Subject: [Python-Dev] Clarification on MRO when inheriting from builtin
	type.
In-Reply-To: <20140428055945.311dea37@x34f>
References: <20140428055945.311dea37@x34f>
Message-ID: <CAPTjJmqzdFFcNcjA_jMAES7juRnoxy0JFMC-mL=YW5ftMJCkJA@mail.gmail.com>

On Mon, Apr 28, 2014 at 12:59 PM, Paul Sokolovsky <pmiscml at gmail.com> wrote:
> From the output, "User" class as expected does not override
> list.append(), but does override list.__str__(). Is this behavior
> documented somewhere (complete arrangement)? What's the rationale
> behind it?

In Python 3.4 (don't know about other versions), list.__str__ doesn't
exist; when you call str([1,2,3]) it calls object.__str__. The MRO for
C is (C, list, User, object) so anything from list (eg append) takes
precedence over anything from User, but anything list doesn't have
will fall through to User before catching object.

ChrisA

From rosuav at gmail.com  Mon Apr 28 05:33:58 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 28 Apr 2014 13:33:58 +1000
Subject: [Python-Dev] Clarification on MRO when inheriting from builtin
	type.
In-Reply-To: <20140428062553.694c2e80@x34f>
References: <20140428055945.311dea37@x34f>
 <CAPTjJmqzdFFcNcjA_jMAES7juRnoxy0JFMC-mL=YW5ftMJCkJA@mail.gmail.com>
 <20140428062553.694c2e80@x34f>
Message-ID: <CAPTjJmoz01oLX7OWeLnwv1YbAqGK3RFH4ECQ_szQ5Tds68iztg@mail.gmail.com>

On Mon, Apr 28, 2014 at 1:25 PM, Paul Sokolovsky <pmiscml at gmail.com> wrote:
>
> Thanks for quick response! I see that list.__repr__ exists, and test
> using it works "as expected". Hopefully, such stuff can be treated as
> implementation-specific details...

The language defines method lookups and the MRO and such, and then the
class chooses what to define. If your classes aren't specifically
designed with inheritance in mind, multiple inheritance will always
get messy (no matter what language you're in!); even single
inheritance can get messy very quickly if the classes aren't designed
to coexist (there's no safe way to refactor a superclass method
without knowing what subclasses might be overriding).

ChrisA

From pmiscml at gmail.com  Mon Apr 28 05:25:53 2014
From: pmiscml at gmail.com (Paul Sokolovsky)
Date: Mon, 28 Apr 2014 06:25:53 +0300
Subject: [Python-Dev] Clarification on MRO when inheriting from builtin
 type.
In-Reply-To: <CAPTjJmqzdFFcNcjA_jMAES7juRnoxy0JFMC-mL=YW5ftMJCkJA@mail.gmail.com>
References: <20140428055945.311dea37@x34f>
 <CAPTjJmqzdFFcNcjA_jMAES7juRnoxy0JFMC-mL=YW5ftMJCkJA@mail.gmail.com>
Message-ID: <20140428062553.694c2e80@x34f>

Hello,

On Mon, 28 Apr 2014 13:13:53 +1000
Chris Angelico <rosuav at gmail.com> wrote:

> On Mon, Apr 28, 2014 at 12:59 PM, Paul Sokolovsky <pmiscml at gmail.com>
> wrote:
> > From the output, "User" class as expected does not override
> > list.append(), but does override list.__str__(). Is this behavior
> > documented somewhere (complete arrangement)? What's the rationale
> > behind it?
> 
> In Python 3.4 (don't know about other versions), list.__str__ doesn't
> exist; when you call str([1,2,3]) it calls object.__str__. The MRO for
> C is (C, list, User, object) so anything from list (eg append) takes
> precedence over anything from User, but anything list doesn't have
> will fall through to User before catching object.

Thanks for quick response! I see that list.__repr__ exists, and test
using it works "as expected". Hopefully, such stuff can be treated as
implementation-specific details...

> 
> ChrisA

-- 
Best regards,
 Paul                          mailto:pmiscml at gmail.com

From victor.stinner at gmail.com  Mon Apr 28 11:08:25 2014
From: victor.stinner at gmail.com (Victor Stinner)
Date: Mon, 28 Apr 2014 11:08:25 +0200
Subject: [Python-Dev] Add calloc-like memory allocators to Python
Message-ID: <CAMpsgwbjMPAUoZZ69sMastXtvAQ--CaiNTrEj+gaEr1sCBJyKg@mail.gmail.com>

Hi,

We made progress on the following issue and the latest patch
(calloc-5.patch) is ready for a review:
http://bugs.python.org/issue21233

This issue should help numpy to reuse Python memory allocators to use
the new tracemalloc module of Python 3.4. The issue is only for Python
3.5.

It was also discussed that numpy might require a memory allocator with
a specified alignment for SIMD instructions. This topic is discussed
in a different issue:
http://bugs.python.org/issue18835


Summary of calloc-5.patch:

- add the following functions:

  * void* PyMem_RawCalloc(size_t nelem, size_t elsize)
  * void* PyMem_Calloc(size_t nelem, size_t elsize)
  * void* PyObject_Calloc(size_t nelem, size_t elsize)
  * PyObject* _PyObject_GC_Calloc(size_t basicsize)

- add "void* calloc(void *ctx, size_t nelem, size_t elsize)" field to
the PyMemAllocator structure
- optimize bytes(n) and bytearray(n) to allocate objects using
calloc() instead of malloc()
- update tracemalloc to trace also calloc()
- document new functions and add unit tests for the calloc "hook" (in _testcapi)

Victor

From python-dev at mgmiller.net  Mon Apr 28 09:52:48 2014
From: python-dev at mgmiller.net (Mike Miller)
Date: Mon, 28 Apr 2014 19:52:48 +1200
Subject: [Python-Dev] Python  2.7.7. on Windows
In-Reply-To: <535DF27E.3000108@wetafx.co.nz>
References: <535DF27E.3000108@wetafx.co.nz>
Message-ID: <535E08D0.2020500@mgmiller.net>

Greetings,

I've just woken up and noticed Python 2.7.7 is on track to be released, and in a 
rather unique event contains a few security enhancements
in addition to the usual fixes:

http://legacy.python.org/dev/peps/pep-0466/

I thought this might be a good time to make a final plea to fix a
long-standing security issue in the installer on Windows.  By default it
installs Python to the root folder, thereby bypassing filesystem permissions:

http://bugs.python.org/issue1284316

The main rationale given (for not using the standard %ProgramFiles%) has been 
that the full path to python is too long to type, and ease of use is more 
important than the security benefits given by following Windows conventions.

However, adding python to the PATH variable is an alternative solution that
would result in even fewer keystrokes needing to be typed at a console, while
maintaining system security.

As this is an installer setting and not a code change, I gather that the
opportunities for code breakage should be fewer.  Please consider updating
this setting to a more secure and friendly default, for 2.7.7 and 3.5+ as well.

-Mike


From jimjjewett at gmail.com  Mon Apr 28 17:32:13 2014
From: jimjjewett at gmail.com (Jim J. Jewett)
Date: Mon, 28 Apr 2014 11:32:13 -0400
Subject: [Python-Dev] API and process questions (sparked by Claudiu Popa on
	16104
Message-ID: <CA+OGgf47B1pRNY1fL6Pi_iqn3_VanpgvmVt7Fs6GLR3aJrtKPg@mail.gmail.com>

(1)  Should fixes to a docstring go in with a patch, even if they
aren't related to the changing functionality?

Bytestring compilation has several orthogonal parameters.  Most -- but
not all -- are documented in the docstring.  (Specifically, there is
no explanation of the rx parameter which acts as a filter, and no
mention that symbolic links to directories are ignored.)

It is best if a commit changes one small thing at a time.
On the other hand, Nick recently posted that the minimal overhead of a
patch commit is about half an hour.

Is that overhead enough to override the one-issue-per-patch guideline?

(2)  The patch adds new functionality to use multiple processes in
parallel.  The normal parameter values are integers indicating how
many processes to use.  The parameter also needs two special values --
one to indicate "use os.cpu_count", and the other to indicate "don't
use multiprocessing at all".

(A)  Is there a Best Practices for this situation, with two odd cases?

(B)  Claudiu originally copied the API from a similar APU for
regrtest.  What is the barrier for "do it sensibly" vs "stick with
precedent elsewhere"?  (Apparently regrtest treats any negative number
as a request for the cpu_count calculation;  I suspect that "-5" is
more likely to be an escaping error for "5" than it is to be a real
request for auto-calculation that just happened to choose -5 instead
of -1.)

(C)  How important is it to keep the API consistent between a
top-level CLI command and the internal implementation?  At the the
moment, the request for cpu_count is handled in the CLI wrapper, and
not available to interactive callers.  On the other hand, interactive
callers could just call cpu_count themselves...

(D)  How important is it to maintain consistency with other uses of
the same tool -- multiprocessing has its own was of requesting
auto-calculation.  (So someone used to multiprocessing might assume
that "None" meant to auto-calculate, as opposed to "don't use
multiprocessing at all.")

-jJ

From bcannon at gmail.com  Mon Apr 28 17:42:29 2014
From: bcannon at gmail.com (Brett Cannon)
Date: Mon, 28 Apr 2014 15:42:29 +0000
Subject: [Python-Dev] API and process questions (sparked by Claudiu Popa on
	16104
References: <CA+OGgf47B1pRNY1fL6Pi_iqn3_VanpgvmVt7Fs6GLR3aJrtKPg@mail.gmail.com>
Message-ID: <CAP1=2W40T4ft0Om7kXVXcVEQ3C7KEsP2pWkW7Vd5zqf9+sqc2g@mail.gmail.com>

On Mon Apr 28 2014 at 11:32:58 AM, Jim J. Jewett <jimjjewett at gmail.com>
wrote:

> (1)  Should fixes to a docstring go in with a patch, even if they
> aren't related to the changing functionality?
>

It should probably be its own commit.


>
> Bytestring compilation has several orthogonal parameters.  Most -- but
> not all -- are documented in the docstring.  (Specifically, there is
> no explanation of the rx parameter which acts as a filter, and no
> mention that symbolic links to directories are ignored.)
>
> It is best if a commit changes one small thing at a time.
> On the other hand, Nick recently posted that the minimal overhead of a
> patch commit is about half an hour.
>
> Is that overhead enough to override the one-issue-per-patch guideline?
>

That's typical for backported patches, not if something is only going into
default. That being said, if the change is truly small and you "sneak" it
in I don't think anyone is going to make you revert the patch to do it as
more atomic commits.


>
> (2)  The patch adds new functionality to use multiple processes in
> parallel.  The normal parameter values are integers indicating how
> many processes to use.  The parameter also needs two special values --
> one to indicate "use os.cpu_count", and the other to indicate "don't
> use multiprocessing at all".
>
> (A)  Is there a Best Practices for this situation, with two odd cases?
>

No. In this situation I would consider 0 or -1 for "use os.cpu_count' and
None for "don't use multi-processing".


>
> (B)  Claudiu originally copied the API from a similar APU for
> regrtest.  What is the barrier for "do it sensibly" vs "stick with
> precedent elsewhere"?  (Apparently regrtest treats any negative number
> as a request for the cpu_count calculation;  I suspect that "-5" is
> more likely to be an escaping error for "5" than it is to be a real
> request for auto-calculation that just happened to choose -5 instead
> of -1.)
>

Regrtest doesn't count as an API to stick to. =)


>
> (C)  How important is it to keep the API consistent between a
> top-level CLI command and the internal implementation?  At the the
> moment, the request for cpu_count is handled in the CLI wrapper, and
> not available to interactive callers.  On the other hand, interactive
> callers could just call cpu_count themselves...
>

It doesn't hurt, but we typically don't promote CLIs for stdlib modules so
there isn't much precedent.


>
> (D)  How important is it to maintain consistency with other uses of
> the same tool -- multiprocessing has its own was of requesting
> auto-calculation.  (So someone used to multiprocessing might assume
> that "None" meant to auto-calculate, as opposed to "don't use
> multiprocessing at all.")
>

That's a judgment call. If there is already no consistency go with the one
that makes the most sense. If consistency exists across the stdlib then
stay consistent.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140428/8e99fa5b/attachment.html>

From pcmanticore at gmail.com  Mon Apr 28 17:45:37 2014
From: pcmanticore at gmail.com (Claudiu Popa)
Date: Mon, 28 Apr 2014 18:45:37 +0300
Subject: [Python-Dev] API and process questions (sparked by Claudiu Popa
 on 16104
In-Reply-To: <CA+OGgf47B1pRNY1fL6Pi_iqn3_VanpgvmVt7Fs6GLR3aJrtKPg@mail.gmail.com>
References: <CA+OGgf47B1pRNY1fL6Pi_iqn3_VanpgvmVt7Fs6GLR3aJrtKPg@mail.gmail.com>
Message-ID: <CAMy=CLqGp9NvD=OGbfFqk0UcRnmUnPs+PgZRt5WdwUsmn8cW0Q@mail.gmail.com>

On Mon, Apr 28, 2014 at 6:32 PM, Jim J. Jewett <jimjjewett at gmail.com> wrote:
> (1)  Should fixes to a docstring go in with a patch, even if they
> aren't related to the changing functionality?
>
> Bytestring compilation has several orthogonal parameters.  Most -- but
> not all -- are documented in the docstring.  (Specifically, there is
> no explanation of the rx parameter which acts as a filter, and no
> mention that symbolic links to directories are ignored.)
>
> It is best if a commit changes one small thing at a time.
> On the other hand, Nick recently posted that the minimal overhead of a
> patch commit is about half an hour.
>
> Is that overhead enough to override the one-issue-per-patch guideline?
>
> (2)  The patch adds new functionality to use multiple processes in
> parallel.  The normal parameter values are integers indicating how
> many processes to use.  The parameter also needs two special values --
> one to indicate "use os.cpu_count", and the other to indicate "don't
> use multiprocessing at all".
>
> (A)  Is there a Best Practices for this situation, with two odd cases?
>
> (B)  Claudiu originally copied the API from a similar APU for
> regrtest.  What is the barrier for "do it sensibly" vs "stick with
> precedent elsewhere"?  (Apparently regrtest treats any negative number
> as a request for the cpu_count calculation;  I suspect that "-5" is
> more likely to be an escaping error for "5" than it is to be a real
> request for auto-calculation that just happened to choose -5 instead
> of -1.)
>
> (C)  How important is it to keep the API consistent between a
> top-level CLI command and the internal implementation?  At the the
> moment, the request for cpu_count is handled in the CLI wrapper, and
> not available to interactive callers.  On the other hand, interactive
> callers could just call cpu_count themselves...
>
> (D)  How important is it to maintain consistency with other uses of
> the same tool -- multiprocessing has its own was of requesting
> auto-calculation.  (So someone used to multiprocessing might assume
> that "None" meant to auto-calculate, as opposed to "don't use
> multiprocessing at all.")
>


Thanks for writing this, Jim. Hopefully, the answers to these
questions will solve our disagreements
regarding the API.

From sturla.molden at gmail.com  Mon Apr 28 18:41:44 2014
From: sturla.molden at gmail.com (Sturla Molden)
Date: Mon, 28 Apr 2014 16:41:44 +0000 (UTC)
Subject: [Python-Dev] Python 2.7.7. on Windows
References: <535E08D0.2020500@mgmiller.net>
Message-ID: <771463774420395726.352685sturla.molden-gmail.com@news.gmane.org>

Mike Miller <python-dev at mgmiller.net> wrote:

> The main rationale given (for not using the standard %ProgramFiles%) has been 
> that the full path to python is too long to type, and ease of use is more 
> important than the security benefits given by following Windows conventions.

"C:\Program Files\Python27" contains an empty space in the path. If you
want to randomly break build tools for C extensions, then go ahead and
change it. 

Sturla


From cf.natali at gmail.com  Mon Apr 28 18:56:06 2014
From: cf.natali at gmail.com (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=)
Date: Mon, 28 Apr 2014 17:56:06 +0100
Subject: [Python-Dev] API and process questions (sparked by Claudiu Popa
 on 16104
In-Reply-To: <CAP1=2W40T4ft0Om7kXVXcVEQ3C7KEsP2pWkW7Vd5zqf9+sqc2g@mail.gmail.com>
References: <CA+OGgf47B1pRNY1fL6Pi_iqn3_VanpgvmVt7Fs6GLR3aJrtKPg@mail.gmail.com>
 <CAP1=2W40T4ft0Om7kXVXcVEQ3C7KEsP2pWkW7Vd5zqf9+sqc2g@mail.gmail.com>
Message-ID: <CAH_1eM0VQmd3bj53vJhXS7hNDuWjha99V7mSx1KidBS-ByJj5Q@mail.gmail.com>

>> (2)  The patch adds new functionality to use multiple processes in
>> parallel.  The normal parameter values are integers indicating how
>> many processes to use.  The parameter also needs two special values --
>> one to indicate "use os.cpu_count", and the other to indicate "don't
>> use multiprocessing at all".
>>
>> (A)  Is there a Best Practices for this situation, with two odd cases?
>
>
> No. In this situation I would consider 0 or -1 for "use os.cpu_count' and
> None for "don't use multi-processing".

Why would the user care if multiprocessing is used behind the scene?
It would be strange for processes=1 to fail if multiprocessing is not
available.

If you set a default value of 1, then compileall() will work
regardless of whether multiprocessing is available.

In short:
processes <= 0: use os.cpu_count()
processes == 1 (default): just use normal sequential compiling
processes > 1: use multiprocessing

There's no reason to introduce None. Or am I missing something?

From Steve.Dower at microsoft.com  Mon Apr 28 19:12:13 2014
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Mon, 28 Apr 2014 17:12:13 +0000
Subject: [Python-Dev] Python  2.7.7. on Windows
In-Reply-To: <535E08D0.2020500@mgmiller.net>
References: <535DF27E.3000108@wetafx.co.nz> <535E08D0.2020500@mgmiller.net>
Message-ID: <789792ad6e7c4d63ac74e33bcc3f50c5@BLUPR03MB389.namprd03.prod.outlook.com>

Mike Miller wrote:
> I thought this might be a good time to make a final plea to fix a
> long-standing security issue in the installer on Windows. By default it installs
> Python to the root folder, thereby bypassing filesystem permissions:
>
>http://bugs.python.org/issue1284316

This would be an incredibly painful change that would surprise and hurt a lot of
people.

Mark Hammond's last comment on the linked issue sums up my view as well:
"My take is still that Python is a tool, not an app. People writing an app they
wish to distribute using Python should include Python in their package (ie, not
rely on an installed version) and these apps should conform with the
guidelines."

Yes, it is possible for a non-admin user to install arbitrary packages into a
place where an admin user may inadvertently run it, thereby providing escalation
of privilege. On the other hand, that applies to a lot of development tools,
especially since most users on Windows these days are actually limited
administrators - ANYTHING they install could run when they elevate a certain
process.

I'm all for encouraging apps to redistribute Python (as most do on Windows). In
this case, they can easily apply the correct security settings, typically by
installing pythonxy.dll with their application and ignoring any global settings
(PYTHONPATH, registry keys, etc.).

On the other hand, Python from python.org is a tool that should only be
installed by those who are prepared to manage it. On Windows it is easy enough
to have a second, secured copy for your admin scripts and an unsecured copy for
non-admin tasks.

> The main rationale given (for not using the standard %ProgramFiles%) has been
> that the full path to python is too long to type, and ease of use is more
> important than the security benefits given by following Windows conventions.
>
> However, adding python to the PATH variable is an alternative solution that
> would result in even fewer keystrokes needing to be typed at a console, while
> maintaining system security.
>
> As this is an installer setting and not a code change, I gather that the
> opportunities for code breakage should be fewer. Please consider updating this
> setting to a more secure and friendly default, for 2.7.7 and 3.5+ as well.

I'm not sure what change you are proposing here... doesn't the installer already
have an option to add to PATH? I'm sure I keep disabling it. I don't want
"python.exe" on my PATH because I have 10+ versions installed at any one time. I
have my own set of aliases because even the py.exe launcher can't handle my
setup :) The info we've gotten from our users (Python Tools for Visual Studio)
shows that most have two or more versions of Python installed, so having one or
both of them in PATH or PATHEXT just adds ambiguity.

The other rationales are .pyc generation and package installation. Unlike PATH,
these are not red herrings, but they do only apply to developers and not end
users (where the developer has done his/her job properly).

Speaking as the one who is likely to take over from Martin as the Windows build
manager (the only people arguing so far are my employer's lawyers, but I'm
winning that argument), I don't see any change that can be made to 2.7.7 apart
from adding a link in the installer to a documentation page on how to secure the
Python installation. At this point, 2.7.6->2.7.7 should be an incredibly safe
upgrade, and there's no way to safely change the default installation location
or the ACLs on the install directory.

As for 3.5, I have some ideas that I will raise with python-dev once I've had a
chance to think through all the issues (think proper per-user installs, redist
modules, etc.). More secure installations would certainly be on the table, but
practicality very easily beats purity here IMHO.

Cheers,
Steve

From cf.natali at gmail.com  Mon Apr 28 19:20:25 2014
From: cf.natali at gmail.com (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=)
Date: Mon, 28 Apr 2014 18:20:25 +0100
Subject: [Python-Dev] API and process questions (sparked by Claudiu Popa
 on 16104
In-Reply-To: <CAH_1eM0VQmd3bj53vJhXS7hNDuWjha99V7mSx1KidBS-ByJj5Q@mail.gmail.com>
References: <CA+OGgf47B1pRNY1fL6Pi_iqn3_VanpgvmVt7Fs6GLR3aJrtKPg@mail.gmail.com>
 <CAP1=2W40T4ft0Om7kXVXcVEQ3C7KEsP2pWkW7Vd5zqf9+sqc2g@mail.gmail.com>
 <CAH_1eM0VQmd3bj53vJhXS7hNDuWjha99V7mSx1KidBS-ByJj5Q@mail.gmail.com>
Message-ID: <CAH_1eM0d-Woch027K9qi4DzBVqSEd7JDok-cC7rbr=SpsTaiXQ@mail.gmail.com>

And incidentally, I think that the argument *processes* should be
renamed to *workers*, or *jobs* (like in make), and any mention of
multiprocessing in the documentation should be removed (if any):
multiprocessing is an implementation detail.
When I type:
make -jN

I don't really care that make is using fork() ;-)

From jimjjewett at gmail.com  Mon Apr 28 19:29:05 2014
From: jimjjewett at gmail.com (Jim J. Jewett)
Date: Mon, 28 Apr 2014 13:29:05 -0400
Subject: [Python-Dev] API and process questions (sparked by Claudiu Popa
 on 16104
In-Reply-To: <CAH_1eM0VQmd3bj53vJhXS7hNDuWjha99V7mSx1KidBS-ByJj5Q@mail.gmail.com>
References: <CA+OGgf47B1pRNY1fL6Pi_iqn3_VanpgvmVt7Fs6GLR3aJrtKPg@mail.gmail.com>
 <CAP1=2W40T4ft0Om7kXVXcVEQ3C7KEsP2pWkW7Vd5zqf9+sqc2g@mail.gmail.com>
 <CAH_1eM0VQmd3bj53vJhXS7hNDuWjha99V7mSx1KidBS-ByJj5Q@mail.gmail.com>
Message-ID: <CA+OGgf4weBk1NsXBSqi1g7tDE-=7rfkzd5bZn1MXiHWSggECEQ@mail.gmail.com>

On Mon, Apr 28, 2014 at 12:56 PM, Charles-Fran?ois Natali
<cf.natali at gmail.com> wrote:
> Why would the user care if multiprocessing is used behind the scene?

Err ... that was another set of questions that I forgot to ask.

(A)  Why bother raising an error if multiprocessing is unavailable?
After all, there is a perfectly fine fallback...

On other hand, errors should not pass silently.  If a user has
explicitly asked for multiprocessing, there should be some notice that
it didn't happen.  And builds are presumably something that a
developer will monitor to respond to the Exception.

(A1)  What sort of Error?  I'm inclined to raise the original
ImportError, but the patch prefers a ValueError.

(B)  Assuming the exception, I suppose your question adds a 3rd
special case of "Whatever the system suggests, and I don't care
whether or not it involves multiprocessing."

> It would be strange for processes=1 to fail if multiprocessing is not
> available.

As Claudiu pointed out, processes=1 should really mean 1 worker
process, which is still different from "do everything in the main
process".  I'm not sure that level of control is really worth the
complexity, but I'm not certain it isn't.

> processes <= 0: use os.cpu_count()

I could understand doing that for 0 or -1; what is the purpose of
doing it for both, let alone for -4?

Are we at the point where the parameter should just take positive
integers or one of a set of specified string values?

-jJ

From cf.natali at gmail.com  Mon Apr 28 19:54:23 2014
From: cf.natali at gmail.com (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=)
Date: Mon, 28 Apr 2014 18:54:23 +0100
Subject: [Python-Dev] file objects guarantees
Message-ID: <CAH_1eM38V5+kCddjETCxAF4JR=CoXPKqGzdgKjqDGjSnC+Yr6Q@mail.gmail.com>

Hi,

What's meant exactly by a "file object"?

Let me be more specific: for example, pickle.dump() accepts a "file object".

Looking at the code, it doesn't check the return value of its write() method.

So it assumes that write() should always write the whole data (not
partial write).

Same thing for read, it assumes there won't be short reads.

A sample use case would be passing a socket.makefile() to pickle: it
works, because makefile() returns a BufferedReader/Writer which takes
care of short read/write.

But the documentation just says "file object". And if you have a look
the file object definition in the glossary:
https://docs.python.org/3.5/glossary.html#term-file-object

"""
There are actually three categories of file objects: raw binary files,
buffered binary files and text files. Their interfaces are defined in
the io module. The canonical way to create a file object is by using
the open() function.
"""

So someone passing e.g. a raw binary file - which doesn't handle short
reads/writes - would run into trouble.

It's the same thing for e.g. GzipFile, and probably many others.

Would it make sense to add a note somewhere?

From pmiscml at gmail.com  Mon Apr 28 19:45:48 2014
From: pmiscml at gmail.com (Paul Sokolovsky)
Date: Mon, 28 Apr 2014 20:45:48 +0300
Subject: [Python-Dev] Multiple inheritance from builtin (C) types [still]
 supported in Python3?
Message-ID: <20140428204548.5eaa28e4@x34f>

Hello,

It's more or less known fact (ref: google) that you can't inherit from
multiple generic builtin (as in "coded in C") types:

class C(dict, list): pass

TypeError: multiple bases have instance lay-out conflict

However, more detailed googling led me to this page of a book:
http://books.google.com.ua/books?id=JnR9hQA3SncC&pg=PA104&lpg=PA104 ,
which suggest that there might be adhoc support for some native types
to serve as multiple bases together, but it doesn't provide an example.
The book is apparently focused on Python2.

I tried to look at typeobject.c:best_base(), which throws the quoted
error above, and the only special rules I could quickly decipher from
it is that it's possible to multi-inherit from a class and its
subclass. Intuitively, it can be understood - it makes no sense to do
that on the same inheritance level, but can happen with recursive
inheritance, and then there's no need for conflict - both classes can be
"merged" into subclass.

Indeed, following works:

import _collections
class Foo(_collections.defaultdict, dict): pass

(opposite order gives MRO conflict)


So, is that it, or disjoint native types are supported as bases
somehow? Also, would someone know if a class-subclass case happens for
example in stdlib?

As the previous question,
https://mail.python.org/pipermail/python-dev/2014-April/134342.html
this one is to help set adequate requirements for implementing multiple
inheritance in MicroPython.


Thanks,
 Paul                          mailto:pmiscml at gmail.com

From cf.natali at gmail.com  Mon Apr 28 19:59:25 2014
From: cf.natali at gmail.com (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=)
Date: Mon, 28 Apr 2014 18:59:25 +0100
Subject: [Python-Dev] API and process questions (sparked by Claudiu Popa
 on 16104
In-Reply-To: <CA+OGgf4weBk1NsXBSqi1g7tDE-=7rfkzd5bZn1MXiHWSggECEQ@mail.gmail.com>
References: <CA+OGgf47B1pRNY1fL6Pi_iqn3_VanpgvmVt7Fs6GLR3aJrtKPg@mail.gmail.com>
 <CAP1=2W40T4ft0Om7kXVXcVEQ3C7KEsP2pWkW7Vd5zqf9+sqc2g@mail.gmail.com>
 <CAH_1eM0VQmd3bj53vJhXS7hNDuWjha99V7mSx1KidBS-ByJj5Q@mail.gmail.com>
 <CA+OGgf4weBk1NsXBSqi1g7tDE-=7rfkzd5bZn1MXiHWSggECEQ@mail.gmail.com>
Message-ID: <CAH_1eM3svnXHT_sspoyzquJnjM+tfRc+PtqaRMz+5T2SBqCB1Q@mail.gmail.com>

2014-04-28 18:29 GMT+01:00 Jim J. Jewett <jimjjewett at gmail.com>:
> On Mon, Apr 28, 2014 at 12:56 PM, Charles-Fran?ois Natali
> <cf.natali at gmail.com> wrote:
>> Why would the user care if multiprocessing is used behind the scene?
>
> Err ... that was another set of questions that I forgot to ask.
>
> (A)  Why bother raising an error if multiprocessing is unavailable?
> After all, there is a perfectly fine fallback...
>
> On other hand, errors should not pass silently.  If a user has
> explicitly asked for multiprocessing, there should be some notice that
> it didn't happen.  And builds are presumably something that a
> developer will monitor to respond to the Exception.

The point I'm making is that he's not asking for multiprocessing, he's
asking for parallel build.
If you pass 1 (or keep the default value), there's no fallback
involved: the code should simply proceed sequentially.

> (A1)  What sort of Error?  I'm inclined to raise the original
> ImportError, but the patch prefers a ValueError.

NotImplementedError would maybe make sense.

> As Claudiu pointed out, processes=1 should really mean 1 worker
> process, which is still different from "do everything in the main
> process".  I'm not sure that level of control is really worth the
> complexity, but I'm not certain it isn't.

I disagree. If you pass job =1 (and not processes = 1), then you don't
care whether multiprocessing is available or not: you just do
everything in your main process. It would be quite wasteful to create
a single child process!

>> processes <= 0: use os.cpu_count()
>
> I could understand doing that for 0 or -1; what is the purpose of
> doing it for both, let alone for -4?
>
> Are we at the point where the parameter should just take positive
> integers or one of a set of specified string values?

Honestly, as long as it accepts 0, I'm happy.

From chris.barker at noaa.gov  Mon Apr 28 20:12:48 2014
From: chris.barker at noaa.gov (Chris Barker)
Date: Mon, 28 Apr 2014 11:12:48 -0700
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <20140427164039.75e0f041@limelight.wooz.org>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
 <ljfcvm$ek7$1@ger.gmane.org> <20140427124039.27125a72@limelight.wooz.org>
 <CALGmxEKDoc5_Xqc1h4vhQdFOVRFsXFnHajbShCv_z2dR2EngLQ@mail.gmail.com>
 <20140427164039.75e0f041@limelight.wooz.org>
Message-ID: <CALGmxE+esoq=F95OpfFq+pvZK2MEmsm=Akwaxvi0jWPuD5afYA@mail.gmail.com>

On Sun, Apr 27, 2014 at 1:40 PM, Barry Warsaw <barry at python.org> wrote:

> On Apr 27, 2014, at 12:34 PM, Chris Barker wrote:
> >foo = long_function_name(var_one,
> >                                          var_two,
> >                                          var_three,
> >                                          var_four)
>
> Wow, do you really indent those 42 columns?
>

No -- stupid variable-width font!

I don't think anyone should write code with variable width fonts, and I'd
rather not do email that way either, but gmail is making it tough these
days..

Sorry for the added confusion.

This actually is forbidden because you're not using "vertical alignment"
> when
> there is an argument on the first line.  You would have to line up
> `var_two`
> right under `var_one` to be compliant.


yeah -- that was the intent....

oh well.

>That is, I find that if the argument list is too long for one line, then
> >splitting it out to only one argument per line is much more readable to
> me.
>
> Sure.  The PEP outlines ways to do that.


not really -- it allows it:

# Aligned with opening delimiter.
foo = long_function_name(var_one, var_two,
                         var_three, var_four)

but all the examples have more than one variable per line...my point is
that I think that should be discouraged.

i.e. I think the above should be:

# Aligned with opening delimiter.
foo = long_function_name(var_one,
                         var_two,
                         var_three,
                         var_four)

(done with fixed-width font this time -- but it may not look right in your
mail reader..)

though I doubt there would consensus on requiring that -- but many of us
learn more from examples than the specification, so maybe I'll submit a
patch with an example like that.

-Chris

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140428/6446da85/attachment.html>

From chris.barker at noaa.gov  Mon Apr 28 20:15:42 2014
From: chris.barker at noaa.gov (Chris Barker)
Date: Mon, 28 Apr 2014 11:15:42 -0700
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <20140428013457.GJ4273@ando>
References: <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
 <ljfcvm$ek7$1@ger.gmane.org> <20140427124039.27125a72@limelight.wooz.org>
 <CALGmxEKDoc5_Xqc1h4vhQdFOVRFsXFnHajbShCv_z2dR2EngLQ@mail.gmail.com>
 <ljjpa7$2g0$1@ger.gmane.org> <20140428013457.GJ4273@ando>
Message-ID: <CALGmxELn=pcM1+wbEf5Kwa-qj2C3k0LmX9eO0pet874JP+NMZw@mail.gmail.com>

On Sun, Apr 27, 2014 at 6:34 PM, Steven D'Aprano <steve at pearwood.info>wrote:

> > I would agree with having at least one example done with one arg per
> line.
>
> Is it really necessary? I think that one-arg-per-line is an obvious
> variation of the existing example.
>

not really -- a lot of folks learn more from following examples. If all teh
examples are the same in some aspect, we'll tend to think that that's
intentional, and that is at least the recommended method....


> If the only example given was one-arg-per-line, then the reader might
> wrongly assume that *only* one arg was allowed.


indeed -- but maybe less clear but the the fact that none of the examples
show one argument per line, the implication is that that is discouraged.

very had to document a widely variable standard!

-Chris

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140428/0437c6f5/attachment.html>

From solipsis at pitrou.net  Mon Apr 28 20:23:12 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 28 Apr 2014 20:23:12 +0200
Subject: [Python-Dev] Python  2.7.7. on Windows
References: <535DF27E.3000108@wetafx.co.nz>
	<535E08D0.2020500@mgmiller.net>
Message-ID: <20140428202312.6903d62a@fsol>

On Mon, 28 Apr 2014 19:52:48 +1200
Mike Miller <python-dev at mgmiller.net> wrote:
> 
> I thought this might be a good time to make a final plea to fix a
> long-standing security issue in the installer on Windows.  By default it
> installs Python to the root folder, thereby bypassing filesystem permissions:
> 
> http://bugs.python.org/issue1284316

Regardless of whether this change this or not, we certainly cannot
change it in a bugfix version. So, 3.5 at the earliest it would be.

Regards

Antoine.



From guido at python.org  Mon Apr 28 20:21:03 2014
From: guido at python.org (Guido van Rossum)
Date: Mon, 28 Apr 2014 11:21:03 -0700
Subject: [Python-Dev] file objects guarantees
In-Reply-To: <CAH_1eM38V5+kCddjETCxAF4JR=CoXPKqGzdgKjqDGjSnC+Yr6Q@mail.gmail.com>
References: <CAH_1eM38V5+kCddjETCxAF4JR=CoXPKqGzdgKjqDGjSnC+Yr6Q@mail.gmail.com>
Message-ID: <CAP7+vJ+i0WmMzPpZW05f1Hth+Nip8mQavwVsSbzN+jbq-Xko9A@mail.gmail.com>

File objects historically have a pretty vague spec. E.g. it's not defined
which methods are supported beyond read() and readline() (for readers) or
write() (for writers).

Short writes shouldn't be allowed (a regular file object's write() doesn't
even return a value for this reason). This means that a raw (unbuffered)

Short reads *should* be expected, since behavior around these varies
widely. If you see code that doesn't expect them please file bugs.

The glossary item doesn't provide much guidance for would-be implementers
of compliant file file objects, and the interface defined in the io module
is too large (e.g. nobody cares to implement readinto()).

I think we should clarify that raw (unbuffered) file objects are not safe.
I don't care about preventing this explicitly though -- when you see "file
object" you should think "duck typing" and program accordingly. (Reading
and writing are already distinct interfaces; ditto for text vs. bytes.)


On Mon, Apr 28, 2014 at 10:54 AM, Charles-Fran?ois Natali <
cf.natali at gmail.com> wrote:

> Hi,
>
> What's meant exactly by a "file object"?
>
> Let me be more specific: for example, pickle.dump() accepts a "file
> object".
>
> Looking at the code, it doesn't check the return value of its write()
> method.
>
> So it assumes that write() should always write the whole data (not
> partial write).
>
> Same thing for read, it assumes there won't be short reads.
>
> A sample use case would be passing a socket.makefile() to pickle: it
> works, because makefile() returns a BufferedReader/Writer which takes
> care of short read/write.
>
> But the documentation just says "file object". And if you have a look
> the file object definition in the glossary:
> https://docs.python.org/3.5/glossary.html#term-file-object
>
> """
> There are actually three categories of file objects: raw binary files,
> buffered binary files and text files. Their interfaces are defined in
> the io module. The canonical way to create a file object is by using
> the open() function.
> """
>
> So someone passing e.g. a raw binary file - which doesn't handle short
> reads/writes - would run into trouble.
>
> It's the same thing for e.g. GzipFile, and probably many others.
>
> Would it make sense to add a note somewhere?
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140428/d4d3dbcc/attachment.html>

From solipsis at pitrou.net  Mon Apr 28 20:24:58 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 28 Apr 2014 20:24:58 +0200
Subject: [Python-Dev] Multiple inheritance from builtin (C) types
 [still] supported in Python3?
References: <20140428204548.5eaa28e4@x34f>
Message-ID: <20140428202458.65de5f13@fsol>

On Mon, 28 Apr 2014 20:45:48 +0300
Paul Sokolovsky <pmiscml at gmail.com> wrote:
> 
> So, is that it, or disjoint native types are supported as bases
> somehow? Also, would someone know if a class-subclass case happens for
> example in stdlib?

Well, for instance this trivial example works:

>>> class C(list, object): pass
... 
>>> 

Basically, if two classes have compatible layouts, you can inherit from
both at once.

Regards

Antoine.



From guido at python.org  Mon Apr 28 20:36:01 2014
From: guido at python.org (Guido van Rossum)
Date: Mon, 28 Apr 2014 11:36:01 -0700
Subject: [Python-Dev] Multiple inheritance from builtin (C) types
 [still] supported in Python3?
In-Reply-To: <20140428202458.65de5f13@fsol>
References: <20140428204548.5eaa28e4@x34f> <20140428202458.65de5f13@fsol>
Message-ID: <CAP7+vJL0p-8jgAtF1Ro43NW7R9denEFmXSb88WNKrHH=TxwtiA@mail.gmail.com>

Antoine's example works because list inherits from object. The more general
rule "compatible layout" only allows the rarest of cases to work --
basically the two classes involved must have a common base class and one of
the classes must not add any C-level fields to that base class's layout. I
would never count on this except in cases where this possibility is
documented (e.g. when one class is designed to be a mix-in for the other).


On Mon, Apr 28, 2014 at 11:24 AM, Antoine Pitrou <solipsis at pitrou.net>wrote:

> On Mon, 28 Apr 2014 20:45:48 +0300
> Paul Sokolovsky <pmiscml at gmail.com> wrote:
> >
> > So, is that it, or disjoint native types are supported as bases
> > somehow? Also, would someone know if a class-subclass case happens for
> > example in stdlib?
>
> Well, for instance this trivial example works:
>
> >>> class C(list, object): pass
> ...
> >>>
>
> Basically, if two classes have compatible layouts, you can inherit from
> both at once.
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140428/835689d9/attachment.html>

From nad at acm.org  Mon Apr 28 20:43:05 2014
From: nad at acm.org (Ned Deily)
Date: Mon, 28 Apr 2014 11:43:05 -0700
Subject: [Python-Dev] API and process questions (sparked by Claudiu Popa
	on 16104
References: <CA+OGgf47B1pRNY1fL6Pi_iqn3_VanpgvmVt7Fs6GLR3aJrtKPg@mail.gmail.com>
 <CAP1=2W40T4ft0Om7kXVXcVEQ3C7KEsP2pWkW7Vd5zqf9+sqc2g@mail.gmail.com>
 <CAH_1eM0VQmd3bj53vJhXS7hNDuWjha99V7mSx1KidBS-ByJj5Q@mail.gmail.com>
 <CA+OGgf4weBk1NsXBSqi1g7tDE-=7rfkzd5bZn1MXiHWSggECEQ@mail.gmail.com>
Message-ID: <nad-946B51.11430528042014@news.gmane.org>

In article 
<CA+OGgf4weBk1NsXBSqi1g7tDE-=7rfkzd5bZn1MXiHWSggECEQ at mail.gmail.com>,
 "Jim J. Jewett" <jimjjewett at gmail.com> wrote:
> As Claudiu pointed out, processes=1 should really mean 1 worker
> process, which is still different from "do everything in the main
> process".  I'm not sure that level of control is really worth the
> complexity, but I'm not certain it isn't.

For regrtest, there is an important difference between "do everything in 
the main process" and "do one test at a time in a subprocess", namely 
the inadvertent global side-effects that running some tests have on 
other tests.  The latter option is much safer and gives more 
reproducible test results.  For compileall, I don't think there is a big 
difference between the two cases such that the regrtest semantics need 
to be followed.

-- 
 Ned Deily,
 nad at acm.org


From pmiscml at gmail.com  Mon Apr 28 20:42:02 2014
From: pmiscml at gmail.com (Paul Sokolovsky)
Date: Mon, 28 Apr 2014 21:42:02 +0300
Subject: [Python-Dev] Multiple inheritance from builtin (C) types
 [still] supported in Python3?
In-Reply-To: <20140428202458.65de5f13@fsol>
References: <20140428204548.5eaa28e4@x34f>
	<20140428202458.65de5f13@fsol>
Message-ID: <20140428214202.417ae99d@x34f>

Hello,

On Mon, 28 Apr 2014 20:24:58 +0200
Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Mon, 28 Apr 2014 20:45:48 +0300
> Paul Sokolovsky <pmiscml at gmail.com> wrote:
> > 
> > So, is that it, or disjoint native types are supported as bases
> > somehow? Also, would someone know if a class-subclass case happens
> > for example in stdlib?
> 
> Well, for instance this trivial example works:
> 
> >>> class C(list, object): pass
> ... 
> >>> 

Well, it's easy to treat "object" class as a special-case, "null"
class. So, let's re-formulate questions above with "where such
native base classes are not 'object'".

> 
> Basically, if two classes have compatible layouts, you can inherit
> from both at once.

How is "compatible layout" defined? Or "layout" for that matter at
all?

> 
> Regards
> 
> Antoine.


-- 
Best regards,
 Paul                          mailto:pmiscml at gmail.com

From solipsis at pitrou.net  Mon Apr 28 21:02:24 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 28 Apr 2014 21:02:24 +0200
Subject: [Python-Dev] Multiple inheritance from builtin (C) types
 [still] supported in Python3?
In-Reply-To: <20140428214202.417ae99d@x34f>
References: <20140428204548.5eaa28e4@x34f> <20140428202458.65de5f13@fsol>
 <20140428214202.417ae99d@x34f>
Message-ID: <20140428210224.02f19f37@fsol>


Hi Paul,

On Mon, 28 Apr 2014 21:42:02 +0300
Paul Sokolovsky <pmiscml at gmail.com> wrote:
> > 
> > Basically, if two classes have compatible layouts, you can inherit
> > from both at once.
> 
> How is "compatible layout" defined? Or "layout" for that matter at
> all?

See Guido's answer. I don't think it's documented anywhere, but you can
find the relevant code somewhere in Objects/typeobject.c (it's quite a
mouthful, though :-)).

(IIRC, "layout" is determined by tp_basicsize, tp_itemsize, the
number of __slots__, and other things perhaps)

Regards

Antoine.

From barry at python.org  Mon Apr 28 21:03:21 2014
From: barry at python.org (Barry Warsaw)
Date: Mon, 28 Apr 2014 15:03:21 -0400
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <CALGmxE+esoq=F95OpfFq+pvZK2MEmsm=Akwaxvi0jWPuD5afYA@mail.gmail.com>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
 <ljfcvm$ek7$1@ger.gmane.org>
 <20140427124039.27125a72@limelight.wooz.org>
 <CALGmxEKDoc5_Xqc1h4vhQdFOVRFsXFnHajbShCv_z2dR2EngLQ@mail.gmail.com>
 <20140427164039.75e0f041@limelight.wooz.org>
 <CALGmxE+esoq=F95OpfFq+pvZK2MEmsm=Akwaxvi0jWPuD5afYA@mail.gmail.com>
Message-ID: <20140428150321.3f908a1e@anarchist.wooz.org>

On Apr 28, 2014, at 11:12 AM, Chris Barker wrote:

>No -- stupid variable-width font!
>
>I don't think anyone should write code with variable width fonts, and I'd
>rather not do email that way either, but gmail is making it tough these
>days..

Ouch.  I'm sure it's gmail being "helpful" the way
<http://wiki.list.org/x/2IA9> is similarly helpful.

>> Sure.  The PEP outlines ways to do that.
>
>not really -- it allows it:
>
># Aligned with opening delimiter.
>foo = long_function_name(var_one, var_two,
>                         var_three, var_four)
>
>but all the examples have more than one variable per line...my point is
>that I think that should be discouraged.
>
>i.e. I think the above should be:
>
># Aligned with opening delimiter.
>foo = long_function_name(var_one,
>                         var_two,
>                         var_three,
>                         var_four)
>
>(done with fixed-width font this time -- but it may not look right in your
>mail reader..)

Fortunately, my mail stack is sane. :)

>though I doubt there would consensus on requiring that -- but many of us
>learn more from examples than the specification, so maybe I'll submit a
>patch with an example like that.

I also very much doubt you'd get consensus on that as a requirement, and I
would oppose the PEP taking a stand one way or the other.  I'm not sure that
the PEP needs an example to illustrate its acceptability.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140428/418f811b/attachment.sig>

From guido at python.org  Mon Apr 28 21:08:40 2014
From: guido at python.org (Guido van Rossum)
Date: Mon, 28 Apr 2014 12:08:40 -0700
Subject: [Python-Dev] Multiple inheritance from builtin (C) types
 [still] supported in Python3?
In-Reply-To: <20140428214202.417ae99d@x34f>
References: <20140428204548.5eaa28e4@x34f> <20140428202458.65de5f13@fsol>
 <20140428214202.417ae99d@x34f>
Message-ID: <CAP7+vJJgUp31VoDwP-+B0HPCcnLUr=W8Prh5YMsLFkZBR=xW=w@mail.gmail.com>

On Mon, Apr 28, 2014 at 11:42 AM, Paul Sokolovsky <pmiscml at gmail.com> wrote:

Well, it's easy to treat "object" class as a special-case, "null"
> class.


But the implementation doesn't, at least not for the question you're asking.


> So, let's re-formulate questions above with "where such
> native base classes are not 'object'".
>
> >
> > Basically, if two classes have compatible layouts, you can inherit
> > from both at once.
>
> How is "compatible layout" defined? Or "layout" for that matter at
> all?
>

The layout is what the C struct defining the object looks like. These are
typically defined in headers in the Include directory (e.g. listobject.h).

The definition of compatible layout is defined by the C code that gives the
error message when it's incompatible. Sorry, I don't recall exactly where
that is, so I recommend that you just look at CPython's source tree. (I
know you have a personal desire not to look at CPython, but I can't help
you without referring to it anyway.)

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140428/d5c9c3ae/attachment.html>

From guido at python.org  Mon Apr 28 21:10:35 2014
From: guido at python.org (Guido van Rossum)
Date: Mon, 28 Apr 2014 12:10:35 -0700
Subject: [Python-Dev] Multiple inheritance from builtin (C) types
 [still] supported in Python3?
In-Reply-To: <20140428210224.02f19f37@fsol>
References: <20140428204548.5eaa28e4@x34f> <20140428202458.65de5f13@fsol>
 <20140428214202.417ae99d@x34f> <20140428210224.02f19f37@fsol>
Message-ID: <CAP7+vJLH7aF5DXj8AZ-cTBKnm7w4DjSBz45w-ZKisYj=0=j2DQ@mail.gmail.com>

On Mon, Apr 28, 2014 at 12:02 PM, Antoine Pitrou <solipsis at pitrou.net>wrote:

>
> On Mon, 28 Apr 2014 21:42:02 +0300
> Paul Sokolovsky <pmiscml at gmail.com> wrote:
> > >
> > > Basically, if two classes have compatible layouts, you can inherit
> > > from both at once.
> >
> > How is "compatible layout" defined? Or "layout" for that matter at
> > all?
>
> See Guido's answer. I don't think it's documented anywhere, but you can
> find the relevant code somewhere in Objects/typeobject.c (it's quite a
> mouthful, though :-)).
>
> (IIRC, "layout" is determined by tp_basicsize, tp_itemsize, the
> number of __slots__, and other things perhaps)
>

IIRC the actual inheritance pattern also goes into it. Two structs that
each add an identical new field to a common base class's struct should
*not* be considered compatible.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140428/cfcf411c/attachment.html>

From solipsis at pitrou.net  Mon Apr 28 21:56:27 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 28 Apr 2014 21:56:27 +0200
Subject: [Python-Dev] Commit-ready patches needing review
References: <877g6a1tc9.fsf@vostro.rath.org>
Message-ID: <20140428215627.4b511202@fsol>

On Sun, 27 Apr 2014 12:10:46 -0700
Nikolaus Rath <Nikolaus at rath.org> wrote:
> 
> * http://bugs.python.org/issue21057 (TextIOWrapper does not support
>   reading bytearrays or memoryviews)

I've reviewed this one.

Regards

Antoine.



From python-dev at mgmiller.net  Mon Apr 28 22:07:57 2014
From: python-dev at mgmiller.net (Mike Miller)
Date: Tue, 29 Apr 2014 08:07:57 +1200
Subject: [Python-Dev] Python  2.7.7. on Windows
In-Reply-To: <789792ad6e7c4d63ac74e33bcc3f50c5@BLUPR03MB389.namprd03.prod.outlook.com>
References: <535DF27E.3000108@wetafx.co.nz> <535E08D0.2020500@mgmiller.net>
 <789792ad6e7c4d63ac74e33bcc3f50c5@BLUPR03MB389.namprd03.prod.outlook.com>
Message-ID: <535EB51D.4020403@mgmiller.net>


On 04/29/2014 05:12 AM, Steve Dower wrote:
> This would be an incredibly painful change that would surprise and hurt a lot of
> people.

Hi, I think "incredibly painful" is overstating the case a bit. ;)  We're 
talking about an installer default, a setting that would still be changeable as 
it always has, that by definition only will affect brand new installs.

> Yes, it is possible for a non-admin user to install arbitrary packages into a
> place where an admin user may inadvertently run it, thereby providing escalation
> of privilege. On the other hand, that applies to a lot of development tools,
> especially since most users on Windows these days are actually limited
> administrators - ANYTHING they install could run when they elevate a certain
> process.

None of Microsoft's Dev tools install to C:\, rather to ProgramFiles.  The fact 
that another security issue may exist is not a good justification for creating 
additional.

> On the other hand, Python from python.org is a tool that should only be
> installed by those who are prepared to manage it. On Windows it is easy enough
> to have a second, secured copy for your admin scripts and an unsecured copy for
> non-admin tasks.

This sounds like the perspective of someone highly technical, forgetting that 
new users will be installing python as well and vastly outnumber us.  "Normal 
people" need help to avoid security issues.

Microsoft's guidelines on where to install software are clear, and don't make 
exceptions that "tools" should be installed to the root of the drive to bypass 
file system permissions, for convenience.

> I'm not sure what change you are proposing here... doesn't the installer already
> have an option to add to PATH? I'm sure I keep disabling it.

No, it does not.  Unless it got slipped in when I wasn't looking.

It should be an option though, I think everyone would agree.

> "python.exe" on my PATH because I have 10+ versions installed at any one time. I

Remember, python-dev's are not the target users of this package, and are a 
rather minuscule fraction of the user base.

> Python installation. At this point, 2.7.6->2.7.7 should be an incredibly safe
> upgrade, and there's no way to safely change the default installation location

This would continue to be the case, as the installer will recognize the 
previously installed Python 2.7 and use its setting.  This should affect only 
*brand new installs.*

> or the ACLs on the install directory.

No ACLs would be affected or changed or even thought about.  Simply installing 
to the correct folder (on new installs) will accomplish the goal.

In short, this design of restricted permissions (read-only) for binaries and 
PATH conveniences goes back decades under Unix and other OS's.  MS Windows has 
finally caught up in the security department in the last few releases.  Please 
don't keep us back in the "bad old days" of DOS where everything was installed 
to the root folder.

-- 
-Mike


From ezio.melotti at gmail.com  Mon Apr 28 22:18:15 2014
From: ezio.melotti at gmail.com (Ezio Melotti)
Date: Mon, 28 Apr 2014 23:18:15 +0300
Subject: [Python-Dev] API and process questions (sparked by Claudiu Popa
 on 16104
In-Reply-To: <CA+OGgf47B1pRNY1fL6Pi_iqn3_VanpgvmVt7Fs6GLR3aJrtKPg@mail.gmail.com>
References: <CA+OGgf47B1pRNY1fL6Pi_iqn3_VanpgvmVt7Fs6GLR3aJrtKPg@mail.gmail.com>
Message-ID: <CACBhJdF=pfTGdU3+94Tqb0z7CUtYxiq1s6r-+bAt9CUbEEE1qw@mail.gmail.com>

On Mon, Apr 28, 2014 at 6:32 PM, Jim J. Jewett <jimjjewett at gmail.com> wrote:
> (1)  Should fixes to a docstring go in with a patch, even if they
> aren't related to the changing functionality?
>
> [...]
>
> It is best if a commit changes one small thing at a time.
> On the other hand, Nick recently posted that the minimal overhead of a
> patch commit is about half an hour.
>

It could be much less.  As an example,
http://bugs.python.org/issue19943 has been closed 9 minutes after the
report, it didn't have a patch and the fix had to be
applied/grafted/merged on 3 branches.  The time passed between the
first commit and the push is less than a minute too.  Clearly the time
increases if you have to run the test suite or if there is a merge
conflict/push race, and further decreases if there is a simple
typo-fix on default only and a patch already available.

> Is that overhead enough to override the one-issue-per-patch guideline?
>

That said, if the main fix should go only on default and it has a
smaller unrelated fix that should also go on default only, then doing
it together is generally OK (for some value of "unrelated" -- the fix
should still be small and near the code you touched for the main fix).
 If the unrelated fix needs to be ported on all the branches (or in
general in branches where the main fix shouldn't go), it's better to
have two separate patches.  If you commit the smaller fix together
with the bigger one, and then decide to backport it, you will have to
do a null merge and it gets slightly more complicated.


Best Regards,
Ezio Melotti

From brian at python.org  Mon Apr 28 22:38:36 2014
From: brian at python.org (Brian Curtin)
Date: Mon, 28 Apr 2014 15:38:36 -0500
Subject: [Python-Dev] Python 2.7.7. on Windows
In-Reply-To: <535EB51D.4020403@mgmiller.net>
References: <535DF27E.3000108@wetafx.co.nz> <535E08D0.2020500@mgmiller.net>
 <789792ad6e7c4d63ac74e33bcc3f50c5@BLUPR03MB389.namprd03.prod.outlook.com>
 <535EB51D.4020403@mgmiller.net>
Message-ID: <CAD+XWwqmKuDQ_88RRR_YksfbfLBYwra=xLrcbYT0+ty+X6aFuw@mail.gmail.com>

On Mon, Apr 28, 2014 at 3:07 PM, Mike Miller <python-dev at mgmiller.net> wrote:
>
> On 04/29/2014 05:12 AM, Steve Dower wrote:
>>
>> This would be an incredibly painful change that would surprise and hurt a
>> lot of
>> people.
>
>
> Hi, I think "incredibly painful" is overstating the case a bit. ;)  We're
> talking about an installer default, a setting that would still be changeable
> as it always has, that by definition only will affect brand new installs.
>
>
>> Yes, it is possible for a non-admin user to install arbitrary packages
>> into a
>> place where an admin user may inadvertently run it, thereby providing
>> escalation
>> of privilege. On the other hand, that applies to a lot of development
>> tools,
>> especially since most users on Windows these days are actually limited
>> administrators - ANYTHING they install could run when they elevate a
>> certain
>> process.
>
>
> None of Microsoft's Dev tools install to C:\, rather to ProgramFiles.  The
> fact that another security issue may exist is not a good justification for
> creating additional.
>
>
>> On the other hand, Python from python.org is a tool that should only be
>> installed by those who are prepared to manage it. On Windows it is easy
>> enough
>> to have a second, secured copy for your admin scripts and an unsecured
>> copy for
>> non-admin tasks.
>
>
> This sounds like the perspective of someone highly technical, forgetting
> that new users will be installing python as well and vastly outnumber us.
> "Normal people" need help to avoid security issues.
>
> Microsoft's guidelines on where to install software are clear, and don't
> make exceptions that "tools" should be installed to the root of the drive to
> bypass file system permissions, for convenience.
>
>
>> I'm not sure what change you are proposing here... doesn't the installer
>> already
>> have an option to add to PATH? I'm sure I keep disabling it.
>
>
> No, it does not.  Unless it got slipped in when I wasn't looking.
>
> It should be an option though, I think everyone would agree.

The option to add the current install to your path was added 3.3.

>> "python.exe" on my PATH because I have 10+ versions installed at any one
>> time. I
>
>
> Remember, python-dev's are not the target users of this package, and are a
> rather minuscule fraction of the user base.

Knowing which Python you want on your path and that you want it on
your path at all is somewhat of an advanced usage. While beginners do
want to just open up cmd and type "python" and have it work, there are
better ways to accomplish that which don't involve system-wide path
manipulation or installation changes.

Several PC manufacturers have been known to use Python for various
system utilities, which is how Python sometimes ends up in the path on
your grandma's Dell*. Even for a beginner who just wants "python" to
work, we need to be careful to not wreck their entire system by
helping them get our fresh Python install to show up.

A more reasonable way to help beginners would be to create a shortcut
somewhere that starts up cmd with a modified path. They can do
whatever they want to do within that context without modifying their
entire system. If they learn that they later want their system-wide
path manipulated, they can do that within the installer or will known
how to do that themselves.

* watch Dave Beazley's PyCon 2014 talk for a good story involving one
of those manufacturer installed Pythons:
https://www.youtube.com/watch?v=RZ4Sn-Y7AP8

From pcmanticore at gmail.com  Mon Apr 28 22:24:16 2014
From: pcmanticore at gmail.com (Claudiu Popa)
Date: Mon, 28 Apr 2014 23:24:16 +0300
Subject: [Python-Dev] API and process questions (sparked by Claudiu Popa
 on 16104
In-Reply-To: <CACBhJdF=pfTGdU3+94Tqb0z7CUtYxiq1s6r-+bAt9CUbEEE1qw@mail.gmail.com>
References: <CA+OGgf47B1pRNY1fL6Pi_iqn3_VanpgvmVt7Fs6GLR3aJrtKPg@mail.gmail.com>
 <CACBhJdF=pfTGdU3+94Tqb0z7CUtYxiq1s6r-+bAt9CUbEEE1qw@mail.gmail.com>
Message-ID: <CAMy=CLpi+P95F6m74_6_4We-m5o2XnG1xnPD98dX4uShBCbPTA@mail.gmail.com>

This issue raised too much bikeshedding. To wrap it up, I'll modify
the patch with the following:

- processes renamed to workers
- `workers` defaults to 1
- When `workers` is equal to 0, then `os.cpu_count` will be used
- When `workers` > 1, multiple processes will be used
- When `workers` == 1, run normally (no multiple processes)
- Negative values really should raise a ValueError (as
multiprocessing.Pool and soon
  concurrent.futures.Thread/ProcessPoolExecutor)
- Will raise NotImplementedError if multiprocessing can't be used
(when `workers` equals to 0 or > 1)

If anyone agrees with the above, then I'll modify the patch. This will
be its last iteration, any other bikeshedding
should be addressed by the core dev who'll apply it.

Thank you.

From python-dev at mgmiller.net  Mon Apr 28 22:12:58 2014
From: python-dev at mgmiller.net (Mike Miller)
Date: Tue, 29 Apr 2014 08:12:58 +1200
Subject: [Python-Dev] Python 2.7.7. on Windows
In-Reply-To: <mailman.51040.1398707666.18129.python-dev@python.org>
References: <mailman.51040.1398707666.18129.python-dev@python.org>
Message-ID: <535EB64A.1000803@mgmiller.net>

Hi,

Those bugs were fixed 9 years ago or so around 2005. I use python in 
ProgramFiles every day without issue.  If another bug has crept in somewhere, it 
can be fixed.

-Mike


> ------------------------------
> From: Sturla Molden <sturla.molden at gmail.com>
> To: python-dev at python.org
> Subject: Re: [Python-Dev] Python 2.7.7. on Windows
> Message-ID:
> 	<771463774420395726.352685sturla.molden-gmail.com at news.gmane.org>
>
> "C:\Program Files\Python27" contains an empty space in the path. If you
> want to randomly break build tools for C extensions, then go ahead and
> change it.
>
> Sturla

From python-dev at mgmiller.net  Mon Apr 28 22:18:22 2014
From: python-dev at mgmiller.net (Mike Miller)
Date: Tue, 29 Apr 2014 08:18:22 +1200
Subject: [Python-Dev] Python-Dev Digest, Vol 129, Issue 81
In-Reply-To: <mailman.51065.1398710600.18129.python-dev@python.org>
References: <mailman.51065.1398710600.18129.python-dev@python.org>
Message-ID: <535EB78E.1080004@mgmiller.net>

Hi, note the pep, it makes allowances for security enhancements.

-Mike

 > Message: 5 Date: Mon, 28 Apr 2014 20:23:12 +0200
 > From: Antoine Pitrou <solipsis at pitrou.net>
 > To: python-dev at python.org Subject:
 > Re: [Python-Dev] Python 2.7.7. on Windows
 > Message-ID: <20140428202312.6903d62a at fsol>

 > Regardless of whether this change this or not, we certainly cannot
 > change it in a bugfix version. So, 3.5 at the earliest it would be.

 > Regards

 > Antoine.

From python-dev at mgmiller.net  Mon Apr 28 22:56:10 2014
From: python-dev at mgmiller.net (Mike Miller)
Date: Tue, 29 Apr 2014 08:56:10 +1200
Subject: [Python-Dev] Python 2.7.7. on Windows
In-Reply-To: <CAD+XWwqmKuDQ_88RRR_YksfbfLBYwra=xLrcbYT0+ty+X6aFuw@mail.gmail.com>
References: <535DF27E.3000108@wetafx.co.nz>	<535E08D0.2020500@mgmiller.net>	<789792ad6e7c4d63ac74e33bcc3f50c5@BLUPR03MB389.namprd03.prod.outlook.com>	<535EB51D.4020403@mgmiller.net>
 <CAD+XWwqmKuDQ_88RRR_YksfbfLBYwra=xLrcbYT0+ty+X6aFuw@mail.gmail.com>
Message-ID: <535EC06A.40107@mgmiller.net>


On 04/29/2014 08:38 AM, Brian Curtin wrote:
> The option to add the current install to your path was added 3.3.

Ok, thanks.  So there is some precedent it would be useful.

>> Remember, python-dev's are not the target users of this package, and are a
>> rather minuscule fraction of the user base.
>
> Knowing which Python you want on your path and that you want it on
> your path at all is somewhat of an advanced usage.
 > ...
> They can do
> whatever they want to do within that context without modifying their
> entire system. If they learn that they later want their system-wide
> path manipulated...

Installing python at all is a "system-wide" change.  Why not default to a known 
safe, ready to use configuration, rather than a "tool-box" that needs assembly?

> * watch Dave Beazley's PyCon 2014 talk for a good story involving one
> of those manufacturer installed Pythons:
> https://www.youtube.com/watch?v=RZ4Sn-Y7AP8

Thanks, I'm trying to get thru all the talk will watch that shortly. ;)

Still, edge cases regarding manufactures should not override the needs of the 
majority of users.

-Mike

From bcannon at gmail.com  Mon Apr 28 23:01:45 2014
From: bcannon at gmail.com (Brett Cannon)
Date: Mon, 28 Apr 2014 21:01:45 +0000
Subject: [Python-Dev] Python-Dev Digest, Vol 129, Issue 81
References: <mailman.51065.1398710600.18129.python-dev@python.org>
 <535EB78E.1080004@mgmiller.net>
Message-ID: <CAP1=2W5xCadXck2Rh1oboD+nEDn+0pUnYG6e-FPT+fHsL-nn-g@mail.gmail.com>

On Mon Apr 28 2014 at 4:58:35 PM, Mike Miller <python-dev at mgmiller.net>
wrote:

> Hi, note the pep, it makes allowances for security enhancements.
>

The PEP in question is about fixing fundamentally broken security issues in
Python 2.7 (e.g. updating OpenSSL). Tweaking where Python is installed by
default on Windows is not fundamentally broken, it's a difference of
opinion as to whether the current defaults are the best option or not.

-Brett


>
> -Mike
>
>  > Message: 5 Date: Mon, 28 Apr 2014 20:23:12 +0200
>  > From: Antoine Pitrou <solipsis at pitrou.net>
>  > To: python-dev at python.org Subject:
>  > Re: [Python-Dev] Python 2.7.7. on Windows
>  > Message-ID: <20140428202312.6903d62a at fsol>
>
>  > Regardless of whether this change this or not, we certainly cannot
>  > change it in a bugfix version. So, 3.5 at the earliest it would be.
>
>  > Regards
>
>  > Antoine.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140428/d2905269/attachment.html>

From tjreedy at udel.edu  Mon Apr 28 23:34:56 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 28 Apr 2014 17:34:56 -0400
Subject: [Python-Dev] API and process questions (sparked by Claudiu Popa
	on 16104
In-Reply-To: <CAMy=CLpi+P95F6m74_6_4We-m5o2XnG1xnPD98dX4uShBCbPTA@mail.gmail.com>
References: <CA+OGgf47B1pRNY1fL6Pi_iqn3_VanpgvmVt7Fs6GLR3aJrtKPg@mail.gmail.com>
 <CACBhJdF=pfTGdU3+94Tqb0z7CUtYxiq1s6r-+bAt9CUbEEE1qw@mail.gmail.com>
 <CAMy=CLpi+P95F6m74_6_4We-m5o2XnG1xnPD98dX4uShBCbPTA@mail.gmail.com>
Message-ID: <ljmhj5$67a$1@ger.gmane.org>

On 4/28/2014 4:24 PM, Claudiu Popa wrote:
> This issue raised too much bikeshedding. To wrap it up, I'll modify
> the patch with the following:
>
> - processes renamed to workers
> - `workers` defaults to 1
> - When `workers` is equal to 0, then `os.cpu_count` will be used
> - When `workers` > 1, multiple processes will be used
> - When `workers` == 1, run normally (no multiple processes)

I presume you mean for this to be the default.

> - Negative values really should raise a ValueError (as
> multiprocessing.Pool and soon
>    concurrent.futures.Thread/ProcessPoolExecutor)

Yay!

> - Will raise NotImplementedError if multiprocessing can't be used
> (when `workers` equals to 0 or > 1)
>
> If anyone agrees with the above, then I'll modify the patch.

Having read the thread, though not much of the issue, these look to me 
like the best choices for a clean, comprehensible API.

 > This will
> be its last iteration, any other bikeshedding
> should be addressed by the core dev who'll apply it.

-- 
Terry Jan Reedy


From tjreedy at udel.edu  Tue Apr 29 00:01:32 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 28 Apr 2014 18:01:32 -0400
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <CALGmxE+esoq=F95OpfFq+pvZK2MEmsm=Akwaxvi0jWPuD5afYA@mail.gmail.com>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
 <ljfcvm$ek7$1@ger.gmane.org> <20140427124039.27125a72@limelight.wooz.org>
 <CALGmxEKDoc5_Xqc1h4vhQdFOVRFsXFnHajbShCv_z2dR2EngLQ@mail.gmail.com>
 <20140427164039.75e0f041@limelight.wooz.org>
 <CALGmxE+esoq=F95OpfFq+pvZK2MEmsm=Akwaxvi0jWPuD5afYA@mail.gmail.com>
Message-ID: <ljmj51$vfu$1@ger.gmane.org>

On 4/28/2014 2:12 PM, Chris Barker wrote:

> I don't think anyone should write code with variable width fonts,

The problem is that fixed pitch does not work well for even a half-way 
complete unicode font and I don't know that there are any available. As 
far as I know, my Windows 7 only came with one unicode font that Idle 
finds. An advantage from my viewpoint is that spaces in this font are 
narrower that the average character, making multiple-of-4 indents and 
spaces around operators look pretty good to me.

Given that PEP8 generally disallows adding spaces to make things line 
up, as in the following (a style I used  to adhere to)

a     = 134543344  # big number
bonge = 2          # short number

it seems to me consistent to use the multiple-of-4 hanging indent style 
and not try to make things line up with an odd number of spaces in this 
one special case.

-- 
Terry Jan Reedy



From tjreedy at udel.edu  Tue Apr 29 00:06:03 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 28 Apr 2014 18:06:03 -0400
Subject: [Python-Dev] Python-Dev Digest, Vol 129, Issue 81
In-Reply-To: <CAP1=2W5xCadXck2Rh1oboD+nEDn+0pUnYG6e-FPT+fHsL-nn-g@mail.gmail.com>
References: <mailman.51065.1398710600.18129.python-dev@python.org>
 <535EB78E.1080004@mgmiller.net>
 <CAP1=2W5xCadXck2Rh1oboD+nEDn+0pUnYG6e-FPT+fHsL-nn-g@mail.gmail.com>
Message-ID: <ljmjdf$vfu$2@ger.gmane.org>

On 4/28/2014 5:01 PM, Brett Cannon wrote:
>
>
> On Mon Apr 28 2014 at 4:58:35 PM, Mike Miller <python-dev at mgmiller.net
> <mailto:python-dev at mgmiller.net>> wrote:
>
>     Hi, note the pep, it makes allowances for security enhancements.
>
>
> The PEP in question is about fixing fundamentally broken security issues
> in Python 2.7 (e.g. updating OpenSSL). Tweaking where Python is
> installed by default on Windows is not fundamentally broken, it's a
> difference of opinion as to whether the current defaults are the best
> option or not.

Besides which, the PEP was cut down to a specific list of changes. Any 
more would need a separate dicussion.

-- 
Terry Jan Reedy


From rdmurray at bitdance.com  Tue Apr 29 00:38:42 2014
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 28 Apr 2014 18:38:42 -0400
Subject: [Python-Dev] API and process questions (sparked by Claudiu Popa
	on 16104
In-Reply-To: <CAMy=CLpi+P95F6m74_6_4We-m5o2XnG1xnPD98dX4uShBCbPTA@mail.gmail.com>
References: <CA+OGgf47B1pRNY1fL6Pi_iqn3_VanpgvmVt7Fs6GLR3aJrtKPg@mail.gmail.com>
 <CACBhJdF=pfTGdU3+94Tqb0z7CUtYxiq1s6r-+bAt9CUbEEE1qw@mail.gmail.com>
 <CAMy=CLpi+P95F6m74_6_4We-m5o2XnG1xnPD98dX4uShBCbPTA@mail.gmail.com>
Message-ID: <20140428223842.78EE1250D0C@webabinitio.net>

On Mon, 28 Apr 2014 23:24:16 +0300, Claudiu Popa <pcmanticore at gmail.com> wrote:
> - Will raise NotImplementedError if multiprocessing can't be used
> (when `workers` equals to 0 or > 1)

I think the most common use case for this ability will be "run with
the appropriate number of processes for the system I'm on", where
'the appropriate number' is 1 (the main process) if multiprocessing
is not available.  Otherwise the tool calling compileall would have to
figure out how to "catch the error" (how do you do that when invoking
a CLI?) and re-run the script using '1` itself.

How you spell this I don't really care, but I think the above is the
most common use case.

--David

From rdmurray at bitdance.com  Tue Apr 29 00:44:17 2014
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 28 Apr 2014 18:44:17 -0400
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <ljmj51$vfu$1@ger.gmane.org>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
 <ljfcvm$ek7$1@ger.gmane.org> <20140427124039.27125a72@limelight.wooz.org>
 <CALGmxEKDoc5_Xqc1h4vhQdFOVRFsXFnHajbShCv_z2dR2EngLQ@mail.gmail.com>
 <20140427164039.75e0f041@limelight.wooz.org>
 <CALGmxE+esoq=F95OpfFq+pvZK2MEmsm=Akwaxvi0jWPuD5afYA@mail.gmail.com>
 <ljmj51$vfu$1@ger.gmane.o rg>
Message-ID: <20140428224417.EDBB8250D0C@webabinitio.net>

On Mon, 28 Apr 2014 18:01:32 -0400, Terry Reedy <tjreedy at udel.edu> wrote:
> On 4/28/2014 2:12 PM, Chris Barker wrote:
> 
> > I don't think anyone should write code with variable width fonts,
> 
> The problem is that fixed pitch does not work well for even a half-way 
> complete unicode font and I don't know that there are any available. As 
> far as I know, my Windows 7 only came with one unicode font that Idle 
> finds. An advantage from my viewpoint is that spaces in this font are 
> narrower that the average character, making multiple-of-4 indents and 
> spaces around operators look pretty good to me.

My unix fix-width terminal font handles most unicode (even a lot of
non-bmp stuff...though I have no idea if it is readable :).

I have no idea what font it is, to tell you the truth.

--David

From chris.barker at noaa.gov  Tue Apr 29 01:13:07 2014
From: chris.barker at noaa.gov (Chris Barker)
Date: Mon, 28 Apr 2014 16:13:07 -0700
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <ljmj51$vfu$1@ger.gmane.org>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
 <ljfcvm$ek7$1@ger.gmane.org> <20140427124039.27125a72@limelight.wooz.org>
 <CALGmxEKDoc5_Xqc1h4vhQdFOVRFsXFnHajbShCv_z2dR2EngLQ@mail.gmail.com>
 <20140427164039.75e0f041@limelight.wooz.org>
 <CALGmxE+esoq=F95OpfFq+pvZK2MEmsm=Akwaxvi0jWPuD5afYA@mail.gmail.com>
 <ljmj51$vfu$1@ger.gmane.org>
Message-ID: <CALGmxE+uWZDcaa=Wt1eAH9RhzFN1J5Xj60FcVBL2wjfxanKrHA@mail.gmail.com>

On Mon, Apr 28, 2014 at 3:01 PM, Terry Reedy <tjreedy at udel.edu> wrote:

> I don't think anyone should write code with variable width fonts,
>>
>
> The problem is that fixed pitch does not work well for even a half-way
> complete unicode font and I don't know that there are any available.
>
...

> Given that PEP8 generally disallows adding spaces to make things line up,
> as in the following (a style I used  to adhere to)
>
> a     = 134543344  # big number
> bonge = 2          # short number
>

Is ironic that that looks like hell in my variable-width font email client?

it seems to me consistent to use the multiple-of-4 hanging indent style and
> not try to make things line up with an odd number of spaces in this one
> special case.


things aren't going to generally line up with a non-fixed width font no
matter how you slice it, so I guess if you need to use one for your code,
you're just stuck. But I suppose the "line up with the opening bracket"
style isn't a good idea in that case. All the more reason not to
standardize anymore that PEP 8 already does.

-Chris


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140428/cbddf16b/attachment.html>

From tjreedy at udel.edu  Tue Apr 29 01:20:02 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 28 Apr 2014 19:20:02 -0400
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <CALGmxE+uWZDcaa=Wt1eAH9RhzFN1J5Xj60FcVBL2wjfxanKrHA@mail.gmail.com>
References: <5358BABE.6030300@simplistix.co.uk>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk> <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
 <ljfcvm$ek7$1@ger.gmane.org> <20140427124039.27125a72@limelight.wooz.org>
 <CALGmxEKDoc5_Xqc1h4vhQdFOVRFsXFnHajbShCv_z2dR2EngLQ@mail.gmail.com>
 <20140427164039.75e0f041@limelight.wooz.org>
 <CALGmxE+esoq=F95OpfFq+pvZK2MEmsm=Akwaxvi0jWPuD5afYA@mail.gmail.com>
 <ljmj51$vfu$1@ger.gmane.org>
 <CALGmxE+uWZDcaa=Wt1eAH9RhzFN1J5Xj60FcVBL2wjfxanKrHA@mail.gmail.com>
Message-ID: <535EE222.8000701@udel.edu>

On 4/28/2014 7:13 PM, Chris Barker wrote:
> On Mon, Apr 28, 2014 at 3:01 PM, Terry Reedy <tjreedy at udel.edu
> <mailto:tjreedy at udel.edu>> wrote:
>
>         I don't think anyone should write code with variable width fonts,
>
>     The problem is that fixed pitch does not work well for even a
>     half-way complete unicode font and I don't know that there are any
>     available.
> ...
>
>     Given that PEP8 generally disallows adding spaces to make things
>     line up, as in the following (a style I used  to adhere to)
>
>     a     = 134543344  # big number
>     bonge = 2          # short number
>
> Is ironic that that looks like hell in my variable-width font email client?

I believe that has been given as a reason to not do what I used to do. 
The only things guaranteed to actually line up are the initial graphic 
characters of lines with the exact same indent.

tjr



From chris.barker at noaa.gov  Tue Apr 29 01:20:08 2014
From: chris.barker at noaa.gov (Chris Barker)
Date: Mon, 28 Apr 2014 16:20:08 -0700
Subject: [Python-Dev] Python 2.7.7. on Windows
In-Reply-To: <535EC06A.40107@mgmiller.net>
References: <535DF27E.3000108@wetafx.co.nz> <535E08D0.2020500@mgmiller.net>
 <789792ad6e7c4d63ac74e33bcc3f50c5@BLUPR03MB389.namprd03.prod.outlook.com>
 <535EB51D.4020403@mgmiller.net>
 <CAD+XWwqmKuDQ_88RRR_YksfbfLBYwra=xLrcbYT0+ty+X6aFuw@mail.gmail.com>
 <535EC06A.40107@mgmiller.net>
Message-ID: <CALGmxEL7AUEWUNs0kzxnh221aDfXNsrG9YFc=ygZv4YU5P_Ssw@mail.gmail.com>

On Mon, Apr 28, 2014 at 1:56 PM, Mike Miller <python-dev at mgmiller.net>wrote:

> * watch Dave Beazley's PyCon 2014 talk for a good story involving one
>
>> of those manufacturer installed Pythons:
>> https://www.youtube.com/watch?v=RZ4Sn-Y7AP8
>>
>
> Thanks, I'm trying to get thru all the talk will watch that shortly. ;)
>

fun talk -- but start at about 18:00 minutes if you want the  relevant
bit...

By the way, some people (or at least ESRI, with ArcMap) Install and use
their own python in: C:\python27 -- if you dont want to interat with that,
you need to put it somewhere else!

I'd argue that any OEM or third party app that uses the standard location
or puts it on the PATH be default is in error, but what can you so?

Redhat used to have #!/usr/bin/env python on their admin scripts, which was
really painful if you wanted to upgrade anything. Don't know it they've
fixed that....

-Chris


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140428/c75a46b7/attachment-0001.html>

From pmiscml at gmail.com  Tue Apr 29 01:26:17 2014
From: pmiscml at gmail.com (Paul Sokolovsky)
Date: Tue, 29 Apr 2014 02:26:17 +0300
Subject: [Python-Dev] Multiple inheritance from builtin (C) types
 [still] supported in Python3?
In-Reply-To: <CAP7+vJJgUp31VoDwP-+B0HPCcnLUr=W8Prh5YMsLFkZBR=xW=w@mail.gmail.com>
References: <20140428204548.5eaa28e4@x34f> <20140428202458.65de5f13@fsol>
 <20140428214202.417ae99d@x34f>
 <CAP7+vJJgUp31VoDwP-+B0HPCcnLUr=W8Prh5YMsLFkZBR=xW=w@mail.gmail.com>
Message-ID: <20140429022617.5dbe2ebf@x34f>

Hello,

On Mon, 28 Apr 2014 12:08:40 -0700
Guido van Rossum <guido at python.org> wrote:

[]

> > How is "compatible layout" defined? Or "layout" for that matter at
> > all?
> >
> 
> The layout is what the C struct defining the object looks like. These
> are typically defined in headers in the Include directory (e.g.
> listobject.h).
> 
> The definition of compatible layout is defined by the C code that
> gives the error message when it's incompatible. Sorry, I don't recall
> exactly where that is, so I recommend that you just look at CPython's
> source tree. (I know you have a personal desire not to look at
> CPython, but I can't help you without referring to it anyway.)

Well, sure I did, as I mentioned, but as that's first time I see that
code (that specific piece is in typeobject.c:extra_ivars()), it would
take quite some time be sure I understand all aspects of it. Thanks for
confirming that it's governed essentially by CPython implementation
details and not some language-level semantics like metaclasses (I
mentioned them because error message in Python2 did so, though Python3
doesn't refer to metaclasses).

An example would really help me to get a feel of the issue, but I
assume lack of them means that there's no well-known idiom where such
inheritance is used, and that's good enough on its own. I also tried to
figure how it's important to support such multi-base cases, so the code
I write didn't require complete rewrite if it hits one day, but
everything seems to turn out to be pretty extensible.

> 
> -- 
> --Guido van Rossum (python.org/~guido)

Thanks!

-- 
Best regards,
 Paul                          mailto:pmiscml at gmail.com

From Steve.Dower at microsoft.com  Tue Apr 29 02:04:24 2014
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Tue, 29 Apr 2014 00:04:24 +0000
Subject: [Python-Dev] Python  2.7.7. on Windows
In-Reply-To: <535EB51D.4020403@mgmiller.net>
References: <535DF27E.3000108@wetafx.co.nz> <535E08D0.2020500@mgmiller.net>
 <789792ad6e7c4d63ac74e33bcc3f50c5@BLUPR03MB389.namprd03.prod.outlook.com>
 <535EB51D.4020403@mgmiller.net>
Message-ID: <d8de496d5345489aa7ee336ee0d0275e@BLUPR03MB389.namprd03.prod.outlook.com>

> Mike Miller wrote:
> On 04/29/2014 05:12 AM, Steve Dower wrote:
>> This would be an incredibly painful change that would surprise and
>> hurt a lot of people.
> 
> Hi, I think "incredibly painful" is overstating the case a bit. ;) We're talking
> about an installer default, a setting that would still be changeable as it
> always has, that by definition only will affect brand new installs.

Good point about it only affecting new installs, though that still constitutes a lot of installs.
 
>> Yes, it is possible for a non-admin user to install arbitrary packages
>> into a place where an admin user may inadvertently run it, thereby
>> providing escalation of privilege. On the other hand, that applies to
>> a lot of development tools, especially since most users on Windows
>> these days are actually limited administrators - ANYTHING they install
>> could run when they elevate a certain process.
> 
> None of Microsoft's Dev tools install to C:\, rather to ProgramFiles. The fact
> that another security issue may exist is not a good justification for creating
> additional.

The fact that the mitigations are well known by the people who have to worry about them is a good justification for not creating a compatibility issue. It's easy for IT admins to install Python in a way that the files are read-only, the .pyc and .pyo files are already there, and user site packages will be used by default (I think that last one is easy?). The good IT admins even know that they need to do this - perhaps we can help educate the bad admins? (FWIW, if you have admin privileges on your own machine, YOU are the IT admin. Are you a good one or a bad one?)

>> On the other hand, Python from python.org is a tool that should only
>> be installed by those who are prepared to manage it. On Windows it is
>> easy enough to have a second, secured copy for your admin scripts and
>> an unsecured copy for non-admin tasks.
> 
> This sounds like the perspective of someone highly technical, forgetting that
> new users will be installing python as well and vastly outnumber us. "Normal
> people" need help to avoid security issues.

One place where mitigations are added are the distributions (Canopy, Anaconda, etc.). These packages redistribute Python and install to different locations with different permissions (I'm not promising that they all do it properly, but they have the opportunity to do so). The reference implementation of CPython is typically not the best option for "normal people", who are much better served by one of these bundles.

> Microsoft's guidelines on where to install software are clear, and don't make
> exceptions that "tools" should be installed to the root of the drive to bypass
> file system permissions, for convenience.

I'm well aware of the guidelines, hence the practicality vs. purity comment. I'm fairly certain that the installation to the root was originally about ease of command-line access rather than bypassing permissions - the permissions probably didn't exist for the first few versions of Python (Python for DOS obviously didn't care... maybe it's always been about backwards compatibility?)

At this point, installing into the root is about not breaking everyone who *knows* that Python installs into the root directory and always has. 

>> I'm not sure what change you are proposing here... doesn't the
>> installer already have an option to add to PATH? I'm sure I keep disabling it.
> 
> No, it does not. Unless it got slipped in when I wasn't looking.
> 
> It should be an option though, I think everyone would agree.

Thanks Brett for pointing out that it arrived in Python 3. I'm sure it would be an acceptable addition to 2.7.7, though you'd need to get it in before RC and you'd also need to find someone who is keen and able to keep making 2.7 installers for Windows. Right now, we don't have anyone who is both.

>> "python.exe" on my PATH because I have 10+ versions installed at any
>> one time.
> 
> Remember, python-dev's are not the target users of this package, and are a
> rather minuscule fraction of the user base.

My day job revolves around making Python easier to use for non "python-dev"s, especially those who have multiple versions of Python installed simultaneously (as I mentioned, over 50% of our user base). Having PATH or PATHEXT set globally is a source of much confusion. (Though a global PYTHONPATH causes more problems, but that isn't part of this discussion.)

>> Python installation. At this point, 2.7.6->2.7.7 should be an
>> incredibly safe upgrade, and there's no way to safely change the
>> default installation location
> 
> This would continue to be the case, as the installer will recognize the
> previously installed Python 2.7 and use its setting. This should affect only
> *brand new installs.*

For example, next time you redeploy your hosted service on a clean VM with the latest security patches? The next time someone does a clean uninstall-reinstall because they've been burned by upgrades before? The next time someone arrives at work and IT has given them a new image and Python is "gone"? I can guarantee that you will break someone

>> or the ACLs on the install directory.
> 
> No ACLs would be affected or changed or even thought about. Simply installing to
> the correct folder (on new installs) will accomplish the goal.

The effective permissions change, and now commands that didn't require elevation do. Tools that didn't need to be manifested for UAC have to be recompiled. Scripts that used to work will simply break. This is the problem, not whether the permissions are explicitly set as part of the install process.

> In short, this design of restricted permissions (read-only) for binaries and
> PATH conveniences goes back decades under Unix and other OS's. MS Windows has
> finally caught up in the security department in the last few releases. Please
> don't keep us back in the "bad old days" of DOS where everything was installed
> to the root folder.

My employer would be most unhappy if I wasn't trying to encourage good Windows support :)

At the same time, my employer, colleagues, friends, collaborators and most people on python-dev would all be unhappy if I made everyone angry at Python. At risk of saying it too often, practicality beats purity.

Cheers,
Steve

> --
> -Mike
>

From brian at python.org  Tue Apr 29 02:14:16 2014
From: brian at python.org (Brian Curtin)
Date: Mon, 28 Apr 2014 19:14:16 -0500
Subject: [Python-Dev] Python 2.7.7. on Windows
In-Reply-To: <d8de496d5345489aa7ee336ee0d0275e@BLUPR03MB389.namprd03.prod.outlook.com>
References: <535DF27E.3000108@wetafx.co.nz> <535E08D0.2020500@mgmiller.net>
 <789792ad6e7c4d63ac74e33bcc3f50c5@BLUPR03MB389.namprd03.prod.outlook.com>
 <535EB51D.4020403@mgmiller.net>
 <d8de496d5345489aa7ee336ee0d0275e@BLUPR03MB389.namprd03.prod.outlook.com>
Message-ID: <CAD+XWwoPNmEL4xQhj=D3aWkeC3TjT53upBPusg07d5A3EOnPiA@mail.gmail.com>

On Mon, Apr 28, 2014 at 7:04 PM, Steve Dower <Steve.Dower at microsoft.com> wrote:
>> Mike Miller wrote:
>> On 04/29/2014 05:12 AM, Steve Dower wrote:
>>> This would be an incredibly painful change that would surprise and
>>> hurt a lot of people.
>>
>> Hi, I think "incredibly painful" is overstating the case a bit. ;) We're talking
>> about an installer default, a setting that would still be changeable as it
>> always has, that by definition only will affect brand new installs.
>
> Good point about it only affecting new installs, though that still constitutes a lot of installs.
>
>>> Yes, it is possible for a non-admin user to install arbitrary packages
>>> into a place where an admin user may inadvertently run it, thereby
>>> providing escalation of privilege. On the other hand, that applies to
>>> a lot of development tools, especially since most users on Windows
>>> these days are actually limited administrators - ANYTHING they install
>>> could run when they elevate a certain process.
>>
>> None of Microsoft's Dev tools install to C:\, rather to ProgramFiles. The fact
>> that another security issue may exist is not a good justification for creating
>> additional.
>
> The fact that the mitigations are well known by the people who have to worry about them is a good justification for not creating a compatibility issue. It's easy for IT admins to install Python in a way that the files are read-only, the .pyc and .pyo files are already there, and user site packages will be used by default (I think that last one is easy?). The good IT admins even know that they need to do this - perhaps we can help educate the bad admins? (FWIW, if you have admin privileges on your own machine, YOU are the IT admin. Are you a good one or a bad one?)
>
>>> On the other hand, Python from python.org is a tool that should only
>>> be installed by those who are prepared to manage it. On Windows it is
>>> easy enough to have a second, secured copy for your admin scripts and
>>> an unsecured copy for non-admin tasks.
>>
>> This sounds like the perspective of someone highly technical, forgetting that
>> new users will be installing python as well and vastly outnumber us. "Normal
>> people" need help to avoid security issues.
>
> One place where mitigations are added are the distributions (Canopy, Anaconda, etc.). These packages redistribute Python and install to different locations with different permissions (I'm not promising that they all do it properly, but they have the opportunity to do so). The reference implementation of CPython is typically not the best option for "normal people", who are much better served by one of these bundles.
>
>> Microsoft's guidelines on where to install software are clear, and don't make
>> exceptions that "tools" should be installed to the root of the drive to bypass
>> file system permissions, for convenience.
>
> I'm well aware of the guidelines, hence the practicality vs. purity comment. I'm fairly certain that the installation to the root was originally about ease of command-line access rather than bypassing permissions - the permissions probably didn't exist for the first few versions of Python (Python for DOS obviously didn't care... maybe it's always been about backwards compatibility?)
>
> At this point, installing into the root is about not breaking everyone who *knows* that Python installs into the root directory and always has.
>
>>> I'm not sure what change you are proposing here... doesn't the
>>> installer already have an option to add to PATH? I'm sure I keep disabling it.
>>
>> No, it does not. Unless it got slipped in when I wasn't looking.
>>
>> It should be an option though, I think everyone would agree.
>
> Thanks Brett for pointing out that it arrived in Python 3. I'm sure it would be an acceptable addition to 2.7.7, though you'd need to get it in before RC and you'd also need to find someone who is keen and able to keep making 2.7 installers for Windows. Right now, we don't have anyone who is both.

If it's an acceptable change to the release manager (Benjamin?), and
if there's actually time before the RC (I don't know when it is
planned), I am willing to backport my 3.3 change to get this in the
2.7 installer.

However, I'm not currently setup to make release installers -- I think
I need a signing certificate or something like that.

From benjamin at python.org  Tue Apr 29 02:18:59 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 28 Apr 2014 17:18:59 -0700
Subject: [Python-Dev] Python 2.7.7. on Windows
In-Reply-To: <CAD+XWwoPNmEL4xQhj=D3aWkeC3TjT53upBPusg07d5A3EOnPiA@mail.gmail.com>
References: <535DF27E.3000108@wetafx.co.nz> <535E08D0.2020500@mgmiller.net>
 <789792ad6e7c4d63ac74e33bcc3f50c5@BLUPR03MB389.namprd03.prod.outlook.com>
 <535EB51D.4020403@mgmiller.net>
 <d8de496d5345489aa7ee336ee0d0275e@BLUPR03MB389.namprd03.prod.outlook.com>
 <CAD+XWwoPNmEL4xQhj=D3aWkeC3TjT53upBPusg07d5A3EOnPiA@mail.gmail.com>
Message-ID: <1398730739.14732.111466989.40EAD063@webmail.messagingengine.com>

On Mon, Apr 28, 2014, at 17:14, Brian Curtin wrote:
> If it's an acceptable change to the release manager (Benjamin?), and
> if there's actually time before the RC (I don't know when it is
> planned), I am willing to backport my 3.3 change to get this in the
> 2.7 installer.

That's fine.

> 
> However, I'm not currently setup to make release installers -- I think
> I need a signing certificate or something like that.

Apparently the current PSF one just expired, so we're in the process of
procuring a new one.

From stephen at xemacs.org  Tue Apr 29 03:30:24 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 29 Apr 2014 10:30:24 +0900
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <20140428224417.EDBB8250D0C@webabinitio.net>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
 <ljfcvm$ek7$1@ger.gmane.org>
 <20140427124039.27125a72@limelight.wooz.org>
 <CALGmxEKDoc5_Xqc1h4vhQdFOVRFsXFnHajbShCv_z2dR2EngLQ@mail.gmail.com>
 <20140427164039.75e0f041@limelight.wooz.org>
 <CALGmxE+esoq=F95OpfFq+pvZK2MEmsm=Akwaxvi0jWPuD5afYA@mail.gmail.com>
 <ljmj51$vfu$1@ger.gmane.o rg>
 <20140428224417.EDBB8250D0C@webabinitio.net>
Message-ID: <87vbtsncr3.fsf@uwakimon.sk.tsukuba.ac.jp>

R. David Murray writes:

 > My unix fix-width terminal font handles most unicode (even a lot of
 > non-bmp stuff...though I have no idea if it is readable :).

Oh, I bet you do.  With a true fixed-width Unicode font, it's the
*Latin-character* text that's painful, if not unreadable, because the
aspect ratio needs to be close to 1.0 to handle ideographs, rather
than the 0.5 or less common with Latin fonts.

XTerm has an option to treat ideographs as double width.  That does
the trick (at least, I don't know any scripts that really really want
an aspect ratio of 0.7 or 2.0).


From stephen at xemacs.org  Tue Apr 29 03:30:36 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 29 Apr 2014 10:30:36 +0900
Subject: [Python-Dev] pep8 reasoning
In-Reply-To: <CALGmxE+esoq=F95OpfFq+pvZK2MEmsm=Akwaxvi0jWPuD5afYA@mail.gmail.com>
References: <5358BABE.6030300@simplistix.co.uk>
 <20140424095931.2fe5d048@anarchist.wooz.org>
 <CAPTjJmrWQRxp-Ba-N6SErEK6j4yz3Xk0jCT5cHu41kcN_2bt0g@mail.gmail.com>
 <CAExdVNnrgZexcXr78PESQ+ADzqQUNKa9Gm66sdVjFKxBzVPxiw@mail.gmail.com>
 <20140425014001.GA31627@gensokyo.amherst.edu>
 <CAPTjJmpm9X1t2fcYt-LDTrB6fyFXYoDN4WcKSxtazWLENb7mBg@mail.gmail.com>
 <20140424230338.1d11c6c9@limelight.wooz.org>
 <535A5080.6030406@simplistix.co.uk>
 <87y4yto650.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140425103647.1eafca9f@anarchist.wooz.org>
 <CADiSq7eo3q=8PKoEmh=Rv-sK99f6_F=N1zngqfODB03RaeESkQ@mail.gmail.com>
 <CAPFzasPDtG9Y+NyW06v+8R+DXR97d1e64g8hY=OS5A_vZnsoPw@mail.gmail.com>
 <535ABD9B.8040807@stoneleaf.us> <535AD900.6090606@oddbird.net>
 <ljfcvm$ek7$1@ger.gmane.org>
 <20140427124039.27125a72@limelight.wooz.org>
 <CALGmxEKDoc5_Xqc1h4vhQdFOVRFsXFnHajbShCv_z2dR2EngLQ@mail.gmail.com>
 <20140427164039.75e0f041@limelight.wooz.org>
 <CALGmxE+esoq=F95OpfFq+pvZK2MEmsm=Akwaxvi0jWPuD5afYA@mail.gmail.com>
Message-ID: <87tx9cncqr.fsf@uwakimon.sk.tsukuba.ac.jp>


 > On Mon, Apr 28, 2014 at 11:12 AM, Chris Barker wrote:

 > not really -- it allows it:
 > 
 > # Aligned with opening delimiter.
 > foo = long_function_name(var_one, var_two,
 >                          var_three, var_four)
 > 
 > but all the examples have more than one variable per line...my point is
 > that I think that should be discouraged.
 > 
 > i.e. I think the above should be:
 > 
 > # Aligned with opening delimiter.
 > foo = long_function_name(var_one,
 >                          var_two,
 >                          var_three,
 >                          var_four)

I don't have a problem with discouraging it, but it should not be
flagged in pep8 (OK in pep8 --wink-wink-nudge-nudge-say-no-more-eh,
though!)

I usually do use the style you prefer, especially when the arguments
want to be commented, but in many cases the arguments are semantically
tuples.  Eg,

    rect1 = paint_rectangle(point[1], point[0],        # top, left
                            textheight + 2 * padding, textwidth + 2 * padding,
                            chartreuse)


From stephen at xemacs.org  Tue Apr 29 05:07:00 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 29 Apr 2014 12:07:00 +0900
Subject: [Python-Dev] Python  2.7.7. on Windows
In-Reply-To: <535EB51D.4020403@mgmiller.net>
References: <535DF27E.3000108@wetafx.co.nz> <535E08D0.2020500@mgmiller.net>
 <789792ad6e7c4d63ac74e33bcc3f50c5@BLUPR03MB389.namprd03.prod.outlook.com>
 <535EB51D.4020403@mgmiller.net>
Message-ID: <87ppk0n8a3.fsf@uwakimon.sk.tsukuba.ac.jp>

Mike Miller writes:

 > Microsoft's guidelines on where to install software are clear, and
 > don't make exceptions that "tools" should be installed to the root
 > of the drive to bypass file system permissions, for convenience.

But there's the rub.  In this case, Microsoft doesn't have *security*,
it has "guidelines".  They are *still* guidelines, not security,
*exactly* because it's convenient for somebody.  The fact that taking
advantage of that convenience has the side effect of bypassing
filesystem permissions is unfortunate (and a bug in Windows IMO).

Note that if users actually paid attention to these guidelines, we'd
be getting complaints from *them*, not from you.  I don't recall ever
seeing that.  That implies that "normal users" will install anything
anwhere anyway.

If it's that unimportant to Microsoft, I see insufficient reason why
we should risk confusing those "normal users" who already have Python
2.7 installed (and as pointed out, they *are* at risk precisely
because the proposal changes the default install location).

From Nikolaus at rath.org  Tue Apr 29 05:48:11 2014
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Mon, 28 Apr 2014 20:48:11 -0700
Subject: [Python-Dev] bytes with trailing \0?
Message-ID: <87ha5cyex0.fsf@vostro.rath.org>

Hello,

I was surprised to find the following in bytesobject.c:

,----
| [...]
|    As always, an extra byte is allocated for a trailing \0 byte (newsize
|    does *not* include that), and a trailing \0 byte is stored.
| */
| 
| int
| _PyBytes_Resize(PyObject **pv, Py_ssize_t newsize)
| {
| [...]
| 
`----

Does this mean that bytes objects are internally stored with a trailing
\0? Why is that? Isn't that just wasting a byte, because \0 might also
be in the middle of the byte sequence, and the bytes objects stores its
length explicitly anyway?


Best,
-Nikolaus
-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             ?Time flies like an arrow, fruit flies like a Banana.?

From steve at pearwood.info  Tue Apr 29 05:52:52 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 29 Apr 2014 13:52:52 +1000
Subject: [Python-Dev] Python  2.7.7. on Windows
In-Reply-To: <87ppk0n8a3.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <535DF27E.3000108@wetafx.co.nz> <535E08D0.2020500@mgmiller.net>
 <789792ad6e7c4d63ac74e33bcc3f50c5@BLUPR03MB389.namprd03.prod.outlook.com>
 <535EB51D.4020403@mgmiller.net> <87ppk0n8a3.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20140429035250.GP4273@ando>

On Tue, Apr 29, 2014 at 12:07:00PM +0900, Stephen J. Turnbull wrote:
> Mike Miller writes:
> 
>  > Microsoft's guidelines on where to install software are clear, and
>  > don't make exceptions that "tools" should be installed to the root
>  > of the drive to bypass file system permissions, for convenience.
> 
> But there's the rub.  In this case, Microsoft doesn't have *security*,
> it has "guidelines".  They are *still* guidelines, not security,
> *exactly* because it's convenient for somebody.  The fact that taking
> advantage of that convenience has the side effect of bypassing
> filesystem permissions is unfortunate (and a bug in Windows IMO).
> 
> Note that if users actually paid attention to these guidelines, we'd
> be getting complaints from *them*, not from you.  I don't recall ever
> seeing that.  That implies that "normal users" will install anything
> anwhere anyway.

I don't think that argument is terribly useful. If people waited for 
"normal users" to complain before doing something about Heartbeat, we'd 
be in a pretty pickle. "Normal users" don't understand the technology 
well enough to have a valid threat model or judge the consequences, and 
they are confused by a mixture of ignorance, misinformation and hype. 
It's up to technical users to lead, not to follow.


> If it's that unimportant to Microsoft, 

I think that's unfair. I'm not a MS fan, not even close. I think their 
business practices in the past have been reprehensible. But if there is 
anyone who takes backwards-compatibility even more seriously than 
Python-Dev, it is them. You should give them the courtesy of assuming 
that their decision is not based on apathy, but on *exactly* the same 
reasoning that *you* apply below:

> I see insufficient reason why
> we should risk confusing those "normal users" who already have Python
> 2.7 installed (and as pointed out, they *are* at risk precisely
> because the proposal changes the default install location).

And thus security vulnerabilities never get fixed :-)

I don't have an opinion on the importance or magnitude of this security 
vulnerability, the risk of confusion, or whether it should be fixed or 
not. But I wonder why the installer is ignoring the OS's guidelines for 
where software should be installed? If this were Apple we were talking 
about, would we ignore their guidelines? Or on Linux, would we blithly 
install Python in / instead of (say) /usr/local/bin? I don't think so. I 
would have thought that the mere fact that Microsoft disapproves of 
installing applications into the root should be good enough reason to 
not do it. In the absense of an extremely compelling reason not to do 
so, we should be a "good citizen" regardless of the OS.



-- 
Steven

From v+python at g.nevcal.com  Tue Apr 29 06:15:57 2014
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Mon, 28 Apr 2014 21:15:57 -0700
Subject: [Python-Dev] Python  2.7.7. on Windows
In-Reply-To: <20140429035250.GP4273@ando>
References: <535DF27E.3000108@wetafx.co.nz> <535E08D0.2020500@mgmiller.net>
 <789792ad6e7c4d63ac74e33bcc3f50c5@BLUPR03MB389.namprd03.prod.outlook.com>
 <535EB51D.4020403@mgmiller.net> <87ppk0n8a3.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140429035250.GP4273@ando>
Message-ID: <535F277D.4080201@g.nevcal.com>

On 4/28/2014 8:52 PM, Steven D'Aprano wrote:
> I think that's unfair. I'm not a MS fan, not even close. I think their
> business practices in the past have been reprehensible. But if there is
> anyone who takes backwards-compatibility even more seriously than
> Python-Dev, it is them.
I guess there is no one who takes backwards-compatibility even more 
seriously than Python-Dev.

I have perfectly good DOS and Windows programs that no longer run on 
supported Windows versions. XP->Vista was a bigger break than Python 
2->3, from my perspective.

And then if you get into the command line programs that have vanished or 
take different options, breaking batch files, and similar facilities, it 
gets even worse.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140428/fe664d34/attachment.html>

From guido at python.org  Tue Apr 29 06:34:47 2014
From: guido at python.org (Guido van Rossum)
Date: Mon, 28 Apr 2014 21:34:47 -0700
Subject: [Python-Dev] bytes with trailing \0?
In-Reply-To: <87ha5cyex0.fsf@vostro.rath.org>
References: <87ha5cyex0.fsf@vostro.rath.org>
Message-ID: <CAP7+vJJJUH1hWqLi17YfNPxtitADQFpFw7LQEtDYuA9Y3B3bLg@mail.gmail.com>

It is to improve the experience of passing bytes to a C function that
expects a trailing \0. For example syscalls taking filenames. The wrapper
must still check for embedded \0 but the bytes don't need to be copied.

On Monday, April 28, 2014, Nikolaus Rath <Nikolaus at rath.org> wrote:

> Hello,
>
> I was surprised to find the following in bytesobject.c:
>
> ,----
> | [...]
> |    As always, an extra byte is allocated for a trailing \0 byte (newsize
> |    does *not* include that), and a trailing \0 byte is stored.
> | */
> |
> | int
> | _PyBytes_Resize(PyObject **pv, Py_ssize_t newsize)
> | {
> | [...]
> |
> `----
>
> Does this mean that bytes objects are internally stored with a trailing
> \0? Why is that? Isn't that just wasting a byte, because \0 might also
> be in the middle of the byte sequence, and the bytes objects stores its
> length explicitly anyway?
>
>
> Best,
> -Nikolaus
> --
> GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
> Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
>
>              ?Time flies like an arrow, fruit flies like a Banana.?
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org <javascript:;>
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (on iPad)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140428/cb255b49/attachment-0001.html>

From ncoghlan at gmail.com  Tue Apr 29 08:50:34 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 29 Apr 2014 02:50:34 -0400
Subject: [Python-Dev] API and process questions (sparked by Claudiu Popa
 on 16104
In-Reply-To: <CACBhJdF=pfTGdU3+94Tqb0z7CUtYxiq1s6r-+bAt9CUbEEE1qw@mail.gmail.com>
References: <CA+OGgf47B1pRNY1fL6Pi_iqn3_VanpgvmVt7Fs6GLR3aJrtKPg@mail.gmail.com>
 <CACBhJdF=pfTGdU3+94Tqb0z7CUtYxiq1s6r-+bAt9CUbEEE1qw@mail.gmail.com>
Message-ID: <CADiSq7eFy3LMvpd+0M+_hkZN54GqucPs=8xotktDGLROK5Mhsg@mail.gmail.com>

On 28 Apr 2014 16:19, "Ezio Melotti" <ezio.melotti at gmail.com> wrote:
>
> On Mon, Apr 28, 2014 at 6:32 PM, Jim J. Jewett <jimjjewett at gmail.com>
wrote:
> > (1)  Should fixes to a docstring go in with a patch, even if they
> > aren't related to the changing functionality?
> >
> > [...]
> >
> > It is best if a commit changes one small thing at a time.
> > On the other hand, Nick recently posted that the minimal overhead of a
> > patch commit is about half an hour.
> >
>
> It could be much less.  As an example,
> http://bugs.python.org/issue19943 has been closed 9 minutes after the
> report, it didn't have a patch and the fix had to be
> applied/grafted/merged on 3 branches.  The time passed between the
> first commit and the push is less than a minute too.  Clearly the time
> increases if you have to run the test suite or if there is a merge
> conflict/push race, and further decreases if there is a simple
> typo-fix on default only and a patch already available.

The 30 minute guesstimate was for doing it right for a patch you didn't
write yourself:
- final recheck that all the required pieces are there
- patch import & merge commands
- run the tests for every affected branch
- coming up with the NEWS entry & commit message

It's generally much, much faster for changes I write myself - another
undesirable incentive since it further encourages us to work on our own
changes over incorporating patches from others.

> > Is that overhead enough to override the one-issue-per-patch guideline?
>
> That said, if the main fix should go only on default and it has a
> smaller unrelated fix that should also go on default only, then doing
> it together is generally OK (for some value of "unrelated" -- the fix
> should still be small and near the code you touched for the main fix).
>  If the unrelated fix needs to be ported on all the branches (or in
> general in branches where the main fix shouldn't go), it's better to
> have two separate patches.  If you commit the smaller fix together
> with the bigger one, and then decide to backport it, you will have to
> do a null merge and it gets slightly more complicated.

What Ezio said :)

I definitely do coarser commits for CPython than I do for beaker-project.org,
specifically because Gerrit provides decent tools for reviewing and merging
a patch series, and we don't currently have anything like that in the
CPython workflow.

Cheers,
Nick.

>
>
> Best Regards,
> Ezio Melotti
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140429/f1be6dd9/attachment.html>

From stephen at xemacs.org  Tue Apr 29 08:56:02 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 29 Apr 2014 15:56:02 +0900
Subject: [Python-Dev] Python  2.7.7. on Windows
In-Reply-To: <20140429035250.GP4273@ando>
References: <535DF27E.3000108@wetafx.co.nz> <535E08D0.2020500@mgmiller.net>
 <789792ad6e7c4d63ac74e33bcc3f50c5@BLUPR03MB389.namprd03.prod.outlook.com>
 <535EB51D.4020403@mgmiller.net>
 <87ppk0n8a3.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140429035250.GP4273@ando>
Message-ID: <87oazkmxod.fsf@uwakimon.sk.tsukuba.ac.jp>

Steven D'Aprano writes:
 > On Tue, Apr 29, 2014 at 12:07:00PM +0900, Stephen J. Turnbull wrote:

 > > Note that if users actually paid attention to these guidelines, we'd
 > > be getting complaints from *them*, not from you.  I don't recall ever
 > > seeing that.  That implies that "normal users" will install anything
 > > anwhere anyway.
 > 
 > I don't think that argument is terribly useful. If people waited for 
 > "normal users"

I'm not suggesting that we wait for such reports in any case.  My
point is that the "security hole" is apparently wide open and nobody,
not Microsoft and not corporate IT, is doing anything to close it.  If
it were, it would inconvenience users and we'd hear about it.  I infer
they assess the threat to be essentially zero.

 > > If it's that unimportant to Microsoft, 
 > 
 > I think that's unfair.

What's unfair?  Mr. Miller is evidently concerned about users who have
completely left their security up to their OS vendor.  Isn't reference
to the importance of an alleged security hole to that vendor both fair
and relevant to a decision to make a backwards-incompatible change in
a Python bugfix release, even in the installer?

 > You should give them the courtesy of assuming that their decision
 > is not based on apathy,

My point is that Microsoft has *not* made a decision, but left it up
to anybody who has software they hope Windows users will install --
both Python-Dev and crackers.  I infer they do not consider this a
security issue worthy of notice.

 > And thus security vulnerabilities never get fixed :-)

I have no objection *at all* to making the change in the next feature
release.  I think the "good citizenship" argument is more than
sufficient, but of course I'll leave it up to the release manager.  As
for bugfix releases, given the arguments above, I want a stronger
argument than "Microsoft guidelines", that's all.  I don't even ask
for a CVE. :-)

 > I would have thought that the mere fact that Microsoft disapproves
 > of installing applications into the root should be good enough
 > reason to not do it.

I'm not defending the failure to follow the guideline in the Python
2.x.0 releases (IIUC the guideline pre-existed 2.7).  I'm questioning
whether it is a sufficient reason to make a backwards-incompatible
change in a bugfix release.

My take is that Microsoft itself doesn't think it's very important.

Regards,

From python-dev at mgmiller.net  Tue Apr 29 12:43:48 2014
From: python-dev at mgmiller.net (Mike Miller)
Date: Tue, 29 Apr 2014 22:43:48 +1200
Subject: [Python-Dev] Python 2.7.7. on Windows
In-Reply-To: <CAPkN8xKQZKCvRf0GopFueTLGKGzcWm1rhNSESbPoGMXZM6fyRg@mail.gmail.com>
References: <535DF27E.3000108@wetafx.co.nz> <535E08D0.2020500@mgmiller.net>
 <789792ad6e7c4d63ac74e33bcc3f50c5@BLUPR03MB389.namprd03.prod.outlook.com>
 <535EB51D.4020403@mgmiller.net>
 <CAPkN8xKQZKCvRf0GopFueTLGKGzcWm1rhNSESbPoGMXZM6fyRg@mail.gmail.com>
Message-ID: <535F8264.2000508@mgmiller.net>

Hi,

Every change has pluses and minuses.  I can't guarantee 100% benefits, only 
trying to make the case that the benefits here outweigh them.

Since we are talking about humans, I'd gather most of them trying to install 
something on Windows will have heard about ProgramFiles and not be too bothered 
at its inclusion in the path.

-Mike

On 04/29/2014 06:31 PM, anatoly techtonik wrote:
> You don't take into account many tutorials and internal docs that make the
> job of new users easier by using this default. Human processes will be
> broken, newbies will suffer.
>

From python-dev at mgmiller.net  Tue Apr 29 12:27:16 2014
From: python-dev at mgmiller.net (Mike Miller)
Date: Tue, 29 Apr 2014 22:27:16 +1200
Subject: [Python-Dev] Python  2.7.7. on Windows
In-Reply-To: <d8de496d5345489aa7ee336ee0d0275e@BLUPR03MB389.namprd03.prod.outlook.com>
References: <535DF27E.3000108@wetafx.co.nz> <535E08D0.2020500@mgmiller.net>
 <789792ad6e7c4d63ac74e33bcc3f50c5@BLUPR03MB389.namprd03.prod.outlook.com>
 <535EB51D.4020403@mgmiller.net>
 <d8de496d5345489aa7ee336ee0d0275e@BLUPR03MB389.namprd03.prod.outlook.com>
Message-ID: <535F7E84.3040002@mgmiller.net>

Hi,

Stepping back a bit...

I doubt you'd take the idea this far, but that Python should need assembly by 
professionals before use doesn't match its "Batteries Included" spirit, nor the 
PC revolution for that matter.

The reason I brought up the subject at 2.7.7 is because there are greater 
changes than normal in this release, and people are expecting security fixes.
This is realistically the last chance to fix this vulnerability in 2.X.

We should help people do the right thing and be safe, while keeping the 
possibility to customize.

Yes, I can imagine a few people in the world unhappy with any change, but a 
large majority of "millions" will not notice and reap the benefits.  There are 
also quite a few people unhappy the change hasn't been made (see the bug).

However, those that want a less safe setting on brand new installs may still 
change it manually, script it with the command-line parameter, or could stay 
with 2.7.6 until they do.  The porting effort is tiny and would effect few 
compared to those who'd benefit.

-Mike

p.s. I'm available to help with maintaining the 2.7 installer, unfortunately 
I've only used Inno so not sure how long it would take me to ramp up.


On 04/29/2014 12:04 PM, Steve Dower wrote:

From python-dev at mgmiller.net  Tue Apr 29 13:14:42 2014
From: python-dev at mgmiller.net (Mike Miller)
Date: Tue, 29 Apr 2014 23:14:42 +1200
Subject: [Python-Dev] Python  2.7.7. on Windows
In-Reply-To: <87ppk0n8a3.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <535DF27E.3000108@wetafx.co.nz>	<535E08D0.2020500@mgmiller.net>	<789792ad6e7c4d63ac74e33bcc3f50c5@BLUPR03MB389.namprd03.prod.outlook.com>	<535EB51D.4020403@mgmiller.net>
 <87ppk0n8a3.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <535F89A2.50102@mgmiller.net>


On 04/29/2014 03:07 PM, Stephen J. Turnbull wrote:
 > I have no objection *at all* to making the change in the next feature
 > release.  I think the "good citizenship" argument is more than
 > sufficient, ...
 > I'm questioning whether it is a sufficient reason to make a backwards-
 > incompatible change in a bugfix release.

Normally I would completely agree.

However, this bug has been shitcanned for a decade.  This is the last chance to 
fix this bug in a branch that's going to be supported until 2020!  An unknown 
portion of Python 2.x users will never upgrade to 3.

Do we continue on this path in good conscience for at least six more years?

-Mike

From techtonik at gmail.com  Tue Apr 29 08:31:18 2014
From: techtonik at gmail.com (anatoly techtonik)
Date: Tue, 29 Apr 2014 09:31:18 +0300
Subject: [Python-Dev] Python 2.7.7. on Windows
In-Reply-To: <535EB51D.4020403@mgmiller.net>
References: <535DF27E.3000108@wetafx.co.nz> <535E08D0.2020500@mgmiller.net>
 <789792ad6e7c4d63ac74e33bcc3f50c5@BLUPR03MB389.namprd03.prod.outlook.com>
 <535EB51D.4020403@mgmiller.net>
Message-ID: <CAPkN8xKQZKCvRf0GopFueTLGKGzcWm1rhNSESbPoGMXZM6fyRg@mail.gmail.com>

On Mon, Apr 28, 2014 at 11:07 PM, Mike Miller <python-dev at mgmiller.net> wrote:
> On 04/29/2014 05:12 AM, Steve Dower wrote:
>>
>> This would be an incredibly painful change that would surprise and hurt a
>> lot of
>> people.
>
>
> Hi, I think "incredibly painful" is overstating the case a bit. ;)  We're
> talking about an installer default, a setting that would still be changeable
> as it always has, that by definition only will affect brand new installs.

You don't take into account many tutorials and internal docs that make the
job of new users easier by using this default. Human processes will be
broken, newbies will suffer.

From pje at telecommunity.com  Tue Apr 29 16:47:26 2014
From: pje at telecommunity.com (PJ Eby)
Date: Tue, 29 Apr 2014 10:47:26 -0400
Subject: [Python-Dev] Multiple inheritance from builtin (C) types
 [still] supported in Python3?
In-Reply-To: <20140429022617.5dbe2ebf@x34f>
References: <20140428204548.5eaa28e4@x34f> <20140428202458.65de5f13@fsol>
 <20140428214202.417ae99d@x34f>
 <CAP7+vJJgUp31VoDwP-+B0HPCcnLUr=W8Prh5YMsLFkZBR=xW=w@mail.gmail.com>
 <20140429022617.5dbe2ebf@x34f>
Message-ID: <CALeMXf6UsgMATSHw1asxrLfxCvamm5xTzwCT2GtOuJ0q03G+=g@mail.gmail.com>

On Mon, Apr 28, 2014 at 7:26 PM, Paul Sokolovsky <pmiscml at gmail.com> wrote:

> Well, sure I did, as I mentioned, but as that's first time I see that
> code (that specific piece is in typeobject.c:extra_ivars()), it would
> take quite some time be sure I understand all aspects of it. Thanks for
> confirming that it's governed essentially by CPython implementation
> details and not some language-level semantics like metaclasses (I
> mentioned them because error message in Python2 did so, though Python3
> doesn't refer to metaclasses).
>
> An example would really help me to get a feel of the issue, but I
> assume lack of them means that there's no well-known idiom where such
> inheritance is used, and that's good enough on its own. I also tried to
> figure how it's important to support such multi-base cases, so the code
> I write didn't require complete rewrite if it hits one day, but
> everything seems to turn out to be pretty extensible.
>

>From memory of the last time I dealt with this, the rules were that you
could mix two classes only if their __slots__ differed from their common
__base__ by *at most* __dict__ and/or __weakref__.  The dict and weakref
slots are special, in that the type structure contains their offsets, which
makes them relocatable in subclasses.  But any other __slots__ aren't
relocatable in subclasses, because the type structure doesn't directly keep
track of the offsets.  (The slot descriptors do.)

But I don't think there's anything in principle that requires this, it's
just the implementation.  You could in theory relocate __slots__ defined
from Python code in order to make a merged subclass.  It's just that the
effective "__slots__" of C code can't be moved, because C code is expecting
to find them at specific offsets.  Therefore, if two types define their own
struct fields, they can't be inherited from unless one is a subtype of the
other.

In the C code (again if I recall correctly), this is done using the
__base__ attribute of the type, which indicates what struct layout the
object will use.  A type can have a larger struct than its base type,
adding its own fields after the base type's struct fields.  (The dict and
weakref fields are added -- if they are added -- *after* the base struct
fields.  If your __base__ already has them, those offsets within the
existing layout are used, which is why them being in another base class's
__slots__ isn't a problem.)

When you create a new type, CPython looks at your bases to find a suitable
__base__.  If two of your bases inherit from each other, the ancestor can
be ignored, keeping the more-derived one as a candidate __base__.  If a
base adds only __dict__ and/or __weakref__ (or neither) to its __base__,
then its __base__ is a candidate (not recursively, though).  If at the end
there is more than one base left standing, then it's an error, since you
have bases with incompatible layouts.

That is not a precise description of the algorithm, but that's the gist of
how it works.  __base__ is a slot on the type object and is tracked at the
C level in order to sort out layouts like this.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140429/55ae6f1f/attachment.html>

From guido at python.org  Tue Apr 29 17:36:18 2014
From: guido at python.org (Guido van Rossum)
Date: Tue, 29 Apr 2014 08:36:18 -0700
Subject: [Python-Dev] Multiple inheritance from builtin (C) types
 [still] supported in Python3?
In-Reply-To: <CALeMXf6UsgMATSHw1asxrLfxCvamm5xTzwCT2GtOuJ0q03G+=g@mail.gmail.com>
References: <20140428204548.5eaa28e4@x34f> <20140428202458.65de5f13@fsol>
 <20140428214202.417ae99d@x34f>
 <CAP7+vJJgUp31VoDwP-+B0HPCcnLUr=W8Prh5YMsLFkZBR=xW=w@mail.gmail.com>
 <20140429022617.5dbe2ebf@x34f>
 <CALeMXf6UsgMATSHw1asxrLfxCvamm5xTzwCT2GtOuJ0q03G+=g@mail.gmail.com>
Message-ID: <CAP7+vJJF-eDDYezmTHc0DrcsLRswYOz_oRY7_S7c0oXRa22Xaw@mail.gmail.com>

Thanks -- your memory is better than mine!
On Apr 29, 2014 8:16 AM, "PJ Eby" <pje at telecommunity.com> wrote:

>
>
>
> On Mon, Apr 28, 2014 at 7:26 PM, Paul Sokolovsky <pmiscml at gmail.com>wrote:
>
>> Well, sure I did, as I mentioned, but as that's first time I see that
>> code (that specific piece is in typeobject.c:extra_ivars()), it would
>> take quite some time be sure I understand all aspects of it. Thanks for
>> confirming that it's governed essentially by CPython implementation
>> details and not some language-level semantics like metaclasses (I
>> mentioned them because error message in Python2 did so, though Python3
>> doesn't refer to metaclasses).
>>
>> An example would really help me to get a feel of the issue, but I
>> assume lack of them means that there's no well-known idiom where such
>> inheritance is used, and that's good enough on its own. I also tried to
>> figure how it's important to support such multi-base cases, so the code
>> I write didn't require complete rewrite if it hits one day, but
>> everything seems to turn out to be pretty extensible.
>>
>
> From memory of the last time I dealt with this, the rules were that you
> could mix two classes only if their __slots__ differed from their common
> __base__ by *at most* __dict__ and/or __weakref__.  The dict and weakref
> slots are special, in that the type structure contains their offsets, which
> makes them relocatable in subclasses.  But any other __slots__ aren't
> relocatable in subclasses, because the type structure doesn't directly keep
> track of the offsets.  (The slot descriptors do.)
>
> But I don't think there's anything in principle that requires this, it's
> just the implementation.  You could in theory relocate __slots__ defined
> from Python code in order to make a merged subclass.  It's just that the
> effective "__slots__" of C code can't be moved, because C code is expecting
> to find them at specific offsets.  Therefore, if two types define their own
> struct fields, they can't be inherited from unless one is a subtype of the
> other.
>
> In the C code (again if I recall correctly), this is done using the
> __base__ attribute of the type, which indicates what struct layout the
> object will use.  A type can have a larger struct than its base type,
> adding its own fields after the base type's struct fields.  (The dict and
> weakref fields are added -- if they are added -- *after* the base struct
> fields.  If your __base__ already has them, those offsets within the
> existing layout are used, which is why them being in another base class's
> __slots__ isn't a problem.)
>
> When you create a new type, CPython looks at your bases to find a suitable
> __base__.  If two of your bases inherit from each other, the ancestor can
> be ignored, keeping the more-derived one as a candidate __base__.  If a
> base adds only __dict__ and/or __weakref__ (or neither) to its __base__,
> then its __base__ is a candidate (not recursively, though).  If at the end
> there is more than one base left standing, then it's an error, since you
> have bases with incompatible layouts.
>
> That is not a precise description of the algorithm, but that's the gist of
> how it works.  __base__ is a slot on the type object and is tracked at the
> C level in order to sort out layouts like this.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140429/9e9b9bc6/attachment.html>

From Steve.Dower at microsoft.com  Tue Apr 29 18:14:06 2014
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Tue, 29 Apr 2014 16:14:06 +0000
Subject: [Python-Dev] Python 2.7.7. on Windows
In-Reply-To: <535F8264.2000508@mgmiller.net>
References: <535DF27E.3000108@wetafx.co.nz> <535E08D0.2020500@mgmiller.net>
 <789792ad6e7c4d63ac74e33bcc3f50c5@BLUPR03MB389.namprd03.prod.outlook.com>
 <535EB51D.4020403@mgmiller.net>
 <CAPkN8xKQZKCvRf0GopFueTLGKGzcWm1rhNSESbPoGMXZM6fyRg@mail.gmail.com>
 <535F8264.2000508@mgmiller.net>
Message-ID: <7b013d3d9478440593ad1e511623ca95@BLUPR03MB389.namprd03.prod.outlook.com>

Mike Miller wrote:
> Every change has pluses and minuses. I can't guarantee 100% benefits, only
> trying to make the case that the benefits here outweigh them.

If this is your case about the benefits, it's a weak case. Feel free to blog about how to secure a Python installation in multi-user environments, or contribute docs to python.org, or just go ahead and make your own secure installer (seriously - there are enterprises/IT departments that will prefer a "secure by default" version rather than having to secure it themselves).

A change that will break many existing users is a huge minus. Even putting this into 3.5 would worry me unless there is a *much* higher-than-usual usage of the prereleases. 

Here are some more minuses beyond those listed on the issue:

* Longer install time (due to having to compile the stdlib to pyc and pyo eagerly)
* Larger installed size (same reason)
* Incompatibility with existing non-elevating package installers (every bdist_wininst is in this category, I believe)
* Incompatibility with tools that don't recognize user site-packages

Incidentally, forcing normal users to elevate to install packages is a bigger security risk than we currently have, since installers (or setup.py's) do not run elevated. Once everyone "knows" that you need to elevate for this (and trust me, people already expect it), *any* package could become the delivery mechanism for malicious code that will be run with admin privileges - no exploits, just social engineering. Right now, we are actually more secure for the vast majority of single-user environments, since people aren't expecting a UAC prompt and will view it with suspicion.

> Since we are talking about humans, I'd gather most of them trying to install
> something on Windows will have heard about ProgramFiles and not be too bothered
> at its inclusion in the path.

Modifying PATH is not recommended by Microsoft (App Paths are - http://msdn.microsoft.com/en-US/ee872121.aspx), and your entire argument in support of using Program Files is that it is Microsoft's recommendation. It's only a minor inconsistency, but be aware that you are weakening your own argument here :)

"Heard about ProgramFiles" and "inclusion in the path" are two very different ideas. My grandmother knows about Program Files (though the "x86" still confuses her), but doesn't know anything about PATH. If someone installed Python for her and her "trick" of typing "news.txt" into the Run dialog* suddenly broke, I'm sure she would be unhappy.

(*She has no such trick, to my knowledge, but this is one of the side-effects of modifying PATH.)

Or if someone dropped a "sol.py" ahead of sol.exe (the user profile directory will work, provided the file is not in App Paths) and the PATHEXT change caused the Python file to be run instead of the EXE. The unintended consequences are significant, which is why I argue that python.org Python should not be installed on my grandmother's computer - there is a difference between the Python development environment and the runtime environment.


My basic argument is not that you are wrong, but that it is too late to change. If we were starting from scratch right now, the reference installers from python.org would absolutely install Python into Program Files. But it is far too late to change it for 2.7.x, and it would require a serious education campaign for 3.5 to make sure as many users as possible are not burned. Now is the time to start publicly making a noise about the risks of the current installer and how to mitigate them (the second part is important - otherwise you are just making noise).

Cheers,
Steve

From stephen at xemacs.org  Tue Apr 29 18:33:08 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 30 Apr 2014 01:33:08 +0900
Subject: [Python-Dev] Python  2.7.7. on Windows
In-Reply-To: <535F89A2.50102@mgmiller.net>
References: <535DF27E.3000108@wetafx.co.nz> <535E08D0.2020500@mgmiller.net>
 <789792ad6e7c4d63ac74e33bcc3f50c5@BLUPR03MB389.namprd03.prod.outlook.com>
 <535EB51D.4020403@mgmiller.net>
 <87ppk0n8a3.fsf@uwakimon.sk.tsukuba.ac.jp>
 <535F89A2.50102@mgmiller.net>
Message-ID: <87k3a8m6yj.fsf@uwakimon.sk.tsukuba.ac.jp>

Mike Miller writes:

 > However, this bug has been shitcanned for a decade.  This is the
 > last chance to fix this bug in a branch that's going to be
 > supported until 2020!

Probably.  I'm not convinced.  But that doesn't really matter.

Your bigger concern is the deafening silence from the senior
committers, which suggests you're going to need to come up with a
stronger argument.  Maybe there's a relevant CVE?




From martin at v.loewis.de  Tue Apr 29 20:53:11 2014
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Tue, 29 Apr 2014 20:53:11 +0200
Subject: [Python-Dev] Python 2.7.7. on Windows
In-Reply-To: <87k3a8m6yj.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <535DF27E.3000108@wetafx.co.nz> <535E08D0.2020500@mgmiller.net>
 <789792ad6e7c4d63ac74e33bcc3f50c5@BLUPR03MB389.namprd03.prod.outlook.com>
 <535EB51D.4020403@mgmiller.net> <87ppk0n8a3.fsf@uwakimon.sk.tsukuba.ac.jp>
 <535F89A2.50102@mgmiller.net> <87k3a8m6yj.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20140429205311.Horde.vtbb8rX0Vwn-pg6xN9dH9A2@webmail.df.eu>


Quoting "Stephen J. Turnbull" <stephen at xemacs.org>:

> Mike Miller writes:
>
>  > However, this bug has been shitcanned for a decade.  This is the
>  > last chance to fix this bug in a branch that's going to be
>  > supported until 2020!
>
> Probably.  I'm not convinced.  But that doesn't really matter.
>
> Your bigger concern is the deafening silence from the senior
> committers, which suggests you're going to need to come up with a
> stronger argument.  Maybe there's a relevant CVE?

*My* silence is easy to explain: I don't maintain Python 2.7 any longer.
So my view doesn't really matter (although I do support Steve's position).

In any case, I think Mike is following a lost cause. If the maintainer in
charge (i.e. Steve) doesn't want to make a specific change, the only way
to force him would be from "higher up", which could be the release manager,
the BDFL, or a PEP.

Regards,
Martin





From cf.natali at gmail.com  Tue Apr 29 22:11:37 2014
From: cf.natali at gmail.com (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=)
Date: Tue, 29 Apr 2014 21:11:37 +0100
Subject: [Python-Dev] API and process questions (sparked by Claudiu Popa
 on 16104
In-Reply-To: <CAMy=CLpi+P95F6m74_6_4We-m5o2XnG1xnPD98dX4uShBCbPTA@mail.gmail.com>
References: <CA+OGgf47B1pRNY1fL6Pi_iqn3_VanpgvmVt7Fs6GLR3aJrtKPg@mail.gmail.com>
 <CACBhJdF=pfTGdU3+94Tqb0z7CUtYxiq1s6r-+bAt9CUbEEE1qw@mail.gmail.com>
 <CAMy=CLpi+P95F6m74_6_4We-m5o2XnG1xnPD98dX4uShBCbPTA@mail.gmail.com>
Message-ID: <CAH_1eM0n-AY80r0H1D5rBu0owEZ3J5Z2TtR_G8RfbVoNNdpivw@mail.gmail.com>

2014-04-28 21:24 GMT+01:00 Claudiu Popa <pcmanticore at gmail.com>:
> [...]
>
> If anyone agrees with the above, then I'll modify the patch. This will
> be its last iteration, any other bikeshedding
> should be addressed by the core dev who'll apply it.

I'm perfectly happy with those proposals.

From python-dev at mgmiller.net  Tue Apr 29 22:21:39 2014
From: python-dev at mgmiller.net (Mike Miller)
Date: Wed, 30 Apr 2014 08:21:39 +1200
Subject: [Python-Dev] Python 2.7.7. on Windows
In-Reply-To: <7b013d3d9478440593ad1e511623ca95@BLUPR03MB389.namprd03.prod.outlook.com>
References: <535DF27E.3000108@wetafx.co.nz> <535E08D0.2020500@mgmiller.net>
 <789792ad6e7c4d63ac74e33bcc3f50c5@BLUPR03MB389.namprd03.prod.outlook.com>
 <535EB51D.4020403@mgmiller.net>
 <CAPkN8xKQZKCvRf0GopFueTLGKGzcWm1rhNSESbPoGMXZM6fyRg@mail.gmail.com>
 <535F8264.2000508@mgmiller.net>
 <7b013d3d9478440593ad1e511623ca95@BLUPR03MB389.namprd03.prod.outlook.com>
Message-ID: <536009D3.7090409@mgmiller.net>


On 04/30/2014 04:14 AM, Steve Dower wrote:
> Here are some more minuses beyond those listed on the issue:

I have to say I'm a bit baffled.  I expected disagreement, but didn't expect 
that multiple reasons against would be made up seemingly at random?  I and a 
company I work for (that distributes Py) have been installing Python to 
ProgramFiles for almost a decade, and can assure that none of those things you 
mention have yet happened.

Also, your security point contradicts industry best practice.  Mac and Linux 
require sudo to install system packages, with user packages and venv as 
alternatives.

> My basic argument is not that you are wrong, but that it is too late to change.

That's a fine argument.

The problem is that we and others like us are not able to move to Py3 (for work) 
and won't be able to for a long-enough time that it could be considered to be 
"never".

-Mike

From python-dev at mgmiller.net  Tue Apr 29 22:25:06 2014
From: python-dev at mgmiller.net (Mike Miller)
Date: Wed, 30 Apr 2014 08:25:06 +1200
Subject: [Python-Dev] Python 2.7.7. on Windows
In-Reply-To: <7b013d3d9478440593ad1e511623ca95@BLUPR03MB389.namprd03.prod.outlook.com>
References: <535DF27E.3000108@wetafx.co.nz> <535E08D0.2020500@mgmiller.net>
 <789792ad6e7c4d63ac74e33bcc3f50c5@BLUPR03MB389.namprd03.prod.outlook.com>
 <535EB51D.4020403@mgmiller.net>
 <CAPkN8xKQZKCvRf0GopFueTLGKGzcWm1rhNSESbPoGMXZM6fyRg@mail.gmail.com>
 <535F8264.2000508@mgmiller.net>
 <7b013d3d9478440593ad1e511623ca95@BLUPR03MB389.namprd03.prod.outlook.com>
Message-ID: <53600AA2.5060608@mgmiller.net>


On 04/30/2014 04:14 AM, Steve Dower wrote:
>> Since we are talking about humans, I'd gather most of them trying to install
>> something on Windows will have heard about ProgramFiles and not be too bothered
>> at its inclusion in the path.
>
> Modifying PATH is not recommended by Microsoft...

Sorry, I meant the install path in that sentence, not the environment var.

-Mike


From stefan-usenet at bytereef.org  Tue Apr 29 23:02:11 2014
From: stefan-usenet at bytereef.org (Stefan Krah)
Date: Tue, 29 Apr 2014 23:02:11 +0200
Subject: [Python-Dev] Python 2.7.7. on Windows
In-Reply-To: <536009D3.7090409@mgmiller.net>
References: <535DF27E.3000108@wetafx.co.nz> <535E08D0.2020500@mgmiller.net>
 <789792ad6e7c4d63ac74e33bcc3f50c5@BLUPR03MB389.namprd03.prod.outlook.com>
 <535EB51D.4020403@mgmiller.net>
 <CAPkN8xKQZKCvRf0GopFueTLGKGzcWm1rhNSESbPoGMXZM6fyRg@mail.gmail.com>
 <535F8264.2000508@mgmiller.net>
 <7b013d3d9478440593ad1e511623ca95@BLUPR03MB389.namprd03.prod.outlook.com>
 <536009D3.7090409@mgmiller.net>
Message-ID: <20140429210210.GA18505@sleipnir.bytereef.org>

Mike Miller <python-dev at mgmiller.net> wrote:
> I have to say I'm a bit baffled.  I expected disagreement, but
> didn't expect that multiple reasons against would be made up
> seemingly at random?  I and a company I work for (that distributes
> Py) have been installing Python to ProgramFiles for almost a decade,
> and can assure that none of those things you mention have yet
> happened.

Relax, I don't think Steve is making things up.  That said, I can confirm
what you wrote:  I've always installed Python to "Program Files" and I've
never had any issues (then again, I'm mostly using Linux).


Stefan Krah




From pmiscml at gmail.com  Wed Apr 30 03:03:52 2014
From: pmiscml at gmail.com (Paul Sokolovsky)
Date: Wed, 30 Apr 2014 04:03:52 +0300
Subject: [Python-Dev] Multiple inheritance from builtin (C) types
 [still] supported in Python3?
In-Reply-To: <CALeMXf6UsgMATSHw1asxrLfxCvamm5xTzwCT2GtOuJ0q03G+=g@mail.gmail.com>
References: <20140428204548.5eaa28e4@x34f> <20140428202458.65de5f13@fsol>
 <20140428214202.417ae99d@x34f>
 <CAP7+vJJgUp31VoDwP-+B0HPCcnLUr=W8Prh5YMsLFkZBR=xW=w@mail.gmail.com>
 <20140429022617.5dbe2ebf@x34f>
 <CALeMXf6UsgMATSHw1asxrLfxCvamm5xTzwCT2GtOuJ0q03G+=g@mail.gmail.com>
Message-ID: <20140430040352.007c8bf3@x34f>

Hello,

On Tue, 29 Apr 2014 10:47:26 -0400
PJ Eby <pje at telecommunity.com> wrote:

[]
> 
> From memory of the last time I dealt with this, the rules were that
> you could mix two classes only if their __slots__ differed from their
> common __base__ by *at most* __dict__ and/or __weakref__.  The dict
> and weakref slots are special, in that the type structure contains
> their offsets, which makes them relocatable in subclasses.  But any
> other __slots__ aren't relocatable in subclasses, because the type
> structure doesn't directly keep track of the offsets.  (The slot
> descriptors do.)
> 
> But I don't think there's anything in principle that requires this,
> it's just the implementation.  You could in theory relocate __slots__
> defined from Python code in order to make a merged subclass.  It's

Ok, so one can gather from this that __slot__'ed class can't be used as
multiple bases, as in this example:

class B1:
    __slots__ = ('a', 'b')

class B2:
    __slots__ = ('a', 'b', 'c')

class C(B2, B1):
    pass

> just that the effective "__slots__" of C code can't be moved, because
> C code is expecting to find them at specific offsets.  Therefore, if
> two types define their own struct fields, they can't be inherited
> from unless one is a subtype of the other.

Well, here it itches to ask if C++-like offsetting of subclass to base
class "this" pointer was considered, but I already feel like a child
asking all those stupid questions. One obvious answer would be that
there's little point to complicate the matter, as it's just emulation
of inheritance via enclosing, and explicit enclosing offers more
flexibility anyway (but then it's less efficient).

[]

> That is not a precise description of the algorithm, but that's the
> gist of how it works.  __base__ is a slot on the type object and is
> tracked at the C level in order to sort out layouts like this.

Thanks much for the detailed description, will serve as a good
reference when in MicroPython we'll hit actual case when CPython
accepts some inheritance patterns and uPy doesn't.


-- 
Best regards,
 Paul                          mailto:pmiscml at gmail.com

From guido at python.org  Wed Apr 30 03:38:42 2014
From: guido at python.org (Guido van Rossum)
Date: Tue, 29 Apr 2014 18:38:42 -0700
Subject: [Python-Dev] Multiple inheritance from builtin (C) types
 [still] supported in Python3?
In-Reply-To: <20140430040352.007c8bf3@x34f>
References: <20140428204548.5eaa28e4@x34f> <20140428202458.65de5f13@fsol>
 <20140428214202.417ae99d@x34f>
 <CAP7+vJJgUp31VoDwP-+B0HPCcnLUr=W8Prh5YMsLFkZBR=xW=w@mail.gmail.com>
 <20140429022617.5dbe2ebf@x34f>
 <CALeMXf6UsgMATSHw1asxrLfxCvamm5xTzwCT2GtOuJ0q03G+=g@mail.gmail.com>
 <20140430040352.007c8bf3@x34f>
Message-ID: <CAP7+vJ+1TJ=B8-oAgCtwvqGE77DFFW5w-KO6wpQzcxj5v=RSSA@mail.gmail.com>

When I redesigned and reimplemented this part of Python inheritance
(somewhere in the 2.1 - 2.3 time frame, I forget the exact timing) I was
well aware of the C++ approach and decided against it, preferring an
approach requiring less compiler assistance that was easier for C
programmers to use and understand. If as a Python programmer you want a
more general multiple inheritance, you just shouldn't use slots. As a C
programmer writing extension modules, the single-inheritance model (which
this effectively is, at that level) is much easier to deal with.


On Tue, Apr 29, 2014 at 6:03 PM, Paul Sokolovsky <pmiscml at gmail.com> wrote:

> Hello,
>
> On Tue, 29 Apr 2014 10:47:26 -0400
> PJ Eby <pje at telecommunity.com> wrote:
>
> []
> >
> > From memory of the last time I dealt with this, the rules were that
> > you could mix two classes only if their __slots__ differed from their
> > common __base__ by *at most* __dict__ and/or __weakref__.  The dict
> > and weakref slots are special, in that the type structure contains
> > their offsets, which makes them relocatable in subclasses.  But any
> > other __slots__ aren't relocatable in subclasses, because the type
> > structure doesn't directly keep track of the offsets.  (The slot
> > descriptors do.)
> >
> > But I don't think there's anything in principle that requires this,
> > it's just the implementation.  You could in theory relocate __slots__
> > defined from Python code in order to make a merged subclass.  It's
>
> Ok, so one can gather from this that __slot__'ed class can't be used as
> multiple bases, as in this example:
>
> class B1:
>     __slots__ = ('a', 'b')
>
> class B2:
>     __slots__ = ('a', 'b', 'c')
>
> class C(B2, B1):
>     pass
>
> > just that the effective "__slots__" of C code can't be moved, because
> > C code is expecting to find them at specific offsets.  Therefore, if
> > two types define their own struct fields, they can't be inherited
> > from unless one is a subtype of the other.
>
> Well, here it itches to ask if C++-like offsetting of subclass to base
> class "this" pointer was considered, but I already feel like a child
> asking all those stupid questions. One obvious answer would be that
> there's little point to complicate the matter, as it's just emulation
> of inheritance via enclosing, and explicit enclosing offers more
> flexibility anyway (but then it's less efficient).
>
> []
>
> > That is not a precise description of the algorithm, but that's the
> > gist of how it works.  __base__ is a slot on the type object and is
> > tracked at the C level in order to sort out layouts like this.
>
> Thanks much for the detailed description, will serve as a good
> reference when in MicroPython we'll hit actual case when CPython
> accepts some inheritance patterns and uPy doesn't.
>
>
> --
> Best regards,
>  Paul                          mailto:pmiscml at gmail.com
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140429/ce784a4f/attachment.html>

From Nikolaus at rath.org  Wed Apr 30 04:02:56 2014
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Tue, 29 Apr 2014 19:02:56 -0700
Subject: [Python-Dev] Aborting unit tests on first failure
Message-ID: <87eh0fy3ov.fsf@vostro.rath.org>

Hello,

I've just run the testsuite of hg tip with

./python -m test -u network,urlfetch -j 8 -G -v

and it finished with

,----
| [...]
| test_extract_dir (test.test_zipfile.TestWithDirectory) ... ok
| test_store_dir (test.test_zipfile.TestWithDirectory) ... ok
| test_different_file (test.test_zipfile.TestsWithMultipleOpens) ... ok
| test_interleaved (test.test_zipfile.TestsWithMultipleOpens) ... ok
| test_same_file (test.test_zipfile.TestsWithMultipleOpens) ... ok
| 
| ----------------------------------------------------------------------
| Ran 163 tests in 14.522s
| 
| OK (skipped=25)
| 368 tests OK.
| 2 tests failed:
|     test_decimal test_itertools
| 1 test altered the execution environment:
|     test___all__
| 17 tests skipped:
|     test_bz2 test_curses test_dbm_gnu test_devpoll test_idle
|     test_kqueue test_msilib test_ossaudiodev test_readline
|     test_startfile test_tcl test_tk test_ttk_guionly test_ttk_textonly
|     test_winreg test_winsound test_zipfile64
`----

I thought the -G option is would cause the test to stop as soon as an
error occured:

|  -G, --failfast        fail as soon as a test fails (only with -v or -W)

But it my case it seems that it actually continued to run all the other
test modules. Did I misunderstand what -G is supposed to do, or is this
a bug in the test runner?

It seems to work fine within a single test module, i.e. if I run

| ./python -m test -u network,urlfetch -j 8 -G -v test_decimal

..then execution stops right after the failed test.


Best,
-Nikolaus

--
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             ?Time flies like an arrow, fruit flies like a Banana.?

From ericsnowcurrently at gmail.com  Wed Apr 30 04:38:44 2014
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Tue, 29 Apr 2014 20:38:44 -0600
Subject: [Python-Dev] Aborting unit tests on first failure
In-Reply-To: <87eh0fy3ov.fsf@vostro.rath.org>
References: <87eh0fy3ov.fsf@vostro.rath.org>
Message-ID: <CALFfu7BgU=vFw-udYf0JaVgrC4rWSaMtJ3_YdKkH-Bq_Ad5CyA@mail.gmail.com>

On Tue, Apr 29, 2014 at 8:02 PM, Nikolaus Rath <Nikolaus at rath.org> wrote:
>
> Hello,
>
> I've just run the testsuite of hg tip with
>
> ./python -m test -u network,urlfetch -j 8 -G -v

"failfast" (from -G) is passed directly to unittest.TextTestRunner
(see test/support/__init__.py:_run_suite()).  However, that's on a
per-process basis.  When running with -j, each test (test case?) is
run in a separate process.  Unless the processes are sharing some
state (which I didn't see), any existing processes will run to
completion.  Furthermore, I don't know if the "pending" tests are
cleared out under "failfast", but it doesn't look like it.

-eric

From apolkosnik at gmail.com  Wed Apr 30 04:45:01 2014
From: apolkosnik at gmail.com (Adam Polkosnik)
Date: Tue, 29 Apr 2014 22:45:01 -0400
Subject: [Python-Dev] [issue6839] zipfile can't extract file
Message-ID: <CA+uCAST4LbhY9qFtfXq94sECAQ8F7u_UMC8Bnd14HX-X4t03zw@mail.gmail.com>

Gentlemen,

I'd like to politely ask for a second pair of eyes on [issue6839]. It's
been dragging for a very long time, and the fix is really a change from a
raise() to a debugging print.

Thanks in advance,
Adam Polkosnik
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140429/f36af5b2/attachment.html>

From jessica.mckellar at gmail.com  Wed Apr 30 04:48:00 2014
From: jessica.mckellar at gmail.com (Jessica McKellar)
Date: Tue, 29 Apr 2014 19:48:00 -0700
Subject: [Python-Dev] [issue6839] zipfile can't extract file
In-Reply-To: <CA+uCAST4LbhY9qFtfXq94sECAQ8F7u_UMC8Bnd14HX-X4t03zw@mail.gmail.com>
References: <CA+uCAST4LbhY9qFtfXq94sECAQ8F7u_UMC8Bnd14HX-X4t03zw@mail.gmail.com>
Message-ID: <CAKDZRcjv7h6GOErY-m40T4FfqwhtfyEeZCsVQHiisymD1AJc3w@mail.gmail.com>

Hi Adam,

Gentlemen,
>

Thanks for contributing to Python! But not everyone on this list is a guy.

Regards,
-Jessica
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140429/f0807337/attachment.html>

From steve at pearwood.info  Wed Apr 30 04:58:32 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 30 Apr 2014 12:58:32 +1000
Subject: [Python-Dev] [issue6839] zipfile can't extract file
In-Reply-To: <CAKDZRcjv7h6GOErY-m40T4FfqwhtfyEeZCsVQHiisymD1AJc3w@mail.gmail.com>
References: <CA+uCAST4LbhY9qFtfXq94sECAQ8F7u_UMC8Bnd14HX-X4t03zw@mail.gmail.com>
 <CAKDZRcjv7h6GOErY-m40T4FfqwhtfyEeZCsVQHiisymD1AJc3w@mail.gmail.com>
Message-ID: <20140430025831.GT4273@ando>

On Tue, Apr 29, 2014 at 07:48:00PM -0700, Jessica McKellar wrote:
> Hi Adam,
> 
> Gentlemen,
> >
> 
> Thanks for contributing to Python! But not everyone on this list is a guy.

And not all of the guys are gentlemen :-)

The term I sometimes use is "gentlefolks", or even just "folks". "Ladies 
and gentlemen" is just too old-fashioned and formal. "Folks" is nicely 
informal and all-inclusive, regardless of sex or class we're all just 
folks.


-- 
Steven

From ncoghlan at gmail.com  Wed Apr 30 06:22:26 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 30 Apr 2014 00:22:26 -0400
Subject: [Python-Dev] Multiple inheritance from builtin (C) types
 [still] supported in Python3?
In-Reply-To: <CAP7+vJ+1TJ=B8-oAgCtwvqGE77DFFW5w-KO6wpQzcxj5v=RSSA@mail.gmail.com>
References: <20140428204548.5eaa28e4@x34f> <20140428202458.65de5f13@fsol>
 <20140428214202.417ae99d@x34f>
 <CAP7+vJJgUp31VoDwP-+B0HPCcnLUr=W8Prh5YMsLFkZBR=xW=w@mail.gmail.com>
 <20140429022617.5dbe2ebf@x34f>
 <CALeMXf6UsgMATSHw1asxrLfxCvamm5xTzwCT2GtOuJ0q03G+=g@mail.gmail.com>
 <20140430040352.007c8bf3@x34f>
 <CAP7+vJ+1TJ=B8-oAgCtwvqGE77DFFW5w-KO6wpQzcxj5v=RSSA@mail.gmail.com>
Message-ID: <CADiSq7fFBDG1Ayoy8qv6FG9aBc-+eHHU55qk130S2HnB_xVM4g@mail.gmail.com>

On 29 April 2014 21:38, Guido van Rossum <guido at python.org> wrote:
> When I redesigned and reimplemented this part of Python inheritance
> (somewhere in the 2.1 - 2.3 time frame, I forget the exact timing) I was
> well aware of the C++ approach and decided against it, preferring an
> approach requiring less compiler assistance that was easier for C
> programmers to use and understand. If as a Python programmer you want a more
> general multiple inheritance, you just shouldn't use slots. As a C
> programmer writing extension modules, the single-inheritance model (which
> this effectively is, at that level) is much easier to deal with.

Also related to the "single inheritance is easier to deal with when
writing C extensions" aspect, C extension code is also far more likely
than Python code to call the base class method implementations
directly - getting hold of super() objects from C to do cooperative
multiple inheritance isn't straightforward (I actually don't know how
to do it myself - I'd have to go trawling through the C API docs to
figure it out).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Wed Apr 30 06:17:54 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 30 Apr 2014 00:17:54 -0400
Subject: [Python-Dev] Python 2.7.7. on Windows
In-Reply-To: <20140429210210.GA18505@sleipnir.bytereef.org>
References: <535DF27E.3000108@wetafx.co.nz> <535E08D0.2020500@mgmiller.net>
 <789792ad6e7c4d63ac74e33bcc3f50c5@BLUPR03MB389.namprd03.prod.outlook.com>
 <535EB51D.4020403@mgmiller.net>
 <CAPkN8xKQZKCvRf0GopFueTLGKGzcWm1rhNSESbPoGMXZM6fyRg@mail.gmail.com>
 <535F8264.2000508@mgmiller.net>
 <7b013d3d9478440593ad1e511623ca95@BLUPR03MB389.namprd03.prod.outlook.com>
 <536009D3.7090409@mgmiller.net>
 <20140429210210.GA18505@sleipnir.bytereef.org>
Message-ID: <CADiSq7faM32VPA4VJLain_Z9n7tNMPU=d3h8Th+W7x0dsV+ZLQ@mail.gmail.com>

On 29 April 2014 17:02, Stefan Krah <stefan-usenet at bytereef.org> wrote:
> Mike Miller <python-dev at mgmiller.net> wrote:
>> I have to say I'm a bit baffled.  I expected disagreement, but
>> didn't expect that multiple reasons against would be made up
>> seemingly at random?  I and a company I work for (that distributes
>> Py) have been installing Python to ProgramFiles for almost a decade,
>> and can assure that none of those things you mention have yet
>> happened.
>
> Relax, I don't think Steve is making things up.  That said, I can confirm
> what you wrote:  I've always installed Python to "Program Files" and I've
> never had any issues (then again, I'm mostly using Linux).

It's important to note that the feature backport exceptions in the
network security enhancements PEP were granted specifically because
they had security implications *beyond* the specific systems and
applications still running Python 2.7. Unfortunately, I lost some of
that rationale when I trimmed it down to the more specific proposal:

==============================
The key requirement for a feature to be considered for inclusion in this
policy is that it must have security implications *beyond* the specific
application that is written in Python and the system that application is
running on. Thus the focus on network security protocols, password storage
and related cryptographic infrastructure - Python is a popular choice for
the development of web services and clients, and thus the capabilities of
widely used Python versions have implications for the security design of
other services that may themselves be using newer versions of Python or
other development languages, but need to interoperate with clients or
servers written using older versions of Python.

The intent behind this requirement is to minimise any impact that the
introduction of this policy may have on the stability and compatibility of
maintenance releases. It would be thoroughly counterproductive if end
users became as cautious about updating to new Python 2.7 maintenance
releases as they are about updating to new feature releases within the
same release series.
==============================

I'll find a place to add that back in (not tonight, though), since
it's an important part of the reason Mike's suggested installer
changes are *not* remotely in scope for 2.7.7. However,  that's
currently not obvious to folks that have only read the final version
of the PEP, and didn't see the earlier more open ended versions that
included that text.

Cheers,
Nick.



-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From cf.natali at gmail.com  Wed Apr 30 07:26:32 2014
From: cf.natali at gmail.com (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=)
Date: Wed, 30 Apr 2014 06:26:32 +0100
Subject: [Python-Dev] [issue6839] zipfile can't extract file
In-Reply-To: <20140430025831.GT4273@ando>
References: <CA+uCAST4LbhY9qFtfXq94sECAQ8F7u_UMC8Bnd14HX-X4t03zw@mail.gmail.com>
 <CAKDZRcjv7h6GOErY-m40T4FfqwhtfyEeZCsVQHiisymD1AJc3w@mail.gmail.com>
 <20140430025831.GT4273@ando>
Message-ID: <CAH_1eM1Ma7cAKd_JPju5rRt0ZdR1cfTwwRcF28KVNKUGh_N8pg@mail.gmail.com>

2014-04-30 3:58 GMT+01:00 Steven D'Aprano <steve at pearwood.info>:
> On Tue, Apr 29, 2014 at 07:48:00PM -0700, Jessica McKellar wrote:
>> Hi Adam,
>>
>> Gentlemen,
>> >
>>
>> Thanks for contributing to Python! But not everyone on this list is a guy.
>
> And not all of the guys are gentlemen :-)

And I thought "guys" could be used to address mixed-gender groups (I'm
pretty sure I've heard some "ladies" use it in this setting), but I'm
not a native speaker.

The idea being that one should not infer too much from a salutation
from someone who might not be a native speaker (some languages default
to masculine for a mixed audience), although in this case "Ladies and
gentlemen" is really famous.

In any case, I'm sure he'd like to have his code reviewed by someone,
regardless of its gender!

From greg.ewing at canterbury.ac.nz  Wed Apr 30 08:48:59 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 30 Apr 2014 18:48:59 +1200
Subject: [Python-Dev] Multiple inheritance from builtin (C) types
 [still] supported in Python3?
In-Reply-To: <20140430040352.007c8bf3@x34f>
References: <20140428204548.5eaa28e4@x34f> <20140428202458.65de5f13@fsol>
 <20140428214202.417ae99d@x34f>
 <CAP7+vJJgUp31VoDwP-+B0HPCcnLUr=W8Prh5YMsLFkZBR=xW=w@mail.gmail.com>
 <20140429022617.5dbe2ebf@x34f>
 <CALeMXf6UsgMATSHw1asxrLfxCvamm5xTzwCT2GtOuJ0q03G+=g@mail.gmail.com>
 <20140430040352.007c8bf3@x34f>
Message-ID: <53609CDB.8010303@canterbury.ac.nz>

Paul Sokolovsky wrote:

> Well, here it itches to ask if C++-like offsetting of subclass to base
> class "this" pointer was considered,

I suppose in theory it would be possible to build a new
set of __slot__ descriptors for the subclass. It mightn't
even be all that difficult. My guess would be that it
wasn't considered worth the effort, if it was considered
at all.

-- 
Greg

From guido at python.org  Wed Apr 30 16:30:29 2014
From: guido at python.org (Guido van Rossum)
Date: Wed, 30 Apr 2014 07:30:29 -0700
Subject: [Python-Dev] Multiple inheritance from builtin (C) types
 [still] supported in Python3?
In-Reply-To: <53609CDB.8010303@canterbury.ac.nz>
References: <20140428204548.5eaa28e4@x34f> <20140428202458.65de5f13@fsol>
 <20140428214202.417ae99d@x34f>
 <CAP7+vJJgUp31VoDwP-+B0HPCcnLUr=W8Prh5YMsLFkZBR=xW=w@mail.gmail.com>
 <20140429022617.5dbe2ebf@x34f>
 <CALeMXf6UsgMATSHw1asxrLfxCvamm5xTzwCT2GtOuJ0q03G+=g@mail.gmail.com>
 <20140430040352.007c8bf3@x34f> <53609CDB.8010303@canterbury.ac.nz>
Message-ID: <CAP7+vJLtRzX8HShUNyQCYqEGCJWAzY1hdikM=bxNaxF2eQHA7w@mail.gmail.com>

It would be a disaster if the base class's slot descriptors would be broken
by that though, so the implementation of slot descriptors would have to
become more complicated. (It's worth understanding how __slots__ works. the
interpreter first finds the slot on the class and then calls its __get__
method to retrieve the actual value from the instance. Each slot
descriptor, in the current implementation, knows the offset of its slot
within the instance.


On Tue, Apr 29, 2014 at 11:48 PM, Greg Ewing <greg.ewing at canterbury.ac.nz>wrote:

> Paul Sokolovsky wrote:
>
>  Well, here it itches to ask if C++-like offsetting of subclass to base
>> class "this" pointer was considered,
>>
>
> I suppose in theory it would be possible to build a new
> set of __slot__ descriptors for the subclass. It mightn't
> even be all that difficult. My guess would be that it
> wasn't considered worth the effort, if it was considered
> at all.
>
> --
> Greg
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140430/0dd80954/attachment.html>

From pcmanticore at gmail.com  Wed Apr 30 11:21:31 2014
From: pcmanticore at gmail.com (Claudiu Popa)
Date: Wed, 30 Apr 2014 12:21:31 +0300
Subject: [Python-Dev] API and process questions (sparked by Claudiu Popa
 on 16104
In-Reply-To: <CAH_1eM0n-AY80r0H1D5rBu0owEZ3J5Z2TtR_G8RfbVoNNdpivw@mail.gmail.com>
References: <CA+OGgf47B1pRNY1fL6Pi_iqn3_VanpgvmVt7Fs6GLR3aJrtKPg@mail.gmail.com>
 <CACBhJdF=pfTGdU3+94Tqb0z7CUtYxiq1s6r-+bAt9CUbEEE1qw@mail.gmail.com>
 <CAMy=CLpi+P95F6m74_6_4We-m5o2XnG1xnPD98dX4uShBCbPTA@mail.gmail.com>
 <CAH_1eM0n-AY80r0H1D5rBu0owEZ3J5Z2TtR_G8RfbVoNNdpivw@mail.gmail.com>
Message-ID: <CAMy=CLoiuQoQLVfKPGwfVNTS0V5GbGVzp+W1c-XZzYHMX1sphw@mail.gmail.com>

On Tue, Apr 29, 2014 at 11:11 PM, Charles-Fran?ois Natali
<cf.natali at gmail.com> wrote:
> 2014-04-28 21:24 GMT+01:00 Claudiu Popa <pcmanticore at gmail.com>:
>> [...]
>>
>> If anyone agrees with the above, then I'll modify the patch. This will
>> be its last iteration, any other bikeshedding
>> should be addressed by the core dev who'll apply it.
>
> I'm perfectly happy with those proposals.


I've just updated the issue with the latest patch. If any core dev
could pick it up
and commit it, I'll be thankful.

From francismb at email.de  Wed Apr 30 20:11:49 2014
From: francismb at email.de (francis)
Date: Wed, 30 Apr 2014 20:11:49 +0200
Subject: [Python-Dev] [issue6839] zipfile can't extract file
In-Reply-To: <CAKDZRcjv7h6GOErY-m40T4FfqwhtfyEeZCsVQHiisymD1AJc3w@mail.gmail.com>
References: <CA+uCAST4LbhY9qFtfXq94sECAQ8F7u_UMC8Bnd14HX-X4t03zw@mail.gmail.com>
 <CAKDZRcjv7h6GOErY-m40T4FfqwhtfyEeZCsVQHiisymD1AJc3w@mail.gmail.com>
Message-ID: <53613CE5.8000204@email.de>

On 04/30/2014 04:45 AM, Adam Polkosnik wrote:
 > Gentlemen,
 >

On 04/30/2014 04:48 AM, Jessica McKellar wrote:
> Hi Adam,
>
> Gentlemen,
>>
>
> Thanks for contributing to Python! But not everyone on this list is a guy.
>

That was a fast reply (< 2min -> Ain't you playing with bots
searching for the "Gentlemen" word?). Just a joke ;-)

Regards,
francis




From njs at pobox.com  Wed Apr 30 20:46:46 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Wed, 30 Apr 2014 19:46:46 +0100
Subject: [Python-Dev] [issue6839] zipfile can't extract file
In-Reply-To: <53613CE5.8000204@email.de>
References: <CA+uCAST4LbhY9qFtfXq94sECAQ8F7u_UMC8Bnd14HX-X4t03zw@mail.gmail.com>
 <CAKDZRcjv7h6GOErY-m40T4FfqwhtfyEeZCsVQHiisymD1AJc3w@mail.gmail.com>
 <53613CE5.8000204@email.de>
Message-ID: <CAPJVwBnPJzF1ViNCMpJibJykj3ZZperqa44R5zPQy1-SbQA2jw@mail.gmail.com>

On 30 Apr 2014 19:17, "francis" <francismb at email.de> wrote:
>
> On 04/30/2014 04:45 AM, Adam Polkosnik wrote:
> > Gentlemen,
>
> >
>
> On 04/30/2014 04:48 AM, Jessica McKellar wrote:
>>
>> Hi Adam,
>>
>> Gentlemen,
>>>
>>>
>>
>> Thanks for contributing to Python! But not everyone on this list is a
guy.
>>
>
> That was a fast reply (< 2min -> Ain't you playing with bots
> searching for the "Gentlemen" word?). Just a joke ;-)

When people are defending their right to be included is probably not a
great time to make jokes about them not being human. Tacking a smiley face
on the end just gives the impression that you realize that your words will
come off as aggressive and are trying to preemptively make sure that if
anyone tries to hold you responsible for this then you can claim that it's
their fault for not having a sense of humor. Trying to further defend this
position is just going to dig you in deeper, so let's drop the discussion
here.

-n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140430/06db8fb5/attachment.html>

From francismb at email.de  Wed Apr 30 21:27:17 2014
From: francismb at email.de (francis)
Date: Wed, 30 Apr 2014 21:27:17 +0200
Subject: [Python-Dev] [issue6839] zipfile can't extract file
In-Reply-To: <CAPJVwBnPJzF1ViNCMpJibJykj3ZZperqa44R5zPQy1-SbQA2jw@mail.gmail.com>
References: <CA+uCAST4LbhY9qFtfXq94sECAQ8F7u_UMC8Bnd14HX-X4t03zw@mail.gmail.com>	<CAKDZRcjv7h6GOErY-m40T4FfqwhtfyEeZCsVQHiisymD1AJc3w@mail.gmail.com>	<53613CE5.8000204@email.de>
 <CAPJVwBnPJzF1ViNCMpJibJykj3ZZperqa44R5zPQy1-SbQA2jw@mail.gmail.com>
Message-ID: <53614E95.5050701@email.de>

Hi Nathaniel,

> their fault for not having a sense of humor. Trying to further defend this
> position is just going to dig you in deeper, so let's drop the discussion
> here.

You're right, the formulation wasn't correct. My excuses. I just better 
do the review. Thanks.

Regards,
francis