From greg.ewing at canterbury.ac.nz  Sat Sep  1 02:03:06 2012
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 01 Sep 2012 12:03:06 +1200
Subject: [Python-ideas] Unpack of sequences
In-Reply-To: <50400D71.7000705@pearwood.info>
References: <CAO6hKot1zYo+0XhcJu+c5u_JyjuczEpn11N1+YRr_jAJ_JWPOQ@mail.gmail.com>
	<503E1FCA.7050309@nedbatchelder.com>
	<CAP7+vJLyO0=geteHCpmuYPVUfGu6FeXFEb57p88i+-swb=R-fw@mail.gmail.com>
	<CAEBZo3NMiGMOmi+d=3A3rER1Y3YdLxQJE3n5B2t+Cw9+6bDXQA@mail.gmail.com>
	<CAP7+vJLUDdKJMDVOhOXBn1tL-+hO0wj99XdZwKanVjGLY5jE6Q@mail.gmail.com>
	<503ECF2E.6060400@canterbury.ac.nz> <k1o1su$nj8$1@ger.gmane.org>
	<CAEBZo3MphcModANcEJPshCP=AHvtne8-FgR5hU8pJOuJRDca8g@mail.gmail.com>
	<5040027C.2000001@canterbury.ac.nz> <50400D71.7000705@pearwood.info>
Message-ID: <504150BA.7020004@canterbury.ac.nz>

Steven D'Aprano wrote:

> For sequences:
> 
> a, b, c, d = really_long_sequence[:4]

Sure, there are any number of ways of doing it, but they
all involve some degree of convolution or DRY-violation.

The most direct and obvious way is denied to us, because there
is no way to tell the unpacking operation *not* to check for
the end of the sequence afterwards. Because that unwanted
check is forced on us, we have to perform even more useless
work to get around it. That seems perverse to me.

-- 
Greg



From greg at krypto.org  Sat Sep  1 02:03:54 2012
From: greg at krypto.org (Gregory P. Smith)
Date: Fri, 31 Aug 2012 17:03:54 -0700
Subject: [Python-ideas] Add a "hasmethod()" builtin?
In-Reply-To: <20120830042825.GA13314@ando>
References: <CAP7+vJJ1dVpGCyWuFw+oEYrmvSvYokYnwwPC4u3JWqAk=+5Z2g@mail.gmail.com>
	<20120830042825.GA13314@ando>
Message-ID: <CAGE7PNLjAu5L1fNA_KhKGKGeby8zPapFXA3p2Hjc9vM62Qvbbg@mail.gmail.com>

On Wed, Aug 29, 2012 at 9:28 PM, Steven D'Aprano <steve at pearwood.info>wrote:

> On Wed, Aug 29, 2012 at 03:48:01PM -0700, Guido van Rossum wrote:
> > There's a concept that's sometimes useful when explaining behavior of
> > certain Python operations in terms of simpler ones, and it is "does
> > the class of x define a method m?".
>
> It's not just methods where this is useful. For example, the help()
> quasi-builtin ignores instance attribute x.__doc__ and instead uses
> type(x).__doc__.
>
> I'm not sure that needing this is common enough to justify builtins, but
> I think it would be useful to have hastypeattr and friends (get*, set*
> and del*) in the operator module.
>

+1.

I wouldn't call it hasmethod as described either as that name implies to
most readers the much simpler "it has a callable attribute with this name"
check rather than getting into the differences between an arbitrary
callable attribute, function, or an actual method on the type which most
people do not need to know about (*).

-gps

(*) unless mocking, i ran into a test failure because for 2.6 it was
stubbing out a __special__ method with a mock no longer worked in 2.7. but
that test was better all around when refactored to not need to be that
crazy. :)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120831/b80fe779/attachment.html>

From mikegraham at gmail.com  Sat Sep  1 02:35:56 2012
From: mikegraham at gmail.com (Mike Graham)
Date: Fri, 31 Aug 2012 20:35:56 -0400
Subject: [Python-ideas] Verbose traceback formatting
In-Reply-To: <k1m1b5$ota$1@ger.gmane.org>
References: <CAEBZo3MeWvWzGEPeAEh3zB2V3g7gC1b1rPUJd8zVxFDj8ePuyQ@mail.gmail.com>
	<k1m1b5$ota$1@ger.gmane.org>
Message-ID: <CAEBZo3N4h7fMiho-aKkX5PBeVHwZSoMUATvD-bSnL=TVx5veAw@mail.gmail.com>

On Wed, Aug 29, 2012 at 5:24 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> Part of the problem is in the overly skimpy exception instances
> themselves. They should contain the needed runtime info that one
> cannot find in the code. I would rather you push for more such
> changes.
>
>>> ...
>>>          x = 16
>>>          y = 0
>>> ...
>
>>>ZeroDivisionError: integer division or modulo by zero
>
> This could and, imo, should be changed to include the numerator,
> which is the main extra info included the the verbose traceback.
> Most of the rest strikes me as noise.

I think you have read the example too narrowly. Having the locals of
each frame is _very_ useful--just because I know the terms of the
division does not mean that I can easily connect that information back
five calls ago when I passed the wrong thing. Currently you have to
spin up the debugger (or insert print statements or similar) to get
this information, which is more work than it is to read it and
requires another run of your code (and getting to the same point could
be expensive or tricky).

Mike


From techtonik at gmail.com  Sat Sep  1 08:27:25 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Sat, 1 Sep 2012 09:27:25 +0300
Subject: [Python-ideas] itertools.chunks(iterable, size, fill=None)
In-Reply-To: <jsl3cb$knn$1@dough.gmane.org>
References: <CAPkN8xKq7sfWQt9P2Vaw6B_ZwqKtNLiGdavPT=Y84yBcV08uEg@mail.gmail.com>
	<jsl3cb$knn$1@dough.gmane.org>
Message-ID: <CAPkN8xKM_K=59-Q9jfso+uGLnF=b_zrjvmCzNjEoOcGg+4P0ZQ@mail.gmail.com>

I've run into the necessity of implementing chunks() again. Here is
the code I've made from scratch.

def chunks(seq, size):
  '''Cut sequence into chunks of given size. If `seq` length is
     not divisible by `size` without reminder, last chunk will
     have length less than size.

     >>> list( chunks([1,2,3,4,5,6,7], 3) )
     [[1, 2, 3], [4, 5, 6], [7]]
  '''
  endlen = len(seq)//size
  for i in range(endlen):
    yield [seq[i*size+n] for n in range(size)]
  if len(seq) % size:
    yield seq[endlen*size:]

--
anatoly t.


On Fri, Jun 29, 2012 at 11:32 PM, Georg Brandl <g.brandl at gmx.net> wrote:
> On 26.06.2012 10:03, anatoly techtonik wrote:
>>
>> Now that Python 3 is all about iterators (which is a user killer
>> feature for Python according to StackOverflow -
>> http://stackoverflow.com/questions/tagged/python) would it be nice to
>> introduce more first class functions to work with them? One function
>> to be exact to split string into chunks.
>>
>>      itertools.chunks(iterable, size, fill=None)
>>
>> Which is the 33th most voted Python question on SO -
>>
>> http://stackoverflow.com/questions/312443/how-do-you-split-a-list-into-evenly-sized-chunks-in-python/312464
>>
>> P.S. CC'ing to python-dev@ to notify about the thread in python-ideas.
>>
>
> Anatoly, so far there were no negative votes -- would you care to go
> another step and propose a patch?
>
>
> Georg
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas


From techtonik at gmail.com  Sat Sep  1 08:29:37 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Sat, 1 Sep 2012 09:29:37 +0300
Subject: [Python-ideas] sequence.apply(function)
Message-ID: <CAPkN8xJK2jq59HOU6du4KozMJzONBJ6erApnpdY0_x4A1asPSg@mail.gmail.com>

Idea: Apply function to every element of a sequence and return new
sequence. It's more pythonic than map(), because clearly works only as
a list method.
--
anatoly t.


From steve at pearwood.info  Sat Sep  1 08:56:21 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 01 Sep 2012 16:56:21 +1000
Subject: [Python-ideas] sequence.apply(function)
In-Reply-To: <CAPkN8xJK2jq59HOU6du4KozMJzONBJ6erApnpdY0_x4A1asPSg@mail.gmail.com>
References: <CAPkN8xJK2jq59HOU6du4KozMJzONBJ6erApnpdY0_x4A1asPSg@mail.gmail.com>
Message-ID: <5041B195.7070708@pearwood.info>

On 01/09/12 16:29, anatoly techtonik wrote:
> Idea: Apply function to every element of a sequence and return new
> sequence. It's more pythonic than map(), because clearly works only as
> a list method.

I think you mean "less pythonic".

-1

We already have map, and it works lazily on any iterable. Why do we
need something less efficient and more limited?


-- 
Steven


From ned at nedbatchelder.com  Sat Sep  1 14:00:21 2012
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Sat, 01 Sep 2012 08:00:21 -0400
Subject: [Python-ideas] sequence.apply(function)
In-Reply-To: <CAPkN8xJK2jq59HOU6du4KozMJzONBJ6erApnpdY0_x4A1asPSg@mail.gmail.com>
References: <CAPkN8xJK2jq59HOU6du4KozMJzONBJ6erApnpdY0_x4A1asPSg@mail.gmail.com>
Message-ID: <5041F8D5.6040904@nedbatchelder.com>

On 9/1/2012 2:29 AM, anatoly techtonik wrote:
> Idea: Apply function to every element of a sequence and return new
> sequence. It's more pythonic than map(), because clearly works only as
> a list method.

Python 2 has itertools.imap, and Python 3 has map, both of which do 
exactly what you want.

--Ned.
> --
> anatoly t.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



From miki.tebeka at gmail.com  Sat Sep  1 16:42:19 2012
From: miki.tebeka at gmail.com (Miki Tebeka)
Date: Sat, 1 Sep 2012 07:42:19 -0700 (PDT)
Subject: [Python-ideas] itertools.chunks(iterable, size, fill=None)
In-Reply-To: <CAPkN8xKM_K=59-Q9jfso+uGLnF=b_zrjvmCzNjEoOcGg+4P0ZQ@mail.gmail.com>
References: <CAPkN8xKq7sfWQt9P2Vaw6B_ZwqKtNLiGdavPT=Y84yBcV08uEg@mail.gmail.com>
	<jsl3cb$knn$1@dough.gmane.org>
	<CAPkN8xKM_K=59-Q9jfso+uGLnF=b_zrjvmCzNjEoOcGg+4P0ZQ@mail.gmail.com>
Message-ID: <d1c0db48-cf7b-4d7f-bb8a-fe2f950c4a6c@googlegroups.com>

See the "grouper" example in http://docs.python.org/library/itertools.html

On Friday, August 31, 2012 11:28:33 PM UTC-7, anatoly techtonik wrote:
>
> I've run into the necessity of implementing chunks() again. Here is 
> the code I've made from scratch. 
>
> def chunks(seq, size): 
>   '''Cut sequence into chunks of given size. If `seq` length is 
>      not divisible by `size` without reminder, last chunk will 
>      have length less than size. 
>
>      >>> list( chunks([1,2,3,4,5,6,7], 3) ) 
>      [[1, 2, 3], [4, 5, 6], [7]] 
>   ''' 
>   endlen = len(seq)//size 
>   for i in range(endlen): 
>     yield [seq[i*size+n] for n in range(size)] 
>   if len(seq) % size: 
>     yield seq[endlen*size:] 
>
> -- 
> anatoly t. 
>
>
> On Fri, Jun 29, 2012 at 11:32 PM, Georg Brandl <g.br... at gmx.net<javascript:>> 
> wrote: 
> > On 26.06.2012 10:03, anatoly techtonik wrote: 
> >> 
> >> Now that Python 3 is all about iterators (which is a user killer 
> >> feature for Python according to StackOverflow - 
> >> http://stackoverflow.com/questions/tagged/python) would it be nice to 
> >> introduce more first class functions to work with them? One function 
> >> to be exact to split string into chunks. 
> >> 
> >>      itertools.chunks(iterable, size, fill=None) 
> >> 
> >> Which is the 33th most voted Python question on SO - 
> >> 
> >> 
> http://stackoverflow.com/questions/312443/how-do-you-split-a-list-into-evenly-sized-chunks-in-python/312464 
> >> 
> >> P.S. CC'ing to python-dev@ to notify about the thread in python-ideas. 
> >> 
> > 
> > Anatoly, so far there were no negative votes -- would you care to go 
> > another step and propose a patch? 
> > 
> > 
> > Georg 
> > 
> > _______________________________________________ 
> > Python-ideas mailing list 
> > Python... at python.org <javascript:> 
> > http://mail.python.org/mailman/listinfo/python-ideas 
> _______________________________________________ 
> Python-ideas mailing list 
> Python... at python.org <javascript:> 
> http://mail.python.org/mailman/listinfo/python-ideas 
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120901/c9cd341e/attachment.html>

From python at mrabarnett.plus.com  Sat Sep  1 18:39:50 2012
From: python at mrabarnett.plus.com (MRAB)
Date: Sat, 01 Sep 2012 17:39:50 +0100
Subject: [Python-ideas] itertools.chunks(iterable, size, fill=None)
In-Reply-To: <CAPkN8xKM_K=59-Q9jfso+uGLnF=b_zrjvmCzNjEoOcGg+4P0ZQ@mail.gmail.com>
References: <CAPkN8xKq7sfWQt9P2Vaw6B_ZwqKtNLiGdavPT=Y84yBcV08uEg@mail.gmail.com>
	<jsl3cb$knn$1@dough.gmane.org>
	<CAPkN8xKM_K=59-Q9jfso+uGLnF=b_zrjvmCzNjEoOcGg+4P0ZQ@mail.gmail.com>
Message-ID: <50423A56.5070903@mrabarnett.plus.com>

On 01/09/2012 07:27, anatoly techtonik wrote:
> I've run into the necessity of implementing chunks() again. Here is
> the code I've made from scratch.
>
> def chunks(seq, size):
>    '''Cut sequence into chunks of given size. If `seq` length is
>       not divisible by `size` without reminder, last chunk will
>       have length less than size.
>
>       >>> list( chunks([1,2,3,4,5,6,7], 3) )
>       [[1, 2, 3], [4, 5, 6], [7]]
>    '''
>    endlen = len(seq)//size
>    for i in range(endlen):
>      yield [seq[i*size+n] for n in range(size)]
>    if len(seq) % size:
>      yield seq[endlen*size:]
>
Here's a lazy version:

def chunks(seq, size):
     '''Cut sequence into chunks of given size. If `seq` length is
        not divisible by `size` without reminder, last chunk will
        have length less than size.

        >>> list( chunks([1,2,3,4,5,6,7], 3) )
        [[1, 2, 3], [4, 5, 6], [7]]
     '''
     if size < 1:
         raise ValueError("chunk size less than 1")

     it = iter(seq)

     try:
         while True:
             chunk = []

             for _ in range(size):
                 chunk.append(next(it))

             yield chunk
     except StopIteration:
         if chunk:
             yield chunk



From guido at python.org  Sat Sep  1 19:06:48 2012
From: guido at python.org (Guido van Rossum)
Date: Sat, 1 Sep 2012 19:06:48 +0200
Subject: [Python-ideas] sequence.apply(function)
In-Reply-To: <CAPkN8xJK2jq59HOU6du4KozMJzONBJ6erApnpdY0_x4A1asPSg@mail.gmail.com>
References: <CAPkN8xJK2jq59HOU6du4KozMJzONBJ6erApnpdY0_x4A1asPSg@mail.gmail.com>
Message-ID: <CAP7+vJLwPe3=iNhBtTx7mdSi+pa-TSCUNDHuF0SM6q6KXU2=Ew@mail.gmail.com>

On Sat, Sep 1, 2012 at 8:29 AM, anatoly techtonik <techtonik at gmail.com> wrote:
> Idea: Apply function to every element of a sequence and return new
> sequence. It's more pythonic than map(), because clearly works only as
> a list method.

It's less Pythonic, because every sequence-like type (not just list)
would have to reimplement it.

Similar things get proposed for iterators (e.g. it1 + it2, it[:n],
it[n:]) regularly and they are (and should be) rejected for the same
reason.

-- 
--Guido van Rossum (python.org/~guido)


From michelelacchia at gmail.com  Sat Sep  1 19:16:36 2012
From: michelelacchia at gmail.com (Michele Lacchia)
Date: Sat, 1 Sep 2012 19:16:36 +0200
Subject: [Python-ideas] itertools.chunks(iterable, size, fill=None)
In-Reply-To: <50423A56.5070903@mrabarnett.plus.com>
References: <CAPkN8xKq7sfWQt9P2Vaw6B_ZwqKtNLiGdavPT=Y84yBcV08uEg@mail.gmail.com>
	<jsl3cb$knn$1@dough.gmane.org>
	<CAPkN8xKM_K=59-Q9jfso+uGLnF=b_zrjvmCzNjEoOcGg+4P0ZQ@mail.gmail.com>
	<50423A56.5070903@mrabarnett.plus.com>
Message-ID: <CAFjP7=XxNokYsuxD9AAsVuozKzH4wVRLyuLVZmnDtZ_mj0QyBA@mail.gmail.com>

+ 1 for the lazy version. Why not using itertools.islice instead of the
innermost for loop?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120901/8a083503/attachment.html>

From python at mrabarnett.plus.com  Sat Sep  1 21:02:20 2012
From: python at mrabarnett.plus.com (MRAB)
Date: Sat, 01 Sep 2012 20:02:20 +0100
Subject: [Python-ideas] itertools.chunks(iterable, size, fill=None)
In-Reply-To: <CAFjP7=XxNokYsuxD9AAsVuozKzH4wVRLyuLVZmnDtZ_mj0QyBA@mail.gmail.com>
References: <CAPkN8xKq7sfWQt9P2Vaw6B_ZwqKtNLiGdavPT=Y84yBcV08uEg@mail.gmail.com>
	<jsl3cb$knn$1@dough.gmane.org>
	<CAPkN8xKM_K=59-Q9jfso+uGLnF=b_zrjvmCzNjEoOcGg+4P0ZQ@mail.gmail.com>
	<50423A56.5070903@mrabarnett.plus.com>
	<CAFjP7=XxNokYsuxD9AAsVuozKzH4wVRLyuLVZmnDtZ_mj0QyBA@mail.gmail.com>
Message-ID: <50425BBC.8060409@mrabarnett.plus.com>

On 01/09/2012 18:16, Michele Lacchia wrote:
> + 1 for the lazy version. Why not using itertools.islice instead of the
> innermost for loop?
>
OK, here's a lazy version using islice:


from itertools import islice

def chunks(seq, size):
     '''Cut sequence into chunks of given size. If `seq` length is
        not divisible by `size` without reminder, last chunk will
        have length less than size.

        >>> list( chunks([1,2,3,4,5,6,7], 3) )
        [[1, 2, 3], [4, 5, 6], [7]]
     '''
     if size < 1:
         raise ValueError("chunk size less than 1")

     it = iter(seq)

     while True:
         chunk = list(islice(it, 0, size))
         if not chunk:
             break

         yield chunk



From ubershmekel at gmail.com  Sat Sep  1 23:55:39 2012
From: ubershmekel at gmail.com (Yuval Greenfield)
Date: Sun, 2 Sep 2012 00:55:39 +0300
Subject: [Python-ideas] sequence.apply(function)
In-Reply-To: <CAP7+vJLwPe3=iNhBtTx7mdSi+pa-TSCUNDHuF0SM6q6KXU2=Ew@mail.gmail.com>
References: <CAPkN8xJK2jq59HOU6du4KozMJzONBJ6erApnpdY0_x4A1asPSg@mail.gmail.com>
	<CAP7+vJLwPe3=iNhBtTx7mdSi+pa-TSCUNDHuF0SM6q6KXU2=Ew@mail.gmail.com>
Message-ID: <CANSw7KwTcspEArSH_38ANuzYFeR_DAbWWG5FWXt3EQjP0HBbkw@mail.gmail.com>

On Sat, Sep 1, 2012 at 8:06 PM, Guido van Rossum <guido at python.org> wrote:

> It's less Pythonic, because every sequence-like type (not just list)
> would have to reimplement it.
>
> Similar things get proposed for iterators (e.g. it1 + it2, it[:n],
> it[n:]) regularly and they are (and should be) rejected for the same
> reason.
>
>
Python causes some confusion because some things are methods and others
builtins. Is there a PEP or rationale that defines what goes where?


Yuval Greenfield
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120902/30c4122e/attachment.html>

From solipsis at pitrou.net  Sun Sep  2 00:02:11 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 2 Sep 2012 00:02:11 +0200
Subject: [Python-ideas] sequence.apply(function)
References: <CAPkN8xJK2jq59HOU6du4KozMJzONBJ6erApnpdY0_x4A1asPSg@mail.gmail.com>
	<CAP7+vJLwPe3=iNhBtTx7mdSi+pa-TSCUNDHuF0SM6q6KXU2=Ew@mail.gmail.com>
	<CANSw7KwTcspEArSH_38ANuzYFeR_DAbWWG5FWXt3EQjP0HBbkw@mail.gmail.com>
Message-ID: <20120902000211.1ae5ae0b@pitrou.net>

On Sun, 2 Sep 2012 00:55:39 +0300
Yuval Greenfield <ubershmekel at gmail.com>
wrote:
> On Sat, Sep 1, 2012 at 8:06 PM, Guido van Rossum <guido at python.org> wrote:
> 
> > It's less Pythonic, because every sequence-like type (not just list)
> > would have to reimplement it.
> >
> > Similar things get proposed for iterators (e.g. it1 + it2, it[:n],
> > it[n:]) regularly and they are (and should be) rejected for the same
> > reason.
> >
> >
> Python causes some confusion because some things are methods and others
> builtins. Is there a PEP or rationale that defines what goes where?

When something only applies to a single type or a couple of types, it is
a method. When it is generic enough, it is a builtin.
Of course there are grey areas but that's the basic idea.

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net




From ncoghlan at gmail.com  Sun Sep  2 04:14:50 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 2 Sep 2012 12:14:50 +1000
Subject: [Python-ideas] sequence.apply(function)
In-Reply-To: <20120902000211.1ae5ae0b@pitrou.net>
References: <CAPkN8xJK2jq59HOU6du4KozMJzONBJ6erApnpdY0_x4A1asPSg@mail.gmail.com>
	<CAP7+vJLwPe3=iNhBtTx7mdSi+pa-TSCUNDHuF0SM6q6KXU2=Ew@mail.gmail.com>
	<CANSw7KwTcspEArSH_38ANuzYFeR_DAbWWG5FWXt3EQjP0HBbkw@mail.gmail.com>
	<20120902000211.1ae5ae0b@pitrou.net>
Message-ID: <CADiSq7cNCEoQZkEauN7NSxUS4NZ=3uK=waX6toUfMsk9EZyQYA@mail.gmail.com>

On Sun, Sep 2, 2012 at 8:02 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Sun, 2 Sep 2012 00:55:39 +0300
> Yuval Greenfield <ubershmekel at gmail.com>
> wrote:
>> On Sat, Sep 1, 2012 at 8:06 PM, Guido van Rossum <guido at python.org> wrote:
>>
>> > It's less Pythonic, because every sequence-like type (not just list)
>> > would have to reimplement it.
>> >
>> > Similar things get proposed for iterators (e.g. it1 + it2, it[:n],
>> > it[n:]) regularly and they are (and should be) rejected for the same
>> > reason.
>> >
>> >
>> Python causes some confusion because some things are methods and others
>> builtins. Is there a PEP or rationale that defines what goes where?
>
> When something only applies to a single type or a couple of types, it is
> a method. When it is generic enough, it is a builtin.
> Of course there are grey areas but that's the basic idea.

Yes, it comes down to the fact that we are *very* reluctant to impose
required base classes (I believe the only ones currently enforced
anywhere are object, BaseException and str - everything else should
fall back to a protocol method, ABC or interface specific registration
mechanism. Most interfaces that used to require actual integer objects
are now using operator.index, or one of its C API equivalents).

In Python, we also actively discourage "reopening" classes to add new
methods (this is mostly a cultural thing, though - the language
doesn't actually contain any mechanism to stop you by default,
although it's possible to add such enforcement via metaclasses)

Thus, protocols are born which define "has this behaviour", rather
than "is one of these". That's why we have the len() builtin and
associated __len__() protocol to say "taking the length of this object
is a meaningful operation" rather than mandatory inheritance from a
Container class that has a ".len()" method.

They're most obviously beneficial when there are *multiple* protocols
that can be used to implement a particular behaviour. For example,
with iter(), the __iter__ protocol is only the first option tried. If
that fails, then it will instead check for __getitem__ and if that
exists, return a standard sequence iterator instead. Similarly,
reversed() checks for __reversed__ first, and then checks for __len__
and __getitem__, producing a reverse sequence iterator in the latter
case.

Similarly, next() was moved from a standard method to a builtin
function in 3.x? Why? Mainly to add the "if not found, return this
default value" behaviour. That kind of thing is much easier to add
when the object is only handling a piece of the behaviour, with
additional standard mechanisms around it (in this case, optionally
returning a default value when StopIteration is thrown by the
iterator).

Generators are another good illustration of the principle: For iter()
and next(), they follow the standard protocol and rely on the
corresponding builtins. However, g.send() and g.throw() require deep
integration with the interpreter's eval loop. There's currently no way
to implement either of those behaviours as an ordinary type, thus
they're exposed as ordinary methods, since they're genuinely generator
specific.

As to *why* this is a good thing: procedural APIs encourage low
coupling. Yes, object oriented programming is a good way to scale an
application architecture up to more complicated problems. The issue is
with fetishising OOP to the point where you disallow the creation of
procedural APIs that hide the OOP details. That approach sets a
minimum floor to the complexity of your implementations, as even if
you don't *need* the power of OOP, you're forced to deal with it
because the language doesn't offer anything else, and that way lies
Java. There's a reason Java is significantly more popular on large
enterprise projects than it is in small teams - it takes a certain,
rather high, level of complexity for the reasons behind any of that
boilerplate to start to become clear :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia


From ubershmekel at gmail.com  Sun Sep  2 10:50:07 2012
From: ubershmekel at gmail.com (Yuval Greenfield)
Date: Sun, 2 Sep 2012 11:50:07 +0300
Subject: [Python-ideas] sequence.apply(function)
In-Reply-To: <CADiSq7cNCEoQZkEauN7NSxUS4NZ=3uK=waX6toUfMsk9EZyQYA@mail.gmail.com>
References: <CAPkN8xJK2jq59HOU6du4KozMJzONBJ6erApnpdY0_x4A1asPSg@mail.gmail.com>
	<CAP7+vJLwPe3=iNhBtTx7mdSi+pa-TSCUNDHuF0SM6q6KXU2=Ew@mail.gmail.com>
	<CANSw7KwTcspEArSH_38ANuzYFeR_DAbWWG5FWXt3EQjP0HBbkw@mail.gmail.com>
	<20120902000211.1ae5ae0b@pitrou.net>
	<CADiSq7cNCEoQZkEauN7NSxUS4NZ=3uK=waX6toUfMsk9EZyQYA@mail.gmail.com>
Message-ID: <CANSw7KyaNwM9J0iyRASOSRWhejjUDxBB26MX1EUsb4FUZCvS+g@mail.gmail.com>

On Sun, Sep 2, 2012 at 5:14 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Sun, Sep 2, 2012 at 8:02 AM, Antoine Pitrou <solipsis at pitrou.net>
> wrote:
> > On Sun, 2 Sep 2012 00:55:39 +0300
> > Yuval Greenfield <ubershmekel at gmail.com>
> > wrote:
> >> On Sat, Sep 1, 2012 at 8:06 PM, Guido van Rossum <guido at python.org>
> wrote:
> >>
> >> > It's less Pythonic, because every sequence-like type (not just list)
> >> > would have to reimplement it.
> >> >
> >> > Similar things get proposed for iterators (e.g. it1 + it2, it[:n],
> >> > it[n:]) regularly and they are (and should be) rejected for the same
> >> > reason.
> >> >
> >> >
> >> Python causes some confusion because some things are methods and others
> >> builtins. Is there a PEP or rationale that defines what goes where?
> >
> > When something only applies to a single type or a couple of types, it is
> > a method. When it is generic enough, it is a builtin.
> > Of course there are grey areas but that's the basic idea.
>
> Yes, it comes down to the fact that we are *very* reluctant to impose
> required base classes (I believe the only ones currently enforced
> anywhere are object, BaseException and str - everything else should
> fall back to a protocol method, ABC or interface specific registration
> mechanism. Most interfaces that used to require actual integer objects
> are now using operator.index, or one of its C API equivalents).
>
> In Python, we also actively discourage "reopening" classes to add new
> methods (this is mostly a cultural thing, though - the language
> doesn't actually contain any mechanism to stop you by default,
> although it's possible to add such enforcement via metaclasses)
>
> Thus, protocols are born which define "has this behaviour", rather
> than "is one of these". That's why we have the len() builtin and
> associated __len__() protocol to say "taking the length of this object
> is a meaningful operation" rather than mandatory inheritance from a
> Container class that has a ".len()" method.
>
> They're most obviously beneficial when there are *multiple* protocols
> that can be used to implement a particular behaviour. For example,
> with iter(), the __iter__ protocol is only the first option tried. If
> that fails, then it will instead check for __getitem__ and if that
> exists, return a standard sequence iterator instead. Similarly,
> reversed() checks for __reversed__ first, and then checks for __len__
> and __getitem__, producing a reverse sequence iterator in the latter
> case.
>
> Similarly, next() was moved from a standard method to a builtin
> function in 3.x? Why? Mainly to add the "if not found, return this
> default value" behaviour. That kind of thing is much easier to add
> when the object is only handling a piece of the behaviour, with
> additional standard mechanisms around it (in this case, optionally
> returning a default value when StopIteration is thrown by the
> iterator).
>
> Generators are another good illustration of the principle: For iter()
> and next(), they follow the standard protocol and rely on the
> corresponding builtins. However, g.send() and g.throw() require deep
> integration with the interpreter's eval loop. There's currently no way
> to implement either of those behaviours as an ordinary type, thus
> they're exposed as ordinary methods, since they're genuinely generator
> specific.
>
> As to *why* this is a good thing: procedural APIs encourage low
> coupling. Yes, object oriented programming is a good way to scale an
> application architecture up to more complicated problems. The issue is
> with fetishising OOP to the point where you disallow the creation of
> procedural APIs that hide the OOP details. That approach sets a
> minimum floor to the complexity of your implementations, as even if
> you don't *need* the power of OOP, you're forced to deal with it
> because the language doesn't offer anything else, and that way lies
> Java. There's a reason Java is significantly more popular on large
> enterprise projects than it is in small teams - it takes a certain,
> rather high, level of complexity for the reasons behind any of that
> boilerplate to start to become clear :)
>
> Cheers,
> Nick.
>
>
Thanks, that's some interesting reasoning.

Maybe I'm old fashioned but I like running dir(x) to find out what an
object can do, and the wall of double underscores is hard to read.

Perhaps we could add to the inspect module a "dirprotocols" function which
returns a list of builtins that can be used on an object. I see that the
builtins are listed in e.g. help([]) but on user defined classes it might
be less obvious. Maybe we could just add a dictionary:

inspect.special_methods = {'__len__': len,
 '__getitem__': 'x.__getitem__(y) <==> x[y]',
 '__iter__': iter,
 ... }

and then dirprotocols would be easy to implement.


Yuval
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120902/fcc40544/attachment.html>

From techtonik at gmail.com  Sun Sep  2 13:36:59 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Sun, 2 Sep 2012 14:36:59 +0300
Subject: [Python-ideas] itertools.chunks(iterable, size, fill=None)
In-Reply-To: <d1c0db48-cf7b-4d7f-bb8a-fe2f950c4a6c@googlegroups.com>
References: <CAPkN8xKq7sfWQt9P2Vaw6B_ZwqKtNLiGdavPT=Y84yBcV08uEg@mail.gmail.com>
	<jsl3cb$knn$1@dough.gmane.org>
	<CAPkN8xKM_K=59-Q9jfso+uGLnF=b_zrjvmCzNjEoOcGg+4P0ZQ@mail.gmail.com>
	<d1c0db48-cf7b-4d7f-bb8a-fe2f950c4a6c@googlegroups.com>
Message-ID: <CAPkN8xKNgYWSy=C2rpxyiP+jQgr7k5w61qpRagPSSCoHwHWcyQ@mail.gmail.com>

On Sat, Sep 1, 2012 at 5:42 PM, Miki Tebeka <miki.tebeka at gmail.com> wrote:
> See the "grouper" example in http://docs.python.org/library/itertools.html

As was discussed before, the problem is visibility of the solution,
not the implementation. If we can divide core Python API into levels
where 0 is the less important and 10 is more, then `chunks` should be
level above than it is now.
--
anatoly t.


From stephen at xemacs.org  Mon Sep  3 11:01:44 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 03 Sep 2012 18:01:44 +0900
Subject: [Python-ideas] itertools.chunks(iterable, size, fill=None)
In-Reply-To: <CAPkN8xKNgYWSy=C2rpxyiP+jQgr7k5w61qpRagPSSCoHwHWcyQ@mail.gmail.com>
References: <CAPkN8xKq7sfWQt9P2Vaw6B_ZwqKtNLiGdavPT=Y84yBcV08uEg@mail.gmail.com>
	<jsl3cb$knn$1@dough.gmane.org>
	<CAPkN8xKM_K=59-Q9jfso+uGLnF=b_zrjvmCzNjEoOcGg+4P0ZQ@mail.gmail.com>
	<d1c0db48-cf7b-4d7f-bb8a-fe2f950c4a6c@googlegroups.com>
	<CAPkN8xKNgYWSy=C2rpxyiP+jQgr7k5w61qpRagPSSCoHwHWcyQ@mail.gmail.com>
Message-ID: <87k3wbtr7r.fsf@uwakimon.sk.tsukuba.ac.jp>

anatoly techtonik writes:

 > On Sat, Sep 1, 2012 at 5:42 PM, Miki Tebeka <miki.tebeka at gmail.com> wrote:
 > > See the "grouper" example in http://docs.python.org/library/itertools.html
 > 
 > As was discussed before, the problem is visibility of the solution,

Well, no, it's apparently not.  You should be well aware of the
solution since you were one of the most ardent posters in this thread
the last time it came up.[1]  Yet you say "I had to *re*implement
chunks". IOW, the implementations which you were already aware of were
inappropriate for your situation.  That suggests that no, there are no
generic solutions suitable for the stdlib yet, and you personally
aren't convinced that any of the implementations belong in your own
private library, either.

You really need to get over those humps before you have a case for a
"higher-visibility" placement of any particular implementation.


Footnotes: 
[1]  And I think you cross-posted that time, too, but that's another
issue.  In any case, please stop cross-posting.  Pick one or the
other.  (IMHO, this discussion belongs here on -ideas (or maybe on
python-list), not on python-dev.  Or submit an issue and a patch and
discuss it there.)




From alexander.belopolsky at gmail.com  Mon Sep  3 17:21:50 2012
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Mon, 3 Sep 2012 11:21:50 -0400
Subject: [Python-ideas] Memoryview tolist() method is misleading
Message-ID: <CAP7h-xaooxRvMTEJE+gGw04kfRWWc--sciV5y8SSv-Lg3SgM=g@mail.gmail.com>

Consider this:

>>> memoryview(b'x').cast('B', ()).tolist()
120

The return value of to list() is an int, not a list.

I suggest to deprecate memoryview.tolist() and .tobytes() methods
(soft deprecation - in documentation only) and recommend using list(m)
and  bytes(m) instead.

For the multidimensional (and 0-dimensional) views, I suggest adding
an unpack([depth]) method that would unpack a view into a nested list
of tuples or subviews.  For example a single-byte scalar should unpack
as follows:

>>> m = memoryview(b'x').cast('B', ())
>>> m.unpack()
(120,)

consistent with

>>> struct.unpack_from(m.format, m)
(120,)


From solipsis at pitrou.net  Mon Sep  3 18:35:08 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 3 Sep 2012 18:35:08 +0200
Subject: [Python-ideas] 0-dimensional memoryviews?
References: <CAP7h-xaooxRvMTEJE+gGw04kfRWWc--sciV5y8SSv-Lg3SgM=g@mail.gmail.com>
Message-ID: <20120903183508.585c0e46@pitrou.net>

On Mon, 3 Sep 2012 11:21:50 -0400
Alexander Belopolsky
<alexander.belopolsky at gmail.com> wrote:
> Consider this:
> 
> >>> memoryview(b'x').cast('B', ()).tolist()
> 120
> 
> The return value of to list() is an int, not a list.
> 
> I suggest to deprecate memoryview.tolist() and .tobytes() methods
> (soft deprecation - in documentation only) and recommend using list(m)
> and  bytes(m) instead.
> 
> For the multidimensional (and 0-dimensional) views, I suggest adding
> an unpack([depth]) method that would unpack a view into a nested list
> of tuples or subviews.

Is there any point in 0-dimensional views? Wouldn't it be clearer if we
simply mandated a strictly positive number of dimensions?

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net




From alexander.belopolsky at gmail.com  Mon Sep  3 18:55:32 2012
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Mon, 3 Sep 2012 12:55:32 -0400
Subject: [Python-ideas] 0-dimensional memoryviews?
In-Reply-To: <20120903183508.585c0e46@pitrou.net>
References: <CAP7h-xaooxRvMTEJE+gGw04kfRWWc--sciV5y8SSv-Lg3SgM=g@mail.gmail.com>
	<20120903183508.585c0e46@pitrou.net>
Message-ID: <CAP7h-xZ68RO=cnTMfZPsW5BrJ8VEf3U3e_n6BSYri=xEc0ULKg@mail.gmail.com>

On Mon, Sep 3, 2012 at 12:35 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Is there any point in 0-dimensional views? Wouldn't it be clearer if we
> simply mandated a strictly positive number of dimensions?

0-d arrays (scalars) are very important in numpy and it took a
significant effort to get the semantics right.   I would argue that in
nonnumeric contexts, 0-d case is even more important than 1d.  For
example, users of ctypes are rarely exposed to arrays, but often see
nontrivial scalars.


From Athelopuce at gmx.fr  Thu Sep  6 14:30:04 2012
From: Athelopuce at gmx.fr (Alexandre Bosc)
Date: Thu, 06 Sep 2012 14:30:04 +0200
Subject: [Python-ideas] Add annotations to global statement
Message-ID: <5048974C.6080901@gmx.fr>

I think the annotations of parameters and  return value of a function, 
a  useful practice for the user of the function.
As a function can modify or create global variables, and as it's 
important for the end user,   I would appreciate to add annotations in 
the global statement.

An  annotation syntax similar to that of parameters  could be employed :

     global var : expression
     global var1 : expression1, var2 : expression2,...


cheers,

Alex (geoscience modeler)


From rob.cliffe at btinternet.com  Thu Sep  6 15:17:03 2012
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Thu, 06 Sep 2012 14:17:03 +0100
Subject: [Python-ideas] Add annotations to global statement
In-Reply-To: <5048974C.6080901@gmx.fr>
References: <5048974C.6080901@gmx.fr>
Message-ID: <5048A24F.9090101@btinternet.com>


On 06/09/2012 13:30, Alexandre Bosc wrote:
> I think the annotations of parameters and  return value of a function, 
> a useful practice for the user of the function.
> As a function can modify or create global variables, and as it's 
> important for the end user,   I would appreciate to add annotations in 
> the global statement.
>
> An  annotation syntax similar to that of parameters  could be employed :
>
>     global var : expression
>     global var1 : expression1, var2 : expression2,...
>
>
> cheers,
>
> Alex (geoscience modeler)
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
>
Just to clarify:

Are you proposing that
     global var : expression
is equivalent to
     global var
     var = expression
?

If so, wouldn't the syntax
     global var = expression
be more readable?
Rob Cliffe




From mikegraham at gmail.com  Thu Sep  6 15:26:16 2012
From: mikegraham at gmail.com (Mike Graham)
Date: Thu, 6 Sep 2012 09:26:16 -0400
Subject: [Python-ideas] Add annotations to global statement
In-Reply-To: <5048A24F.9090101@btinternet.com>
References: <5048974C.6080901@gmx.fr>
	<5048A24F.9090101@btinternet.com>
Message-ID: <CAEBZo3MZE9-AvN2om9_ofDn5Z4PqytwxLZ1GXbELukJXa+ZbyA@mail.gmail.com>

On Thu, Sep 6, 2012 at 8:30 AM, Alexandre Bosc <Athelopuce at gmx.fr> wrote:
> I think the annotations of parameters and  return value of a function, a
> useful practice for the user of the function.
> As a function can modify or create global variables, and as it's important
> for the end user,   I would appreciate to add annotations in the global
> statement.
>
> An  annotation syntax similar to that of parameters  could be employed :
>
>     global var : expression
>     global var1 : expression1, var2 : expression2,...
>
>
> cheers,
>
> Alex (geoscience modeler)

I cannot think of a case where having this feature would be of real
help making a well-designed Python program.

On Thu, Sep 6, 2012 at 9:17 AM, Rob Cliffe <rob.cliffe at btinternet.com> wrote:
> Just to clarify:
>
> Are you proposing that
>     global var : expression
> is equivalent to
>     global var
>     var = expression
> ?
>
> If so, wouldn't the syntax
>     global var = expression
> be more readable?
> Rob Cliffe


I believe the suggestion is that the expression becomes metadata
associated with the name var. Are you familiar with function
annotations, with which the parallel was drawn?
http://www.python.org/dev/peps/pep-3107/


Mike


From Athelopuce at gmx.fr  Thu Sep  6 15:29:57 2012
From: Athelopuce at gmx.fr (Alexandre Bosc)
Date: Thu, 06 Sep 2012 15:29:57 +0200
Subject: [Python-ideas] Add annotations to global statement
In-Reply-To: <5048A24F.9090101@btinternet.com>
References: <5048974C.6080901@gmx.fr> <5048A24F.9090101@btinternet.com>
Message-ID: <5048A555.6080707@gmx.fr>

Le 06. 09. 12 15:17, Rob Cliffe a ?crit :
>
> On 06/09/2012 13:30, Alexandre Bosc wrote:
>> I think the annotations of parameters and return value of a function, 
>> a useful practice for the user of the function.
>> As a function can modify or create global variables, and as it's 
>> important for the end user,   I would appreciate to add annotations 
>> in the global statement.
>>
>> An  annotation syntax similar to that of parameters  could be employed :
>>
>>     global var : expression
>>     global var1 : expression1, var2 : expression2,...
>>
>>
>> cheers,
>>
>> Alex (geoscience modeler)
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> http://mail.python.org/mailman/listinfo/python-ideas
>>
>>
> Just to clarify:
>
> Are you proposing that
>     global var : expression
> is equivalent to
>     global var
>     var = expression
> ?
>
> If so, wouldn't the syntax
>     global var = expression
> be more readable?
> Rob Cliffe
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
Hi Rob,

No what I propose is  :

     def f(...):
        global var: expression
        ....
is equivalent to

     def f(...):
         ...

     f.__annotations__['var']=expression

is it clear ?


Alex,


From mikegraham at gmail.com  Fri Sep  7 21:51:21 2012
From: mikegraham at gmail.com (Mike Graham)
Date: Fri, 7 Sep 2012 15:51:21 -0400
Subject: [Python-ideas] reprs of recursive datastructures.
Message-ID: <CAEBZo3O2umaYO2bNc=+4vsPqfegFGLmOouPDJ3mC6a9LEwXWLA@mail.gmail.com>

With the Python 3 loosening of where ... can occur, this somewhat
suboptimal behaviour occurs

>>> x = []
>>> x.append(x)
>>> x
[[...]]
>>> eval(repr(x))
[[Ellipsis]]

Is this something that can be improved? Is it something worth improving?

Mike


From tjreedy at udel.edu  Fri Sep  7 23:57:31 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 07 Sep 2012 17:57:31 -0400
Subject: [Python-ideas] reprs of recursive datastructures.
In-Reply-To: <CAEBZo3O2umaYO2bNc=+4vsPqfegFGLmOouPDJ3mC6a9LEwXWLA@mail.gmail.com>
References: <CAEBZo3O2umaYO2bNc=+4vsPqfegFGLmOouPDJ3mC6a9LEwXWLA@mail.gmail.com>
Message-ID: <k2dqkp$2k1$1@ger.gmane.org>

On 9/7/2012 3:51 PM, Mike Graham wrote:
> With the Python 3 loosening of where ... can occur, this somewhat
> suboptimal behaviour occurs
>
>>>> x = []
>>>> x.append(x)
>>>> x
> [[...]]
>>>> eval(repr(x))
> [[Ellipsis]]

I believe ... was used for representations before it became Ellipsis 
literal. In any case, the representation is now ambiguous. It is not 
possible to reliably invert a many-to-one function.

> Is this something that can be improved?

Change the recursive substitution so there is no ambiguity. For 
instance, use the unicode ellipsis character instead of '...', Since the 
output is unicode and may contain non-ascii chars anyway, that might be 
considered.
 >>> '\u2026'
'?'
 >>> [[...]]
[[Ellipsis]]
 >>> [[?]]
SyntaxError: invalid character in identifier

If not that, pick anything else giving a syntax error.
 >>> [[,,,]]
SyntaxError: invalid syntax
 >>> [[. . .]]
SyntaxError: invalid syntax

> Is it something worth improving?

I think so. Ambiguity is bad, and the substituted representation is 
something of a fib, so it should not mimic something that is valid. 
eval(representation of recursive structure) should either correctly 
evaluate by re-creating the recursive structure represented* or it 
should raise an error.

* That would mean that the same expression should be valid in code also. 
An interesting idea, and a deep can of worms. I believe it would require 
that '. . .' or whatever be recognizable syntax but not a named object, 
as the latter would re-introduce the same ambiguity.

-- 
Terry Jan Reedy




From tjreedy at udel.edu  Sat Sep  8 00:07:37 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 07 Sep 2012 18:07:37 -0400
Subject: [Python-ideas] reprs of recursive datastructures.
In-Reply-To: <k2dqkp$2k1$1@ger.gmane.org>
References: <CAEBZo3O2umaYO2bNc=+4vsPqfegFGLmOouPDJ3mC6a9LEwXWLA@mail.gmail.com>
	<k2dqkp$2k1$1@ger.gmane.org>
Message-ID: <k2dr7n$6bh$1@ger.gmane.org>

On 9/7/2012 5:57 PM, Terry Reedy wrote:
> On 9/7/2012 3:51 PM, Mike Graham wrote:
>> With the Python 3 loosening of where ... can occur, this somewhat
>> suboptimal behaviour occurs
>>
>>>>> x = []
>>>>> x.append(x)
>>>>> x
>> [[...]]
>>>>> eval(repr(x))
>> [[Ellipsis]]
>
> I believe ... was used for representations before it became Ellipsis
> literal. In any case, the representation is now ambiguous. It is not
> possible to reliably invert a many-to-one function.
>
>> Is this something that can be improved?
>
> Change the recursive substitution so there is no ambiguity. For
> instance, use the unicode ellipsis character instead of '...', Since the
> output is unicode and may contain non-ascii chars anyway, that might be
> considered.
>  >>> '\u2026'
> '?'
>  >>> [[...]]
> [[Ellipsis]]
>  >>> [[?]]
> SyntaxError: invalid character in identifier
>
> If not that, pick anything else giving a syntax error.
>  >>> [[,,,]]
> SyntaxError: invalid syntax
>  >>> [[. . .]]
> SyntaxError: invalid syntax

Or probably the simplest, just use 4 periods
 >>> [[....]]
SyntaxError: invalid syntax

>
>> Is it something worth improving?
>
> I think so. Ambiguity is bad, and the substituted representation is
> something of a fib, so it should not mimic something that is valid.
> eval(representation of recursive structure) should either correctly
> evaluate by re-creating the recursive structure represented* or it
> should raise an error.
>
> * That would mean that the same expression should be valid in code also.
> An interesting idea, and a deep can of worms. I believe it would require
> that '. . .' or whatever be recognizable syntax but not a named object,
> as the latter would re-introduce the same ambiguity.
>


-- 
Terry Jan Reedy




From alexander.belopolsky at gmail.com  Sat Sep  8 04:55:36 2012
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Fri, 7 Sep 2012 22:55:36 -0400
Subject: [Python-ideas] reprs of recursive datastructures.
In-Reply-To: <k2dr7n$6bh$1@ger.gmane.org>
References: <CAEBZo3O2umaYO2bNc=+4vsPqfegFGLmOouPDJ3mC6a9LEwXWLA@mail.gmail.com>
	<k2dqkp$2k1$1@ger.gmane.org> <k2dr7n$6bh$1@ger.gmane.org>
Message-ID: <CAP7h-xZgwLMWAwUqaifqjM_GJs7nrpSaZiyJK7d0N+m7J7Rq+w@mail.gmail.com>

On Fri, Sep 7, 2012 at 6:07 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> Or probably the simplest, just use 4 periods
>>>> [[....]]

or two:   [[..]]


From guido at python.org  Sat Sep  8 08:27:41 2012
From: guido at python.org (Guido van Rossum)
Date: Sat, 8 Sep 2012 08:27:41 +0200
Subject: [Python-ideas] reprs of recursive datastructures.
In-Reply-To: <CAP7h-xZgwLMWAwUqaifqjM_GJs7nrpSaZiyJK7d0N+m7J7Rq+w@mail.gmail.com>
References: <CAEBZo3O2umaYO2bNc=+4vsPqfegFGLmOouPDJ3mC6a9LEwXWLA@mail.gmail.com>
	<k2dqkp$2k1$1@ger.gmane.org> <k2dr7n$6bh$1@ger.gmane.org>
	<CAP7h-xZgwLMWAwUqaifqjM_GJs7nrpSaZiyJK7d0N+m7J7Rq+w@mail.gmail.com>
Message-ID: <CAP7+vJJ0gjMRgGCGmg5jQYyufwUF8ORjnA9p4sTPYVZq4KJScw@mail.gmail.com>

Can someone explain what problem we are trying to solve? I fail to
uderstand what's wrong with the current behavior...

-- 
Sent from Gmail Mobile
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120908/39a7d43f/attachment.html>

From tjreedy at udel.edu  Sat Sep  8 09:23:14 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 08 Sep 2012 03:23:14 -0400
Subject: [Python-ideas] reprs of recursive datastructures.
In-Reply-To: <CAP7+vJJ0gjMRgGCGmg5jQYyufwUF8ORjnA9p4sTPYVZq4KJScw@mail.gmail.com>
References: <CAEBZo3O2umaYO2bNc=+4vsPqfegFGLmOouPDJ3mC6a9LEwXWLA@mail.gmail.com>
	<k2dqkp$2k1$1@ger.gmane.org> <k2dr7n$6bh$1@ger.gmane.org>
	<CAP7h-xZgwLMWAwUqaifqjM_GJs7nrpSaZiyJK7d0N+m7J7Rq+w@mail.gmail.com>
	<CAP7+vJJ0gjMRgGCGmg5jQYyufwUF8ORjnA9p4sTPYVZq4KJScw@mail.gmail.com>
Message-ID: <k2erpg$p19$1@ger.gmane.org>

On 9/8/2012 2:27 AM, Guido van Rossum wrote:
> Can someone explain what problem we are trying to solve? I fail to
> uderstand what's wrong with the current behavior...

Pairs of different things have the same representation, making the 
representation ambiguous to both people and the interpreter. Moreover, 
the interpreter's guess is usually wrong.

In particular, the representations of recursive lists use what is now 
the Ellipsis literal '...', so they are also valid list displays for a 
non-recursive nested list containing Ellipsis. The interpreter always 
reads ... as the Ellipsis literal, which it nearly always is not what is 
meant.

It would be trivial to tweak the representations of recursive lists so 
they are not valid list displays.

---
Terry Jan Reedy


-- 
Terry Jan Reedy



From steve at pearwood.info  Sat Sep  8 09:45:54 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 08 Sep 2012 17:45:54 +1000
Subject: [Python-ideas] reprs of recursive datastructures.
In-Reply-To: <CAP7+vJJ0gjMRgGCGmg5jQYyufwUF8ORjnA9p4sTPYVZq4KJScw@mail.gmail.com>
References: <CAEBZo3O2umaYO2bNc=+4vsPqfegFGLmOouPDJ3mC6a9LEwXWLA@mail.gmail.com>
	<k2dqkp$2k1$1@ger.gmane.org> <k2dr7n$6bh$1@ger.gmane.org>
	<CAP7h-xZgwLMWAwUqaifqjM_GJs7nrpSaZiyJK7d0N+m7J7Rq+w@mail.gmail.com>
	<CAP7+vJJ0gjMRgGCGmg5jQYyufwUF8ORjnA9p4sTPYVZq4KJScw@mail.gmail.com>
Message-ID: <504AF7B2.7090703@pearwood.info>

On 08/09/12 16:27, Guido van Rossum wrote:
> Can someone explain what problem we are trying to solve? I fail to
> uderstand what's wrong with the current behavior...


I believe that some people think that if you eval the repr of a
recursive list, the result should be an equally recursive list. But
it isn't:

py> x = [1, 2, 3]
py> x.append(x)
py> eval(repr(x)) == x
False

I think they are misguided in their expectation. There is no way
to write a single expression using list literals which generates
a recursive list, so why would you expect eval to produce one?

Furthermore, list reprs of recursive lists have been ambiguous
for years. This code works identically in 2.4 and 3.2:

py> a = []; a.append(a)
py> b = []; b.append(b)
py> x = [[], []]; x[0].append(x); x[1].append(x)
py> y = [a, b]
py> x == y
False
py> repr(x) == repr(y)
True


eval(repr(x)) == x is not a guaranteed invariant, it is a "nice
to have". -1 on trying to fix this.



-- 
Steven


From steve at pearwood.info  Sat Sep  8 10:06:41 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 08 Sep 2012 18:06:41 +1000
Subject: [Python-ideas] reprs of recursive datastructures.
In-Reply-To: <k2erpg$p19$1@ger.gmane.org>
References: <CAEBZo3O2umaYO2bNc=+4vsPqfegFGLmOouPDJ3mC6a9LEwXWLA@mail.gmail.com>
	<k2dqkp$2k1$1@ger.gmane.org> <k2dr7n$6bh$1@ger.gmane.org>
	<CAP7h-xZgwLMWAwUqaifqjM_GJs7nrpSaZiyJK7d0N+m7J7Rq+w@mail.gmail.com>
	<CAP7+vJJ0gjMRgGCGmg5jQYyufwUF8ORjnA9p4sTPYVZq4KJScw@mail.gmail.com>
	<k2erpg$p19$1@ger.gmane.org>
Message-ID: <504AFC91.4050707@pearwood.info>

On 08/09/12 17:23, Terry Reedy wrote:

> In particular, the representations of recursive lists use what is
>now the Ellipsis literal '...', so they are also valid list displays
>for a non-recursive nested list containing Ellipsis. The interpreter
>always reads ... as the Ellipsis literal, which it nearly always is
>not what is meant.

I'm not sure that you are right to assume that recursive lists are
more common than lists containing Ellipsis. Neither are exactly common,
and at least a few people use Ellipsis as a ready-made sentinel value
that isn't None.


> It would be trivial to tweak the representations of recursive lists
>so they are not valid list displays.

Ah, I had not realised that you wanted eval(repr(x)) to fail if x was
recursive. That's more reasonable than expecting it to generate x.

Changing the repr of recursive lists will break doctests.

And frankly, my aesthetic sense would be hurt if the repr of a
recursive list used something other than ... for the part not displayed.
An ellipsis is the right symbol to use when skipping part of the
display, and an ellipsis is three dots, not two or four. A unicode ?
would be acceptable, except I understand that buildins must be ASCII.

I don't think this is genuinely enough of a problem that it needs fixing.


-- 
Steven


From ncoghlan at gmail.com  Sat Sep  8 10:16:23 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 8 Sep 2012 18:16:23 +1000
Subject: [Python-ideas] reprs of recursive datastructures.
In-Reply-To: <504AF7B2.7090703@pearwood.info>
References: <CAEBZo3O2umaYO2bNc=+4vsPqfegFGLmOouPDJ3mC6a9LEwXWLA@mail.gmail.com>
	<k2dqkp$2k1$1@ger.gmane.org> <k2dr7n$6bh$1@ger.gmane.org>
	<CAP7h-xZgwLMWAwUqaifqjM_GJs7nrpSaZiyJK7d0N+m7J7Rq+w@mail.gmail.com>
	<CAP7+vJJ0gjMRgGCGmg5jQYyufwUF8ORjnA9p4sTPYVZq4KJScw@mail.gmail.com>
	<504AF7B2.7090703@pearwood.info>
Message-ID: <CADiSq7c6J3NPqEyo4gt-Zz1Z8F-+sCVGRLjAt7ACdTFRs29k3Q@mail.gmail.com>

On Sat, Sep 8, 2012 at 5:45 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> On 08/09/12 16:27, Guido van Rossum wrote:
>>
>> Can someone explain what problem we are trying to solve? I fail to
>> uderstand what's wrong with the current behavior...
>
>
>
> I believe that some people think that if you eval the repr of a
> recursive list, the result should be an equally recursive list.

No, the problem is that you get the *wrong answer* instead of an exception.

Python 2:

>>> x = []
>>> x.append(x)
>>> x
[[...]]
>>> eval(repr(x))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 1
    [[...]]
      ^
SyntaxError: invalid syntax

Python 3:

>>> x = []
>>> x.append(x)
>>> x
[[...]]
>>> eval(repr(x))
[[Ellipsis]]

As pointed out earlier, this is due to the fact that the previously
illegal notation used to indicate the recursive reference is now valid
syntax. The simplest fix is to just introduce alternative notation for
the self-reference that will reintroduce the desired syntax error,
such as "<...>" or "<self>".

Cheers,
Nick.
-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia


From guido at python.org  Sat Sep  8 11:49:29 2012
From: guido at python.org (Guido van Rossum)
Date: Sat, 8 Sep 2012 11:49:29 +0200
Subject: [Python-ideas] reprs of recursive datastructures.
In-Reply-To: <504AF11A.80700@udel.edu>
References: <CAEBZo3O2umaYO2bNc=+4vsPqfegFGLmOouPDJ3mC6a9LEwXWLA@mail.gmail.com>
	<k2dqkp$2k1$1@ger.gmane.org> <k2dr7n$6bh$1@ger.gmane.org>
	<CAP7h-xZgwLMWAwUqaifqjM_GJs7nrpSaZiyJK7d0N+m7J7Rq+w@mail.gmail.com>
	<CAP7+vJJ0gjMRgGCGmg5jQYyufwUF8ORjnA9p4sTPYVZq4KJScw@mail.gmail.com>
	<504AF11A.80700@udel.edu>
Message-ID: <CAP7+vJLiDmEQ34KMWuVzUbjbR=J+sQrUNh49G+q-ydNx=Kf2Tg@mail.gmail.com>

On Sat, Sep 8, 2012 at 9:17 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 9/8/2012 2:27 AM, Guido van Rossum wrote:
>> Can someone explain what problem we are trying to solve? I fail to
>> uderstand what's wrong with the current behavior...

> Pairs of different things have the same representation, making the
> representation ambiguous to both people and the interpreter.

Well yeah, when designing a repr() we usually have to compromise. E.g.
if you render a class instance it often shows the class name but not
the module name (e.g. decimal.Decimal.)

> Moreover, the interpreter's guess is usually wrong.

The requirement that the interpreter can evaluate a repr() and return
a similar value is pretty weak, and I'm not sure that in this case the
fact that copying the output back into the interpreter returns an
object of a different share matters much to anyone.

A subtler but similar bug appears with lists containing multiple
references to the same sublist, e.g.

>>> a = [1, 2]
>>> b = [a, a]
>>> b
[[1, 2], [1, 2]]
>>> b[0].append(3)
>>> b
[[1, 2, 3], [1, 2, 3]]
>>> x = [[1, 2], [1, 2]]
>>> x[0].append(3)
>>> x
[[1, 2, 3], [1, 2]]
>>>

I don't think we should attempt to fix this particular one -- first of
all, the analysis would be tricky (there could be a user-defined
object involved) and second of all, I can't think of a solution that
still produces a valid expression (except perhaps a very ugly one).

> In particular, the representations of recursive lists use what is now the
> Ellipsis literal '...', so they are also valid list displays for a
> non-recursive nested list containing Ellipsis. The interpreter always reads
> ... as the Ellipsis literal, which it nearly always is not what is meant.

But when does it ever matter?

> It would be trivial to tweak the representations of recursive lists so they
> are not valid list displays.

To what purpose? I still don't understand what the actual use case is
where you think that will produce a better experience for the user.

-- 
--Guido van Rossum (python.org/~guido)


From ncoghlan at gmail.com  Sat Sep  8 12:02:04 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 8 Sep 2012 20:02:04 +1000
Subject: [Python-ideas] reprs of recursive datastructures.
In-Reply-To: <CAP7+vJLiDmEQ34KMWuVzUbjbR=J+sQrUNh49G+q-ydNx=Kf2Tg@mail.gmail.com>
References: <CAEBZo3O2umaYO2bNc=+4vsPqfegFGLmOouPDJ3mC6a9LEwXWLA@mail.gmail.com>
	<k2dqkp$2k1$1@ger.gmane.org> <k2dr7n$6bh$1@ger.gmane.org>
	<CAP7h-xZgwLMWAwUqaifqjM_GJs7nrpSaZiyJK7d0N+m7J7Rq+w@mail.gmail.com>
	<CAP7+vJJ0gjMRgGCGmg5jQYyufwUF8ORjnA9p4sTPYVZq4KJScw@mail.gmail.com>
	<504AF11A.80700@udel.edu>
	<CAP7+vJLiDmEQ34KMWuVzUbjbR=J+sQrUNh49G+q-ydNx=Kf2Tg@mail.gmail.com>
Message-ID: <CADiSq7edZ7O1qPaLE_m6WRJmdjHZNSfn3pz1eC6WGABo4Rzxdw@mail.gmail.com>

On Sat, Sep 8, 2012 at 7:49 PM, Guido van Rossum <guido at python.org> wrote:
> To what purpose? I still don't understand what the actual use case is
> where you think that will produce a better experience for the user.

The thing I don't like is that the current display flat out lies about
the sequence contents - it displays a terminal constant ("..."),
rather than a clear marker that a recursive loop was detected.

The case of multiple references to a different list is not the same,
as then the repr() at least still accurately reflects what you would
get when iterating over the data structure.

So, my perspective is if attempting to naively flatten the list would
create an infinite loop, then I want evaluating the representation to
throw a syntax error the way it did in Python 2.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia


From tjreedy at udel.edu  Sat Sep  8 21:02:23 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 08 Sep 2012 15:02:23 -0400
Subject: [Python-ideas] reprs of recursive datastructures.
In-Reply-To: <CADiSq7edZ7O1qPaLE_m6WRJmdjHZNSfn3pz1eC6WGABo4Rzxdw@mail.gmail.com>
References: <CAEBZo3O2umaYO2bNc=+4vsPqfegFGLmOouPDJ3mC6a9LEwXWLA@mail.gmail.com>
	<k2dqkp$2k1$1@ger.gmane.org> <k2dr7n$6bh$1@ger.gmane.org>
	<CAP7h-xZgwLMWAwUqaifqjM_GJs7nrpSaZiyJK7d0N+m7J7Rq+w@mail.gmail.com>
	<CAP7+vJJ0gjMRgGCGmg5jQYyufwUF8ORjnA9p4sTPYVZq4KJScw@mail.gmail.com>
	<504AF11A.80700@udel.edu>
	<CAP7+vJLiDmEQ34KMWuVzUbjbR=J+sQrUNh49G+q-ydNx=Kf2Tg@mail.gmail.com>
	<CADiSq7edZ7O1qPaLE_m6WRJmdjHZNSfn3pz1eC6WGABo4Rzxdw@mail.gmail.com>
Message-ID: <k2g4oc$tec$1@ger.gmane.org>

On 9/8/2012 6:02 AM, Nick Coghlan wrote:
> On Sat, Sep 8, 2012 at 7:49 PM, Guido van Rossum <guido at python.org> wrote:
>> To what purpose? I still don't understand what the actual use case is
>> where you think that will produce a better experience for the user.
>
> The thing I don't like is that the current display flat out lies about
> the sequence contents - it displays a terminal constant ("..."),
> rather than a clear marker that a recursive loop was detected.
>
> The case of multiple references to a different list is not the same,
> as then the repr() at least still accurately reflects what you would
> get when iterating over the data structure.
>
> So, my perspective is if attempting to naively flatten the list would
> create an infinite loop, then I want evaluating the representation to
> throw a syntax error the way it did in Python 2.

This expresses what I was trying to say better than I did. When '...' 
was chosen for recursive structures, it make the result not-legal-code, 
as it should be. The 3.0 incorporation of '...' as legal syntax, 
created, in a sense, a reversion. So that suggests revising the 
recursion marker.

That said, there is the issue of doctests, so I would only change in 3.4.

-- 
Terry Jan Reedy



From alexander.belopolsky at gmail.com  Sat Sep  8 21:53:23 2012
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Sat, 8 Sep 2012 15:53:23 -0400
Subject: [Python-ideas] reprs of recursive datastructures.
In-Reply-To: <k2g4oc$tec$1@ger.gmane.org>
References: <CAEBZo3O2umaYO2bNc=+4vsPqfegFGLmOouPDJ3mC6a9LEwXWLA@mail.gmail.com>
	<k2dqkp$2k1$1@ger.gmane.org> <k2dr7n$6bh$1@ger.gmane.org>
	<CAP7h-xZgwLMWAwUqaifqjM_GJs7nrpSaZiyJK7d0N+m7J7Rq+w@mail.gmail.com>
	<CAP7+vJJ0gjMRgGCGmg5jQYyufwUF8ORjnA9p4sTPYVZq4KJScw@mail.gmail.com>
	<504AF11A.80700@udel.edu>
	<CAP7+vJLiDmEQ34KMWuVzUbjbR=J+sQrUNh49G+q-ydNx=Kf2Tg@mail.gmail.com>
	<CADiSq7edZ7O1qPaLE_m6WRJmdjHZNSfn3pz1eC6WGABo4Rzxdw@mail.gmail.com>
	<k2g4oc$tec$1@ger.gmane.org>
Message-ID: <FE4036BD-10EA-4F9E-848D-3AFAEDB44C09@gmail.com>



On Sep 8, 2012, at 3:02 PM, Terry Reedy <tjreedy at udel.edu> wrote:

> 
> That said, there is the issue of doctests, so I would only change in 3.4.

Note that in doctest displays, ellipsis has yet another meaning.  I agree that this is 3.4 material, and the solution should probably be something in <>. 

From guido at python.org  Sat Sep  8 21:59:11 2012
From: guido at python.org (Guido van Rossum)
Date: Sat, 8 Sep 2012 21:59:11 +0200
Subject: [Python-ideas] reprs of recursive datastructures.
In-Reply-To: <FE4036BD-10EA-4F9E-848D-3AFAEDB44C09@gmail.com>
References: <CAEBZo3O2umaYO2bNc=+4vsPqfegFGLmOouPDJ3mC6a9LEwXWLA@mail.gmail.com>
	<k2dqkp$2k1$1@ger.gmane.org> <k2dr7n$6bh$1@ger.gmane.org>
	<CAP7h-xZgwLMWAwUqaifqjM_GJs7nrpSaZiyJK7d0N+m7J7Rq+w@mail.gmail.com>
	<CAP7+vJJ0gjMRgGCGmg5jQYyufwUF8ORjnA9p4sTPYVZq4KJScw@mail.gmail.com>
	<504AF11A.80700@udel.edu>
	<CAP7+vJLiDmEQ34KMWuVzUbjbR=J+sQrUNh49G+q-ydNx=Kf2Tg@mail.gmail.com>
	<CADiSq7edZ7O1qPaLE_m6WRJmdjHZNSfn3pz1eC6WGABo4Rzxdw@mail.gmail.com>
	<k2g4oc$tec$1@ger.gmane.org>
	<FE4036BD-10EA-4F9E-848D-3AFAEDB44C09@gmail.com>
Message-ID: <CAP7+vJJZUPk1oX3Pzkf+LWcZm4nCaYf+E4AWtsnUO5z9Az=uHg@mail.gmail.com>

Ok, I can live with <...>.

On Saturday, September 8, 2012, Alexander Belopolsky wrote:

>
>
> On Sep 8, 2012, at 3:02 PM, Terry Reedy <tjreedy at udel.edu <javascript:;>>
> wrote:
>
> >
> > That said, there is the issue of doctests, so I would only change in 3.4.
>
> Note that in doctest displays, ellipsis has yet another meaning.  I agree
> that this is 3.4 material, and the solution should probably be something in
> <>.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org <javascript:;>
> http://mail.python.org/mailman/listinfo/python-ideas
>


-- 
Sent from Gmail Mobile
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120908/ac2d7bc1/attachment.html>

From python at mrabarnett.plus.com  Sat Sep  8 23:26:26 2012
From: python at mrabarnett.plus.com (MRAB)
Date: Sat, 08 Sep 2012 22:26:26 +0100
Subject: [Python-ideas] reprs of recursive datastructures.
In-Reply-To: <CAP7+vJJZUPk1oX3Pzkf+LWcZm4nCaYf+E4AWtsnUO5z9Az=uHg@mail.gmail.com>
References: <CAEBZo3O2umaYO2bNc=+4vsPqfegFGLmOouPDJ3mC6a9LEwXWLA@mail.gmail.com>
	<k2dqkp$2k1$1@ger.gmane.org> <k2dr7n$6bh$1@ger.gmane.org>
	<CAP7h-xZgwLMWAwUqaifqjM_GJs7nrpSaZiyJK7d0N+m7J7Rq+w@mail.gmail.com>
	<CAP7+vJJ0gjMRgGCGmg5jQYyufwUF8ORjnA9p4sTPYVZq4KJScw@mail.gmail.com>
	<504AF11A.80700@udel.edu>
	<CAP7+vJLiDmEQ34KMWuVzUbjbR=J+sQrUNh49G+q-ydNx=Kf2Tg@mail.gmail.com>
	<CADiSq7edZ7O1qPaLE_m6WRJmdjHZNSfn3pz1eC6WGABo4Rzxdw@mail.gmail.com>
	<k2g4oc$tec$1@ger.gmane.org>
	<FE4036BD-10EA-4F9E-848D-3AFAEDB44C09@gmail.com>
	<CAP7+vJJZUPk1oX3Pzkf+LWcZm4nCaYf+E4AWtsnUO5z9Az=uHg@mail.gmail.com>
Message-ID: <504BB802.5050603@mrabarnett.plus.com>

On 08/09/2012 20:59, Guido van Rossum wrote:
> On Saturday, September 8, 2012, Alexander Belopolsky wrote:
>
>
>
>     On Sep 8, 2012, at 3:02 PM, Terry Reedy <tjreedy at udel.edu
>     <javascript:;>> wrote:
>
>      >
>      > That said, there is the issue of doctests, so I would only change
>     in 3.4.
>
>     Note that in doctest displays, ellipsis has yet another meaning.  I
>     agree that this is 3.4 material, and the solution should probably be
>     something in <>.
>
> Ok, I can live with <...>.
>
Should we also guarantee that it'll never have any other meaning in
Python so that this will never happen again?


From alexander.belopolsky at gmail.com  Sun Sep  9 00:06:14 2012
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Sat, 8 Sep 2012 18:06:14 -0400
Subject: [Python-ideas] reprs of recursive datastructures.
In-Reply-To: <504BB802.5050603@mrabarnett.plus.com>
References: <CAEBZo3O2umaYO2bNc=+4vsPqfegFGLmOouPDJ3mC6a9LEwXWLA@mail.gmail.com>
	<k2dqkp$2k1$1@ger.gmane.org> <k2dr7n$6bh$1@ger.gmane.org>
	<CAP7h-xZgwLMWAwUqaifqjM_GJs7nrpSaZiyJK7d0N+m7J7Rq+w@mail.gmail.com>
	<CAP7+vJJ0gjMRgGCGmg5jQYyufwUF8ORjnA9p4sTPYVZq4KJScw@mail.gmail.com>
	<504AF11A.80700@udel.edu>
	<CAP7+vJLiDmEQ34KMWuVzUbjbR=J+sQrUNh49G+q-ydNx=Kf2Tg@mail.gmail.com>
	<CADiSq7edZ7O1qPaLE_m6WRJmdjHZNSfn3pz1eC6WGABo4Rzxdw@mail.gmail.com>
	<k2g4oc$tec$1@ger.gmane.org>
	<FE4036BD-10EA-4F9E-848D-3AFAEDB44C09@gmail.com>
	<CAP7+vJJZUPk1oX3Pzkf+LWcZm4nCaYf+E4AWtsnUO5z9Az=uHg@mail.gmail.com>
	<504BB802.5050603@mrabarnett.plus.com>
Message-ID: <CAP7h-xbvQYYFOyTtNiF6_mPj+wD_8VYJrwCziCd4cvTr-AcmDg@mail.gmail.com>

On Sat, Sep 8, 2012 at 5:26 PM, MRAB <python at mrabarnett.plus.com> wrote:
>>
>>     Note that in doctest displays, ellipsis has yet another meaning.  I
>>     agree that this is 3.4 material, and the solution should probably be
>>     something in <>.
>>
>> Ok, I can live with <...>.
>>
> Should we also guarantee that it'll never have any other meaning in
> Python so that this will never happen again?

I was probably not very clear about the problem of having ellipsis
appear as expected output in doctests.  The problem is that '...' has
a special meaning for doctests:

"""
When specified, an ellipsis marker (...) in the expected output can
match any substring in the actual output. ...
""" http://docs.python.org/py3k/library/doctest.html#doctest.ELLIPSIS

This means that <...> will match any angle bracketed repr.  Note that
lists are not the only types affected by this issue.  Dicts, for
example, have the same problem:

>>> d = {}
>>> d[0] = d
>>> d
{0: {...}}
>>> eval('{0: {...}}')
{0: {Ellipsis}}

It is possible the other mutable container types are similarly
affected.  It looks like this problem requires some more though.

If we ever decide to allow non-ASCII characters in repr, my vote for
repr of recursive list will be

>>> "[[\N{ANTICLOCKWISE GAPPED CIRCLE ARROW}]]"
'[[?]]'


>>> "\N{WHITE SMILING FACE}"
'?'


From python at mrabarnett.plus.com  Sun Sep  9 00:27:24 2012
From: python at mrabarnett.plus.com (MRAB)
Date: Sat, 08 Sep 2012 23:27:24 +0100
Subject: [Python-ideas] reprs of recursive datastructures.
In-Reply-To: <CAP7h-xbvQYYFOyTtNiF6_mPj+wD_8VYJrwCziCd4cvTr-AcmDg@mail.gmail.com>
References: <CAEBZo3O2umaYO2bNc=+4vsPqfegFGLmOouPDJ3mC6a9LEwXWLA@mail.gmail.com>
	<k2dqkp$2k1$1@ger.gmane.org> <k2dr7n$6bh$1@ger.gmane.org>
	<CAP7h-xZgwLMWAwUqaifqjM_GJs7nrpSaZiyJK7d0N+m7J7Rq+w@mail.gmail.com>
	<CAP7+vJJ0gjMRgGCGmg5jQYyufwUF8ORjnA9p4sTPYVZq4KJScw@mail.gmail.com>
	<504AF11A.80700@udel.edu>
	<CAP7+vJLiDmEQ34KMWuVzUbjbR=J+sQrUNh49G+q-ydNx=Kf2Tg@mail.gmail.com>
	<CADiSq7edZ7O1qPaLE_m6WRJmdjHZNSfn3pz1eC6WGABo4Rzxdw@mail.gmail.com>
	<k2g4oc$tec$1@ger.gmane.org>
	<FE4036BD-10EA-4F9E-848D-3AFAEDB44C09@gmail.com>
	<CAP7+vJJZUPk1oX3Pzkf+LWcZm4nCaYf+E4AWtsnUO5z9Az=uHg@mail.gmail.com>
	<504BB802.5050603@mrabarnett.plus.com>
	<CAP7h-xbvQYYFOyTtNiF6_mPj+wD_8VYJrwCziCd4cvTr-AcmDg@mail.gmail.com>
Message-ID: <504BC64C.1050700@mrabarnett.plus.com>

On 08/09/2012 23:06, Alexander Belopolsky wrote:
> On Sat, Sep 8, 2012 at 5:26 PM, MRAB <python at mrabarnett.plus.com> wrote:
>>>
>>>     Note that in doctest displays, ellipsis has yet another meaning.  I
>>>     agree that this is 3.4 material, and the solution should probably be
>>>     something in <>.
>>>
>>> Ok, I can live with <...>.
>>>
>> Should we also guarantee that it'll never have any other meaning in
>> Python so that this will never happen again?
>
> I was probably not very clear about the problem of having ellipsis
> appear as expected output in doctests.  The problem is that '...' has
> a special meaning for doctests:
>
> """
> When specified, an ellipsis marker (...) in the expected output can
> match any substring in the actual output. ...
> """ http://docs.python.org/py3k/library/doctest.html#doctest.ELLIPSIS
>
> This means that <...> will match any angle bracketed repr.  Note that
> lists are not the only types affected by this issue.  Dicts, for
> example, have the same problem:
>
>>>> d = {}
>>>> d[0] = d
>>>> d
> {0: {...}}
>>>> eval('{0: {...}}')
> {0: {Ellipsis}}
>
> It is possible the other mutable container types are similarly
> affected.  It looks like this problem requires some more though.
>
> If we ever decide to allow non-ASCII characters in repr, my vote for
> repr of recursive list will be
>
>>>> "[[\N{ANTICLOCKWISE GAPPED CIRCLE ARROW}]]"
> '[[?]]'
>
Or:

 >>> "[[\N{CLOCKWISE GAPPED CIRCLE ARROW}]]"
'[[?]]'

>
>>>> "\N{WHITE SMILING FACE}"
> '?'
>



From cs at zip.com.au  Sun Sep  9 01:20:20 2012
From: cs at zip.com.au (Cameron Simpson)
Date: Sun, 9 Sep 2012 09:20:20 +1000
Subject: [Python-ideas] reprs of recursive datastructures.
In-Reply-To: <504BC64C.1050700@mrabarnett.plus.com>
References: <504BC64C.1050700@mrabarnett.plus.com>
Message-ID: <20120908232020.GA2900@cskk.homeip.net>

On 08Sep2012 23:27, MRAB <python at mrabarnett.plus.com> wrote:
| On 08/09/2012 23:06, Alexander Belopolsky wrote:
| > If we ever decide to allow non-ASCII characters in repr, my vote for
| > repr of recursive list will be
| >
| >>>> "[[\N{ANTICLOCKWISE GAPPED CIRCLE ARROW}]]"
| > '[[?]]'
| >
| Or:
| 
|  >>> "[[\N{CLOCKWISE GAPPED CIRCLE ARROW}]]"
| '[[?]]'
[...]

These are sublime!

Personally I find the former one more intuitively expressive of a
recursion, probably because the arrow points "left" (in my current font,
anyway; how variable is this?) and therefore towards the stuff already
recited. The latter arrow seems to point "right" or "forwards", no so
recursive to my intuition.

| >>>> "\N{WHITE SMILING FACE}"
| > '?'

Cute but a -1 from me; less intuitive meaning.

Cheers,
-- 
Cameron Simpson <cs at zip.com.au>

To understand recursion, you must first understand recursion.


From dholth at gmail.com  Sun Sep  9 05:41:43 2012
From: dholth at gmail.com (Daniel Holth)
Date: Sat, 8 Sep 2012 23:41:43 -0400
Subject: [Python-ideas] bdist naming scheme (compatibility tags) PEP
In-Reply-To: <CAG8k2+5Eygfk5MFx35OG98RhxcrVpDXC+0Ph9S+ecG4ahG5WWw@mail.gmail.com>
References: <CAG8k2+7BYcg5RRnUhjY6h7jKsZUCeYCMp-mUohPSSsh5kmaQFw@mail.gmail.com>
	<50236FD5.2060404@egenix.com>
	<CAG8k2+5Eygfk5MFx35OG98RhxcrVpDXC+0Ph9S+ecG4ahG5WWw@mail.gmail.com>
Message-ID: <CAG8k2+59BaH_a4FcDy-H731b0g8Gk-9urWr4VvJ_bkQ3AZ-RMA@mail.gmail.com>

Re: http://www.python.org/dev/peps/pep-0425/ "Compatibility tags for
built distributions"

Progress towards a proper set of rules for generating the tags a
Python implementation is likely to support.

This system of being willing to install older built distributions is
intended to solve the frustrating problem with eggs that you would
have to build a new egg for each Python release, even for a
pure-Python egg that probably runs fine on a newer Python.

In order of preference the tags are:

- built for the current implementation and its preferred ABI and architecture
- for the current implementation and tagged with just the major
version number (explicitly tagged as cross-version compatible)
- for the current implementation, and any of the lesser minor
revisions (cp26..cp20)

- for the current language version (py27)
- for the current language major version (py2)
- for any of the current language minor versions (py26..py20)

Importantly "py2" means "expected to work across minor releases" and
is not shorthand for "py20". Practically it means the packager
overrode the default tag.

For PyPy, I think "pp19" for the current version makes more sense than
"pp27" since they add important runtime features without changing the
version of the Python language they support (like stackless
emulation). I don't know how their versions will work when PyPy for
Python 3 is released. Other Python implementations seem to follow the
CPython version numbers more closely.

For PyPy it may be appropriate to cross major versions when going back
to generate the list of older packages one is willing to install. For
CPython it is a bit overkill to go all the way back to Python 2.0; the
"all the way back to the last major revision" rule is really for the 2
- 3 split.


List of supported or "willing to install" tags for CPython 3.2 (an mu build):

[('cp32', 'cp32mu', 'linux_x86_64'),
 ('cp3', 'none', 'any'),
 ('cp31', 'none', 'any'),
 ('cp30', 'none', 'any'),
 ('py32', 'none', 'any'),
 ('py3', 'none', 'any'),
 ('py31', 'none', 'any'),
 ('py30', 'none', 'any')]

For CPython 2.7:

[('cp27', 'none', 'linux_x86_64'),
 ('cp2', 'none', 'any'),
 ('cp26', 'none', 'any'),
 ('cp25', 'none', 'any'),
 ('cp24', 'none', 'any'),
 ('cp23', 'none', 'any'),
 ('cp22', 'none', 'any'),
 ('cp21', 'none', 'any'),
 ('cp20', 'none', 'any'),
 ('py27', 'none', 'any'),
 ('py2', 'none', 'any'),
 ('py26', 'none', 'any'),
 ('py25', 'none', 'any'),
 ('py24', 'none', 'any'),
 ('py23', 'none', 'any'),
 ('py22', 'none', 'any'),
 ('py21', 'none', 'any'),
 ('py20', 'none', 'any')]


From raymond.hettinger at gmail.com  Sun Sep  9 07:04:34 2012
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Sun, 9 Sep 2012 01:04:34 -0400
Subject: [Python-ideas] reprs of recursive datastructures.
In-Reply-To: <CADiSq7edZ7O1qPaLE_m6WRJmdjHZNSfn3pz1eC6WGABo4Rzxdw@mail.gmail.com>
References: <CAEBZo3O2umaYO2bNc=+4vsPqfegFGLmOouPDJ3mC6a9LEwXWLA@mail.gmail.com>
	<k2dqkp$2k1$1@ger.gmane.org> <k2dr7n$6bh$1@ger.gmane.org>
	<CAP7h-xZgwLMWAwUqaifqjM_GJs7nrpSaZiyJK7d0N+m7J7Rq+w@mail.gmail.com>
	<CAP7+vJJ0gjMRgGCGmg5jQYyufwUF8ORjnA9p4sTPYVZq4KJScw@mail.gmail.com>
	<504AF11A.80700@udel.edu>
	<CAP7+vJLiDmEQ34KMWuVzUbjbR=J+sQrUNh49G+q-ydNx=Kf2Tg@mail.gmail.com>
	<CADiSq7edZ7O1qPaLE_m6WRJmdjHZNSfn3pz1eC6WGABo4Rzxdw@mail.gmail.com>
Message-ID: <CF275729-EAA3-4CF4-AAB0-20FA831D3AAF@gmail.com>


On Sep 8, 2012, at 6:02 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> The thing I don't like is that the current display flat out lies about
> the sequence contents - it displays a terminal constant ("..."),
> rather than a clear marker that a recursive loop was detected.

It's more of an equivocation than a flat-out lie ;-)

It is an equivocation because "..." is legitimately used
for multiple purposes (in English text for "and so on ...",
in long established use in Python to denote recursive reprs,
in doctest as a placeholder for elided result text, and in
its newest role as the Ellipsis terminal constant).

It seems to me that the first three roles are well-established
and are reasonably consistent with one another.    Further,
each of those roles an important task.

In contrast, the new role as a terminal constant for an
Ellipsis singleton is brand-new, not very important,
and doesn't even have a clear semantic role (what it is
"supposed" to mean?).

Changing the first three uses just so it won't conflict with the last
seems like the tail wagging the dog.

I agree Steven that this isn't a problem worth solving.

As Alexander pointed-out, the ... punctuation can be used in two
distinct ways inside doctests (as part of expected output or as
a placeholder for elided content).  A consequence is that 
there won't be a reliable automated way to convert existing 
doctests for a new notation for recursive reprs. 

ISTM that changes which break tests are worse than other
changes because the process of upgrading from one Python
version to the next is so heavily dependent getting existing tests
to pass.  The tests are your safety net during upgrades -- breaking them
makes upgrading less palatable.


Raymond




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120909/73103a56/attachment.html>

From ncoghlan at gmail.com  Sun Sep  9 14:16:24 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 9 Sep 2012 22:16:24 +1000
Subject: [Python-ideas] bdist naming scheme (compatibility tags) PEP
In-Reply-To: <CAG8k2+59BaH_a4FcDy-H731b0g8Gk-9urWr4VvJ_bkQ3AZ-RMA@mail.gmail.com>
References: <CAG8k2+7BYcg5RRnUhjY6h7jKsZUCeYCMp-mUohPSSsh5kmaQFw@mail.gmail.com>
	<50236FD5.2060404@egenix.com>
	<CAG8k2+5Eygfk5MFx35OG98RhxcrVpDXC+0Ph9S+ecG4ahG5WWw@mail.gmail.com>
	<CAG8k2+59BaH_a4FcDy-H731b0g8Gk-9urWr4VvJ_bkQ3AZ-RMA@mail.gmail.com>
Message-ID: <CADiSq7eU46twFdBV6+7NFbek9ddhX7RnJUvUcvZOgujthqL+vA@mail.gmail.com>

On Sun, Sep 9, 2012 at 1:41 PM, Daniel Holth <dholth at gmail.com> wrote:
> Re: http://www.python.org/dev/peps/pep-0425/ "Compatibility tags for
> built distributions"
>
> Progress towards a proper set of rules for generating the tags a
> Python implementation is likely to support.
>
> This system of being willing to install older built distributions is
> intended to solve the frustrating problem with eggs that you would
> have to build a new egg for each Python release, even for a
> pure-Python egg that probably runs fine on a newer Python.

Yep, those rules look sensible to me (and thanks for clarifying the
intended semantics of the "py2" and "py3" version markers)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia


From simon.sapin at kozea.fr  Mon Sep 10 11:21:49 2012
From: simon.sapin at kozea.fr (Simon Sapin)
Date: Mon, 10 Sep 2012 11:21:49 +0200
Subject: [Python-ideas] issue15824
In-Reply-To: <k1og84$rde$1@ger.gmane.org>
References: <CAE=pOyHwjb21uJOMiAT-k8dHbmZkKLOn=SHJBWNKqGx7qCfo2w@mail.gmail.com>
	<k1og84$rde$1@ger.gmane.org>
Message-ID: <504DB12D.4060506@kozea.fr>

Le 30/08/2012 21:51, Serhiy Storchaka a ?crit :
> On 30.08.12 22:03, Ben Toews wrote:
>> I think this is really clunky. I don't see any reason why we should be
>> using a type that doesn't support item assignment and needs to be
>> casted to a another type in order to make changes.
>
> Mutable urlparse result is backward incompatible. For now this result
> can be used as dict key.


Note that namedtuples *are* immutables, but they have a _replace method 
that returns a new namedtuple.

This method could also be extended to handle the properties that are 
computed dynamically (username, password, host, port).

Regards,
-- 
Simon Sapin


From masklinn at masklinn.net  Thu Sep 13 15:15:03 2012
From: masklinn at masklinn.net (Masklinn)
Date: Thu, 13 Sep 2012 15:15:03 +0200
Subject: [Python-ideas] Optional kwarg making attrgetter & itemgetter always
	return a tuple
Message-ID: <F747C5D0-8C98-4E68-8287-07A2576656BD@masklinn.net>

attrgetter and itemgetter are both very useful functions, but both have
a significant pitfall if the arguments passed in are validated but not
controlled: if receiving the arguments (list of attributes, keys or
indexes) from an external source and *-applying it, if the external
source passes a sequence of one element both functions will in turn
return an element rather than a singleton (1-element tuple).

This means such code, for instance code "slicing" a matrix of some sort
to get only some columns and getting the slicing information from its
caller (in situation where extracting a single column may be perfectly
sensible) will have to implement a manual dispatch between a "manual"
getitem (or getattr) and an itemgetter (resp. attrgetter) call, e.g.

    slicer = (operator.itemgetter(*indices) if len(indices) > 1
              else lambda ar: [ar[indices[0]])

This makes for more verbose and less straightforward code, I think it
would be useful to such situations if attrgetter and itemgetter could be
forced into always returning a tuple by way of an optional argument:

    # works the same no matter what len(indices) is
    slicer = operator.itemgetter(*indices, force_tuple=True)

which in the example equivalences[0] would be an override (to False) of
the `len` check (`len(items) == 1` would become `len(items) == 1 and not
force_tuple`)

The argument is backward-compatible as neither function currently
accepts any keyword argument.

Uncertainty note: whether force_tuple (or whatever its name is)
silences the error generated when len(indices) == 0, and returns
a null tuple rather than raising a TypeError.

[0] http://docs.python.org/dev/library/operator.html#operator.attrgetter


From tjreedy at udel.edu  Thu Sep 13 21:11:22 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 13 Sep 2012 15:11:22 -0400
Subject: [Python-ideas] Optional kwarg making attrgetter & itemgetter
 always return a tuple
In-Reply-To: <F747C5D0-8C98-4E68-8287-07A2576656BD@masklinn.net>
References: <F747C5D0-8C98-4E68-8287-07A2576656BD@masklinn.net>
Message-ID: <k2tb5h$ha9$1@ger.gmane.org>

On 9/13/2012 9:15 AM, Masklinn wrote:
> attrgetter and itemgetter are both very useful functions, but both have
> a significant pitfall if the arguments passed in are validated but not
> controlled: if receiving the arguments (list of attributes, keys or
> indexes) from an external source and *-applying it, if the external
> source passes a sequence of one element both functions will in turn
> return an element rather than a singleton (1-element tuple).
>
> This means such code, for instance code "slicing" a matrix of some sort
> to get only some columns and getting the slicing information from its
> caller (in situation where extracting a single column may be perfectly
> sensible) will have to implement a manual dispatch between a "manual"
> getitem (or getattr) and an itemgetter (resp. attrgetter) call, e.g.
>
>      slicer = (operator.itemgetter(*indices) if len(indices) > 1
>                else lambda ar: [ar[indices[0]])
>
> This makes for more verbose and less straightforward code, I think it
> would be useful to such situations if attrgetter and itemgetter could be
> forced into always returning a tuple by way of an optional argument:
>
>      # works the same no matter what len(indices) is
>      slicer = operator.itemgetter(*indices, force_tuple=True)
>
> which in the example equivalences[0] would be an override (to False) of
> the `len` check (`len(items) == 1` would become `len(items) == 1 and not
> force_tuple`)
>
> The argument is backward-compatible as neither function currently
> accepts any keyword argument.
>
> Uncertainty note: whether force_tuple (or whatever its name is)
> silences the error generated when len(indices) == 0, and returns
> a null tuple rather than raising a TypeError.
>
> [0] http://docs.python.org/dev/library/operator.html#operator.attrgetter

This seems like a plausible idea. The actual C version requires one 
argument. The Python equivalent in the doc does not (hence the different 
signature), as it would return an empty tuple for empty *items.

-- 
Terry Jan Reedy



From poalman at gmail.com  Thu Sep 13 23:05:22 2012
From: poalman at gmail.com (Paul Wiseman)
Date: Thu, 13 Sep 2012 22:05:22 +0100
Subject: [Python-ideas] syntax to continue into the next subsequent except
	block
Message-ID: <CACgdh2jh4MDJ0Rye1-HeUHy2iU1oe1JqnTTShb66rP3B9Mrg9w@mail.gmail.com>

I think it would be useful if there was a way to skip into the next except
block, perhaps with continue as I think it's currently always illegal to
use in an except block. I don't believe there's currently a way to do this.

This is my reasoning, often there's multiple reasons for exceptions that
raise the same exception, as an example an IOError might get raised for
lots of different reasons. If you want to handle one or several of these
reasons, you have to catch all exceptions of this type, but there's not
really a way to "put back" the exception if it isn't the type you were
after. For instance

try:
    operation()
except IOError as err:
    if err.errno == 2:
        do_something()
    else:
        continue #This would continue the except down to the next check,
except Exception
except Exception as err:
    logger.error("Error performing operation: {}".format(err.message)")
    some_clean_up()
    raise


The current alternatives to get this behaviour I don't believe are as nice,
but maybe I'm missing something

This works but clearly not as nice with nested try excepts,
try:
    try:
        operation()
    except IOError as err:
        if err.errno == 2:
            do_something()
        else:
            raise
except Exception as err:
    logger.error("Error performing operation: {}".format(err.message))
    some_clean_up()
    raise

This is clearly a not very good and un-dry solution:
try:
    operation()
except IOError as err:
    if err.errno == 2:
        do_something()
    else:
        logger.error("Error performing operation: {}".format(err.message))
        some_clean_up()
        raise
except Exception as err:
    logger.error("Error performing operation: {}".format(err.message))
    some_clean_up()
    raise

There's the option of using a context manager, but personally I don't think
it's as explicit or as obvious as a try except block, but maybe others
would disagree
class SpecificCaseErrorHandler(object):
    def __enter__(self):
        pass

    def __exit__(self, exc_type, exc_value, tb):
        if exc_type is not None:
            if exc_type is IOError and exc_value.errno == 2:
                do_something()
                return True
                  logger.error("Error performing operation:
{}".format(err.message))
            some_clean_up()

with SpecificCaseErrorHandler():
    operation()
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120913/c87a5993/attachment.html>

From scott+python-ideas at scottdial.com  Fri Sep 14 00:28:56 2012
From: scott+python-ideas at scottdial.com (Scott Dial)
Date: Thu, 13 Sep 2012 18:28:56 -0400
Subject: [Python-ideas] syntax to continue into the next subsequent
 except block
In-Reply-To: <CACgdh2jh4MDJ0Rye1-HeUHy2iU1oe1JqnTTShb66rP3B9Mrg9w@mail.gmail.com>
References: <CACgdh2jh4MDJ0Rye1-HeUHy2iU1oe1JqnTTShb66rP3B9Mrg9w@mail.gmail.com>
Message-ID: <50525E28.2040409@scottdial.com>

On 9/13/2012 5:05 PM, Paul Wiseman wrote:
> I think it would be useful if there was a way to skip into the next
> except block, perhaps with continue as I think it's currently always
> illegal to use in an except block. I don't believe there's currently a
> way to do this.
> 
> This is my reasoning, often there's multiple reasons for exceptions that
> raise the same exception, as an example an IOError might get raised for
> lots of different reasons. If you want to handle one or several of these
> reasons, you have to catch all exceptions of this type, but there's not
> really a way to "put back" the exception if it isn't the type you were
> after. For instance
> 
> try:
>     operation()
> except IOError as err:
>     if err.errno == 2:
>         do_something()
>     else:
>         continue #This would continue the except down to the next check,
> except Exception
> except Exception as err:
>     logger.error("Error performing operation: {}".format(err.message)")
>     some_clean_up()
>     raise
> 

"continue" already has a meaning that would make this ambiguous:

for i in range(10):
    try:
        raise IOError()
    except IOError as err:
        continue

Also, I would inevitably write what you want as:

try:
    operation()
except Exception as err:
    if isinstance(err, IOError):
        if err.errno == 2:
            do_something()
    else:
        logger.error(
            "Error performing operation: {}".format(err.message)")
        some_clean_up()
        raise

-- 
Scott Dial
scott at scottdial.com


From tjreedy at udel.edu  Fri Sep 14 01:46:05 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 13 Sep 2012 19:46:05 -0400
Subject: [Python-ideas] syntax to continue into the next subsequent
	except block
In-Reply-To: <CACgdh2jh4MDJ0Rye1-HeUHy2iU1oe1JqnTTShb66rP3B9Mrg9w@mail.gmail.com>
References: <CACgdh2jh4MDJ0Rye1-HeUHy2iU1oe1JqnTTShb66rP3B9Mrg9w@mail.gmail.com>
Message-ID: <k2tr8k$kcg$1@ger.gmane.org>

On 9/13/2012 5:05 PM, Paul Wiseman wrote:
> I think it would be useful if there was a way to skip into the next
> except block, perhaps with continue as I think it's currently always
> illegal to use in an except block. I don't believe there's currently a
> way to do this.
>
> This is my reasoning, often there's multiple reasons for exceptions that
> raise the same exception, as an example an IOError might get raised for
> lots of different reasons. If you want to handle one or several of these
> reasons, you have to catch all exceptions of this type, but there's not
> really a way to "put back" the exception if it isn't the type you were
> after. For instance
>
> try:
>      operation()
> except IOError as err:
>      if err.errno == 2:
>          do_something()
>      else:
>          continue #This would continue the except down to the next
> check, except Exception
> except Exception as err:
>      logger.error("Error performing operation: {}".format(err.message)")
>      some_clean_up()
>      raise

> The current alternatives to get this behaviour I don't believe are as
> nice, but maybe I'm missing something

As you already know, raise puts the exception back, in a sense

try:
   try:
     operation()
   except IOError as err:
     if err.errno == 2:
        do_something()
     else:
       raise
except Exception as err:
      logger.error("Error performing operation: {}".format(err.message)")
      some_clean_up()
      raise

or probably better

try:
     operation()
except Exception as err:
     if isinstance(err, IOError) and err.errno == 2:
        do_something()
     else:
         logger.error("Error performing operation: {}".format(err.message)")
         some_clean_up()
         raise

-- 
Terry Jan Reedy



From steve at pearwood.info  Fri Sep 14 03:20:38 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 14 Sep 2012 11:20:38 +1000
Subject: [Python-ideas] Optional kwarg making attrgetter & itemgetter
 always return a tuple
In-Reply-To: <F747C5D0-8C98-4E68-8287-07A2576656BD@masklinn.net>
References: <F747C5D0-8C98-4E68-8287-07A2576656BD@masklinn.net>
Message-ID: <50528666.7090400@pearwood.info>

On 13/09/12 23:15, Masklinn wrote:
> attrgetter and itemgetter are both very useful functions, but both have
> a significant pitfall if the arguments passed in are validated but not
> controlled: if receiving the arguments (list of attributes, keys or
> indexes) from an external source and *-applying it, if the external
> source passes a sequence of one element both functions will in turn
> return an element rather than a singleton (1-element tuple).

For those who, like me, had to read this three or four times to work out
what Masklinn is talking about, I think he is referring to the fact that
attrgetter and itemgetter both return a single element if passed a single
index, otherwise they return a tuple of results.

If a call itemgetter(*args)(some_list) returns a tuple, was that tuple
a single element (and args contained a single index) or was the tuple
a collection of individual elements (and args contained multiple
indexes)?

py> itemgetter(*[1])(['a', ('b', 'c'), 'd'])
('b', 'c')
py> itemgetter(*[1, 2])(['a', 'b', 'c', 'd'])
('b', 'c')


> This means such code, for instance code "slicing" a matrix of some sort
> to get only some columns and getting the slicing information from its
> caller (in situation where extracting a single column may be perfectly
> sensible) will have to implement a manual dispatch between a "manual"
> getitem (or getattr) and an itemgetter (resp. attrgetter) call, e.g.
>
>      slicer = (operator.itemgetter(*indices) if len(indices)>  1
>                else lambda ar: [ar[indices[0]])


Why is this a problem? If you don't like writing this out in place, write
it once in a helper function. Not every short code snippet needs to be in
the standard library.


> This makes for more verbose and less straightforward code, I think it
> would be useful to such situations if attrgetter and itemgetter could be
> forced into always returning a tuple by way of an optional argument:

-1

There is no need to add extra complexity to itemgetter and attrgetter for
something best solved in your code. Write a helper:

def slicer(*indexes):
     getter = itemgetter(*indexes)
     if len(indexes) == 1:
         return lambda seq: (getter(seq), )  # Wrap in a tuple.
     return getter



-- 
Steven


From stephen at xemacs.org  Fri Sep 14 03:52:39 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 14 Sep 2012 10:52:39 +0900
Subject: [Python-ideas] syntax to continue into the next
	subsequent	except block
In-Reply-To: <k2tr8k$kcg$1@ger.gmane.org>
References: <CACgdh2jh4MDJ0Rye1-HeUHy2iU1oe1JqnTTShb66rP3B9Mrg9w@mail.gmail.com>
	<k2tr8k$kcg$1@ger.gmane.org>
Message-ID: <87k3vxqsk8.fsf@uwakimon.sk.tsukuba.ac.jp>

Terry Reedy writes:

 > try:
 >    try:

Ugh-ugh.<0.5 wink>

 > try:
 >      operation()
 > except Exception as err:
 >      if isinstance(err, IOError) and err.errno == 2:

Ugh.<0.5 wink>

Not your fault, but these constructions are pretty ugly IMO, I have to
go with the OP on that.

ISTR there were discussions of "qualified except" clauses here maybe
6mo to 1yr ago?  That is, they'd look something like

try:
     operation()
except IOError as err if err.errno == 2:
     do_something()
except Exception:
     logger.error("Error performing operation: {}".format(err.message)")
     some_clean_up()
     raise

Again ISTR that this got spiked for some reason, but maybe it will be
of use to the OP in formulating his next idea.  Sorry for the lack of
precise reference.


From ncoghlan at gmail.com  Fri Sep 14 04:32:04 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 14 Sep 2012 12:32:04 +1000
Subject: [Python-ideas] syntax to continue into the next subsequent
	except block
In-Reply-To: <87k3vxqsk8.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CACgdh2jh4MDJ0Rye1-HeUHy2iU1oe1JqnTTShb66rP3B9Mrg9w@mail.gmail.com>
	<k2tr8k$kcg$1@ger.gmane.org>
	<87k3vxqsk8.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CADiSq7dDz6UU1O0jBba_11LYSakyH-8EPoc7bS=76fkSwvg26w@mail.gmail.com>

On Fri, Sep 14, 2012 at 11:52 AM, Stephen J. Turnbull
<stephen at xemacs.org> wrote:
> ISTR there were discussions of "qualified except" clauses here maybe
> 6mo to 1yr ago?  That is, they'd look something like
>
> try:
>      operation()
> except IOError as err if err.errno == 2:
>      do_something()
> except Exception:
>      logger.error("Error performing operation: {}".format(err.message)")
>      some_clean_up()
>      raise
>
> Again ISTR that this got spiked for some reason, but maybe it will be
> of use to the OP in formulating his next idea.  Sorry for the lack of
> precise reference.

They were one of the ideas discussed when Antoine was writing PEP
3151. As I recall, nobody could think of any good use cases that
didn't involve errno checking, and PEP 3151 provides a far more
elegant (and cross-platform) solution to most problems that require
errno checking in versions prior to 3.3.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia


From greg.ewing at canterbury.ac.nz  Fri Sep 14 02:26:59 2012
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 14 Sep 2012 12:26:59 +1200
Subject: [Python-ideas] syntax to continue into the next subsequent
 except block
In-Reply-To: <CACgdh2jh4MDJ0Rye1-HeUHy2iU1oe1JqnTTShb66rP3B9Mrg9w@mail.gmail.com>
References: <CACgdh2jh4MDJ0Rye1-HeUHy2iU1oe1JqnTTShb66rP3B9Mrg9w@mail.gmail.com>
Message-ID: <505279D3.9070709@canterbury.ac.nz>

Paul Wiseman wrote:
> try:
>     operation()
> except IOError as err:
>     if err.errno == 2:
>         do_something()
>     else:
>         continue #This would continue the except down to the next check,

I think a better way to address this would be to allow guard
expressions on the except clauses.

    try:
       operation()
    except IOError as err if err.errno == 2:
       ...

-- 
Greg


From masklinn at masklinn.net  Fri Sep 14 09:43:38 2012
From: masklinn at masklinn.net (Masklinn)
Date: Fri, 14 Sep 2012 09:43:38 +0200
Subject: [Python-ideas] Optional kwarg making attrgetter & itemgetter
	always return a tuple
In-Reply-To: <50528666.7090400@pearwood.info>
References: <F747C5D0-8C98-4E68-8287-07A2576656BD@masklinn.net>
	<50528666.7090400@pearwood.info>
Message-ID: <61323E14-01CE-4A1E-9533-292B093F51BC@masklinn.net>

On 2012-09-14, at 03:20 , Steven D'Aprano wrote:
>> This means such code, for instance code "slicing" a matrix of some sort
>> to get only some columns and getting the slicing information from its
>> caller (in situation where extracting a single column may be perfectly
>> sensible) will have to implement a manual dispatch between a "manual"
>> getitem (or getattr) and an itemgetter (resp. attrgetter) call, e.g.
>> 
>>     slicer = (operator.itemgetter(*indices) if len(indices)>  1
>>               else lambda ar: [ar[indices[0]])
> 
> 
> Why is this a problem?

Because it adds significant complexity to the code, and that's for the
trivial version of itemgetter, attrgetter also does keypath resolution
so the code is nowhere near this simple.

It's also anything but obvious what this snippet does on its own.

> If you don't like writing this out in place, write
> it once in a helper function. Not every short code snippet needs to be in
> the standard library.

It's not really "every short code snippet" in this case, it's a way to
avoid a sometimes deleterious special case and irregularity of the stdlib.

>> This makes for more verbose and less straightforward code, I think it
>> would be useful to such situations if attrgetter and itemgetter could be
>> forced into always returning a tuple by way of an optional argument:
> 
> -1
> 
> There is no need to add extra complexity to itemgetter and attrgetter for
> something best solved in your code.

I don't agree with this statement, the stdlib flag adds very little
extra complexity, way less than the original irregularity/special case
and way less than necessary to do it outside the stdlib. Furthermore, it
makes the solution (to having a regular output behavior for
(attr|item)getter) far more obvious and makes the code itself much simpler
to read.



From steve at pearwood.info  Fri Sep 14 11:02:54 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 14 Sep 2012 19:02:54 +1000
Subject: [Python-ideas] Optional kwarg making attrgetter & itemgetter
 always return a tuple
In-Reply-To: <61323E14-01CE-4A1E-9533-292B093F51BC@masklinn.net>
References: <F747C5D0-8C98-4E68-8287-07A2576656BD@masklinn.net>
	<50528666.7090400@pearwood.info>
	<61323E14-01CE-4A1E-9533-292B093F51BC@masklinn.net>
Message-ID: <5052F2BE.5070507@pearwood.info>

On 14/09/12 17:43, Masklinn wrote:
> On 2012-09-14, at 03:20 , Steven D'Aprano wrote:
>>> This means such code, for instance code "slicing" a matrix of some sort
>>> to get only some columns and getting the slicing information from its
>>> caller (in situation where extracting a single column may be perfectly
>>> sensible) will have to implement a manual dispatch between a "manual"
>>> getitem (or getattr) and an itemgetter (resp. attrgetter) call, e.g.
>>>
>>>      slicer = (operator.itemgetter(*indices) if len(indices)>   1
>>>                else lambda ar: [ar[indices[0]])
>>
>>
>> Why is this a problem?
>
> Because it adds significant complexity to the code,

I don't consider that to be *significant* complexity.


> and that's for the
> trivial version of itemgetter, attrgetter also does keypath resolution
> so the code is nowhere near this simple.

I don't understand what you mean by "keypath resolution". attrgetter
simply looks up the attribute(s) by name, just like obj.name would do. It
has the same API as itemgetter, except with attribute names instead of
item indexes.


> It's also anything but obvious what this snippet does on its own.

Once you get past the ternary if operator, the complexity is pretty much
entirely in the call to itemgetter. You don't even use itemgetter in the
else clause! Beyond the call to itemgetter, it's trivially simple Python
code.

slicer = operator.itemgetter(*indices, force_tuple=flag)

is equally mysterious to anyone who doesn't know what itemgetter does.


>> If you don't like writing this out in place, write
>> it once in a helper function. Not every short code snippet needs to be in
>> the standard library.
>
> It's not really "every short code snippet" in this case, it's a way to
> avoid a sometimes deleterious special case and irregularity of the stdlib.


I disagree that this is a "sometimes deleterious special case". itemgetter
and attrgetter have two APIs:

itemgetter(index)(L) => element
itemgetter(index, index, ...)(L) => tuple of elements

and likewise for attrgetter:

attrgetter(name)(L) => attribute
attrgetter(name, name, ...)(L) => tuple of attributes

Perhaps it would have been better if there were four functions rather than
two. Or if the second API were:

itemgetter(sequence_of_indexes)(L) => tuple of elements
attrgetter(sequence_of_names)(L) => tuple of attributes

so that the two getters always took a single argument, and dispatched on
whether that argument is an atomic value or a sequence. But either way,
it is not what I consider a "special case" so much as two related non-
special cases.

But let's not argue about definitions. Special case or not, can you
demonstrate that the situation is not only deleterious, but cannot be
reasonably fixed with a helper function?

Whenever you call itemgetter, there is no ambiguity because you always know
whether you are calling it with a single index or multiple indexes.



>>> This makes for more verbose and less straightforward code, I think it
>>> would be useful to such situations if attrgetter and itemgetter could be
>>> forced into always returning a tuple by way of an optional argument:
>>
>> -1
>>
>> There is no need to add extra complexity to itemgetter and attrgetter for
>> something best solved in your code.
>
> I don't agree with this statement, the stdlib flag adds very little
> extra complexity, way less than the original irregularity/special case

Whether or not it is empirically less than the complexity already there in
itemgetter, it would still be adding extra complexity. It simply isn't
possible to end up with *less* complexity by *adding* features.

(Complexity is not always a bad thing. If we wanted to program in something
simple, we would program using a Turing machine.)

The reader now has to consider "what does the force_tuple argument do?"
which is not necessarily trivial nor obvious. I expect a certain number of
beginners who don't read documentation will assume that you have to do this:

slicer = itemgetter(1, 2, 3, force_tuple=False)

if they want to pass something other than a tuple to slicer. Don't imagine
that adding an additional argument will make itemgetter and attrgetter
*simpler* to understand.


To me, a major red-flag for your suggested API can be seen here:

itemgetter(1, 2, 3, 4, force_tuple=False)

What should this do? I consider all the alternatives to be less than
ideal:

- ignore the explicit keyword argument and return a tuple anyway
- raise an exception

To say nothing of more... imaginative... semantics:

- return a list, or a set, anything but a tuple
- return a single element instead of four (but which one?)

The suggested API is not as straight-forward as you seem to think it is.


> and way less than necessary to do it outside the stdlib. Furthermore, it
> makes the solution (to having a regular output behavior for
> (attr|item)getter) far more obvious and makes the code itself much simpler
> to read.

The only thing I will grant is that it aids in discoverability of a
solution: you don't have to think of the (trivial) solution yourself, you
just need to read the documentation. But I don't see either the problem
or the solution to be great enough to justify adding an argument, writing
new documentation, and doubling the number of tests for both itemgetter and
attrgetter.



-- 
Steven


From masklinn at masklinn.net  Fri Sep 14 11:29:47 2012
From: masklinn at masklinn.net (Masklinn)
Date: Fri, 14 Sep 2012 11:29:47 +0200
Subject: [Python-ideas] Optional kwarg making attrgetter & itemgetter
	always return a tuple
In-Reply-To: <5052F2BE.5070507@pearwood.info>
References: <F747C5D0-8C98-4E68-8287-07A2576656BD@masklinn.net>
	<50528666.7090400@pearwood.info>
	<61323E14-01CE-4A1E-9533-292B093F51BC@masklinn.net>
	<5052F2BE.5070507@pearwood.info>
Message-ID: <B211DBA5-5E4C-443B-92F7-65CEEFE68B1B@masklinn.net>

On 2012-09-14, at 11:02 , Steven D'Aprano wrote
>> and that's for the
>> trivial version of itemgetter, attrgetter also does keypath resolution
>> so the code is nowhere near this simple.
> 
> I don't understand what you mean by "keypath resolution". attrgetter
> simply looks up the attribute(s) by name, just like obj.name would do. It
> has the same API as itemgetter, except with attribute names instead of
> item indexes.

It takes dotted paths, not just attribute names

>> It's also anything but obvious what this snippet does on its own.
> 
> Once you get past the ternary if operator, the complexity is pretty much
> entirely in the call to itemgetter. You don't even use itemgetter in the
> else clause! Beyond the call to itemgetter, it's trivially simple Python
> code.
> 
> slicer = operator.itemgetter(*indices, force_tuple=flag)
> 
> is equally mysterious to anyone who doesn't know what itemgetter does.

I would expect either foreknowledge or reading up on it to be obvious
in the context of its usage.

>>> If you don't like writing this out in place, write
>>> it once in a helper function. Not every short code snippet needs to be in
>>> the standard library.
>> 
>> It's not really "every short code snippet" in this case, it's a way to
>> avoid a sometimes deleterious special case and irregularity of the stdlib.
> 
> 
> I disagree that this is a "sometimes deleterious special case". itemgetter
> and attrgetter have two APIs:
> 
> itemgetter(index)(L) => element
> itemgetter(index, index, ...)(L) => tuple of elements
> 
> and likewise for attrgetter:
> 
> attrgetter(name)(L) => attribute
> attrgetter(name, name, ...)(L) => tuple of attributes
> 
> Perhaps it would have been better if there were four functions rather than
> two. Or if the second API were:
> 
> itemgetter(sequence_of_indexes)(L) => tuple of elements
> attrgetter(sequence_of_names)(L) => tuple of attributes
> 
> so that the two getters always took a single argument, and dispatched on
> whether that argument is an atomic value or a sequence. But either way,
> it is not what I consider a "special case" so much as two related non-
> special cases.

Which conflict for a sequence of length 1, which is the very reason
why I started this thread.

> But let's not argue about definitions. Special case or not, can you
> demonstrate that the situation is not only deleterious, but cannot be
> reasonably fixed with a helper function?

Which as usual hinges on the definition of "reasonably", of course the
situation can be "fixed" (with "reasonably" being a wholly personal
value judgement) with a helper function or a reimplementation of an
(attr|item)getter-like function from scratch. As it can pretty much
always be. I don't see that as a very useful benchmark.

> Whenever you call itemgetter, there is no ambiguity because you always know
> whether you are calling it with a single index or multiple indexes.

That is not quite correct, even ignoring that you have to call `len` to
do so when the indices are provided by a third party, the correct code
gets yet more complex as the third party could provide an iterator which
would have to be reified before being passed to len(), increasing the
complexity of the "helper" yet again.

>>>> This makes for more verbose and less straightforward code, I think it
>>>> would be useful to such situations if attrgetter and itemgetter could be
>>>> forced into always returning a tuple by way of an optional argument:
>>> 
>>> -1
>>> 
>>> There is no need to add extra complexity to itemgetter and attrgetter for
>>> something best solved in your code.
>> 
>> I don't agree with this statement, the stdlib flag adds very little
>> extra complexity, way less than the original irregularity/special case
> 
> Whether or not it is empirically less than the complexity already there in
> itemgetter, it would still be adding extra complexity. It simply isn't
> possible to end up with *less* complexity by *adding* features.

At no point did I deny that, as far as I know or can see.

> (Complexity is not always a bad thing. If we wanted to program in something
> simple, we would program using a Turing machine.)
> 
> The reader now has to consider "what does the force_tuple argument do?"
> which is not necessarily trivial nor obvious. I expect a certain number of
> beginners who don't read documentation will assume that you have to do this:
> 
> slicer = itemgetter(1, 2, 3, force_tuple=False)
> 
> if they want to pass something other than a tuple to slicer. Don't imagine
> that adding an additional argument will make itemgetter and attrgetter
> *simpler* to understand.
> 
> 
> To me, a major red-flag for your suggested API can be seen here:
> 
> itemgetter(1, 2, 3, 4, force_tuple=False)
> 
> What should this do?

The exact same as `itemgetter(1, 2, 3, 4)`, since `force_tuple` defaults
to False.

> I consider all the alternatives to be less than
> ideal:
> 
> - ignore the explicit keyword argument and return a tuple anyway
> - raise an exception
> 
> To say nothing of more... imaginative... semantics:
> 
> - return a list, or a set, anything but a tuple
> - return a single element instead of four (but which one?)

I have trouble seeing how such interpretations can be drawn up from
explicitly providing the default value for the argument. Does anyone
really expect dict.get(key, None) to always return None?

> The suggested API is not as straight-forward as you seem to think it is.

It's simply a proposal to fix what I see as an issue (as befits to
python-ideas), you're getting way too hung up on something which can
quite trivially be discussed and changed.

>> and way less than necessary to do it outside the stdlib. Furthermore, it
>> makes the solution (to having a regular output behavior for
>> (attr|item)getter) far more obvious and makes the code itself much simpler
>> to read.
> 
> The only thing I will grant is that it aids in discoverability of a
> solution

It also aids in the discoverability of the problem in the first place, and
in limiting the surprise when unexpectedly encountering it for the first
time.



From wuwei23 at gmail.com  Fri Sep 14 11:41:43 2012
From: wuwei23 at gmail.com (alex23)
Date: Fri, 14 Sep 2012 02:41:43 -0700 (PDT)
Subject: [Python-ideas] Optional kwarg making attrgetter & itemgetter
	always return a tuple
In-Reply-To: <F747C5D0-8C98-4E68-8287-07A2576656BD@masklinn.net>
References: <F747C5D0-8C98-4E68-8287-07A2576656BD@masklinn.net>
Message-ID: <e79f71ad-46c2-4be3-919a-d093f42de6fb@s9g2000pbh.googlegroups.com>

On Sep 13, 11:15?pm, Masklinn <maskl... at masklinn.net> wrote:
> ? ? # works the same no matter what len(indices) is
> ? ? slicer = operator.itemgetter(*indices, force_tuple=True)

I'd be inclined to write that as:

    slicer = force_tuple(operator.itemgetter(*indices))

With force_tuple then just being another decorator.


From ncoghlan at gmail.com  Fri Sep 14 13:01:04 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 14 Sep 2012 21:01:04 +1000
Subject: [Python-ideas] Optional kwarg making attrgetter & itemgetter
 always return a tuple
In-Reply-To: <F747C5D0-8C98-4E68-8287-07A2576656BD@masklinn.net>
References: <F747C5D0-8C98-4E68-8287-07A2576656BD@masklinn.net>
Message-ID: <CADiSq7fYVR_s3wH8z2HWmHDqO-zjHKGgtxzYkzOSNMocUQp0Zg@mail.gmail.com>

On Thu, Sep 13, 2012 at 11:15 PM, Masklinn <masklinn at masklinn.net> wrote:
> attrgetter and itemgetter are both very useful functions, but both have
> a significant pitfall if the arguments passed in are validated but not
> controlled: if receiving the arguments (list of attributes, keys or
> indexes) from an external source and *-applying it, if the external
> source passes a sequence of one element both functions will in turn
> return an element rather than a singleton (1-element tuple).

Both attrgetter and itemgetter are really designed to be called with
*literal* arguments, not via *args. In particular, they are designed
to be useful as arguments bound to a "key" parameter, where the object
vs singleton tuple distinction doesn't matter.

If that behaviour is not desirable, *write a different function* that
does what you want, and don't use itemgetter or attrgetter at all.
These tools are designed as convenience functions for a particular use
case (specifically sorting, and similar ordering operations). Outside
those use cases, you will need to drop back down to the underlying
building blocks and produce your *own* tool from the same raw
materials.

For example:

    def my_itemgetter(*subscripts):
        def f(obj):
            return tuple(obj[x] for x in subscripts)
        return f

I agree attrgetter is slightly more complex due to the fact that it
*also* handles chained lookups, where getattr does not, but that's a
matter of making the case for providing chained lookup (or even
str.format style field value lookup) as a more readily accessible
building block, not for making the attrgetter API more complicated.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia


From masklinn at masklinn.net  Fri Sep 14 13:36:39 2012
From: masklinn at masklinn.net (Masklinn)
Date: Fri, 14 Sep 2012 13:36:39 +0200
Subject: [Python-ideas] Optional kwarg making attrgetter & itemgetter
	always return a tuple
In-Reply-To: <CADiSq7fYVR_s3wH8z2HWmHDqO-zjHKGgtxzYkzOSNMocUQp0Zg@mail.gmail.com>
References: <F747C5D0-8C98-4E68-8287-07A2576656BD@masklinn.net>
	<CADiSq7fYVR_s3wH8z2HWmHDqO-zjHKGgtxzYkzOSNMocUQp0Zg@mail.gmail.com>
Message-ID: <93B4E6EE-8556-45E2-9F40-BCF7D6609328@masklinn.net>

On 2012-09-14, at 13:01 , Nick Coghlan wrote:
> On Thu, Sep 13, 2012 at 11:15 PM, Masklinn <masklinn at masklinn.net> wrote:
>> attrgetter and itemgetter are both very useful functions, but both have
>> a significant pitfall if the arguments passed in are validated but not
>> controlled: if receiving the arguments (list of attributes, keys or
>> indexes) from an external source and *-applying it, if the external
>> source passes a sequence of one element both functions will in turn
>> return an element rather than a singleton (1-element tuple).
> 
> Both attrgetter and itemgetter are really designed to be called with
> *literal* arguments, not via *args. In particular, they are designed
> to be useful as arguments bound to a "key" parameter, where the object
> vs singleton tuple distinction doesn't matter.

It was my understanding that they are also designed to be useful for
mapping (such a usage is shown in itemgetter's examples), which is
a superset of the use case outlined here.

> If that behaviour is not desirable, *write a different function* that
> does what you want, and don't use itemgetter or attrgetter at all.
> These tools are designed as convenience functions

And save for one stumbling block, they are utilities I love for their
convenience and their plain clarity of purpose.


From oscar.j.benjamin at gmail.com  Fri Sep 14 15:23:53 2012
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Fri, 14 Sep 2012 14:23:53 +0100
Subject: [Python-ideas] Optional kwarg making attrgetter & itemgetter
 always return a tuple
In-Reply-To: <93B4E6EE-8556-45E2-9F40-BCF7D6609328@masklinn.net>
References: <F747C5D0-8C98-4E68-8287-07A2576656BD@masklinn.net>
	<CADiSq7fYVR_s3wH8z2HWmHDqO-zjHKGgtxzYkzOSNMocUQp0Zg@mail.gmail.com>
	<93B4E6EE-8556-45E2-9F40-BCF7D6609328@masklinn.net>
Message-ID: <CAHVvXxRAByXPYdRZ_y39gnTKrFL2HDwJn8nqVs9oJFaaa7SCRQ@mail.gmail.com>

On 14 September 2012 12:36, Masklinn <masklinn at masklinn.net> wrote:

> On 2012-09-14, at 13:01 , Nick Coghlan wrote:
> > On Thu, Sep 13, 2012 at 11:15 PM, Masklinn <masklinn at masklinn.net>
> wrote:
> >> attrgetter and itemgetter are both very useful functions, but both have
> >> a significant pitfall if the arguments passed in are validated but not
> >> controlled: if receiving the arguments (list of attributes, keys or
> >> indexes) from an external source and *-applying it, if the external
> >> source passes a sequence of one element both functions will in turn
> >> return an element rather than a singleton (1-element tuple).
> >
> > Both attrgetter and itemgetter are really designed to be called with
> > *literal* arguments, not via *args. In particular, they are designed
> > to be useful as arguments bound to a "key" parameter, where the object
> > vs singleton tuple distinction doesn't matter.
>
> It was my understanding that they are also designed to be useful for
> mapping (such a usage is shown in itemgetter's examples), which is
> a superset of the use case outlined here.
>
> > If that behaviour is not desirable, *write a different function* that
> > does what you want, and don't use itemgetter or attrgetter at all.
> > These tools are designed as convenience functions
>

I can see why you would expect different behaviour here, though. I tend not
to think of the functions in the operator module as convenience functions
but as *efficient* nameable functions referring to operations that are
normally invoked with a non-function syntax. Which is more convenient out
of the following:

1) using operator
import operator
result = sorted(values, key=operator.attrgetter('name'))

2) using lambda
result = sorted(values, key=lambda v: v.name)

I don't think that the operator module is convenient and I think that it
damages readability in many cases. My primary reason for choosing it in
some cases is that it is more efficient than the lambda expression.

There is no special syntax for 'get several items as a tuple'. I didn't
know about this extended use for attrgetter, itemgetter. I can't see any
other functions in the operator module (abs, add, and_, ...) that extend
the semantics of the operation they are supposed to represent in this way.

In general it is bad to conflate scalar/sequence semantics so that a caller
should get a different type of object depending on the length of a
sequence. I can see how practicality beats purity in adding this feature
for people who want to use these functions for sorting by a couple of
elements/attributes. I think it would have been better though to add these
as separate functions itemsgetter and attrsgetter that always return tuples.

Oscar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120914/c48a1cf3/attachment.html>

From stefan at bytereef.org  Fri Sep 14 22:00:09 2012
From: stefan at bytereef.org (Stefan Krah)
Date: Fri, 14 Sep 2012 22:00:09 +0200
Subject: [Python-ideas] Memoryview tolist() method is misleading
Message-ID: <20120914200009.GA29952@sleipnir.bytereef.org>

Alexander Belopolsky wrote:
> Consider this:
> >>> memoryview(b'x').cast('B', ()).tolist()
> 120
>
> The return value of to list() is an int, not a list.


That's because NumPy's tolist() does the same thing:

>>> x = numpy.array(120, dtype='B')
>>> x
array(120, dtype=uint8)
>>> x.tolist()
120


If you implement tolist() recursively like in _testbuffer.c and choose
the zeroth dimension as the base case, you arrive at single elements.
So at least it's not completely unnatural.


Stefan Krah




From jimjjewett at gmail.com  Fri Sep 14 23:02:31 2012
From: jimjjewett at gmail.com (Jim Jewett)
Date: Fri, 14 Sep 2012 17:02:31 -0400
Subject: [Python-ideas] Optional kwarg making attrgetter & itemgetter
 always return a tuple
In-Reply-To: <CAHVvXxRAByXPYdRZ_y39gnTKrFL2HDwJn8nqVs9oJFaaa7SCRQ@mail.gmail.com>
References: <F747C5D0-8C98-4E68-8287-07A2576656BD@masklinn.net>
	<CADiSq7fYVR_s3wH8z2HWmHDqO-zjHKGgtxzYkzOSNMocUQp0Zg@mail.gmail.com>
	<93B4E6EE-8556-45E2-9F40-BCF7D6609328@masklinn.net>
	<CAHVvXxRAByXPYdRZ_y39gnTKrFL2HDwJn8nqVs9oJFaaa7SCRQ@mail.gmail.com>
Message-ID: <CA+OGgf4gJW6qg_Ykbo_ZsSpfy5ZYb1sHzmRWbUOP0t+LaeQCYQ@mail.gmail.com>

On 9/14/12, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote:

> I can see why you would expect different behaviour here, though. I tend not
> to think of the functions in the operator module as convenience functions
> but as *efficient* nameable functions referring to operations that are
> normally invoked with a non-function syntax. Which is more convenient out
> of the following:

> 1) using operator
> import operator
> result = sorted(values, key=operator.attrgetter('name'))

I would normally write that as

    from operator import attrgetter as attr
    ... # may use it several times

    result=sorted(values, key=attr('name'))

which is about the best I could hope for, without being able to use
the dot itself.

> 2) using lambda
> result = sorted(values, key=lambda v: v.name)

And I honestly think that would be worse, even if lambda didn't have a
code smell.  It focuses attention on the fact that you're creating a
callable, instead of on the fact that you're grabbing the name
attribute.

> In general it is bad to conflate scalar/sequence semantics so that a caller
> should get a different type of object depending on the length of a
> sequence.

Yeah, but that can't really be solved well in python, except maybe by
never extending an API to handle sequences.  I would personally not
consider that an improvement.

Part of the problem is that the cleanest way to take a variable number
of arguments is to turn them into a sequence under the covers (*args),
even if they weren't passed that way.

-jJ


From dholth at gmail.com  Fri Sep 14 23:24:13 2012
From: dholth at gmail.com (Daniel Holth)
Date: Fri, 14 Sep 2012 17:24:13 -0400
Subject: [Python-ideas] Why is there no way to pass PYTHONPATH on the
	command line?
Message-ID: <CAG8k2+411N3h9Q+H73mWmsf9_EhamQkXz0QVzE7Lj-6hmgnGLw@mail.gmail.com>

Why is there no way to pass PYTHONPATH on the command line? Oversight
or intentional?

Given path_item/something.py

python -p path_item -c "import something; something.foo()"

I am aware that the __main__.py behavior lessens the need for this
significantly.


From mikegraham at gmail.com  Sat Sep 15 00:10:17 2012
From: mikegraham at gmail.com (Mike Graham)
Date: Fri, 14 Sep 2012 18:10:17 -0400
Subject: [Python-ideas] Why is there no way to pass PYTHONPATH on the
 command line?
In-Reply-To: <CAG8k2+411N3h9Q+H73mWmsf9_EhamQkXz0QVzE7Lj-6hmgnGLw@mail.gmail.com>
References: <CAG8k2+411N3h9Q+H73mWmsf9_EhamQkXz0QVzE7Lj-6hmgnGLw@mail.gmail.com>
Message-ID: <CAEBZo3MhF8QDiV1Xet4d-01Pp8vOyfbYNFdJ_MgogFRGWg5h3A@mail.gmail.com>

On Fri, Sep 14, 2012 at 5:24 PM, Daniel Holth <dholth at gmail.com> wrote:
> Why is there no way to pass PYTHONPATH on the command line? Oversight
> or intentional?
>
> Given path_item/something.py
>
> python -p path_item -c "import something; something.foo()"
>
> I am aware that the __main__.py behavior lessens the need for this
> significantly.

PYTHONPATH=whatever python -c "import something; something.foo()"

Mike


From dholth at gmail.com  Sat Sep 15 00:11:29 2012
From: dholth at gmail.com (Daniel Holth)
Date: Fri, 14 Sep 2012 18:11:29 -0400
Subject: [Python-ideas] Why is there no way to pass PYTHONPATH on the
 command line?
In-Reply-To: <CAEBZo3MhF8QDiV1Xet4d-01Pp8vOyfbYNFdJ_MgogFRGWg5h3A@mail.gmail.com>
References: <CAG8k2+411N3h9Q+H73mWmsf9_EhamQkXz0QVzE7Lj-6hmgnGLw@mail.gmail.com>
	<CAEBZo3MhF8QDiV1Xet4d-01Pp8vOyfbYNFdJ_MgogFRGWg5h3A@mail.gmail.com>
Message-ID: <CAG8k2+7GGLw7xuG6XSRUz5sPzrdYwUcQXL4Q02_xJAJ-MgMF0Q@mail.gmail.com>

On Fri, Sep 14, 2012 at 6:10 PM, Mike Graham <mikegraham at gmail.com> wrote:
> On Fri, Sep 14, 2012 at 5:24 PM, Daniel Holth <dholth at gmail.com> wrote:
>> Why is there no way to pass PYTHONPATH on the command line? Oversight
>> or intentional?
>>
>> Given path_item/something.py
>>
>> python -p path_item -c "import something; something.foo()"
>>
>> I am aware that the __main__.py behavior lessens the need for this
>> significantly.
>
> PYTHONPATH=whatever python -c "import something; something.foo()"
>
> Mike

Yes, I've heard of that one. Does it work on Windows too?


From p.f.moore at gmail.com  Sat Sep 15 00:40:31 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 14 Sep 2012 23:40:31 +0100
Subject: [Python-ideas] Why is there no way to pass PYTHONPATH on the
 command line?
In-Reply-To: <CAG8k2+7GGLw7xuG6XSRUz5sPzrdYwUcQXL4Q02_xJAJ-MgMF0Q@mail.gmail.com>
References: <CAG8k2+411N3h9Q+H73mWmsf9_EhamQkXz0QVzE7Lj-6hmgnGLw@mail.gmail.com>
	<CAEBZo3MhF8QDiV1Xet4d-01Pp8vOyfbYNFdJ_MgogFRGWg5h3A@mail.gmail.com>
	<CAG8k2+7GGLw7xuG6XSRUz5sPzrdYwUcQXL4Q02_xJAJ-MgMF0Q@mail.gmail.com>
Message-ID: <CACac1F_p3ybv7TfWghpRxC++_G5r+R8OQf9G4zSqyqiox-qBCg@mail.gmail.com>

On 14 September 2012 23:11, Daniel Holth <dholth at gmail.com> wrote:
> On Fri, Sep 14, 2012 at 6:10 PM, Mike Graham <mikegraham at gmail.com> wrote:
>> On Fri, Sep 14, 2012 at 5:24 PM, Daniel Holth <dholth at gmail.com> wrote:
>>> Why is there no way to pass PYTHONPATH on the command line? Oversight
>>> or intentional?
>>>
>>> Given path_item/something.py
>>>
>>> python -p path_item -c "import something; something.foo()"
>>>
>>> I am aware that the __main__.py behavior lessens the need for this
>>> significantly.
>>
>> PYTHONPATH=whatever python -c "import something; something.foo()"
>>
>> Mike
>
> Yes, I've heard of that one. Does it work on Windows too?

No. Sadly.

Paul.


From timothy.c.delaney at gmail.com  Sat Sep 15 00:47:23 2012
From: timothy.c.delaney at gmail.com (Tim Delaney)
Date: Sat, 15 Sep 2012 08:47:23 +1000
Subject: [Python-ideas] Why is there no way to pass PYTHONPATH on the
 command line?
In-Reply-To: <CACac1F_p3ybv7TfWghpRxC++_G5r+R8OQf9G4zSqyqiox-qBCg@mail.gmail.com>
References: <CAG8k2+411N3h9Q+H73mWmsf9_EhamQkXz0QVzE7Lj-6hmgnGLw@mail.gmail.com>
	<CAEBZo3MhF8QDiV1Xet4d-01Pp8vOyfbYNFdJ_MgogFRGWg5h3A@mail.gmail.com>
	<CAG8k2+7GGLw7xuG6XSRUz5sPzrdYwUcQXL4Q02_xJAJ-MgMF0Q@mail.gmail.com>
	<CACac1F_p3ybv7TfWghpRxC++_G5r+R8OQf9G4zSqyqiox-qBCg@mail.gmail.com>
Message-ID: <CAN8CLg=AOP3TvQ_0oHSjyw=00n2WY6AGmTX=VxrR9b+qXRNDOg@mail.gmail.com>

On 15 September 2012 08:40, Paul Moore <p.f.moore at gmail.com> wrote:

> On 14 September 2012 23:11, Daniel Holth <dholth at gmail.com> wrote:
> > On Fri, Sep 14, 2012 at 6:10 PM, Mike Graham <mikegraham at gmail.com>
> wrote:
> >> PYTHONPATH=whatever python -c "import something; something.foo()"
> >>
> >> Mike
> >
> > Yes, I've heard of that one. Does it work on Windows too?
>
> No. Sadly.


Unless you're using either cygwin or (my preference) msys <
http://www.mingw.org/wiki/MSYS>.

Tim Delaney
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120915/2bc59c49/attachment.html>

From ericsnowcurrently at gmail.com  Sat Sep 15 00:48:15 2012
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Fri, 14 Sep 2012 16:48:15 -0600
Subject: [Python-ideas] Why is there no way to pass PYTHONPATH on the
 command line?
In-Reply-To: <CAG8k2+411N3h9Q+H73mWmsf9_EhamQkXz0QVzE7Lj-6hmgnGLw@mail.gmail.com>
References: <CAG8k2+411N3h9Q+H73mWmsf9_EhamQkXz0QVzE7Lj-6hmgnGLw@mail.gmail.com>
Message-ID: <CALFfu7C-q2FbaFkcMOL8BH4Hur_nH0ffG0uUO-1oen8Su4uEyA@mail.gmail.com>

On Fri, Sep 14, 2012 at 3:24 PM, Daniel Holth <dholth at gmail.com> wrote:
> Why is there no way to pass PYTHONPATH on the command line? Oversight
> or intentional?
>
> Given path_item/something.py
>
> python -p path_item -c "import something; something.foo()"

Do you mean something along the lines of
http://bugs.python.org/issue13475?  This would add --path0 and
--nopath0 as commandline options.  Of course, this would only be in
Python 3.4 (if it makes it in at all).

-eric


From guido at python.org  Sat Sep 15 01:03:39 2012
From: guido at python.org (Guido van Rossum)
Date: Fri, 14 Sep 2012 16:03:39 -0700
Subject: [Python-ideas] Why is there no way to pass PYTHONPATH on the
 command line?
In-Reply-To: <CALFfu7C-q2FbaFkcMOL8BH4Hur_nH0ffG0uUO-1oen8Su4uEyA@mail.gmail.com>
References: <CAG8k2+411N3h9Q+H73mWmsf9_EhamQkXz0QVzE7Lj-6hmgnGLw@mail.gmail.com>
	<CALFfu7C-q2FbaFkcMOL8BH4Hur_nH0ffG0uUO-1oen8Su4uEyA@mail.gmail.com>
Message-ID: <CAP7+vJJSYyhuG+x5TCe7hApA=7WcrJn_wWpPiWwvnA9aFrqssQ@mail.gmail.com>

On Fri, Sep 14, 2012 at 3:48 PM, Eric Snow <ericsnowcurrently at gmail.com> wrote:
> On Fri, Sep 14, 2012 at 3:24 PM, Daniel Holth <dholth at gmail.com> wrote:
>> Why is there no way to pass PYTHONPATH on the command line? Oversight
>> or intentional?
>>
>> Given path_item/something.py
>>
>> python -p path_item -c "import something; something.foo()"
>
> Do you mean something along the lines of
> http://bugs.python.org/issue13475?  This would add --path0 and
> --nopath0 as commandline options.  Of course, this would only be in
> Python 3.4 (if it makes it in at all).

Why is the proposal --path0 and not --path? (The only utility I know
with a command line option spelled as "foo0" is find, where -print0
means "like -print, but append \0 instead of \n".)

-- 
--Guido van Rossum (python.org/~guido)


From ericsnowcurrently at gmail.com  Sat Sep 15 01:17:46 2012
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Fri, 14 Sep 2012 17:17:46 -0600
Subject: [Python-ideas] Why is there no way to pass PYTHONPATH on the
 command line?
In-Reply-To: <CAP7+vJJSYyhuG+x5TCe7hApA=7WcrJn_wWpPiWwvnA9aFrqssQ@mail.gmail.com>
References: <CAG8k2+411N3h9Q+H73mWmsf9_EhamQkXz0QVzE7Lj-6hmgnGLw@mail.gmail.com>
	<CALFfu7C-q2FbaFkcMOL8BH4Hur_nH0ffG0uUO-1oen8Su4uEyA@mail.gmail.com>
	<CAP7+vJJSYyhuG+x5TCe7hApA=7WcrJn_wWpPiWwvnA9aFrqssQ@mail.gmail.com>
Message-ID: <CALFfu7B4BeqVhKA5hJyssPj6+KfR4f_Jm9SD9aWoQoXbusZNSA@mail.gmail.com>

On Fri, Sep 14, 2012 at 5:03 PM, Guido van Rossum <guido at python.org> wrote:
> On Fri, Sep 14, 2012 at 3:48 PM, Eric Snow <ericsnowcurrently at gmail.com> wrote:
>> ... issue13475 ...
> Why is the proposal --path0 and not --path? (The only utility I know
> with a command line option spelled as "foo0" is find, where -print0
> means "like -print, but append \0 instead of \n".)

It's meant to allow you to explicitly manage the sys.path[0] value
that gets set implicitly.  That it fits Daniel's use case (as I
understand it) is not coincidental, but it's not meant as a wholesale
substitute for PYTHONPATH.  That would be a separate proposal (which I
believe exists in the tracker already).

-eric


From dholth at gmail.com  Sat Sep 15 01:34:55 2012
From: dholth at gmail.com (Daniel Holth)
Date: Fri, 14 Sep 2012 19:34:55 -0400
Subject: [Python-ideas] Why is there no way to pass PYTHONPATH on the
 command line?
In-Reply-To: <CALFfu7B4BeqVhKA5hJyssPj6+KfR4f_Jm9SD9aWoQoXbusZNSA@mail.gmail.com>
References: <CAG8k2+411N3h9Q+H73mWmsf9_EhamQkXz0QVzE7Lj-6hmgnGLw@mail.gmail.com>
	<CALFfu7C-q2FbaFkcMOL8BH4Hur_nH0ffG0uUO-1oen8Su4uEyA@mail.gmail.com>
	<CAP7+vJJSYyhuG+x5TCe7hApA=7WcrJn_wWpPiWwvnA9aFrqssQ@mail.gmail.com>
	<CALFfu7B4BeqVhKA5hJyssPj6+KfR4f_Jm9SD9aWoQoXbusZNSA@mail.gmail.com>
Message-ID: <CAG8k2+6wufrUaU2jG==kd0OnUd7S6ceM3xQu4tE0UVBe8=AU_Q@mail.gmail.com>

Surely it should be the same as python path, at the front, and use - E to
ignore the environment
On Sep 14, 2012 7:17 PM, "Eric Snow" <ericsnowcurrently at gmail.com> wrote:

> On Fri, Sep 14, 2012 at 5:03 PM, Guido van Rossum <guido at python.org>
> wrote:
> > On Fri, Sep 14, 2012 at 3:48 PM, Eric Snow <ericsnowcurrently at gmail.com>
> wrote:
> >> ... issue13475 ...
> > Why is the proposal --path0 and not --path? (The only utility I know
> > with a command line option spelled as "foo0" is find, where -print0
> > means "like -print, but append \0 instead of \n".)
>
> It's meant to allow you to explicitly manage the sys.path[0] value
> that gets set implicitly.  That it fits Daniel's use case (as I
> understand it) is not coincidental, but it's not meant as a wholesale
> substitute for PYTHONPATH.  That would be a separate proposal (which I
> believe exists in the tracker already).
>
> -eric
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120914/ee11ce8d/attachment.html>

From greg at krypto.org  Sat Sep 15 08:28:59 2012
From: greg at krypto.org (Gregory P. Smith)
Date: Fri, 14 Sep 2012 23:28:59 -0700
Subject: [Python-ideas] syntax to continue into the next subsequent
	except block
In-Reply-To: <505279D3.9070709@canterbury.ac.nz>
References: <CACgdh2jh4MDJ0Rye1-HeUHy2iU1oe1JqnTTShb66rP3B9Mrg9w@mail.gmail.com>
	<505279D3.9070709@canterbury.ac.nz>
Message-ID: <CAGE7PNKm3pV-xRZOTBx_HL=s10ChzCG6ARzqgJJO3hmOgOXq0A@mail.gmail.com>

On Thu, Sep 13, 2012 at 5:26 PM, Greg Ewing <greg.ewing at canterbury.ac.nz>wrote:

> Paul Wiseman wrote:
>
>> try:
>>     operation()
>> except IOError as err:
>>     if err.errno == 2:
>>         do_something()
>>     else:
>>         continue #This would continue the except down to the next check,
>>
>
> I think a better way to address this would be to allow guard
> expressions on the except clauses.
>
>
>    try:
>       operation()
>    except IOError as err if err.errno == 2:
>       ...


I like that.  Granted, what are the use cases beyond the errno one that PEP
3151 largely addresses?


>
>
> --
> Greg
>
> ______________________________**_________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/**mailman/listinfo/python-ideas<http://mail.python.org/mailman/listinfo/python-ideas>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120914/dcc5ed88/attachment.html>

From poalman at gmail.com  Sat Sep 15 10:15:53 2012
From: poalman at gmail.com (Paul Wiseman)
Date: Sat, 15 Sep 2012 09:15:53 +0100
Subject: [Python-ideas] syntax to continue into the next subsequent
	except block
In-Reply-To: <CADiSq7dDz6UU1O0jBba_11LYSakyH-8EPoc7bS=76fkSwvg26w@mail.gmail.com>
References: <CACgdh2jh4MDJ0Rye1-HeUHy2iU1oe1JqnTTShb66rP3B9Mrg9w@mail.gmail.com>
	<k2tr8k$kcg$1@ger.gmane.org>
	<87k3vxqsk8.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7dDz6UU1O0jBba_11LYSakyH-8EPoc7bS=76fkSwvg26w@mail.gmail.com>
Message-ID: <CACgdh2igirUpwvvc04+psN60aFa0ZcCv1nmGNiE6pNdv3n8bXQ@mail.gmail.com>

On 14 September 2012 03:32, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Fri, Sep 14, 2012 at 11:52 AM, Stephen J. Turnbull
> <stephen at xemacs.org> wrote:
> > ISTR there were discussions of "qualified except" clauses here maybe
> > 6mo to 1yr ago?  That is, they'd look something like
> >
> > try:
> >      operation()
> > except IOError as err if err.errno == 2:
> >      do_something()
> > except Exception:
> >      logger.error("Error performing operation: {}".format(err.message)")
> >      some_clean_up()
> >      raise
> >
> > Again ISTR that this got spiked for some reason, but maybe it will be
> > of use to the OP in formulating his next idea.  Sorry for the lack of
> > precise reference.
>
> They were one of the ideas discussed when Antoine was writing PEP
> 3151. As I recall, nobody could think of any good use cases that
> didn't involve errno checking, and PEP 3151 provides a far more
> elegant (and cross-platform) solution to most problems that require
> errno checking in versions prior to 3.3.
>
>
Ah I didn't know about that, maybe I chose a bad example with IOError.

The reason that got me thinking is I had to handle specific
S3ResponseErrors from boto.

the S3ResponseError exception class has a code attribute (or errorcode, i
forget exactly).

Surely in this case and a lot of other modules different exceptions are
grouped together that are likely to want to be handle differently?

URLError and HTTPError I'm sure fall into this catagory


> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120915/48259b15/attachment.html>

From poalman at gmail.com  Sat Sep 15 10:21:40 2012
From: poalman at gmail.com (Paul Wiseman)
Date: Sat, 15 Sep 2012 09:21:40 +0100
Subject: [Python-ideas] syntax to continue into the next subsequent
	except block
In-Reply-To: <87k3vxqsk8.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CACgdh2jh4MDJ0Rye1-HeUHy2iU1oe1JqnTTShb66rP3B9Mrg9w@mail.gmail.com>
	<k2tr8k$kcg$1@ger.gmane.org>
	<87k3vxqsk8.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CACgdh2hCL=L4z5siVY-co0D6BXAiYycu-+gwup7q=_28=+fBew@mail.gmail.com>

On 14 September 2012 02:52, Stephen J. Turnbull <stephen at xemacs.org> wrote:

> Terry Reedy writes:
>
>  > try:
>  >    try:
>
> Ugh-ugh.<0.5 wink>
>
>  > try:
>  >      operation()
>  > except Exception as err:
>  >      if isinstance(err, IOError) and err.errno == 2:
>
> Ugh.<0.5 wink>
>
> Not your fault, but these constructions are pretty ugly IMO, I have to
> go with the OP on that.
>
> ISTR there were discussions of "qualified except" clauses here maybe
> 6mo to 1yr ago?  That is, they'd look something like
>
> try:
>      operation()
> except IOError as err if err.errno == 2:
>      do_something()
> except Exception:
>      logger.error("Error performing operation: {}".format(err.message)")
>      some_clean_up()
>      raise
>
> Again ISTR that this got spiked for some reason, but maybe it will be
> of use to the OP in formulating his next idea.  Sorry for the lack of
> precise reference.
>

I like that "qualified except". Almost goes without saying it's a much
better idea/solution that my idea of a continue (which has already pointed
out to be flawed- I'm not sure why now I thought it was always a syntax
error)


> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120915/8e834961/attachment.html>

From rob.cliffe at btinternet.com  Sat Sep 15 12:20:49 2012
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Sat, 15 Sep 2012 11:20:49 +0100
Subject: [Python-ideas] syntax to continue into the next subsequent
 except block
In-Reply-To: <CACgdh2hCL=L4z5siVY-co0D6BXAiYycu-+gwup7q=_28=+fBew@mail.gmail.com>
References: <CACgdh2jh4MDJ0Rye1-HeUHy2iU1oe1JqnTTShb66rP3B9Mrg9w@mail.gmail.com>
	<k2tr8k$kcg$1@ger.gmane.org>
	<87k3vxqsk8.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CACgdh2hCL=L4z5siVY-co0D6BXAiYycu-+gwup7q=_28=+fBew@mail.gmail.com>
Message-ID: <50545681.5040406@btinternet.com>


On 15/09/2012 09:21, Paul Wiseman wrote:
> On 14 September 2012 02:52, Stephen J. Turnbull <stephen at xemacs.org 
> <mailto:stephen at xemacs.org>> wrote:
>
>     Terry Reedy writes:
>
>      > try:
>      >    try:
>
>     Ugh-ugh.<0.5 wink>
>
>      > try:
>      >      operation()
>      > except Exception as err:
>      >      if isinstance(err, IOError) and err.errno == 2:
>
>     Ugh.<0.5 wink>
>
>     Not your fault, but these constructions are pretty ugly IMO, I have to
>     go with the OP on that.
>
>     ISTR there were discussions of "qualified except" clauses here maybe
>     6mo to 1yr ago?  That is, they'd look something like
>
>     try:
>          operation()
>     except IOError as err if err.errno == 2:
>          do_something()
>     except Exception:
>          logger.error("Error performing operation:
>     {}".format(err.message)")
>          some_clean_up()
>          raise
>
>     Again ISTR that this got spiked for some reason, but maybe it will be
>     of use to the OP in formulating his next idea.  Sorry for the lack of
>     precise reference.
>
>
> I like that "qualified except". Almost goes without saying it's a much 
> better idea/solution that my idea of a continue (which has already 
> pointed out to be flawed- I'm not sure why now I thought it was always 
> a syntax error)
I really like this qualified except!  It's meaning is intuitively 
obvious - AND it's useful.
Rob Cliffe
>
>     _______________________________________________
>     Python-ideas mailing list
>     Python-ideas at python.org <mailto:Python-ideas at python.org>
>     http://mail.python.org/mailman/listinfo/python-ideas
>
>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120915/6e8725b3/attachment.html>

From oscar.j.benjamin at gmail.com  Sat Sep 15 13:09:12 2012
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Sat, 15 Sep 2012 12:09:12 +0100
Subject: [Python-ideas] Optional kwarg making attrgetter & itemgetter
 always return a tuple
In-Reply-To: <CAHVvXxTXjsm=XWWdv6C0YppFmcMzsm+a-3UO0DEcA1fZ4eca2w@mail.gmail.com>
References: <F747C5D0-8C98-4E68-8287-07A2576656BD@masklinn.net>
	<CADiSq7fYVR_s3wH8z2HWmHDqO-zjHKGgtxzYkzOSNMocUQp0Zg@mail.gmail.com>
	<93B4E6EE-8556-45E2-9F40-BCF7D6609328@masklinn.net>
	<CAHVvXxRAByXPYdRZ_y39gnTKrFL2HDwJn8nqVs9oJFaaa7SCRQ@mail.gmail.com>
	<CA+OGgf4gJW6qg_Ykbo_ZsSpfy5ZYb1sHzmRWbUOP0t+LaeQCYQ@mail.gmail.com>
	<CAHVvXxTXjsm=XWWdv6C0YppFmcMzsm+a-3UO0DEcA1fZ4eca2w@mail.gmail.com>
Message-ID: <CAHVvXxQs4pAH+50E-wccFGgoMuT3u89wTO6SNvLG7uhdfF8Stg@mail.gmail.com>

On Sep 14, 2012 10:02 PM, "Jim Jewett" <jimjjewett at gmail.com> wrote:
>
> On 9/14/12, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote:
>
> > I can see why you would expect different behaviour here, though. I tend
not
> > to think of the functions in the operator module as convenience
functions
> > but as *efficient* nameable functions referring to operations that are
> > normally invoked with a non-function syntax. Which is more convenient
out
> > of the following:
>
> > 1) using operator
> > import operator
> > result = sorted(values, key=operator.attrgetter('name'))
>
> I would normally write that as
>
>     from operator import attrgetter as attr
>     ... # may use it several times
>
>     result=sorted(values, key=attr('name'))
>
> which is about the best I could hope for, without being able to use
> the dot itself.

To be clear, I wasn't complaining about the inconvenience of importing and
referring to attrgetter. I was saying that if the obvious alternative
(lambda functions) is at least as convenient then it's odd to describe
itemgetter/attrgetter as convenience functions.

> > 2) using lambda

> > result = sorted(values, key=lambda v: v.name)
>
> And I honestly think that would be worse, even if lambda didn't have a
> code smell.  It focuses attention on the fact that you're creating a
> callable, instead of on the fact that you're grabbing the name
> attribute.

I disagree here. I find the fact that a lambda function shows me the
expression I would normally use to get the quantity I'm interested in makes
it easier for me to read. When I look at it I don't see it as a callable
function but as an expression that I'm passing for use somewhere else.

>
> > In general it is bad to conflate scalar/sequence semantics so that a
caller
> > should get a different type of object depending on the length of a
> > sequence.
>
> Yeah, but that can't really be solved well in python, except maybe by
> never extending an API to handle sequences.  I would personally not
> consider that an improvement.
>
> Part of the problem is that the cleanest way to take a variable number
> of arguments is to turn them into a sequence under the covers (*args),
> even if they weren't passed that way.
>
> -jJ

You can extend an API to support sequences by adding a new entry point.
This is a common idiom in python: think list.append vs list.extend.

Oscar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120915/f08135a4/attachment.html>

From ncoghlan at gmail.com  Sat Sep 15 14:43:59 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 15 Sep 2012 22:43:59 +1000
Subject: [Python-ideas] Optional kwarg making attrgetter & itemgetter
 always return a tuple
In-Reply-To: <93B4E6EE-8556-45E2-9F40-BCF7D6609328@masklinn.net>
References: <F747C5D0-8C98-4E68-8287-07A2576656BD@masklinn.net>
	<CADiSq7fYVR_s3wH8z2HWmHDqO-zjHKGgtxzYkzOSNMocUQp0Zg@mail.gmail.com>
	<93B4E6EE-8556-45E2-9F40-BCF7D6609328@masklinn.net>
Message-ID: <CADiSq7cYZRRUiKmN4AYHiuA_QT3vjh1=Yf0rvpO88YUzLg38mQ@mail.gmail.com>

On Fri, Sep 14, 2012 at 9:36 PM, Masklinn <masklinn at masklinn.net> wrote:
> On 2012-09-14, at 13:01 , Nick Coghlan wrote:
>> Both attrgetter and itemgetter are really designed to be called with
>> *literal* arguments, not via *args. In particular, they are designed
>> to be useful as arguments bound to a "key" parameter, where the object
>> vs singleton tuple distinction doesn't matter.
>
> It was my understanding that they are also designed to be useful for
> mapping (such a usage is shown in itemgetter's examples), which is
> a superset of the use case outlined here.

The "key" style usage was definitely the primary motivator, which is
why the ambiguity in the *args case wasn't noticed. If it *had* been
noticed, the multiple argument support likely never would have been
added.

As it is, the *only* case where the ambiguity causes problems is when
you want to use *args with these functions. Since they weren't built
with that style of usage in mind, they don't handle it well. Making
them even *more* complicated to work around an earlier design mistake
doesn't seem like a good idea.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia


From cs at zip.com.au  Sun Sep 16 04:20:28 2012
From: cs at zip.com.au (Cameron Simpson)
Date: Sun, 16 Sep 2012 12:20:28 +1000
Subject: [Python-ideas] syntax to continue into the next subsequent
 except block
In-Reply-To: <CACgdh2igirUpwvvc04+psN60aFa0ZcCv1nmGNiE6pNdv3n8bXQ@mail.gmail.com>
References: <CACgdh2igirUpwvvc04+psN60aFa0ZcCv1nmGNiE6pNdv3n8bXQ@mail.gmail.com>
Message-ID: <20120916022028.GA8527@cskk.homeip.net>

On 15Sep2012 09:15, Paul Wiseman <poalman at gmail.com> wrote:
| On 14 September 2012 03:32, Nick Coghlan <ncoghlan at gmail.com> wrote:
| > On Fri, Sep 14, 2012 at 11:52 AM, Stephen J. Turnbull
| > <stephen at xemacs.org> wrote:
| > > ISTR there were discussions of "qualified except" clauses here maybe
| > > 6mo to 1yr ago?  That is, they'd look something like
| > > try:
| > >      operation()
| > > except IOError as err if err.errno == 2:
[...]
| > > Again ISTR that this got spiked for some reason, but maybe it will be
| > > of use to the OP in formulating his next idea.  Sorry for the lack of
| > > precise reference.
| >
| > They were one of the ideas discussed when Antoine was writing PEP
| > 3151. As I recall, nobody could think of any good use cases that
| > didn't involve errno checking, and PEP 3151 provides a far more
| > elegant (and cross-platform) solution to most problems that require
| > errno checking in versions prior to 3.3.
| >
| Ah I didn't know about that, maybe I chose a bad example with IOError.
| 
| The reason that got me thinking is I had to handle specific
| S3ResponseErrors from boto.
| the S3ResponseError exception class has a code attribute (or errorcode, i
| forget exactly).

I have to say I find this supportive. I think the reason that there were
no use cases that don't involve errno is that most exceptions don't
provide fine grained failure information. IOError/OSError's errno is
the main exception.

Personally I think it is a shame that exceptions are generally so
uninspectable:

  raise ValueError("arbitrary prose here")

Who here thinks that is useful to a _program_ for inspecting issues?
It's the equivalent of the useless bug report "foo is broken!"

When we're raising an exception for something that should not occur,
indicating a programming bug, the above is fine; for a system failure
requiring a programmed and appropriate response, the above is pretty
vague.

OSError and IOError fall pretty clearly into the latter category, and so
does S3ResponseError and so (should) a bunch of other exceptions raised
by libraries.

So in a better quality exception environment, the "if" qualifier would
have many more use cases that Antoine encounted while writing his PEP?

So I'm a big +1 for the:

  except foo as bar if wibble:

syntax. We do it for list comprehensions, we (kicking and screaming) do
it for the tertiary choice operator (eschewing C's ?: notation) and I
think we should do it here.

Cheers,
-- 
Cameron Simpson <cs at zip.com.au>

Outside of a dog, a book is a man's best friend.  Inside of a dog, it's too
dark to read. - Groucho Marx


From yselivanov.ml at gmail.com  Sun Sep 16 19:16:34 2012
From: yselivanov.ml at gmail.com (Yury Selivanov)
Date: Sun, 16 Sep 2012 13:16:34 -0400
Subject: [Python-ideas] syntax to continue into the next subsequent
	except block
In-Reply-To: <20120916022028.GA8527@cskk.homeip.net>
References: <CACgdh2igirUpwvvc04+psN60aFa0ZcCv1nmGNiE6pNdv3n8bXQ@mail.gmail.com>
	<20120916022028.GA8527@cskk.homeip.net>
Message-ID: <AD48FCEB-91AF-42D1-B125-3DD847E51545@gmail.com>

On 2012-09-15, at 10:20 PM, Cameron Simpson <cs at zip.com.au> wrote:

> On 15Sep2012 09:15, Paul Wiseman <poalman at gmail.com> wrote:
> | On 14 September 2012 03:32, Nick Coghlan <ncoghlan at gmail.com> wrote:
> | > On Fri, Sep 14, 2012 at 11:52 AM, Stephen J. Turnbull
> | > <stephen at xemacs.org> wrote:
> | > > ISTR there were discussions of "qualified except" clauses here maybe
> | > > 6mo to 1yr ago?  That is, they'd look something like
> | > > try:
> | > >      operation()
> | > > except IOError as err if err.errno == 2:
> [...]
> | > > Again ISTR that this got spiked for some reason, but maybe it will be
> | > > of use to the OP in formulating his next idea.  Sorry for the lack of
> | > > precise reference.
> | >
> | > They were one of the ideas discussed when Antoine was writing PEP
> | > 3151. As I recall, nobody could think of any good use cases that
> | > didn't involve errno checking, and PEP 3151 provides a far more
> | > elegant (and cross-platform) solution to most problems that require
> | > errno checking in versions prior to 3.3.
> | >
> | Ah I didn't know about that, maybe I chose a bad example with IOError.
> | 
> | The reason that got me thinking is I had to handle specific
> | S3ResponseErrors from boto.
> | the S3ResponseError exception class has a code attribute (or errorcode, i
> | forget exactly).
> 
> I have to say I find this supportive. I think the reason that there were
> no use cases that don't involve errno is that most exceptions don't
> provide fine grained failure information. IOError/OSError's errno is
> the main exception.
> 
> Personally I think it is a shame that exceptions are generally so
> uninspectable:
> 
>  raise ValueError("arbitrary prose here")

So you want to write code like:

   except ValueError as ex if 'foo is wrong' in ex.args[0]:

?

This thread started with IOError and its errno attribute, and for those
exact cases I find 'except .. if' approach quite useful.  But now, in 
3.3 with PEP 3151 we have a much more granular exceptions tree, so instead
of writing

   except IOError as ex if ex.errno == errno.ENOENT:

you will write:

   except FileNotFoundError as ex:

And that's actually how this class of problems should be addressed:
instead of adding attributes to exceptions and exception guards to language - 
just design your exception classes better.  

We have multiple inheritance after all, the perfect method of classifying 
objects/exceptions, why should we code information about the exception class 
to some attribute?

If some library unifies all types of exceptions in one 'S3ResponseError'
exception - that's the problem of the library design.

Big -1.

-
Yury



From dholth at gmail.com  Sun Sep 16 20:17:08 2012
From: dholth at gmail.com (Daniel Holth)
Date: Sun, 16 Sep 2012 14:17:08 -0400
Subject: [Python-ideas] bdist naming scheme (compatibility tags) PEP
In-Reply-To: <50236FD5.2060404@egenix.com>
References: <CAG8k2+7BYcg5RRnUhjY6h7jKsZUCeYCMp-mUohPSSsh5kmaQFw@mail.gmail.com>
	<50236FD5.2060404@egenix.com>
Message-ID: <CAG8k2+5ZKwWhMNu_6MAP8yTNPPHV46hrW4HU8m+utNPZT6HaHg@mail.gmail.com>

>> Platform Tag
>> ------------
>>
>> The platform tag is simply `distutils.util.get_platform()` with all
>> hyphens `-` and periods `.` replaced with underscore `_`.
>
> This part is going to cause problems. distutils is good at identifying
> Linux and Windows and giving them sensible platform names, but it
> doesn't do a good job for other OSes.

http://www.python.org/dev/peps/pep-0425/

I've updated this part of the PEP with some examples of
get_platform(), and have simplified the "list of supported tags"
section by removing the "all older versions of Python with the same
major version" logic from the PEP. It is still allowed, but it is not
needed in the PEP.

I would love to expound on the correct implementation of
get_platform() for all major platforms. I do not know anything about
the other platforms. A BSD and OSX expert will necessarily have to
write that part of the specification.

Daniel Holth

(If you think the list of supported tags is long, go read about the
Google spell-correct algorithm that pre-computes every spelling
mistake for every word so it can tell you which correctly spelled word
is the closest to your typo)


From guido at python.org  Sun Sep 16 21:04:25 2012
From: guido at python.org (Guido van Rossum)
Date: Sun, 16 Sep 2012 12:04:25 -0700
Subject: [Python-ideas] syntax to continue into the next subsequent
	except block
In-Reply-To: <AD48FCEB-91AF-42D1-B125-3DD847E51545@gmail.com>
References: <CACgdh2igirUpwvvc04+psN60aFa0ZcCv1nmGNiE6pNdv3n8bXQ@mail.gmail.com>
	<20120916022028.GA8527@cskk.homeip.net>
	<AD48FCEB-91AF-42D1-B125-3DD847E51545@gmail.com>
Message-ID: <CAP7+vJL2jXnWc8LztU29E2XgZKJ_knyLbV1MeZnhm6sSA71ucg@mail.gmail.com>

The suggestion to add ad-hoc "if <condition>" clauses to random parts
of the syntax doesn't appeal to me at all.

If your code becomes too complex without this you're probably doing
something else wrong.

-- 
--Guido van Rossum (python.org/~guido)


From joshua.landau.ws at gmail.com  Sun Sep 16 22:51:41 2012
From: joshua.landau.ws at gmail.com (Joshua Landau)
Date: Sun, 16 Sep 2012 21:51:41 +0100
Subject: [Python-ideas] syntax to continue into the next subsequent
	except block
In-Reply-To: <CAP7+vJL2jXnWc8LztU29E2XgZKJ_knyLbV1MeZnhm6sSA71ucg@mail.gmail.com>
References: <CACgdh2igirUpwvvc04+psN60aFa0ZcCv1nmGNiE6pNdv3n8bXQ@mail.gmail.com>
	<20120916022028.GA8527@cskk.homeip.net>
	<AD48FCEB-91AF-42D1-B125-3DD847E51545@gmail.com>
	<CAP7+vJL2jXnWc8LztU29E2XgZKJ_knyLbV1MeZnhm6sSA71ucg@mail.gmail.com>
Message-ID: <CAN1F8qURDYvT616X7GqEKdXj4ERhqjr0if0PEfGv0k+JwknAfA@mail.gmail.com>

Instead of adding syntax, we could make except use isinstance.

This means that those people who are using exceptions with
poor hierarchies can override the classes with their own. This makes for an
ugly setup but a very readable result. If you put the class definitions in
another file and import them, it may end up looking quite clean.

Please forgive me if I am doing this wrong, as I've never done this sort of
thing before.

class SpecificOSErrorType(type):
 def __instancecheck__(cls, othercls):
if isinstance(othercls, OSError):
 if othercls.errno == self.errno:
return True
 return False

class FileExistsOSError(OSError, metaclass=SpecificOSErrorType):
 errno = 17


a = OSError(17, None)
b = OSError(10, None)
print(a.errno) # >> 17
print(b.errno) # >> 10
print(isinstance(a, FileExistsOSError)) # >> True
print(isinstance(b, FileExistsOSError)) # >> False

try:
raise FileExistsOSError
except OSError as e:
print(e.errno) # >> 17

import os
try:
os.mkdir("src")
except FileExistsOSError: # Fails
print("Could not make directory: File already exists")

Advantages:
- No syntax change
- Clean result (as would be expected if the hierarchy was right from the
start)
- Works with any library, and is backwards compatible
- Would enable forward-compatibility (if this was in 3.2, we
could re-implement the 3.3 hierarchy)
- [Half-point] More general use of this may encourage better exception
hierarchies

Disadvantages:
- It doesn't work yet [*wink*]
- It may be a bad idea to encourage people to override standard
class hierarchies, even if we agree that they are badly designed in 3.2
- Requires use of metaclasses and stuff people shouldn't need to understand
- Setup is extremely ugly and much longer than the replaced code
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120916/1ac37a5a/attachment.html>

From pyideas at rebertia.com  Sun Sep 16 23:32:12 2012
From: pyideas at rebertia.com (Chris Rebert)
Date: Sun, 16 Sep 2012 14:32:12 -0700
Subject: [Python-ideas] syntax to continue into the next subsequent
	except block
In-Reply-To: <CAN1F8qURDYvT616X7GqEKdXj4ERhqjr0if0PEfGv0k+JwknAfA@mail.gmail.com>
References: <CACgdh2igirUpwvvc04+psN60aFa0ZcCv1nmGNiE6pNdv3n8bXQ@mail.gmail.com>
	<20120916022028.GA8527@cskk.homeip.net>
	<AD48FCEB-91AF-42D1-B125-3DD847E51545@gmail.com>
	<CAP7+vJL2jXnWc8LztU29E2XgZKJ_knyLbV1MeZnhm6sSA71ucg@mail.gmail.com>
	<CAN1F8qURDYvT616X7GqEKdXj4ERhqjr0if0PEfGv0k+JwknAfA@mail.gmail.com>
Message-ID: <CAMZYqRSQnna+5cCE3hsg7bHTfur1ER-Uhno_fTZzrjcZX6-FLQ@mail.gmail.com>

On Sun, Sep 16, 2012 at 1:51 PM, Joshua Landau
<joshua.landau.ws at gmail.com> wrote:
> Instead of adding syntax, we could make except use isinstance.
<snip>
> class SpecificOSErrorType(type):
> def __instancecheck__(cls, othercls):
> if isinstance(othercls, OSError):
<snip>
> class FileExistsOSError(OSError, metaclass=SpecificOSErrorType):
<snip>
> Disadvantages:
> - It doesn't work yet [*wink*]

There's actually an already-open bug regarding that:
"Catching virtual subclasses in except clauses"
http://bugs.python.org/issue12029

Cheers,
Chris


From cs at zip.com.au  Mon Sep 17 00:30:08 2012
From: cs at zip.com.au (Cameron Simpson)
Date: Mon, 17 Sep 2012 08:30:08 +1000
Subject: [Python-ideas] syntax to continue into the next subsequent
 except block
In-Reply-To: <AD48FCEB-91AF-42D1-B125-3DD847E51545@gmail.com>
References: <AD48FCEB-91AF-42D1-B125-3DD847E51545@gmail.com>
Message-ID: <20120916223008.GA19129@cskk.homeip.net>

On 16Sep2012 13:16, Yury Selivanov <yselivanov.ml at gmail.com> wrote:
| On 2012-09-15, at 10:20 PM, Cameron Simpson <cs at zip.com.au> wrote:
| > On 15Sep2012 09:15, Paul Wiseman <poalman at gmail.com> wrote:
| > | The reason that got me thinking is I had to handle specific
| > | S3ResponseErrors from boto.
| > | the S3ResponseError exception class has a code attribute (or errorcode, i
| > | forget exactly).
| > 
| > I have to say I find this supportive. I think the reason that there were
| > no use cases that don't involve errno is that most exceptions don't
| > provide fine grained failure information. IOError/OSError's errno is
| > the main exception.
| > 
| > Personally I think it is a shame that exceptions are generally so
| > uninspectable:
| > 
| >  raise ValueError("arbitrary prose here")
| 
| So you want to write code like:
| 
|    except ValueError as ex if 'foo is wrong' in ex.args[0]:

No! Not at all. I was moaning about the lack of inspectability
of most of the exceptions, and ValueError is top of the list.
Of course, it has to be since we raise it for almost any sanity check
failure; there's not nice enumeration of the (unbounded) possible ways
a value may be unsuitable.

| This thread started with IOError and its errno attribute, and for those
| exact cases I find 'except .. if' approach quite useful.  But now, in 
| 3.3 with PEP 3151 we have a much more granular exceptions tree, so instead
| of writing
| 
|    except IOError as ex if ex.errno == errno.ENOENT:
| 
| you will write:
| 
|    except FileNotFoundError as ex:
| 
| And that's actually how this class of problems should be addressed:
| instead of adding attributes to exceptions and exception guards to language - 
| just design your exception classes better.  

I disagree here. Fairly strongly, actually.

FOr the record I am much cooler on the except...if notion than I was
yesterday, +0 or maybe +0.5.

BUT...

OSErrno and IOError are generally built on low level OS APIs, and
returning errno is a highly correct thing to do. It passes our, _with_ the
exception (so it doesn't get maked by another library call, as the global
POSIX error is subject to), the actual OS-level failure that was reported.

Likewise with the S3 exceptions and probably any other well designed
exception response to a library call with an informative failure code.

| We have multiple inheritance after all, the perfect method of classifying 
| objects/exceptions, why should we code information about the exception class 
| to some attribute?
| If some library unifies all types of exceptions in one 'S3ResponseError'
| exception - that's the problem of the library design.

No, not necessarily. Having a ridiculous suite of a billion trite
subclasses to enumerate the return codes from a lower level (or more
"inner") library is just nuts.

The PEP class tree is handy to _group_ an assortment of failure
modes into small groups of the same flavour. But an exact one-to-one
between exception subclasses and errno values? Ghastly. It _doubles_
the cognitive burden on the dev and the code reader, because the
correspondence between the OS-level errno and the subclass names needs to
kept in mind if the program cares about the OS level failure mode. Which
it does if it is bothering to make a fine grained decision at all.

It is all very well to offer an, um, rich suite of subclasses representing
various library failure modes. But to toss the _actual_ library failure
indicator value in favour of a arbitrary and possibly incomplete class
name list? Bad, really bad. The exception _should_ carry with it the
underlying library failure code if the library has such a thing.

| Big -1.

I'm not +1 any more, but still +.
-- 
Cameron Simpson <cs at zip.com.au>

Judging by my employee ID# my employer thinks I am a small filing cabinet,
so I dont think they give a care about my opinions.
        - Michael Jones <michaelj at nafohq.hp.com>


From joshua.landau.ws at gmail.com  Mon Sep 17 01:02:44 2012
From: joshua.landau.ws at gmail.com (Joshua Landau)
Date: Mon, 17 Sep 2012 00:02:44 +0100
Subject: [Python-ideas] syntax to continue into the next subsequent
	except block
In-Reply-To: <20120916223008.GA19129@cskk.homeip.net>
References: <AD48FCEB-91AF-42D1-B125-3DD847E51545@gmail.com>
	<20120916223008.GA19129@cskk.homeip.net>
Message-ID: <CAN1F8qUr0P0oO4DeoVOBOzHCuSx4tJipFab0C8=kM1wSPbg1zw@mail.gmail.com>

On 16 September 2012 23:30, Cameron Simpson <cs at zip.com.au> wrote:

> On 16Sep2012 13:16, Yury Selivanov <yselivanov.ml at gmail.com> wrote:
> | On 2012-09-15, at 10:20 PM, Cameron Simpson <cs at zip.com.au> wrote:
>
<snip>

>  | This thread started with IOError and its errno attribute, and for those
> | exact cases I find 'except .. if' approach quite useful.  But now, in
> | 3.3 with PEP 3151 we have a much more granular exceptions tree, so
> instead
> | of writing
> |
> |    except IOError as ex if ex.errno == errno.ENOENT:
> |
> | you will write:
> |
> |    except FileNotFoundError as ex:
> |
> | And that's actually how this class of problems should be addressed:
> | instead of adding attributes to exceptions and exception guards to
> language -
> | just design your exception classes better.
>

<snip>

OSErrno and IOError are generally built on low level OS APIs, and
> returning errno is a highly correct thing to do. It passes our, _with_ the
> exception (so it doesn't get maked by another library call, as the global
> POSIX error is subject to), the actual OS-level failure that was reported.
>
> Likewise with the S3 exceptions and probably any other well designed
> exception response to a library call with an informative failure code.


"Informative" failure code? FileNotFoundError(...) contains *exactly* the
same information as OSError(errno.ENOENT, ....). A number is not an
informative error code, and never will be.

Additionally, I don't quite follow your first paragraph ("It passes our,
_with_ the exception (so it doesn't get maked by another library call, as
the global POSIX error is subject to),", but from what I can tell it seems
extremely irrelevant to the end-programmer. I don't care what the
OS-level failure is in terms of a number, I care in terms of the actual
problem and what happened. It's not like we're receiving native data from
the OS itself.

You have a point that error codes can be informative. They *can* be.
When you have a ValueError it could come from almost anywhere. However, it
is a rare case for you to need to distinguish between these, and when you
do it is normally for a reason specific enough that two subclasses can
easily account for it all? I wouldn't mind (except that I mind being wrong
:P) you showing me where you do have a sort of structure where you need to
differentiate between many of the same error class yet cannot split it up,
but until you do I don't believe that there's an analog case where this
could help.

| We have multiple inheritance after all, the perfect method of classifying
> | objects/exceptions, why should we code information about the exception
> class
> | to some attribute?
> | If some library unifies all types of exceptions in one 'S3ResponseError'
> | exception - that's the problem of the library design.
>
> No, not necessarily. Having a ridiculous suite of a billion trite
> subclasses to enumerate the return codes from a lower level (or more
> "inner") library is just nuts.
>
> The PEP class tree is handy to _group_ an assortment of failure
> modes into small groups of the same flavour. But an exact one-to-one
> between exception subclasses and errno values? Ghastly. It _doubles_
> the cognitive burden on the dev and the code reader, because the
> correspondence between the OS-level errno and the subclass names needs to
> kept in mind if the program cares about the OS level failure mode. Which
> it does if it is bothering to make a fine grained decision at all.
>
> It is all very well to offer an, um, rich suite of subclasses representing
> various library failure modes. But to toss the _actual_ library failure
> indicator value in favour of a arbitrary and possibly incomplete class
> name list? Bad, really bad. The exception _should_ carry with it the
> underlying library failure code if the library has such a thing.
>

As said above, how is Library.MathError(5) more arbitrary than
Library.UncalculatableMathError()?

If the number of errnos is large [n], then the cognitive burden is already
large [n]. So if instead you have a large number [n] of error classes, how
is the burden less [n == n]? It doesn't add any real effort on any side as
you needed to allocate the numbers anyway, as you need to know the numbers.

Yes, if you have an incomplete name list you will suffer. But so what? Just
cover all your bases. If you are wrapping a program from a lower-level
language, wrap *everything you need*. It's no different to any other aspect
of wrapping libraries.


> | Big -1.
>
> I'm not +1 any more, but still +.
>

I'm negative, but not -1. The problem is: there are bad libraries. I think
the stuff I mentioned already is a better solution though, and it seems
it's not even my idea :).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120917/53cef0a4/attachment.html>

From greg.ewing at canterbury.ac.nz  Mon Sep 17 02:11:14 2012
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 17 Sep 2012 12:11:14 +1200
Subject: [Python-ideas] syntax to continue into the next
 subsequent	except block
In-Reply-To: <CAP7+vJL2jXnWc8LztU29E2XgZKJ_knyLbV1MeZnhm6sSA71ucg@mail.gmail.com>
References: <CACgdh2igirUpwvvc04+psN60aFa0ZcCv1nmGNiE6pNdv3n8bXQ@mail.gmail.com>
	<20120916022028.GA8527@cskk.homeip.net>
	<AD48FCEB-91AF-42D1-B125-3DD847E51545@gmail.com>
	<CAP7+vJL2jXnWc8LztU29E2XgZKJ_knyLbV1MeZnhm6sSA71ucg@mail.gmail.com>
Message-ID: <50566AA2.5000506@canterbury.ac.nz>

Guido van Rossum wrote:
> The suggestion to add ad-hoc "if <condition>" clauses to random parts
> of the syntax doesn't appeal to me at all.

I wouldn't call it a random part of the syntax. This is
not like the proposals to add if-clauses to while loops,
for loops, etc -- they would just be minor syntactic sugar.
This proposal addresses something that is quite awkward to
express using existing constructs.

-- 
Greg



From guido at python.org  Mon Sep 17 06:20:55 2012
From: guido at python.org (Guido van Rossum)
Date: Sun, 16 Sep 2012 21:20:55 -0700
Subject: [Python-ideas] syntax to continue into the next subsequent
	except block
In-Reply-To: <50566AA2.5000506@canterbury.ac.nz>
References: <CACgdh2igirUpwvvc04+psN60aFa0ZcCv1nmGNiE6pNdv3n8bXQ@mail.gmail.com>
	<20120916022028.GA8527@cskk.homeip.net>
	<AD48FCEB-91AF-42D1-B125-3DD847E51545@gmail.com>
	<CAP7+vJL2jXnWc8LztU29E2XgZKJ_knyLbV1MeZnhm6sSA71ucg@mail.gmail.com>
	<50566AA2.5000506@canterbury.ac.nz>
Message-ID: <CAP7+vJKVebw=vVL59h-8YSF_tuWqbcHzfHub4G6T27t8+oU+dg@mail.gmail.com>

On Sun, Sep 16, 2012 at 5:11 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Guido van Rossum wrote:
>>
>> The suggestion to add ad-hoc "if <condition>" clauses to random parts
>> of the syntax doesn't appeal to me at all.
>
>
> I wouldn't call it a random part of the syntax. This is
> not like the proposals to add if-clauses to while loops,
> for loops, etc -- they would just be minor syntactic sugar.
> This proposal addresses something that is quite awkward to
> express using existing constructs.

It can address an important use case and still be a random syntax
change. I'm sure there are possible refactorings of the error handling
from the examples that make it a lot less awkward. I don't think I've
ever had a use case in my own code where I found it particularly
awkward that I couldn't jump from one except clause to the next; I do
remember some cases where I could simply write

  try:
    <code that may fail>
  except <some exception>, err:
    if <on further inspection we don't want to handle it>:
      raise    # re-raise err
    <special handling for some variant of the exception>

This would possibly be combined with other except clauses but the
would be no need for the 'raise' to transfer to one of these.

-- 
--Guido van Rossum (python.org/~guido)


From steve at pearwood.info  Mon Sep 17 06:49:42 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 17 Sep 2012 14:49:42 +1000
Subject: [Python-ideas] syntax to continue into the next
 subsequent	except block
In-Reply-To: <50566AA2.5000506@canterbury.ac.nz>
References: <CACgdh2igirUpwvvc04+psN60aFa0ZcCv1nmGNiE6pNdv3n8bXQ@mail.gmail.com>
	<20120916022028.GA8527@cskk.homeip.net>
	<AD48FCEB-91AF-42D1-B125-3DD847E51545@gmail.com>
	<CAP7+vJL2jXnWc8LztU29E2XgZKJ_knyLbV1MeZnhm6sSA71ucg@mail.gmail.com>
	<50566AA2.5000506@canterbury.ac.nz>
Message-ID: <5056ABE6.5040206@pearwood.info>

On 17/09/12 10:11, Greg Ewing wrote:
> Guido van Rossum wrote:
>> The suggestion to add ad-hoc "if <condition>" clauses to random parts
>> of the syntax doesn't appeal to me at all.
>
> I wouldn't call it a random part of the syntax. This is
> not like the proposals to add if-clauses to while loops,
> for loops, etc -- they would just be minor syntactic sugar.
> This proposal addresses something that is quite awkward to
> express using existing constructs.

I don't think it is quite awkward. Instead of the proposed:


try:
     something()
except ValueError as ex if condition(ex):
     spam()
except ValueError:
     # if not condition, fall through to the next except block
     ham()

this can be easily written as:

try:
     something()
except ValueError as ex:
     if condition(ex):
         spam()
     else:
         ham()

which costs you an indent level, which is not a big deal. (If your
code is so deeply nested that it is a big deal, it is already in
desperate need of refactoring.)


Instead of the original proposal to add a "continue" statement to
skip to the next except block:

try:
     something()
except HTTPError as ex:
     if condition(ex): continue  # skip to the next except clause
     spam()
except URLError as ex:
     ham()


we can do:


try:
     something()
except (HTTPError, URLError) as ex:
     # Technically, we don't even need to list HTTPError, since it
     # is a subclass it will be caught by URLError too.
     if type(ex) is URLError or condition(ex):
         ham()
     else:
         spam()


Have I missed any suggested use-cases?

I don't think any of the existing solutions are that awkward or
unpleasant to require new syntax. They're actually quite
straightforward.



-- 
Steven


From tjreedy at udel.edu  Mon Sep 17 08:29:06 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 17 Sep 2012 02:29:06 -0400
Subject: [Python-ideas] syntax to continue into the next subsequent
	except block
In-Reply-To: <50566AA2.5000506@canterbury.ac.nz>
References: <CACgdh2igirUpwvvc04+psN60aFa0ZcCv1nmGNiE6pNdv3n8bXQ@mail.gmail.com>
	<20120916022028.GA8527@cskk.homeip.net>
	<AD48FCEB-91AF-42D1-B125-3DD847E51545@gmail.com>
	<CAP7+vJL2jXnWc8LztU29E2XgZKJ_knyLbV1MeZnhm6sSA71ucg@mail.gmail.com>
	<50566AA2.5000506@canterbury.ac.nz>
Message-ID: <k36fvn$abn$1@ger.gmane.org>

On 9/16/2012 8:11 PM, Greg Ewing wrote:
> Guido van Rossum wrote:
>> The suggestion to add ad-hoc "if <condition>" clauses to random parts
>> of the syntax doesn't appeal to me at all.
>
> I wouldn't call it a random part of the syntax. This is
> not like the proposals to add if-clauses to while loops,
> for loops, etc -- they would just be minor syntactic sugar.
> This proposal addresses something that is quite awkward to
> express using existing constructs.

I see it as quite similar. The proposal is to lift conditions out of the 
body of a compound statement and put them in the header.

There is always the option to catch everything in one except statement 
and conditionally process or re-raise as desired. Having all exceptions 
be instances of named subclasses of one baseclass makes this easier than 
when exceptions were strings.

-- 
Terry Jan Reedy



From poalman at gmail.com  Mon Sep 17 12:43:04 2012
From: poalman at gmail.com (Paul Wiseman)
Date: Mon, 17 Sep 2012 11:43:04 +0100
Subject: [Python-ideas] syntax to continue into the next subsequent
	except block
In-Reply-To: <5056ABE6.5040206@pearwood.info>
References: <CACgdh2igirUpwvvc04+psN60aFa0ZcCv1nmGNiE6pNdv3n8bXQ@mail.gmail.com>
	<20120916022028.GA8527@cskk.homeip.net>
	<AD48FCEB-91AF-42D1-B125-3DD847E51545@gmail.com>
	<CAP7+vJL2jXnWc8LztU29E2XgZKJ_knyLbV1MeZnhm6sSA71ucg@mail.gmail.com>
	<50566AA2.5000506@canterbury.ac.nz>
	<5056ABE6.5040206@pearwood.info>
Message-ID: <CACgdh2iJ5-CVY+afr8a5b3zmgH9NW3dpH9iMwEhT4PjAzZ5b8g@mail.gmail.com>

On 17 September 2012 05:49, Steven D'Aprano <steve at pearwood.info> wrote:

> On 17/09/12 10:11, Greg Ewing wrote:
>
>> Guido van Rossum wrote:
>>
>>> The suggestion to add ad-hoc "if <condition>" clauses to random parts
>>> of the syntax doesn't appeal to me at all.
>>>
>>
>> I wouldn't call it a random part of the syntax. This is
>> not like the proposals to add if-clauses to while loops,
>> for loops, etc -- they would just be minor syntactic sugar.
>> This proposal addresses something that is quite awkward to
>> express using existing constructs.
>>
>
> I don't think it is quite awkward. Instead of the proposed:
>
>
> try:
>     something()
> except ValueError as ex if condition(ex):
>     spam()
> except ValueError:
>     # if not condition, fall through to the next except block
>     ham()
>
> this can be easily written as:
>
> try:
>     something()
> except ValueError as ex:
>     if condition(ex):
>         spam()
>     else:
>         ham()
>
> which costs you an indent level, which is not a big deal. (If your
> code is so deeply nested that it is a big deal, it is already in
> desperate need of refactoring.)
>
>
> Instead of the original proposal to add a "continue" statement to
> skip to the next except block:
>
> try:
>     something()
> except HTTPError as ex:
>     if condition(ex): continue  # skip to the next except clause
>     spam()
> except URLError as ex:
>     ham()
>
>
> we can do:
>
>
> try:
>     something()
> except (HTTPError, URLError) as ex:
>     # Technically, we don't even need to list HTTPError, since it
>     # is a subclass it will be caught by URLError too.
>     if type(ex) is URLError or condition(ex):
>         ham()
>     else:
>         spam()
>
>
> Have I missed any suggested use-cases?
>
>
What I think made my original case a bit awkward was a general handler for
a parent exception class, and a specific except clause for a subclass (it
was IOError and Exception originally)

with your ValueError case, say you wanted to call ham on every case of a
parent exception to ValueError (I guess it could only be Exception for this
example), but not ValueError where condition was true,

so how would you write this without the except..if?

try:
    something()
except HTTPError as ex if condition(ex):
    spam()
except URLError:
    # if not condition, fall through to the next except block
    ham()

(I switched ValueError for HTTPError because it has more parent classes
that just Exception, because having 'except Exception' probably isn't the
greatest example)

I don't think any of the existing solutions are that awkward or
> unpleasant to require new syntax. They're actually quite
> straightforward.
>
>
>
> --
> Steven
>
> ______________________________**_________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/**mailman/listinfo/python-ideas<http://mail.python.org/mailman/listinfo/python-ideas>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120917/fed3018e/attachment.html>

From poalman at gmail.com  Mon Sep 17 12:51:30 2012
From: poalman at gmail.com (Paul Wiseman)
Date: Mon, 17 Sep 2012 11:51:30 +0100
Subject: [Python-ideas] syntax to continue into the next subsequent
	except block
In-Reply-To: <CAP7+vJKVebw=vVL59h-8YSF_tuWqbcHzfHub4G6T27t8+oU+dg@mail.gmail.com>
References: <CACgdh2igirUpwvvc04+psN60aFa0ZcCv1nmGNiE6pNdv3n8bXQ@mail.gmail.com>
	<20120916022028.GA8527@cskk.homeip.net>
	<AD48FCEB-91AF-42D1-B125-3DD847E51545@gmail.com>
	<CAP7+vJL2jXnWc8LztU29E2XgZKJ_knyLbV1MeZnhm6sSA71ucg@mail.gmail.com>
	<50566AA2.5000506@canterbury.ac.nz>
	<CAP7+vJKVebw=vVL59h-8YSF_tuWqbcHzfHub4G6T27t8+oU+dg@mail.gmail.com>
Message-ID: <CACgdh2iCLN34ebNCpEPdG6Jr6swqfMHPfME5gL62M3UDEQ=PuA@mail.gmail.com>

On 17 September 2012 05:20, Guido van Rossum <guido at python.org> wrote:

> On Sun, Sep 16, 2012 at 5:11 PM, Greg Ewing <greg.ewing at canterbury.ac.nz>
> wrote:
> > Guido van Rossum wrote:
> >>
> >> The suggestion to add ad-hoc "if <condition>" clauses to random parts
> >> of the syntax doesn't appeal to me at all.
> >
> >
> > I wouldn't call it a random part of the syntax. This is
> > not like the proposals to add if-clauses to while loops,
> > for loops, etc -- they would just be minor syntactic sugar.
> > This proposal addresses something that is quite awkward to
> > express using existing constructs.
>
> It can address an important use case and still be a random syntax
> change. I'm sure there are possible refactorings of the error handling
> from the examples that make it a lot less awkward. I don't think I've
> ever had a use case in my own code where I found it particularly
> awkward that I couldn't jump from one except clause to the next; I do
> remember some cases where I could simply write
>
>
I conceded that my idea of jumping from one except clause to the next was a
bad one, but with the except..if idea there would be no jumping; it would
work the same as it currently does where an exception can only enter at
most one except block in the same level of try..excepts.


>   try:
>     <code that may fail>
>   except <some exception>, err:
>     if <on further inspection we don't want to handle it>:
>       raise    # re-raise err
>     <special handling for some variant of the exception>
>
> This would possibly be combined with other except clauses but the
> would be no need for the 'raise' to transfer to one of these.
>
> --
> --Guido van Rossum (python.org/~guido)
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120917/254bf4ed/attachment.html>

From steve at pearwood.info  Mon Sep 17 13:24:21 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 17 Sep 2012 21:24:21 +1000
Subject: [Python-ideas] syntax to continue into the next subsequent
 except block
In-Reply-To: <CACgdh2iJ5-CVY+afr8a5b3zmgH9NW3dpH9iMwEhT4PjAzZ5b8g@mail.gmail.com>
References: <CACgdh2igirUpwvvc04+psN60aFa0ZcCv1nmGNiE6pNdv3n8bXQ@mail.gmail.com>
	<20120916022028.GA8527@cskk.homeip.net>
	<AD48FCEB-91AF-42D1-B125-3DD847E51545@gmail.com>
	<CAP7+vJL2jXnWc8LztU29E2XgZKJ_knyLbV1MeZnhm6sSA71ucg@mail.gmail.com>
	<50566AA2.5000506@canterbury.ac.nz>
	<5056ABE6.5040206@pearwood.info>
	<CACgdh2iJ5-CVY+afr8a5b3zmgH9NW3dpH9iMwEhT4PjAzZ5b8g@mail.gmail.com>
Message-ID: <50570865.9040105@pearwood.info>

On 17/09/12 20:43, Paul Wiseman wrote:

> with your ValueError case, say you wanted to call ham on every case of a
> parent exception to ValueError (I guess it could only be Exception for this
> example), but not ValueError where condition was true,
>
> so how would you write this without the except..if?
>
> try:
>      something()
> except HTTPError as ex if condition(ex):
>      spam()
> except URLError:
>      # if not condition, fall through to the next except block
>      ham()

try:
     something()
except URLError as ex:  # will also catch HTTPError
     # if you want to be explicit, say "except HTTPError, URLError as ex"
     if isinstance(ex, HTTPError) and condition(ex):
         spam()
     else:
         ham()


> (I switched ValueError for HTTPError because it has more parent classes
> that just Exception, because having 'except Exception' probably isn't the
> greatest example)

Sure, no worries.



-- 
Steven


From ncoghlan at gmail.com  Mon Sep 17 14:13:56 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 17 Sep 2012 22:13:56 +1000
Subject: [Python-ideas] syntax to continue into the next subsequent
	except block
In-Reply-To: <CACgdh2iCLN34ebNCpEPdG6Jr6swqfMHPfME5gL62M3UDEQ=PuA@mail.gmail.com>
References: <CACgdh2igirUpwvvc04+psN60aFa0ZcCv1nmGNiE6pNdv3n8bXQ@mail.gmail.com>
	<20120916022028.GA8527@cskk.homeip.net>
	<AD48FCEB-91AF-42D1-B125-3DD847E51545@gmail.com>
	<CAP7+vJL2jXnWc8LztU29E2XgZKJ_knyLbV1MeZnhm6sSA71ucg@mail.gmail.com>
	<50566AA2.5000506@canterbury.ac.nz>
	<CAP7+vJKVebw=vVL59h-8YSF_tuWqbcHzfHub4G6T27t8+oU+dg@mail.gmail.com>
	<CACgdh2iCLN34ebNCpEPdG6Jr6swqfMHPfME5gL62M3UDEQ=PuA@mail.gmail.com>
Message-ID: <CADiSq7eRU3XboSu35kuYoMi6SiruubE9Fdr7NEaTsXFWfk64ag@mail.gmail.com>

On Mon, Sep 17, 2012 at 8:51 PM, Paul Wiseman <poalman at gmail.com> wrote:
> I conceded that my idea of jumping from one except clause to the next was a
> bad one, but with the except..if idea there would be no jumping; it would
> work the same as it currently does where an exception can only enter at most
> one except block in the same level of try..excepts.

The key thing to remember is that the bar for new syntax is *very high*.

The reason it was kicked around as a possibility in the PEP 3151
discussions is that checking errno values is really common and (more
importantly) often not done correctly (leading to overbroad trapping
of errors).

PEP 3151 ultimately chose to solve the problem a different way (i.e.
modifying the standard exception hierarchy) that didn't require new
syntax. In so doing, it also gutted the case for this being a common
problem. It *used to be* a common problem, but most cases of errno
checking are now better handled by catching more specific OSError
subclasses.

It's simply not worth increasing the complexity of the except clause
definition to deal with the new rare cases that are already amenable
to other solutions, in particular just catching the superclass and
then interrogating for additional details.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia


From mal at egenix.com  Mon Sep 17 14:39:13 2012
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 17 Sep 2012 14:39:13 +0200
Subject: [Python-ideas] bdist naming scheme (compatibility tags) PEP
In-Reply-To: <CAG8k2+5ZKwWhMNu_6MAP8yTNPPHV46hrW4HU8m+utNPZT6HaHg@mail.gmail.com>
References: <CAG8k2+7BYcg5RRnUhjY6h7jKsZUCeYCMp-mUohPSSsh5kmaQFw@mail.gmail.com>
	<50236FD5.2060404@egenix.com>
	<CAG8k2+5ZKwWhMNu_6MAP8yTNPPHV46hrW4HU8m+utNPZT6HaHg@mail.gmail.com>
Message-ID: <505719F1.8090405@egenix.com>

Daniel Holth wrote:
>>> Platform Tag
>>> ------------
>>>
>>> The platform tag is simply `distutils.util.get_platform()` with all
>>> hyphens `-` and periods `.` replaced with underscore `_`.
>>
>> This part is going to cause problems. distutils is good at identifying
>> Linux and Windows and giving them sensible platform names, but it
>> doesn't do a good job for other OSes.
> 
> http://www.python.org/dev/peps/pep-0425/

I still don't think that referencing the distutils function
in the PEP is a good idea :-) It would be better to create
a new helper.

> I've updated this part of the PEP with some examples of
> get_platform(),

The string for x86 Linux platforms usually reads "linux-i686".

Some other get_platform() examples:

macosx-10.4-fat - Tiger, fat PPC/i386 build of Python
macosx-10.6-x86_64 - Snow Leopard, x64-only build of Python
freebsd-8.3-RELEASE-p3-i386 - FreeBSD 8.3, SP3, x86
freebsd-8.3-RELEASE-p3-amd64 - FreeBSD 8.3, SP3, x64
cygwin-1.7.9-i686 - Cygwin, x86

For Macs and other platforms that support fat builds it would
be good to have some form which allows defining which architectures
are included in the fat build, e.g. i386, ppc, x86_64.

For FreeBSD, the string could be reduced to remove he "RELEASE-p3-"
bit.

It would probably be a good idea to develop a binary compatibility
checker package on PyPI first before hard coding these things
into the PEP.

Such a package should offer two functions (sketching here):

get_binary_platform_string() -> return a binary platform compatibility
                                string for the current platform

binary_package_compatible(platform_string) -> return True/False depending
                                on whether the current platform is compatible
                                with the given platform_string

The package could then contain all the domain information needed
for the various platforms.

> and have simplified the "list of supported tags"
> section by removing the "all older versions of Python with the same
> major version" logic from the PEP. It is still allowed, but it is not
> needed in the PEP.

One note regarding adding more than one such tag to a file:
adding those extra tags using dots (".") will make parsing the
file name harder. It's probably better to separate them using
a separator such as "_or_".

> I would love to expound on the correct implementation of
> get_platform() for all major platforms. I do not know anything about
> the other platforms. A BSD and OSX expert will necessarily have to
> write that part of the specification.

See above :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 17 2012)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2012-10-29: PyCon DE 2012, Leipzig, Germany ...            42 days to go
2012-10-23: Python Meeting Duesseldorf ...                 36 days to go
2012-08-28: Released mxODBC 3.2.0 ...             http://egenix.com/go31
2012-08-20: Released mxODBC.Connect 2.0.0 ...     http://egenix.com/go30


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From storchaka at gmail.com  Mon Sep 17 14:41:23 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Mon, 17 Sep 2012 15:41:23 +0300
Subject: [Python-ideas] Expansion of the range of small integers
Message-ID: <k375pl$us6$1@ger.gmane.org>

Now in the CPython small integer numbers from -5 up to 256 inclusive are 
preallocated at the start. It allows to reduce memory consumption and 
time of creation of the integers in this range. In particular this 
affects the speed of short enumerations. Increasing the range to the 
maximum (from -32767 to 32767 inclusive), we can speed up longer 
enumerations.

Microbenchmarks:
./python -m timeit  "for i in range(10000): pass"
./python -m timeit  -s "a=[0]*10000"  "for i, x in enumerate(a): pass"
./python -m timeit  -s "a=[0]*10000"  "i=0"  "for x in a: i+=1"
./python -m timeit  -s "a=[0]*10000"  "for i in range(len(a)): x=a[i]"

Results:
  non-patched     patched
    530 usec      337 usec    57%
   1.06 msec      811 usec    31%
   1.34 msec     1.13 msec    19%
   1.42 msec     1.22 msec    16%

Shortcomings:

1) Memory consumption increases by constant 1-1.5 MB. Or half of it if 
the range is expanded only in a positive direction. This is not a 
problem on most modern computers. But would be better if the parameters 
NSMALLPOSINTS and NSMALLNEGINTS have been configurable at build time.

2) A little bit larger Python start time. I was not able to measure the 
difference, it is too small.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: larger_small_ints.diff
Type: text/x-patch
Size: 1981 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120917/39f3052d/attachment.bin>

From steve at pearwood.info  Mon Sep 17 15:09:58 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 17 Sep 2012 23:09:58 +1000
Subject: [Python-ideas] Expansion of the range of small integers
In-Reply-To: <k375pl$us6$1@ger.gmane.org>
References: <k375pl$us6$1@ger.gmane.org>
Message-ID: <50572126.6090900@pearwood.info>

On 17/09/12 22:41, Serhiy Storchaka wrote:

> Now in the CPython small integer numbers from -5 up to 256 inclusive
>are preallocated at the start. It allows to reduce memory consumption
>and time of creation of the integers in this range. In particular this
>affects the speed of short enumerations. Increasing the range to the
>maximum (from -32767 to 32767 inclusive), we can speed up longer
>enumerations.

[...]

> Shortcomings:
>
> 1) Memory consumption increases by constant 1-1.5 MB. Or half of it if
>the range is expanded only in a positive direction. This is not a
> problem on most modern computers. But would be better if the parameters
>NSMALLPOSINTS and NSMALLNEGINTS have been configurable at build time.
>
> 2) A little bit larger Python start time. I was not able to measure the
>difference, it is too small.

There is an additional consequence of this proposed change. I'm not sure
if this counts as an argument for, or against, the change, but beginners
and even some experienced programmers often find the difference between
identity and equality hard to deal with. Caching of small integers already
blurs the distinction:

py> a = 42
py> b = 42
py> a is b
True

Extending that behaviour up to 32767 will further blur the distinction.



-- 
Steven


From solipsis at pitrou.net  Mon Sep 17 15:29:01 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 17 Sep 2012 15:29:01 +0200
Subject: [Python-ideas] Expansion of the range of small integers
References: <k375pl$us6$1@ger.gmane.org>
Message-ID: <20120917152901.21b3ec1b@pitrou.net>

On Mon, 17 Sep 2012 15:41:23 +0300
Serhiy Storchaka <storchaka at gmail.com>
wrote:
> Now in the CPython small integer numbers from -5 up to 256 inclusive are 
> preallocated at the start. It allows to reduce memory consumption and 
> time of creation of the integers in this range. In particular this 
> affects the speed of short enumerations. Increasing the range to the 
> maximum (from -32767 to 32767 inclusive), we can speed up longer 
> enumerations.
> 
> Microbenchmarks:
> ./python -m timeit  "for i in range(10000): pass"
> ./python -m timeit  -s "a=[0]*10000"  "for i, x in enumerate(a): pass"
> ./python -m timeit  -s "a=[0]*10000"  "i=0"  "for x in a: i+=1"
> ./python -m timeit  -s "a=[0]*10000"  "for i in range(len(a)): x=a[i]"
> 
> Results:
>   non-patched     patched
>     530 usec      337 usec    57%
>    1.06 msec      811 usec    31%
>    1.34 msec     1.13 msec    19%
>    1.42 msec     1.22 msec    16%

See also http://bugs.python.org/issue10044

> 1) Memory consumption increases by constant 1-1.5 MB.

That sounds a bit annoying. Is it for a 32-bit or 64-bit build?

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net




From storchaka at gmail.com  Mon Sep 17 15:29:44 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Mon, 17 Sep 2012 16:29:44 +0300
Subject: [Python-ideas] Expansion of the range of small integers
In-Reply-To: <k375pl$us6$1@ger.gmane.org>
References: <k375pl$us6$1@ger.gmane.org>
Message-ID: <k378ka$ot2$1@ger.gmane.org>

Oops, sorry, I have included a few extra minor changes in the patch. 
Here is the correct patch.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: larger_small_ints.diff
Type: text/x-patch
Size: 471 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120917/e11f3dbc/attachment.bin>

From storchaka at gmail.com  Mon Sep 17 15:40:29 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Mon, 17 Sep 2012 16:40:29 +0300
Subject: [Python-ideas] Expansion of the range of small integers
In-Reply-To: <50572126.6090900@pearwood.info>
References: <k375pl$us6$1@ger.gmane.org> <50572126.6090900@pearwood.info>
Message-ID: <k3798f$uq1$1@ger.gmane.org>

On 17.09.12 16:09, Steven D'Aprano wrote:
> Extending that behaviour up to 32767 will further blur the distinction.

This is not an argument either for or against. Beginners will always 
find something to discourage.

 >>> 1000000 is 1000000
True
 >>> a = 1000000
 >>> b = 1000000
 >>> a is b
False

"I think it's better to give users the rope they want than to try and 
prevent them from hanging themselves, since otherwise they'll just use 
the power cords as ropes and electrocute themselves". (GvR)




From storchaka at gmail.com  Mon Sep 17 18:35:01 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Mon, 17 Sep 2012 19:35:01 +0300
Subject: [Python-ideas] Expansion of the range of small integers
In-Reply-To: <20120917152901.21b3ec1b@pitrou.net>
References: <k375pl$us6$1@ger.gmane.org> <20120917152901.21b3ec1b@pitrou.net>
Message-ID: <k37jfp$112$1@ger.gmane.org>

On 17.09.12 16:29, Antoine Pitrou wrote:
> See also http://bugs.python.org/issue10044

This is interesting. But the main trick causes an undefined behavior.

>> 1) Memory consumption increases by constant 1-1.5 MB.
>
> That sounds a bit annoying. Is it for a 32-bit or 64-bit build?

For a 32-bit it is 14*(2**16-257-5) = 913836 B = 0.87 MiB. For a 64-bit 
it should be twice as large (1.74 MiB). If you want expand the range to 
portable maximum.




From tjreedy at udel.edu  Mon Sep 17 19:49:07 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 17 Sep 2012 13:49:07 -0400
Subject: [Python-ideas] Expansion of the range of small integers
In-Reply-To: <k375pl$us6$1@ger.gmane.org>
References: <k375pl$us6$1@ger.gmane.org>
Message-ID: <k37nqn$aed$1@ger.gmane.org>

On 9/17/2012 8:41 AM, Serhiy Storchaka wrote:
> Now in the CPython small integer numbers from -5 up to 256 inclusive are
> preallocated at the start. It allows to reduce memory consumption and
> time of creation of the integers in this range. In particular this
> affects the speed of short enumerations. Increasing the range to the
> maximum (from -32767 to 32767 inclusive), we can speed up longer
> enumerations.

In 2.x before 3.0, the range was about -5 to 10 or so ;-).
It was expanded when bytes were added.

It might be interesting to instrument the int allocator to count 
allocations of ints up to say 10000 in real apps.

> Microbenchmarks:
> ./python -m timeit  "for i in range(10000): pass"
> ./python -m timeit  -s "a=[0]*10000"  "for i, x in enumerate(a): pass"
> ./python -m timeit  -s "a=[0]*10000"  "i=0"  "for x in a: i+=1"
> ./python -m timeit  -s "a=[0]*10000"  "for i in range(len(a)): x=a[i]"
>
> Results:
>   non-patched     patched
>     530 usec      337 usec    57%
>    1.06 msec      811 usec    31%
>    1.34 msec     1.13 msec    19%
>    1.42 msec     1.22 msec    16%
>
> Shortcomings:
>
> 1) Memory consumption increases by constant 1-1.5 MB. Or half of it if
> the range is expanded only in a positive direction. This is not a
> problem on most modern computers. But would be better if the parameters
> NSMALLPOSINTS and NSMALLNEGINTS have been configurable at build time.

They are -- by patching as you did ;-). The general philosophy seems to 
be to discourage user tuning by not making it too easy.
>
> 2) A little bit larger Python start time. I was not able to measure the
> difference, it is too small.

What is hard to guess is the effect on cache hits and misses in real apps.
-- 
Terry Jan Reedy



From christian at python.org  Mon Sep 17 20:28:41 2012
From: christian at python.org (Christian Heimes)
Date: Mon, 17 Sep 2012 20:28:41 +0200
Subject: [Python-ideas] Expansion of the range of small integers
In-Reply-To: <k37nqn$aed$1@ger.gmane.org>
References: <k375pl$us6$1@ger.gmane.org> <k37nqn$aed$1@ger.gmane.org>
Message-ID: <k37q4o$vfd$1@ger.gmane.org>

Am 17.09.2012 19:49, schrieb Terry Reedy:
> In 2.x before 3.0, the range was about -5 to 10 or so ;-).
> It was expanded when bytes were added.

2.2 had -1 to 99. The numbers grew slowly to -5 to 256 in 2.6.

> It might be interesting to instrument the int allocator to count
> allocations of ints up to say 10000 in real apps.

You can easily test how different settings affect the interpreter with
some compiler flags:

CFLAGS="-DCOUNT_ALLOCS=1 -DNSMALLPOSINTS=10000 -DNSMALLNEGINTS=10000"
./configure && make

COUNT_ALLOCS slows down the interpreter a bit. It prints some stats at
shutdown.

> What is hard to guess is the effect on cache hits and misses in real apps.

Real apps may even save memory when they use lots of ints > 256.

Christian




From jimjjewett at gmail.com  Mon Sep 17 21:33:20 2012
From: jimjjewett at gmail.com (Jim Jewett)
Date: Mon, 17 Sep 2012 15:33:20 -0400
Subject: [Python-ideas] syntax to continue into the next subsequent
	except block
In-Reply-To: <CAP7+vJKVebw=vVL59h-8YSF_tuWqbcHzfHub4G6T27t8+oU+dg@mail.gmail.com>
References: <CACgdh2igirUpwvvc04+psN60aFa0ZcCv1nmGNiE6pNdv3n8bXQ@mail.gmail.com>
	<20120916022028.GA8527@cskk.homeip.net>
	<AD48FCEB-91AF-42D1-B125-3DD847E51545@gmail.com>
	<CAP7+vJL2jXnWc8LztU29E2XgZKJ_knyLbV1MeZnhm6sSA71ucg@mail.gmail.com>
	<50566AA2.5000506@canterbury.ac.nz>
	<CAP7+vJKVebw=vVL59h-8YSF_tuWqbcHzfHub4G6T27t8+oU+dg@mail.gmail.com>
Message-ID: <CA+OGgf6r3rcCCoSPZ=Agwy2EbSxLAV3-PKHMAs=EM3kup3DVKw@mail.gmail.com>

On 9/17/12, Guido van Rossum <guido at python.org> wrote:

> I don't think I've
> ever had a use case in my own code where I found it particularly
> awkward that I couldn't jump from one except clause to the next;

I have had cases where I wanted to either
(1)  Say "oops, not really a match, keep looking at the other except clauses"

The except ... if answers this, but I do think virtual subclasses
would be a better solution.  In my case, I would have created a
subclass for file errors that could be fixed automatically.

(2)  If there is an exception [of such and such a type] do X, but
sometimes *also* do Y.

The continue could answer this, but I'm not convinced it would smell
any less than the current workarounds.

-jJ


From jimjjewett at gmail.com  Mon Sep 17 21:39:50 2012
From: jimjjewett at gmail.com (Jim Jewett)
Date: Mon, 17 Sep 2012 15:39:50 -0400
Subject: [Python-ideas] syntax to continue into the next subsequent
	except block
In-Reply-To: <CAN1F8qUr0P0oO4DeoVOBOzHCuSx4tJipFab0C8=kM1wSPbg1zw@mail.gmail.com>
References: <AD48FCEB-91AF-42D1-B125-3DD847E51545@gmail.com>
	<20120916223008.GA19129@cskk.homeip.net>
	<CAN1F8qUr0P0oO4DeoVOBOzHCuSx4tJipFab0C8=kM1wSPbg1zw@mail.gmail.com>
Message-ID: <CA+OGgf7rVCm0FWyQPtoNg9KkFSF-2mpqPG5Y7GaQr1bE1o7ggQ@mail.gmail.com>

On 9/16/12, Joshua Landau <joshua.landau.ws at gmail.com> wrote:
> On 16 September 2012 23:30, Cameron Simpson <cs at zip.com.au> wrote:

>> Having a ridiculous suite of a billion trite
>> subclasses to enumerate the return codes from a lower level (or more
>> "inner") library is just nuts.

If the library created them, it is distinguishing between them --
regardless of whether it distinguishes by name or by number.

> As said above, how is Library.MathError(5) more arbitrary than
> Library.UncalculatableMathError()?

Numbers are more likely to get shifted by accident when someone adds a
new value.

But the point isn't that codes are more arbitrary -- it is that a name
is more helpful when debugging.

> Yes, if you have an incomplete name list you will suffer. But so what? Just
> cover all your bases. If you are wrapping a program from a lower-level
> language, wrap *everything you need*. It's no different to any other aspect
> of wrapping libraries.

And, more to the point, wrap *only* what you need.  If you were
providing the sole wrapper for a library, then you might have a fairly
long translation list.  But if you're just using the library, only
create virtual subclasses for the conditions that you happen to care
about, and name them based on why they matter.

-jJ


From joshua.landau.ws at gmail.com  Mon Sep 17 22:00:42 2012
From: joshua.landau.ws at gmail.com (Joshua Landau)
Date: Mon, 17 Sep 2012 21:00:42 +0100
Subject: [Python-ideas] syntax to continue into the next subsequent
	except block
In-Reply-To: <CA+OGgf7rVCm0FWyQPtoNg9KkFSF-2mpqPG5Y7GaQr1bE1o7ggQ@mail.gmail.com>
References: <AD48FCEB-91AF-42D1-B125-3DD847E51545@gmail.com>
	<20120916223008.GA19129@cskk.homeip.net>
	<CAN1F8qUr0P0oO4DeoVOBOzHCuSx4tJipFab0C8=kM1wSPbg1zw@mail.gmail.com>
	<CA+OGgf7rVCm0FWyQPtoNg9KkFSF-2mpqPG5Y7GaQr1bE1o7ggQ@mail.gmail.com>
Message-ID: <CAN1F8qWM8ZrsbxzktfSKUY5Q3TkmoAS2Ge_ch6iEPn_omwWDRQ@mail.gmail.com>

On 17 September 2012 20:39, Jim Jewett <jimjjewett at gmail.com> wrote:

> On 9/16/12, Joshua Landau <joshua.landau.ws at gmail.com> wrote:
> > On 16 September 2012 23:30, Cameron Simpson <cs at zip.com.au> wrote:
> > As said above, how is Library.MathError(5) more arbitrary than
> > Library.UncalculatableMathError()?
>
> Numbers are more likely to get shifted by accident when someone adds a
> new value.
>
> But the point isn't that codes are more arbitrary -- it is that a name
> is more helpful when debugging.


And I totally agree. My original post was meant to say "how is *
Library.UncalculableMathError()* more arbitrary than *Library.MathError(5)*?"
as a refute to a claim that they are. Thanks for catching that.


> > Yes, if you have an incomplete name list you will suffer. But so what?
> Just
> > cover all your bases. If you are wrapping a program from a lower-level
> > language, wrap *everything you need*. It's no different to any other
> aspect
> > of wrapping libraries.
>
> And, more to the point, wrap *only* what you need.  If you were
> providing the sole wrapper for a library, then you might have a fairly
> long translation list.  But if you're just using the library, only
> create virtual subclasses for the conditions that you happen to care
> about, and name them based on why they matter.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120917/6c96ca5f/attachment.html>

From storchaka at gmail.com  Mon Sep 17 22:44:37 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Mon, 17 Sep 2012 23:44:37 +0300
Subject: [Python-ideas] Expansion of the range of small integers
In-Reply-To: <k37nqn$aed$1@ger.gmane.org>
References: <k375pl$us6$1@ger.gmane.org> <k37nqn$aed$1@ger.gmane.org>
Message-ID: <k3823o$7p5$1@ger.gmane.org>

On 17.09.12 20:49, Terry Reedy wrote:
> It might be interesting to instrument the int allocator to count
> allocations of ints up to say 10000 in real apps.

Here is a patch. Apply it, rebuild (CFLAGS=-DCOUNT_ALLOCS=1 ./configure 
&& make) and run your favorite Python 3 apps.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: count_small_int_allocs.diff
Type: text/x-patch
Size: 2364 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120917/df3582da/attachment.bin>

From jimjjewett at gmail.com  Tue Sep 18 02:14:09 2012
From: jimjjewett at gmail.com (Jim Jewett)
Date: Mon, 17 Sep 2012 20:14:09 -0400
Subject: [Python-ideas] Expansion of the range of small integers
In-Reply-To: <k375pl$us6$1@ger.gmane.org>
References: <k375pl$us6$1@ger.gmane.org>
Message-ID: <CA+OGgf7_UmpJ+g3K246xXiydPkc-g=6VeKf2hOyZ8Ke_jARRkA@mail.gmail.com>

On 9/17/12, Serhiy Storchaka <storchaka at gmail.com> wrote:
> Now in the CPython small integer numbers from -5 up to 256 inclusive are
> preallocated at the start. It allows to reduce memory consumption and
> time of creation of the integers in this range. In particular this
> affects the speed of short enumerations. Increasing the range to the
> maximum (from -32767 to 32767 inclusive), we can speed up longer
> enumerations.

On the other hand, you'll add to the memory pressure; instead of
putting the small integers and some other important objects in a
single page that probably stays loaded, there will be several
different pages to page in and out if they are useful, and to waste
startup time if they aren't.

-jJ


From stephen at xemacs.org  Tue Sep 18 03:08:54 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 18 Sep 2012 10:08:54 +0900
Subject: [Python-ideas] syntax to continue into the next subsequent
 except block
In-Reply-To: <5056ABE6.5040206@pearwood.info>
References: <CACgdh2igirUpwvvc04+psN60aFa0ZcCv1nmGNiE6pNdv3n8bXQ@mail.gmail.com>
	<20120916022028.GA8527@cskk.homeip.net>
	<AD48FCEB-91AF-42D1-B125-3DD847E51545@gmail.com>
	<CAP7+vJL2jXnWc8LztU29E2XgZKJ_knyLbV1MeZnhm6sSA71ucg@mail.gmail.com>
	<50566AA2.5000506@canterbury.ac.nz>
	<5056ABE6.5040206@pearwood.info>
Message-ID: <87r4q0p26x.fsf@uwakimon.sk.tsukuba.ac.jp>

Steven D'Aprano writes:

 > I don't think [the existing syntax] is quite awkward.

Of course it is.  It doesn't fit the semantics, and that is why it is
awkward.  An exception should be caught by an 'except' clause, not by
a conditional statement.  As Antoine's refactoring demonstrates, these
Exceptions that are differentiated by an internal errno are often
conceptually a variety of different exceptions.  I agree with the
suggestion that really we should fix up other Exceptions that are
clearly quite heterogeneous by subclassing more precise exceptions
from them.

But this isn't always possible, and of course may not be backward
compatible.

I don't know how to do it Pythonically, but it would be nice if there
were some way to "subclass Exceptions on the fly".

(other) Steve


From ncoghlan at gmail.com  Tue Sep 18 08:55:16 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 18 Sep 2012 16:55:16 +1000
Subject: [Python-ideas] syntax to continue into the next subsequent
	except block
In-Reply-To: <87r4q0p26x.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CACgdh2igirUpwvvc04+psN60aFa0ZcCv1nmGNiE6pNdv3n8bXQ@mail.gmail.com>
	<20120916022028.GA8527@cskk.homeip.net>
	<AD48FCEB-91AF-42D1-B125-3DD847E51545@gmail.com>
	<CAP7+vJL2jXnWc8LztU29E2XgZKJ_knyLbV1MeZnhm6sSA71ucg@mail.gmail.com>
	<50566AA2.5000506@canterbury.ac.nz>
	<5056ABE6.5040206@pearwood.info>
	<87r4q0p26x.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CADiSq7fa2DRH981YEDfRCGQHfSRHGq3eTqszvTPkym_v+CjL-g@mail.gmail.com>

On Tue, Sep 18, 2012 at 11:08 AM, Stephen J. Turnbull
<stephen at xemacs.org> wrote:
> I don't know how to do it Pythonically, but it would be nice if there
> were some way to "subclass Exceptions on the fly".

Most likely, this will mean fixing the bug that means the ABC
machinery is currently being ignored by the exception machinery. Then
you can do whatever you want to reshape exception hierarchies.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia


From ckaynor at zindagigames.com  Tue Sep 18 19:00:06 2012
From: ckaynor at zindagigames.com (Chris Kaynor)
Date: Tue, 18 Sep 2012 10:00:06 -0700
Subject: [Python-ideas] syntax to continue into the next subsequent
	except block
In-Reply-To: <CADiSq7fa2DRH981YEDfRCGQHfSRHGq3eTqszvTPkym_v+CjL-g@mail.gmail.com>
References: <CACgdh2igirUpwvvc04+psN60aFa0ZcCv1nmGNiE6pNdv3n8bXQ@mail.gmail.com>
	<20120916022028.GA8527@cskk.homeip.net>
	<AD48FCEB-91AF-42D1-B125-3DD847E51545@gmail.com>
	<CAP7+vJL2jXnWc8LztU29E2XgZKJ_knyLbV1MeZnhm6sSA71ucg@mail.gmail.com>
	<50566AA2.5000506@canterbury.ac.nz> <5056ABE6.5040206@pearwood.info>
	<87r4q0p26x.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7fa2DRH981YEDfRCGQHfSRHGq3eTqszvTPkym_v+CjL-g@mail.gmail.com>
Message-ID: <CALvWhxt1vqoM6G=q2yffqNKBAN3TUp17EQSkxrHP+Tk-Ps9AYA@mail.gmail.com>

On Mon, Sep 17, 2012 at 11:55 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Tue, Sep 18, 2012 at 11:08 AM, Stephen J. Turnbull
> <stephen at xemacs.org> wrote:
>> I agree with the suggestion that really we should fix up other Exceptions that are clearly quite heterogeneous by subclassing more precise exceptions from them.
>> But this isn't always possible, and of course may not be backward compatible.

If done well, it should be fully backwards compatible. There is not
reason that you cannot keep the errno (or however its named on the
specific exception class) while still subclassing the exception.

You could have issues with pickled exceptions if you add more details
to the exceptions, and it is not forward compatible: newer code that
gets the older form of exceptions is liable to break, without extreme
care to update the exceptions.

>> I don't know how to do it Pythonically, but it would be nice if there
>> were some way to "subclass Exceptions on the fly".
>
> Most likely, this will mean fixing the bug that means the ABC
> machinery is currently being ignored by the exception machinery. Then
> you can do whatever you want to reshape exception hierarchies.

One trick we've done at work in a few cases is to put a fairly thin
wrapper around the low-level apis that wrap the exceptions into
subclasses. Often, this can be implemented as a simple decorator that
then can be used on a series of functions easily.

>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas


From dholth at gmail.com  Tue Sep 18 21:31:27 2012
From: dholth at gmail.com (Daniel Holth)
Date: Tue, 18 Sep 2012 15:31:27 -0400
Subject: [Python-ideas] bdist naming scheme (compatibility tags) PEP
In-Reply-To: <505719F1.8090405@egenix.com>
References: <CAG8k2+7BYcg5RRnUhjY6h7jKsZUCeYCMp-mUohPSSsh5kmaQFw@mail.gmail.com>
	<50236FD5.2060404@egenix.com>
	<CAG8k2+5ZKwWhMNu_6MAP8yTNPPHV46hrW4HU8m+utNPZT6HaHg@mail.gmail.com>
	<505719F1.8090405@egenix.com>
Message-ID: <CAG8k2+5fhi=nfXqmigBysqOacn7pLmOAksJcR0vSqXvPGZUPmA@mail.gmail.com>

On Mon, Sep 17, 2012 at 8:39 AM, M.-A. Lemburg <mal at egenix.com> wrote:
> Daniel Holth wrote:
>>>> Platform Tag
>>>> ------------
>>>>
>>>> The platform tag is simply `distutils.util.get_platform()` with all
>>>> hyphens `-` and periods `.` replaced with underscore `_`.
>>>
>>> This part is going to cause problems. distutils is good at identifying
>>> Linux and Windows and giving them sensible platform names, but it
>>> doesn't do a good job for other OSes.
>>
>> http://www.python.org/dev/peps/pep-0425/
>
> I still don't think that referencing the distutils function
> in the PEP is a good idea :-) It would be better to create
> a new helper.

How about just sysconfig.get_platform()?


From mal at egenix.com  Wed Sep 19 08:40:53 2012
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 19 Sep 2012 08:40:53 +0200
Subject: [Python-ideas] bdist naming scheme (compatibility tags) PEP
In-Reply-To: <CAG8k2+5fhi=nfXqmigBysqOacn7pLmOAksJcR0vSqXvPGZUPmA@mail.gmail.com>
References: <CAG8k2+7BYcg5RRnUhjY6h7jKsZUCeYCMp-mUohPSSsh5kmaQFw@mail.gmail.com>
	<50236FD5.2060404@egenix.com>
	<CAG8k2+5ZKwWhMNu_6MAP8yTNPPHV46hrW4HU8m+utNPZT6HaHg@mail.gmail.com>
	<505719F1.8090405@egenix.com>
	<CAG8k2+5fhi=nfXqmigBysqOacn7pLmOAksJcR0vSqXvPGZUPmA@mail.gmail.com>
Message-ID: <505968F5.80702@egenix.com>

Daniel Holth wrote:
> On Mon, Sep 17, 2012 at 8:39 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>> Daniel Holth wrote:
>>>>> Platform Tag
>>>>> ------------
>>>>>
>>>>> The platform tag is simply `distutils.util.get_platform()` with all
>>>>> hyphens `-` and periods `.` replaced with underscore `_`.
>>>>
>>>> This part is going to cause problems. distutils is good at identifying
>>>> Linux and Windows and giving them sensible platform names, but it
>>>> doesn't do a good job for other OSes.
>>>
>>> http://www.python.org/dev/peps/pep-0425/
>>
>> I still don't think that referencing the distutils function
>> in the PEP is a good idea :-) It would be better to create
>> a new helper.
> 
> How about just sysconfig.get_platform()?

That's essentially the same function :-) For some reason it's a copy
of the one in distutils.util. I guess an oversight when sysconfig
was created from various parts of distutils.

Both functions are not suitable for the intended purpose, namely
providing enough information to detect binary compatibility.

Given that such information changes more often than we do Python
releases and that this information is domain specific, I think it's
better to maintain a pair of functions for creating such a platform
string and detecting binary compatibility in a separate PyPI module
which can then be pulled in by packaging and installer tools.

The right place for the logic would be the platform module
which was created in much the same way. Like with the above
module, it was crowd-sourced to integrate domain specific
knowledge. We added it to the stdlib after it stabilized.
This is both good and bad. The good part is that it comes
with Python automatically, the bad part that 3rd party code
relying on it now has to deal with several different versions
(for each Python release) and that platform changes are difficult
to get into the module.

This is why I think the PEP should just reference such a new
module and leave the string format and binary compatibility
check details to the module, rather than spell it out in the
PEP.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 19 2012)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2012-10-29: PyCon DE 2012, Leipzig, Germany ...            40 days to go
2012-10-23: Python Meeting Duesseldorf ...                 34 days to go
2012-09-18: Released mxODBC Zope DA 2.1.0 ...     http://egenix.com/go32
2012-08-28: Released mxODBC 3.2.0 ...             http://egenix.com/go31


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From dholth at gmail.com  Wed Sep 19 14:46:19 2012
From: dholth at gmail.com (Daniel Holth)
Date: Wed, 19 Sep 2012 08:46:19 -0400
Subject: [Python-ideas] bdist naming scheme (compatibility tags) PEP
In-Reply-To: <505968F5.80702@egenix.com>
References: <CAG8k2+7BYcg5RRnUhjY6h7jKsZUCeYCMp-mUohPSSsh5kmaQFw@mail.gmail.com>
	<50236FD5.2060404@egenix.com>
	<CAG8k2+5ZKwWhMNu_6MAP8yTNPPHV46hrW4HU8m+utNPZT6HaHg@mail.gmail.com>
	<505719F1.8090405@egenix.com>
	<CAG8k2+5fhi=nfXqmigBysqOacn7pLmOAksJcR0vSqXvPGZUPmA@mail.gmail.com>
	<505968F5.80702@egenix.com>
Message-ID: <CAG8k2+60PM-wxF-imGJr-dMao9T7DW-nqqhCQMkqNibUFqwr+g@mail.gmail.com>

On Wed, Sep 19, 2012 at 2:40 AM, M.-A. Lemburg <mal at egenix.com> wrote:
> Daniel Holth wrote:
>> On Mon, Sep 17, 2012 at 8:39 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>>> Daniel Holth wrote:
>>>>>> Platform Tag
>>>>>> ------------
>>>>>>
>>>>>> The platform tag is simply `distutils.util.get_platform()` with all
>>>>>> hyphens `-` and periods `.` replaced with underscore `_`.
>>>>>
>>>>> This part is going to cause problems. distutils is good at identifying
>>>>> Linux and Windows and giving them sensible platform names, but it
>>>>> doesn't do a good job for other OSes.
>>>>
>>>> http://www.python.org/dev/peps/pep-0425/
>>>
>>> I still don't think that referencing the distutils function
>>> in the PEP is a good idea :-) It would be better to create
>>> a new helper.
>>
>> How about just sysconfig.get_platform()?
>
> That's essentially the same function :-) For some reason it's a copy
> of the one in distutils.util. I guess an oversight when sysconfig
> was created from various parts of distutils.
>
> Both functions are not suitable for the intended purpose, namely
> providing enough information to detect binary compatibility.
>
> Given that such information changes more often than we do Python
> releases and that this information is domain specific, I think it's
> better to maintain a pair of functions for creating such a platform
> string and detecting binary compatibility in a separate PyPI module
> which can then be pulled in by packaging and installer tools.
>
> The right place for the logic would be the platform module
> which was created in much the same way. Like with the above
> module, it was crowd-sourced to integrate domain specific
> knowledge. We added it to the stdlib after it stabilized.
> This is both good and bad. The good part is that it comes
> with Python automatically, the bad part that 3rd party code
> relying on it now has to deal with several different versions
> (for each Python release) and that platform changes are difficult
> to get into the module.
>
> This is why I think the PEP should just reference such a new
> module and leave the string format and binary compatibility
> check details to the module, rather than spell it out in the
> PEP.

The current implementation is at https://bitbucket.org/dholth/wheel

In wheel/bdist_wheel.py and wheel/util.py


From gerald.britton at gmail.com  Wed Sep 19 18:36:10 2012
From: gerald.britton at gmail.com (Gerald Britton)
Date: Wed, 19 Sep 2012 12:36:10 -0400
Subject: [Python-ideas] Why is there no way to pass PYTHONPATH on the
	command line?
Message-ID: <CAPxRSnYp+Nz4bBQ7QpihyKV+AVK=0a6ksBDG9nBD9q3=HdGX5g@mail.gmail.com>

FYI This works in Windows:

C:\Python26>set PYTHONPATH=c:\temp & python
Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit
(Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', 'c:\\temp', 'C:\\WINDOWS\\system32\\python26.zip',
'C:\\Python26\\DLLs', 'C
:\\Python26\\lib', 'C:\\Python26\\lib\\plat-win',
'C:\\Python26\\lib\\lib-tk', '
C:\\Python26', 'C:\\Python26\\lib\\site-packages',
'C:\\Python26\\lib\\site-pack
ages\\wx-2.8-msw-unicode']
>>>

-- 
Gerald Britton
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120919/d19a52ec/attachment.html>

From p.f.moore at gmail.com  Wed Sep 19 20:48:35 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 19 Sep 2012 19:48:35 +0100
Subject: [Python-ideas] Why is there no way to pass PYTHONPATH on the
 command line?
In-Reply-To: <CAPxRSnYp+Nz4bBQ7QpihyKV+AVK=0a6ksBDG9nBD9q3=HdGX5g@mail.gmail.com>
References: <CAPxRSnYp+Nz4bBQ7QpihyKV+AVK=0a6ksBDG9nBD9q3=HdGX5g@mail.gmail.com>
Message-ID: <CACac1F891i2+Qf_KLatWcTomQu-5hY6hH-SbisfoL=+Za4xOgw@mail.gmail.com>

On 19 September 2012 17:36, Gerald Britton <gerald.britton at gmail.com> wrote:
> FYI This works in Windows:
>
> C:\Python26>set PYTHONPATH=c:\temp & python
> Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)]
> on
> win32
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import sys
>>>> sys.path
> ['', 'c:\\temp', 'C:\\WINDOWS\\system32\\python26.zip',
> 'C:\\Python26\\DLLs', 'C
> :\\Python26\\lib', 'C:\\Python26\\lib\\plat-win',
> 'C:\\Python26\\lib\\lib-tk', '
> C:\\Python26', 'C:\\Python26\\lib\\site-packages',
> 'C:\\Python26\\lib\\site-pack
> ages\\wx-2.8-msw-unicode']

But it leaves PYTHONPATH set afterwards, and it overwrites any
PYTHONPATH setting you may have had before.


D:\Data>set PY
PYTHONPATH=C:\Temp

D:\Data>(set PYTHONPATH=D:\ & py -3.2)
Python 3.2.3 (default, Apr 11 2012, 07:15:24) [MSC v.1500 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> ^Z


D:\Data>set PY
PYTHONPATH=D:\

Paul.


From gerald.britton at gmail.com  Wed Sep 19 21:02:45 2012
From: gerald.britton at gmail.com (Gerald Britton)
Date: Wed, 19 Sep 2012 15:02:45 -0400
Subject: [Python-ideas] Why is there no way to pass PYTHONPATH on the
	command line?
Message-ID: <CAPxRSnb7F68wyeHLx8WuxMzV+Q1SfmNpi5J0Y5C4E6ZhACVJMg@mail.gmail.com>

On 19 September 2012 17:36, Gerald Britton <gerald.britton at
gmail.com <http://mail.python.org/mailman/listinfo/python-ideas>>
wrote:
>* FYI This works in Windows:*>**>* C:\Python26>set PYTHONPATH=c:\temp & python*>* Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)]*>* on*>* win32*>* Type "help", "copyright", "credits" or "license" for more information.*>>>>* import sys*>>>>* sys.path*>* ['', 'c:\\temp', 'C:\\WINDOWS\\system32\\python26.zip',*>* 'C:\\Python26\\DLLs', 'C*>* :\\Python26\\lib', 'C:\\Python26\\lib\\plat-win',*>* 'C:\\Python26\\lib\\lib-tk', '*>* C:\\Python26', 'C:\\Python26\\lib\\site-packages',*>* 'C:\\Python26\\lib\\site-pack*>* ages\\wx-2.8-msw-unicode']*
>
> But it leaves PYTHONPATH set afterwards, and it overwrites any
> PYTHONPATH setting you may have had before.


Sure, but that's easy to remedy.  just add

& set *PYTHONPATH=*

to the end.  If you want to get fancy, save the current value in a
temporary variable and use it later to reset the value.

We're really dealing with a limitation of the Windows Shell.  Things
are somewhat better in PowerShell but it's still not like BASH.

Probably better to write a little script that saves the variable
PYTHONPATH, sets it as desired, calls Python, then resets it
afterwards.

-- 
Gerald Britton
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120919/b34935dc/attachment.html>

From gerald.britton at gmail.com  Wed Sep 19 21:13:39 2012
From: gerald.britton at gmail.com (Gerald Britton)
Date: Wed, 19 Sep 2012 15:13:39 -0400
Subject: [Python-ideas] Why is there no way to pass PYTHONPATH on the
	command line?
In-Reply-To: <CAPxRSnb7F68wyeHLx8WuxMzV+Q1SfmNpi5J0Y5C4E6ZhACVJMg@mail.gmail.com>
References: <CAPxRSnb7F68wyeHLx8WuxMzV+Q1SfmNpi5J0Y5C4E6ZhACVJMg@mail.gmail.com>
Message-ID: <CAPxRSnbMWZ+te93c7oWEz1ZcsiVx5P9-M07cnVRgd-M_Fsiy7A@mail.gmail.com>

BTW, this works:

PS C:\Python26> powershell {Set-Item -Path env:PYTHONPATH -value C:\temp;
c:\python26\python}
Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', 'C:\\temp', 'C:\\WINDOWS\\system32\\python26.zip',
'c:\\python26\\DLLs', 'c:\\python26\\lib', 'c:\\python26\\lib\\p
lat-win', 'c:\\python26\\lib\\lib-tk', 'c:\\python26',
'c:\\python26\\lib\\site-packages', 'c:\\python26\\lib\\site-pack
ages\\wx-2.8-msw-unicode']
>>> ^Z

PS C:\Python26> ls env:PYTHONPATH
Get-ChildItem : Cannot find path 'PYTHONPATH' because it does not exist.
At line:1 char:3
+ ls <<<<  env:PYTHONPATH
    + CategoryInfo          : ObjectNotFound: (PYTHONPATH:String)
[Get-ChildItem], ItemNotFoundException
    + FullyQualifiedErrorId :
PathNotFound,Microsoft.PowerShell.Commands.GetChildItemCommand

PS C:\Python26>

On Wed, Sep 19, 2012 at 3:02 PM, Gerald Britton <gerald.britton at gmail.com>wrote:

> On 19 September 2012 17:36, Gerald Britton <gerald.britton at gmail.com <http://mail.python.org/mailman/listinfo/python-ideas>> wrote:
> >* FYI This works in Windows:*>**>* C:\Python26>set PYTHONPATH=c:\temp & python*>* Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)]*>* on*>* win32*>* Type "help", "copyright", "credits" or "license" for more information.*>>>>* import sys*>>>>* sys.path*>* ['', 'c:\\temp', 'C:\\WINDOWS\\system32\\python26.zip',*>* 'C:\\Python26\\DLLs', 'C*>* :\\Python26\\lib', 'C:\\Python26\\lib\\plat-win',*>* 'C:\\Python26\\lib\\lib-tk', '*>* C:\\Python26', 'C:\\Python26\\lib\\site-packages',*>* 'C:\\Python26\\lib\\site-pack*>* ages\\wx-2.8-msw-unicode']*
>>
>> But it leaves PYTHONPATH set afterwards, and it overwrites any
>>
>> PYTHONPATH setting you may have had before.
>
>
> Sure, but that's easy to remedy.  just add
>
> & set *PYTHONPATH=*
>
> to the end.  If you want to get fancy, save the current value in a temporary variable and use it later to reset the value.
>
>
> We're really dealing with a limitation of the Windows Shell.  Things are somewhat better in PowerShell but it's still not like BASH.
>
> Probably better to write a little script that saves the variable PYTHONPATH, sets it as desired, calls Python, then resets it afterwards.
>
> --
> Gerald Britton
>



-- 
Gerald Britton
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120919/9926c58f/attachment.html>

From stephen at xemacs.org  Thu Sep 20 11:17:34 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 20 Sep 2012 18:17:34 +0900
Subject: [Python-ideas] syntax to continue into the next
	subsequent	except block
In-Reply-To: <CALvWhxt1vqoM6G=q2yffqNKBAN3TUp17EQSkxrHP+Tk-Ps9AYA@mail.gmail.com>
References: <CACgdh2igirUpwvvc04+psN60aFa0ZcCv1nmGNiE6pNdv3n8bXQ@mail.gmail.com>
	<20120916022028.GA8527@cskk.homeip.net>
	<AD48FCEB-91AF-42D1-B125-3DD847E51545@gmail.com>
	<CAP7+vJL2jXnWc8LztU29E2XgZKJ_knyLbV1MeZnhm6sSA71ucg@mail.gmail.com>
	<50566AA2.5000506@canterbury.ac.nz>
	<5056ABE6.5040206@pearwood.info>
	<87r4q0p26x.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7fa2DRH981YEDfRCGQHfSRHGq3eTqszvTPkym_v+CjL-g@mail.gmail.com>
	<CALvWhxt1vqoM6G=q2yffqNKBAN3TUp17EQSkxrHP+Tk-Ps9AYA@mail.gmail.com>
Message-ID: <87a9wloxxt.fsf@uwakimon.sk.tsukuba.ac.jp>

Chris Kaynor writes:

 > On Mon, Sep 17, 2012 at 11:55 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
 > > On Tue, Sep 18, 2012 at 11:08 AM, Stephen J. Turnbull
 > > <stephen at xemacs.org> wrote:
 > >> I agree with the suggestion that really we should fix up other
 > >> Exceptions that are clearly quite heterogeneous by subclassing
 > >> more precise exceptions from them.  But this isn't always
 > >> possible, and of course may not be backward compatible.
 > 
 > If done well, it should be fully backwards compatible. There is not
 > reason that you cannot keep the errno (or however its named on the
 > specific exception class) while still subclassing the exception.

Of course that's backward compatible with old code receiving the new
subclassed exceptions.  Maybe I misused the term "backward
compatible", but what I meant was that it's likely to be the case that
the person doing the subclassing will unify groups of error codes he
believes unlikely to need distinguishing in Python applications
(vs. more low-level languages).  Eg, I don't see more than one hundred
new Exceptions in PEP 3151, but only a handful (about 15).



From py.vjp at vivekpowell.com  Thu Sep 20 17:22:30 2012
From: py.vjp at vivekpowell.com (V James Powell)
Date: Thu, 20 Sep 2012 11:22:30 -0400 (EDT)
Subject: [Python-ideas] syntax to continue into the next subsequent
	except block
In-Reply-To: <mailman.53.1348135203.5868.python-ideas@python.org>
References: <mailman.53.1348135203.5868.python-ideas@python.org>
Message-ID: <alpine.DEB.2.00.1209201114400.32409@disigma.com>

> I think it would be useful if there was a way to skip into the next except
> block, perhaps with continue as I think it's currently always illegal to use
> in an except block. I don't believe there's currently a way to do this.

I would like to share that the original suggestion inspired me on the following
(mis-)adventure.

Cheers,
V James Powell

switch = lambda val: type('', (BaseException,), {'val': val})()
case = lambda pred: type('', (type,), {'__subclasscheck__':
   staticmethod(lambda x: pred(x.val))})('',(BaseException,),{})
default = BaseException

#x = '%s'
x = 12
try:raise    switch(x)
except         case( lambda x: x % 2 == 0 ):
 	             print "It's even!"
except         case( lambda x: x % 2 == 1 ):
 	             print "It's odd!"
except         default:
                  print "It's a joke!"


From chrysn at fsfe.org  Sat Sep 22 18:31:06 2012
From: chrysn at fsfe.org (chrysn)
Date: Sat, 22 Sep 2012 18:31:06 +0200
Subject: [Python-ideas] asyncore: included batteries don't fit
Message-ID: <20120922163106.GA18772@hephaistos.amsuess.com>

hello python-ideas,

i'd like to start discussion about the state of asyncore/asynchat's
adaption in the python standard library, with the intention of finding a
roadmap for how to improve things, and of kicking off and coordinating
implementations.

here's the problem (as previously described in [issue15978] and
redirected here, with some additions):

the asyncore module would be much more useful if it were well integrated
in the standard library. in particular, it should be supported by:

* subprocess

* BaseHTTPServer / http.server (and thus, socketserver)

* urllib2 / urllib, http.client

* probably many other network libraries except smtpd, which already uses
  asyncore

* third party libraries (if stdlib leads the way, the ecosystem will
  follow; eg pyserial)

without widespread asyncore support, it is not possible to easily
integrate different servers and services with each other; with asyncore
support, it's just a matter of creating the objects and entering the
main loop. (eg, a http server for controlling a serial device, with a
telnet-like debugging interface).

some examples of the changes required:

* the socketserver documents that it would like to have such a
  framework ("Future work: [...] Standard framework for select-based
  multiplexing"). due to the nature of socketserver based
  implementations (blocking reads), we can't just "add glue so it
  works", but there could be extensions so that implementations can be
  ported to asynchronous socketservers. i've done if for a particular
  case (ported SimpleHTTPServer, but it's a mess of monkey-patching and
  intermediate StringIOs).

* for subprocess, there's a bunch of recipies at [1].

* pyserial (not standard library, but might as well become) can be
  ported quite easily [2].


this touches several modules whose implementations can be handled
independently from each other; i'd implement some of them myself.
terry.reedy redirected me from the issue tracker to this list, hoping
for controversy and alternatives. if you'd like to discuss, throw in
questions, and we'll find a solution. if you'd think talk is cheap, i
can try to work out first sketches.


python already has batteries for nonblocking operation included, and i
say it's doing it right -- let's just make sure the batteries fit in the
other gadgets!


yours truly
chrysn


[1] http://code.activestate.com/recipes/576957-asynchronous-subprocess-using-asyncore/
[2] http://sourceforge.net/tracker/?func=detail&aid=3559321&group_id=46487&atid=446305
[issue15978] http://bugs.python.org/issue15978


-- 
Es ist nicht deine Schuld, dass die Welt ist, wie sie ist -- es w?r' nur
deine Schuld, wenn sie so bleibt.
(You are not to blame for the state of the world, but you would be if
that state persisted.)
  -- Die ?rzte
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120922/dda79107/attachment.pgp>

From phd at phdru.name  Sat Sep 22 18:52:53 2012
From: phd at phdru.name (Oleg Broytman)
Date: Sat, 22 Sep 2012 20:52:53 +0400
Subject: [Python-ideas] asyncore: included batteries don't fit
In-Reply-To: <20120922163106.GA18772@hephaistos.amsuess.com>
References: <20120922163106.GA18772@hephaistos.amsuess.com>
Message-ID: <20120922165253.GA32520@iskra.aviel.ru>

Hi!

On Sat, Sep 22, 2012 at 06:31:06PM +0200, chrysn <chrysn at fsfe.org> wrote:
> the asyncore module would be much more useful if it were well integrated
> in the standard library. in particular, it should be supported by:
> 
> * BaseHTTPServer / http.server (and thus, socketserver)
> 
> * urllib2 / urllib, http.client
> 
> * probably many other network libraries except smtpd, which already uses
>   asyncore

   It seems you want Twisted, no?

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.


From chrysn at fsfe.org  Sat Sep 22 20:27:10 2012
From: chrysn at fsfe.org (chrysn)
Date: Sat, 22 Sep 2012 20:27:10 +0200
Subject: [Python-ideas] asyncore: included batteries don't fit
In-Reply-To: <20120922165253.GA32520@iskra.aviel.ru>
References: <20120922163106.GA18772@hephaistos.amsuess.com>
	<20120922165253.GA32520@iskra.aviel.ru>
Message-ID: <20120922182710.GB18772@hephaistos.amsuess.com>

On Sat, Sep 22, 2012 at 08:52:53PM +0400, Oleg Broytman wrote:
> On Sat, Sep 22, 2012 at 06:31:06PM +0200, chrysn <chrysn at fsfe.org> wrote:
> > the asyncore module would be much more useful if it were well integrated
> > in the standard library. in particular, it should be supported by:
> > 
> > [...]
> 
>    It seems you want Twisted, no?

if these considerations end in twisted being consecrated as the new
asyncore, i'd consider that a valid solution too. then, again,
subprocess and the onboard servers should work well with *that* out of
the box.

best regards
chrysn

-- 
To use raw power is to make yourself infinitely vulnerable to greater powers.
  -- Bene Gesserit axiom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120922/0ddf38e6/attachment.pgp>

From solipsis at pitrou.net  Sat Sep 22 20:46:06 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 22 Sep 2012 20:46:06 +0200
Subject: [Python-ideas] asyncore: included batteries don't fit
References: <20120922163106.GA18772@hephaistos.amsuess.com>
Message-ID: <20120922204606.0b206706@pitrou.net>

On Sat, 22 Sep 2012 18:31:06 +0200
chrysn <chrysn at fsfe.org> wrote:
> hello python-ideas,
> 
> i'd like to start discussion about the state of asyncore/asynchat's
> adaption in the python standard library, with the intention of finding a
> roadmap for how to improve things, and of kicking off and coordinating
> implementations.
> 
> here's the problem (as previously described in [issue15978] and
> redirected here, with some additions):
> 
> the asyncore module would be much more useful if it were well integrated
> in the standard library. in particular, it should be supported by:

SSL support is also lacking:
http://bugs.python.org/issue10084

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net




From phd at phdru.name  Sat Sep 22 20:52:10 2012
From: phd at phdru.name (Oleg Broytman)
Date: Sat, 22 Sep 2012 22:52:10 +0400
Subject: [Python-ideas] asyncore: included batteries don't fit
In-Reply-To: <20120922182710.GB18772@hephaistos.amsuess.com>
References: <20120922163106.GA18772@hephaistos.amsuess.com>
	<20120922165253.GA32520@iskra.aviel.ru>
	<20120922182710.GB18772@hephaistos.amsuess.com>
Message-ID: <20120922185210.GA3082@iskra.aviel.ru>

On Sat, Sep 22, 2012 at 08:27:10PM +0200, chrysn <chrysn at fsfe.org> wrote:
> On Sat, Sep 22, 2012 at 08:52:53PM +0400, Oleg Broytman wrote:
> > On Sat, Sep 22, 2012 at 06:31:06PM +0200, chrysn <chrysn at fsfe.org> wrote:
> > > the asyncore module would be much more useful if it were well integrated
> > > in the standard library. in particular, it should be supported by:
> > > 
> > > [...]
> > 
> >    It seems you want Twisted, no?
> 
> if these considerations end in twisted being consecrated as the new
> asyncore, i'd consider that a valid solution too.

   If you mean that Twisted will be included in the standard library --
then no, I'm sure it will not. Python comes with batteries included, but
Twisted is not a battery, it's rather a power plant. I am sure it will
always be developed and distributed separately.
   And developing asyncore to the level of Twisted would be a
duplication of effort.

> then, again,
> subprocess and the onboard servers should work well with *that* out of
> the box.

   If you want subprocess and Twisted to work together -- you know where
to send patches.

PS. In my not so humble opinion what the standard library really lacks
in this area is a way to combine a few asynchronous libraries with
different mainloops. Think about wxPython+Twisted in one program. But I
have no slightest idea how to approach the problem.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.


From amauryfa at gmail.com  Sat Sep 22 21:50:39 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Sat, 22 Sep 2012 21:50:39 +0200
Subject: [Python-ideas] asyncore: included batteries don't fit
In-Reply-To: <20120922185210.GA3082@iskra.aviel.ru>
References: <20120922163106.GA18772@hephaistos.amsuess.com>
	<20120922165253.GA32520@iskra.aviel.ru>
	<20120922182710.GB18772@hephaistos.amsuess.com>
	<20120922185210.GA3082@iskra.aviel.ru>
Message-ID: <CAGmFidYCCMS-gvTYQZPeL8+9t0uPMKA+gjWm3e1uM1F6rW6fnA@mail.gmail.com>

2012/9/22 Oleg Broytman <phd at phdru.name>:
> PS. In my not so humble opinion what the standard library really lacks
> in this area is a way to combine a few asynchronous libraries with
> different mainloops. Think about wxPython+Twisted in one program. But I
> have no slightest idea how to approach the problem.

Twisted proposes a wxreactor, of course.

-- 
Amaury Forgeot d'Arc


From phd at phdru.name  Sat Sep 22 22:05:42 2012
From: phd at phdru.name (Oleg Broytman)
Date: Sun, 23 Sep 2012 00:05:42 +0400
Subject: [Python-ideas] combine libraries with different mainloops
In-Reply-To: <CAGmFidYCCMS-gvTYQZPeL8+9t0uPMKA+gjWm3e1uM1F6rW6fnA@mail.gmail.com>
References: <20120922163106.GA18772@hephaistos.amsuess.com>
	<20120922165253.GA32520@iskra.aviel.ru>
	<20120922182710.GB18772@hephaistos.amsuess.com>
	<20120922185210.GA3082@iskra.aviel.ru>
	<CAGmFidYCCMS-gvTYQZPeL8+9t0uPMKA+gjWm3e1uM1F6rW6fnA@mail.gmail.com>
Message-ID: <20120922200542.GA5006@iskra.aviel.ru>

On Sat, Sep 22, 2012 at 09:50:39PM +0200, Amaury Forgeot d'Arc <amauryfa at gmail.com> wrote:
> 2012/9/22 Oleg Broytman <phd at phdru.name>:
> > PS. In my not so humble opinion what the standard library really lacks
> > in this area is a way to combine a few asynchronous libraries with
> > different mainloops. Think about wxPython+Twisted in one program. But I
> > have no slightest idea how to approach the problem.
> 
> Twisted proposes a wxreactor, of course.

   And wxPython has a meaning to extend its main loop. But these are
only partial solutions. There are much more libraries with mainloops.
D-Bus/GLib, e.g.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.


From chrysn at fsfe.org  Sat Sep 22 22:16:53 2012
From: chrysn at fsfe.org (chrysn)
Date: Sat, 22 Sep 2012 22:16:53 +0200
Subject: [Python-ideas] asyncore: included batteries don't fit
In-Reply-To: <20120922185210.GA3082@iskra.aviel.ru>
References: <20120922163106.GA18772@hephaistos.amsuess.com>
	<20120922165253.GA32520@iskra.aviel.ru>
	<20120922182710.GB18772@hephaistos.amsuess.com>
	<20120922185210.GA3082@iskra.aviel.ru>
Message-ID: <20120922201653.GC18772@hephaistos.amsuess.com>

On Sat, Sep 22, 2012 at 10:52:10PM +0400, Oleg Broytman wrote:
>    If you mean that Twisted will be included in the standard library --
> then no, I'm sure it will not. Python comes with batteries included, but
> Twisted is not a battery, it's rather a power plant. I am sure it will
> always be developed and distributed separately.
>    And developing asyncore to the level of Twisted would be a
> duplication of effort.

> PS. In my not so humble opinion what the standard library really lacks
> in this area is a way to combine a few asynchronous libraries with
> different mainloops. Think about wxPython+Twisted in one program. But I
> have no slightest idea how to approach the problem.

well, what about python including a battery and a battery plug, then?
asyncore could be the battery, and a interface between asynchronous
libraries the battery plug. users could start developing with batteries,
and when the project grows, just plug it into a power plant.

less analogy, more technical: the asyncore dispatcher to main loop
interface is pretty thin -- there is a (global or explicitly passed)
"map" (a dictionary), mapping file descriptors to objects that can be
readable or writable (or acceptable, not sure if that detail is really
needed that far down). a dispatcher registers to a map, and then the
main loop select()s for events on all files and dispatches them
accordingly.

it won't be as easy as just taking that interface, eg because it lacks
timeouts, but i think it can be the "way to combine a few asynchronous
libraries". (to avoid asyncore becoming a powerplant itself, it could
choose not to implement some features for simplicity. for example, if
asyncore chose to still not implement timeouts, registering timeouts to
a asyncore based main loop would just result in a NotImplementedError
telling the user to get a more powerful main loop.)

i don't want to claim i know how that could work in detail or even if it
could work at all, but if this is interesting for enough people that it
will be used, i'd like to find out.


> > then, again, subprocess and the onboard servers should work well
> > with *that* out of the box.
> 
>    If you want subprocess and Twisted to work together -- you know where
> to send patches.

no, actually -- for now, it'd be a patch to twisted (who'd reply with
"we already have a way of dealing with it"). if asyncore's interface
becomes the battery plug, it'd be a patch to subprocess.


thanks for sharing your ideas
chrysn

-- 
To use raw power is to make yourself infinitely vulnerable to greater powers.
  -- Bene Gesserit axiom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120922/818d5445/attachment.pgp>

From tjreedy at udel.edu  Sun Sep 23 01:04:44 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 22 Sep 2012 19:04:44 -0400
Subject: [Python-ideas] asyncore: included batteries don't fit
In-Reply-To: <20120922204606.0b206706@pitrou.net>
References: <20120922163106.GA18772@hephaistos.amsuess.com>
	<20120922204606.0b206706@pitrou.net>
Message-ID: <k3lg6m$grm$1@ger.gmane.org>

On 9/22/2012 2:46 PM, Antoine Pitrou wrote:
> On Sat, 22 Sep 2012 18:31:06 +0200
> chrysn <chrysn at fsfe.org> wrote:
>> hello python-ideas,
>>
>> i'd like to start discussion about the state of asyncore/asynchat's
>> adaption in the python standard library, with the intention of finding a
>> roadmap for how to improve things, and of kicking off and coordinating
>> implementations.
>>
>> here's the problem (as previously described in [issue15978] and
>> redirected here, with some additions):
>>
>> the asyncore module would be much more useful if it were well integrated
>> in the standard library. in particular, it should be supported by:
>
> SSL support is also lacking:
> http://bugs.python.org/issue10084

chrysn: The issue needs a patch that incorporates Antoine's review. I am 
sure there are other asyncore issues that could use help too.

-- 
Terry Jan Reedy



From dreamingforward at gmail.com  Mon Sep 24 23:49:27 2012
From: dreamingforward at gmail.com (Mark Adam)
Date: Mon, 24 Sep 2012 16:49:27 -0500
Subject: [Python-ideas] file API
Message-ID: <CAMjeLr8P3GeZUwKf8H+AxjCiA0TtEELpDhmxYBuSSHs88JuNcA@mail.gmail.com>

For some time now, I've wanted to suggest a better abstraction for the
<file> type in Python.  It currently uses an antiquated, low-level
C-style interface for moving around in a file, with methods like
tell() and seek().  But after attributes were introduced to Python, it
seems like it should be re-evaluated.

Let file-type have an attribute .pos for position.   Now you can get
rid of the seek() and tell() methods and manipulate the file pointer
by the more standard and familiar arithmetic operations:

>>> file.pos = x0ae1       #move file pointer to an absolute address
>>> file.pos += 1             #increment the file pointer one byte
>>> curr_pos = file.pos  #read current file pointer

You've now simplified the API by the removal of two obscure legacy
methods (where one has to learn the additional concept of "absolute"
and "relative" addressing) and replaced them with a more basic one
called "position".

Thoughts?

markj


From jsbueno at python.org.br  Mon Sep 24 23:55:58 2012
From: jsbueno at python.org.br (Joao S. O. Bueno)
Date: Mon, 24 Sep 2012 18:55:58 -0300
Subject: [Python-ideas] file API
In-Reply-To: <CAMjeLr8P3GeZUwKf8H+AxjCiA0TtEELpDhmxYBuSSHs88JuNcA@mail.gmail.com>
References: <CAMjeLr8P3GeZUwKf8H+AxjCiA0TtEELpDhmxYBuSSHs88JuNcA@mail.gmail.com>
Message-ID: <CAH0mxTQLQ-rQ8HYyxj6Cg2tXKr+fTKkJ+uFthVjNupvxWXpauw@mail.gmail.com>

On 24 September 2012 18:49, Mark Adam <dreamingforward at gmail.com> wrote:
> For some time now, I've wanted to suggest a better abstraction for the
> <file> type in Python.  It currently uses an antiquated, low-level
> C-style interface for moving around in a file, with methods like
> tell() and seek().  But after attributes were introduced to Python, it
> seems like it should be re-evaluated.
>
> Let file-type have an attribute .pos for position.   Now you can get
> rid of the seek() and tell() methods and manipulate the file pointer
> by the more standard and familiar arithmetic operations:
>
>>>> file.pos = x0ae1       #move file pointer to an absolute address
>>>> file.pos += 1             #increment the file pointer one byte
>>>> curr_pos = file.pos  #read current file pointer
>
> You've now simplified the API by the removal of two obscure legacy
> methods (where one has to learn the additional concept of "absolute"
> and "relative" addressing) and replaced them with a more basic one
> called "position".
>
> Thoughts?

-1
This is not so distant from what can be achieved trivially by tell and seek.

Moreover, event though changes in attributes _can_ be made to have
side effects in Python objects, it does not mean it is easier to read and
maintain in every case.

What I think we need is a better way of dealing with constants - the "whence"
attribute for "seek" takes raw ints for "from start", "from end" and
"relative" -
but that is an entirely other subject.

  js
 -><-

>
> markj
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas


From g.rodola at gmail.com  Tue Sep 25 00:31:37 2012
From: g.rodola at gmail.com (=?ISO-8859-1?Q?Giampaolo_Rodol=E0?=)
Date: Mon, 24 Sep 2012 15:31:37 -0700
Subject: [Python-ideas] asyncore: included batteries don't fit
In-Reply-To: <20120922163106.GA18772@hephaistos.amsuess.com>
References: <20120922163106.GA18772@hephaistos.amsuess.com>
Message-ID: <CAFYqXL-rGN_2ZVpLbw575sCem8Ft9WAB1RNpZKexAhoMHKWL5g@mail.gmail.com>

I still think this proposal is too vaguely defined and any effort towards
adding async IO support to existing batteries is premature for different
reasons, first of which the inadequacy of asyncore as the base async
framework to fulfill the task you're proposing.

asyncore is so old and difficult to fix/enhance without breaking backward
compatibility (see for example http://bugs.python.org/issue11273#msg156439)
that relying on it for any modern work is inevitably a bad idea.

>From a chronological standpoint I still think the best thing to do in order
to fix the "python async problem" once and for all is to first define and
possibly implement an "async WSGI interface" describing what a standard
async IO loop/reactor should look like (in terms of API) and how to
integrate with it, see:
http://mail.python.org/pipermail/python-ideas/2012-May/015223.html
http://mail.python.org/pipermail/python-ideas/2012-May/015235.html

>From there the python stdlib *might* grow a new module implementing the
"async WSGI interface" (let's call it asyncore2) and some of the stdlib
batteries such as socketserver can possibly use it.

In my mind this is the ideal long-term scenario but even managing to define
an "async WSGI interface" alone would be a big step forward.

Again, at this point in time what you're proposing looks too vague,
ambitious and premature to me.


--- Giampaolo
http://code.google.com/p/pyftpdlib/
http://code.google.com/p/psutil/
http://code.google.com/p/pysendfile/



2012/9/22 chrysn <chrysn at fsfe.org>

> hello python-ideas,
>
> i'd like to start discussion about the state of asyncore/asynchat's
> adaption in the python standard library, with the intention of finding a
> roadmap for how to improve things, and of kicking off and coordinating
> implementations.
>
> here's the problem (as previously described in [issue15978] and
> redirected here, with some additions):
>
> the asyncore module would be much more useful if it were well integrated
> in the standard library. in particular, it should be supported by:
>
> * subprocess
>
> * BaseHTTPServer / http.server (and thus, socketserver)
>
> * urllib2 / urllib, http.client
>
> * probably many other network libraries except smtpd, which already uses
>   asyncore
>
> * third party libraries (if stdlib leads the way, the ecosystem will
>   follow; eg pyserial)
>
> without widespread asyncore support, it is not possible to easily
> integrate different servers and services with each other; with asyncore
> support, it's just a matter of creating the objects and entering the
> main loop. (eg, a http server for controlling a serial device, with a
> telnet-like debugging interface).
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120924/2991ca3b/attachment.html>

From josiah.carlson at gmail.com  Tue Sep 25 02:02:08 2012
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Mon, 24 Sep 2012 17:02:08 -0700
Subject: [Python-ideas] asyncore: included batteries don't fit
In-Reply-To: <CAFYqXL-rGN_2ZVpLbw575sCem8Ft9WAB1RNpZKexAhoMHKWL5g@mail.gmail.com>
References: <20120922163106.GA18772@hephaistos.amsuess.com>
	<CAFYqXL-rGN_2ZVpLbw575sCem8Ft9WAB1RNpZKexAhoMHKWL5g@mail.gmail.com>
Message-ID: <CADcB0yhAys8hn7B1O9-nnEMR-GDNt3b13F26neBEx+6D-3ho+Q@mail.gmail.com>

Temporarily un-lurking to reply to this thread (which I'll actually be reading).

Giampaolo and I talked about this for a bit over the weekend, and I
have to say that I agree with his perspective.

In particular, to get something better than asyncore, there must be
something minimally better to build upon. I don't really have an
opinion on what that minimally better thing should be named, but I do
agree that having a simple reactor API that has predictable behavior
over the variety of handlers (select, poll, epoll, kqueue, WSAEvent in
Windows, etc.) is necessary.

Now, let's get to brass tacks...

1. Whatever reactors are available, you need to be able to instantiate
multiple of different types of reactors and multiple instances of the
same type of reactor simultaneously (to support multiple threads
handling different groups of reactors, or different reactors for
different types of objects on certain platforms). While this allows
for insanity in the worst-case, we're all consenting adults here, so
shouldn't be limited by reactor singletons. There should be a default
reactor class, which is defined on module/package import (use the
"best" one for the platform).

2. The API must be simple. I am not sure that it can get easier than
Idea #3 from:
http://mail.python.org/pipermail/python-ideas/2012-May/015245.html
I personally like it because it offers a simple upgrade path for
asyncore users (create your asyncore-derived classes, pass it into the
new reactor), while simultaneously defining a relatively easy API for
any 3rd party to integrate with. By offering an easy-to-integrate
method for 3rd parties (that is also sane), there is the added bonus
that 3rd parties are more likely to integrate, rather than replace,
which means more use in the "real world", better bug reports, etc. To
simplify integration further, make the API register(fd, handler,
events=singleton). Passing no events from the caller means "register
me for all events", which will help 3rd parties that aren't great with
handling read/write registration.

3. I don't have a 3rd tack, you can hang things on the wall with 2 ;)

Regards,
 - Josiah

On Mon, Sep 24, 2012 at 3:31 PM, Giampaolo Rodol? <g.rodola at gmail.com> wrote:
>
> I still think this proposal is too vaguely defined and any effort towards
> adding async IO support to existing batteries is premature for different
> reasons, first of which the inadequacy of asyncore as the base async
> framework to fulfill the task you're proposing.
>
> asyncore is so old and difficult to fix/enhance without breaking backward
> compatibility (see for example http://bugs.python.org/issue11273#msg156439)
> that relying on it for any modern work is inevitably a bad idea.
>
> From a chronological standpoint I still think the best thing to do in order
> to fix the "python async problem" once and for all is to first define and
> possibly implement an "async WSGI interface" describing what a standard
> async IO loop/reactor should look like (in terms of API) and how to
> integrate with it, see:
> http://mail.python.org/pipermail/python-ideas/2012-May/015223.html
> http://mail.python.org/pipermail/python-ideas/2012-May/015235.html
>
> From there the python stdlib *might* grow a new module implementing the
> "async WSGI interface" (let's call it asyncore2) and some of the stdlib
> batteries such as socketserver can possibly use it.
>
> In my mind this is the ideal long-term scenario but even managing to define
> an "async WSGI interface" alone would be a big step forward.
>
> Again, at this point in time what you're proposing looks too vague,
> ambitious and premature to me.
>
>
> --- Giampaolo
> http://code.google.com/p/pyftpdlib/
> http://code.google.com/p/psutil/
> http://code.google.com/p/pysendfile/
>
>
>
> 2012/9/22 chrysn <chrysn at fsfe.org>
>>
>> hello python-ideas,
>>
>> i'd like to start discussion about the state of asyncore/asynchat's
>> adaption in the python standard library, with the intention of finding a
>> roadmap for how to improve things, and of kicking off and coordinating
>> implementations.
>>
>> here's the problem (as previously described in [issue15978] and
>> redirected here, with some additions):
>>
>> the asyncore module would be much more useful if it were well integrated
>> in the standard library. in particular, it should be supported by:
>>
>> * subprocess
>>
>> * BaseHTTPServer / http.server (and thus, socketserver)
>>
>> * urllib2 / urllib, http.client
>>
>> * probably many other network libraries except smtpd, which already uses
>>   asyncore
>>
>> * third party libraries (if stdlib leads the way, the ecosystem will
>>   follow; eg pyserial)
>>
>> without widespread asyncore support, it is not possible to easily
>> integrate different servers and services with each other; with asyncore
>> support, it's just a matter of creating the objects and entering the
>> main loop. (eg, a http server for controlling a serial device, with a
>> telnet-like debugging interface).
>
>


From matt at whoosh.ca  Tue Sep 25 18:05:25 2012
From: matt at whoosh.ca (Matt Chaput)
Date: Tue, 25 Sep 2012 12:05:25 -0400
Subject: [Python-ideas] Regex support code as std lib functions
Message-ID: <5061D645.1070403@whoosh.ca>

I'm not sure where regex is in its long march toward replacing re, but I 
just noticed something interesting: the regex module seems to include 
implementations of some useful functions to support its regular 
expression matching, e.g.:

- Levenshtein distance

- Unicode case folding

Both of these would be useful as fast functions in the std lib. If/when 
regex replaces re, any possibility all the useful functions that support 
it could be added to the std lib in the appropriate modules as part of 
integrating it?

Cheers,

Matt


From python at mrabarnett.plus.com  Tue Sep 25 18:31:44 2012
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 25 Sep 2012 17:31:44 +0100
Subject: [Python-ideas] Regex support code as std lib functions
In-Reply-To: <5061D645.1070403@whoosh.ca>
References: <5061D645.1070403@whoosh.ca>
Message-ID: <5061DC70.4070509@mrabarnett.plus.com>

On 2012-09-25 17:05, Matt Chaput wrote:
> I'm not sure where regex is in its long march toward replacing re, but I
> just noticed something interesting: the regex module seems to include
> implementations of some useful functions to support its regular
> expression matching, e.g.:
>
> - Levenshtein distance
>
> - Unicode case folding
>
> Both of these would be useful as fast functions in the std lib. If/when
> regex replaces re, any possibility all the useful functions that support
> it could be added to the std lib in the appropriate modules as part of
> integrating it?
>
Python 3.3 includes case-folding:

 >>> "\N{LATIN SMALL LETTER SHARP S}".casefold()
'ss'

The regex module doesn't support Levenshtein distance as such, instead 
it supports fuzzy (approximate) matching, where you're concerned not so
much about the _minimum_ edit distance as whether there are no more
than a certain number of errors when matching a regex pattern.

It would be more efficient to implement Levenshtein distance separately.


From p.f.moore at gmail.com  Tue Sep 25 22:39:31 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 25 Sep 2012 21:39:31 +0100
Subject: [Python-ideas] bdist naming scheme (compatibility tags) PEP
In-Reply-To: <CADiSq7eU46twFdBV6+7NFbek9ddhX7RnJUvUcvZOgujthqL+vA@mail.gmail.com>
References: <CAG8k2+7BYcg5RRnUhjY6h7jKsZUCeYCMp-mUohPSSsh5kmaQFw@mail.gmail.com>
	<50236FD5.2060404@egenix.com>
	<CAG8k2+5Eygfk5MFx35OG98RhxcrVpDXC+0Ph9S+ecG4ahG5WWw@mail.gmail.com>
	<CAG8k2+59BaH_a4FcDy-H731b0g8Gk-9urWr4VvJ_bkQ3AZ-RMA@mail.gmail.com>
	<CADiSq7eU46twFdBV6+7NFbek9ddhX7RnJUvUcvZOgujthqL+vA@mail.gmail.com>
Message-ID: <CACac1F9yD1FrSOj_dZUfx0iAzZQvixi1auL57=8jSvqHg6qDDA@mail.gmail.com>

On 9 September 2012 13:16, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Sun, Sep 9, 2012 at 1:41 PM, Daniel Holth <dholth at gmail.com> wrote:
>> Re: http://www.python.org/dev/peps/pep-0425/ "Compatibility tags for
>> built distributions"
>>
>> Progress towards a proper set of rules for generating the tags a
>> Python implementation is likely to support.
>>
>> This system of being willing to install older built distributions is
>> intended to solve the frustrating problem with eggs that you would
>> have to build a new egg for each Python release, even for a
>> pure-Python egg that probably runs fine on a newer Python.
>
> Yep, those rules look sensible to me (and thanks for clarifying the
> intended semantics of the "py2" and "py3" version markers)

It's worth noting that there are two somewhat independent cases:

Binary built distributions (containing C extensions, typically). This
is architecture/ABI dependent, and would generally be tagged as
cpXX-abi-arch.
Pure Python built distributions. This is architecture/ABI independent,
and would be tagged as pyXX-none-any or cpXX-none-any (or maybe
cpX-none-any or pyX-none-any).

(I'm ignoring other implementations from lack of knowledge, but I
suspect a similar distinction will be relevant).

Implementations will therefore *only* match built distributions which either:

1. Exactly match implversion-abi-arch (binary built distributions).
There's a slight complication for implementations that support
multiple ABIs, e.g. the stable ABI, but it's minor.
2. Match implversion in a "fuzzy" manner if abi-arch is none-any (pure
python built distributions). The "fuzzy" matching is clearly defined,
as an example for CPython 3.4, try (in this order of preference) cp34
cp3 cp33 cp32 cp31 cp30 py34 py3 py33 py32 py31 py30. [I wonder -
should py34 be preferred over cp32? That's not what the wheel
implementation does]

On this basis, implementations should *not* declare none-any
combinations, as they can be automatically deduced.

One minor question on this, though, is the statement in the PEP "A
user could instruct their installer to fall back to building from an
sdist more or less often by configuring this list of tags". I don't
see what this means - it should probably be either clarified or
omitted.

This isn't particularly a criticism of the PEP, it's just that the
wording tends to obfuscate the essentials by hinting at complexities
that don't really exist in practice. For example, given the above, the
only really meaningful compressed tagset I can imagine is
py2.py3-none-any. Apart from this one use case, which admittedly is
important, the whole compressed tagset capability is unlikely to ever
be needed.

Paul.


From dholth at gmail.com  Tue Sep 25 23:08:59 2012
From: dholth at gmail.com (Daniel Holth)
Date: Tue, 25 Sep 2012 17:08:59 -0400
Subject: [Python-ideas] bdist naming scheme (compatibility tags) PEP
In-Reply-To: <CACac1F9yD1FrSOj_dZUfx0iAzZQvixi1auL57=8jSvqHg6qDDA@mail.gmail.com>
References: <CAG8k2+7BYcg5RRnUhjY6h7jKsZUCeYCMp-mUohPSSsh5kmaQFw@mail.gmail.com>
	<50236FD5.2060404@egenix.com>
	<CAG8k2+5Eygfk5MFx35OG98RhxcrVpDXC+0Ph9S+ecG4ahG5WWw@mail.gmail.com>
	<CAG8k2+59BaH_a4FcDy-H731b0g8Gk-9urWr4VvJ_bkQ3AZ-RMA@mail.gmail.com>
	<CADiSq7eU46twFdBV6+7NFbek9ddhX7RnJUvUcvZOgujthqL+vA@mail.gmail.com>
	<CACac1F9yD1FrSOj_dZUfx0iAzZQvixi1auL57=8jSvqHg6qDDA@mail.gmail.com>
Message-ID: <CAG8k2+7n3HeSZcbuWQKpG5MxbqMmadsU14WvN5WVqmROrjCtHg@mail.gmail.com>

> It's worth noting that there are two somewhat independent cases:
>
> Binary built distributions (containing C extensions, typically). This
> is architecture/ABI dependent, and would generally be tagged as
> cpXX-abi-arch.
> Pure Python built distributions. This is architecture/ABI independent,
> and would be tagged as pyXX-none-any or cpXX-none-any (or maybe
> cpX-none-any or pyX-none-any).
>
> (I'm ignoring other implementations from lack of knowledge, but I
> suspect a similar distinction will be relevant).
>
> Implementations will therefore *only* match built distributions which either:
>
> 1. Exactly match implversion-abi-arch (binary built distributions).
> There's a slight complication for implementations that support
> multiple ABIs, e.g. the stable ABI, but it's minor.
> 2. Match implversion in a "fuzzy" manner if abi-arch is none-any (pure
> python built distributions). The "fuzzy" matching is clearly defined,
> as an example for CPython 3.4, try (in this order of preference) cp34
> cp3 cp33 cp32 cp31 cp30 py34 py3 py33 py32 py31 py30. [I wonder -
> should py34 be preferred over cp32? That's not what the wheel
> implementation does]

I don't think the exact order of the less-preferred options is
critical, as long as you can make up your mind about whether you
prefer packages with or without the C extension. Your Python is not
likely to be compatible with competing py34 and cp32 wheels for the
same version of a distribution. Most distributions will use either the
cpXX style or the pyXX style tags, but not both.

> On this basis, implementations should *not* declare none-any
> combinations, as they can be automatically deduced.

+0. algorithmically at least. It would not be wrong to dial down the
"previous versions" logic quite a bit, too, as far as only doing cp33,
cp3, py33, py3 which would mean "only use packages that are for our
Python or explicitly cross-version".

> One minor question on this, though, is the statement in the PEP "A
> user could instruct their installer to fall back to building from an
> sdist more or less often by configuring this list of tags". I don't
> see what this means - it should probably be either clarified or
> omitted.

In the above "fewer old tags by default" case, if you are on Python
3.3 and don't install cp32 by default, you could say "also install
cp32 for this one package that I know works" by adding the cp32 tag to
the list. This is to be compatible with lazy human packagers.
Similarly when you are a version behind you will sometimes need to
install packages built for the next version of Python.

Or you could remove all binary package tags of the form
*-abi3-linux_x86_64 from the list that your installer uses to consider
whether to download a built package or an sdist from pypi. It would
still download built pure-Python packages.

> This isn't particularly a criticism of the PEP, it's just that the
> wording tends to obfuscate the essentials by hinting at complexities
> that don't really exist in practice. For example, given the above, the
> only really meaningful compressed tagset I can imagine is
> py2.py3-none-any. Apart from this one use case, which admittedly is
> important, the whole compressed tagset capability is unlikely to ever
> be needed.

Who knows. I imagined bundling a windows and a Linux dll in a single
built package, or doing something with OS X fat binaries. If the
shared library has the same __pycache__/name.tag.so style naming then
this feature gets more interesting, if not difficult to package.

It doesn't currently exist in practice.


From p.f.moore at gmail.com  Tue Sep 25 23:41:01 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 25 Sep 2012 22:41:01 +0100
Subject: [Python-ideas] bdist naming scheme (compatibility tags) PEP
In-Reply-To: <CAG8k2+7n3HeSZcbuWQKpG5MxbqMmadsU14WvN5WVqmROrjCtHg@mail.gmail.com>
References: <CAG8k2+7BYcg5RRnUhjY6h7jKsZUCeYCMp-mUohPSSsh5kmaQFw@mail.gmail.com>
	<50236FD5.2060404@egenix.com>
	<CAG8k2+5Eygfk5MFx35OG98RhxcrVpDXC+0Ph9S+ecG4ahG5WWw@mail.gmail.com>
	<CAG8k2+59BaH_a4FcDy-H731b0g8Gk-9urWr4VvJ_bkQ3AZ-RMA@mail.gmail.com>
	<CADiSq7eU46twFdBV6+7NFbek9ddhX7RnJUvUcvZOgujthqL+vA@mail.gmail.com>
	<CACac1F9yD1FrSOj_dZUfx0iAzZQvixi1auL57=8jSvqHg6qDDA@mail.gmail.com>
	<CAG8k2+7n3HeSZcbuWQKpG5MxbqMmadsU14WvN5WVqmROrjCtHg@mail.gmail.com>
Message-ID: <CACac1F-YRHmUeCJsAftmONUTNKvqqpwUB5bNudszvtm3tnavRw@mail.gmail.com>

On 25 September 2012 22:08, Daniel Holth <dholth at gmail.com> wrote:
> I don't think the exact order of the less-preferred options is
> critical, as long as you can make up your mind about whether you
> prefer packages with or without the C extension. Your Python is not
> likely to be compatible with competing py34 and cp32 wheels for the
> same version of a distribution. Most distributions will use either the
> cpXX style or the pyXX style tags, but not both.



>
>> On this basis, implementations should *not* declare none-any
>> combinations, as they can be automatically deduced.
>
> +0. algorithmically at least. It would not be wrong to dial down the
> "previous versions" logic quite a bit, too, as far as only doing cp33,
> cp3, py33, py3 which would mean "only use packages that are for our
> Python or explicitly cross-version".
>
>> One minor question on this, though, is the statement in the PEP "A
>> user could instruct their installer to fall back to building from an
>> sdist more or less often by configuring this list of tags". I don't
>> see what this means - it should probably be either clarified or
>> omitted.
>
> In the above "fewer old tags by default" case, if you are on Python
> 3.3 and don't install cp32 by default, you could say "also install
> cp32 for this one package that I know works" by adding the cp32 tag to
> the list. This is to be compatible with lazy human packagers.
> Similarly when you are a version behind you will sometimes need to
> install packages built for the next version of Python.
>
> Or you could remove all binary package tags of the form
> *-abi3-linux_x86_64 from the list that your installer uses to consider
> whether to download a built package or an sdist from pypi. It would
> still download built pure-Python packages.
>
>> This isn't particularly a criticism of the PEP, it's just that the
>> wording tends to obfuscate the essentials by hinting at complexities
>> that don't really exist in practice. For example, given the above, the
>> only really meaningful compressed tagset I can imagine is
>> py2.py3-none-any. Apart from this one use case, which admittedly is
>> important, the whole compressed tagset capability is unlikely to ever
>> be needed.
>
> Who knows. I imagined bundling a windows and a Linux dll in a single
> built package, or doing something with OS X fat binaries. If the
> shared library has the same __pycache__/name.tag.so style naming then
> this feature gets more interesting, if not difficult to package.
>
> It doesn't currently exist in practice.


From p.f.moore at gmail.com  Tue Sep 25 23:47:22 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 25 Sep 2012 22:47:22 +0100
Subject: [Python-ideas] bdist naming scheme (compatibility tags) PEP
In-Reply-To: <CAG8k2+7n3HeSZcbuWQKpG5MxbqMmadsU14WvN5WVqmROrjCtHg@mail.gmail.com>
References: <CAG8k2+7BYcg5RRnUhjY6h7jKsZUCeYCMp-mUohPSSsh5kmaQFw@mail.gmail.com>
	<50236FD5.2060404@egenix.com>
	<CAG8k2+5Eygfk5MFx35OG98RhxcrVpDXC+0Ph9S+ecG4ahG5WWw@mail.gmail.com>
	<CAG8k2+59BaH_a4FcDy-H731b0g8Gk-9urWr4VvJ_bkQ3AZ-RMA@mail.gmail.com>
	<CADiSq7eU46twFdBV6+7NFbek9ddhX7RnJUvUcvZOgujthqL+vA@mail.gmail.com>
	<CACac1F9yD1FrSOj_dZUfx0iAzZQvixi1auL57=8jSvqHg6qDDA@mail.gmail.com>
	<CAG8k2+7n3HeSZcbuWQKpG5MxbqMmadsU14WvN5WVqmROrjCtHg@mail.gmail.com>
Message-ID: <CACac1F9QhHdbr00c+vWonWnJVWXf6DOWePur+=UJk_q1dq1wYw@mail.gmail.com>

On 25 September 2012 22:08, Daniel Holth <dholth at gmail.com> wrote:
> I don't think the exact order of the less-preferred options is
> critical, as long as you can make up your mind about whether you
> prefer packages with or without the C extension. Your Python is not
> likely to be compatible with competing py34 and cp32 wheels for the
> same version of a distribution. Most distributions will use either the
> cpXX style or the pyXX style tags, but not both.

I think that this is fine, but the PEP needs to be explicit. If it's a
user option, the PEP should say "installers should allow the user to
specify the list of compatibility tags, and the default should be
XXX". If it's static, the PEP should say what it is.

Having different installers make different, incompatible assumptions,
is unpleasant. At present, of course, the only 2 real contenders are
the reference wheel implementation and pip. Others like
distutils2/packaging may follow.

[...]

>> This isn't particularly a criticism of the PEP, it's just that the
>> wording tends to obfuscate the essentials by hinting at complexities
>> that don't really exist in practice. For example, given the above, the
>> only really meaningful compressed tagset I can imagine is
>> py2.py3-none-any. Apart from this one use case, which admittedly is
>> important, the whole compressed tagset capability is unlikely to ever
>> be needed.
>
> Who knows. I imagined bundling a windows and a Linux dll in a single
> built package, or doing something with OS X fat binaries. If the
> shared library has the same __pycache__/name.tag.so style naming then
> this feature gets more interesting, if not difficult to package.
>
> It doesn't currently exist in practice.

The PEP should stick to defining behaviour for things that do exist.
Let those who build clever new options like that work out how to
integrate with this PEP. (On which note, is the "stable ABI" real yet?
On Windows, at least, it talks about a python3.dll, and yet there is
no such thing distributed with Python 3.3, so based on that (what's
the situation on Linux?) I'd be inclined to say that as of this point,
even the stable ABI can be ignored.

Paul.


From chrysn at fsfe.org  Wed Sep 26 10:17:18 2012
From: chrysn at fsfe.org (chrysn)
Date: Wed, 26 Sep 2012 10:17:18 +0200
Subject: [Python-ideas] asyncore: included batteries don't fit
In-Reply-To: <CADcB0yhAys8hn7B1O9-nnEMR-GDNt3b13F26neBEx+6D-3ho+Q@mail.gmail.com>
	<CAFYqXL-rGN_2ZVpLbw575sCem8Ft9WAB1RNpZKexAhoMHKWL5g@mail.gmail.com>
Message-ID: <20120926081718.GA20843@hephaistos.amsuess.com>

On Mon, Sep 24, 2012 at 03:31:37PM -0700, Giampaolo Rodol? wrote:
> From a chronological standpoint I still think the best thing to do in order
> to fix the "python async problem" once and for all is to first define and
> possibly implement an "async WSGI interface" describing what a standard
> async IO loop/reactor should look like (in terms of API) and how to
> integrate with it, see:
> http://mail.python.org/pipermail/python-ideas/2012-May/015223.html
> http://mail.python.org/pipermail/python-ideas/2012-May/015235.html

i wasn't aware that pep 3153 exists. given that, my original intention
of this thread should be re-worded into "let's get pep3153 along!".

i'm not convinced by the api suggested in the first mail, as it sounds
very unix centric (poll, read/write/error). i rather imagined leaving
the details of the callbackable/mainloop interaction to be platform
details. (a win32evtlog event source just couldn't possibly register
with a select() based main loop). i'd prefer to keep the part that
registers with the a main loop concentrated to a very lowlevel common
denominator. for unix, that'd mean that there is a basic callbackable
for "things that receive events because they have a fileno". everything
above that, eg the distinction whether a "w" event means that we can
write() or that we must accept() could happen above that and wouldn't
have to be concerned with the main loop integration any more.

in case (pseudo)code gets the idea over better:

class UnixFilehandle(object):
    def __init__(self, fileno):
        self._fileno = fileno

    def register_with_main_loop(self, mainloop):
        # it might happen that the main loop doesn't support unix
	# filenos. tough luck, in that case -- the developer should
	# select a more suitable main loop.
        mainloop.register_unix_fileno(self._fileno, self)

    def handle_r_event(self): raise NotImplementedError("Not configured to receive that sort of event")
    # if you're sure you'd never receive any anyway, you can
    # not-register them by setting them None in the subclass
    handle_w_event = handle_e_event = handle_r_event

class SocketServer(UnixFilehandle):
    def __init__(self, socket):
        self._socket = socket
	UnixFilehandle.init(socket.fileno())

    def handle_w_event(self):
        self.handle_accept_event(self.socket.accept())

other interfaces parallel to the file handle interface would, for
example, handle unix signals. (built atop of that, like the
accept-handling socket server, could be an that deals with child
processes.) the interface for android might look different again,
because there is no main loop and select never gets called by the
application.

> From there the python stdlib *might* grow a new module implementing the
> "async WSGI interface" (let's call it asyncore2) and some of the stdlib
> batteries such as socketserver can possibly use it.
> 
> In my mind this is the ideal long-term scenario but even managing to define
> an "async WSGI interface" alone would be a big step forward.

i'd welcome such an interface. if asyncore can then be retrofitted to
accept that interface too w/o breaking compatibility, it'd be nice, but
if not, it's asyncore2, then.

> Again, at this point in time what you're proposing looks too vague,
> ambitious and premature to me.

please don't get me wrong -- i'm not proposing anything for immediate
action, i just want to start a thinking process towards a better
integrated stdlib.


On Mon, Sep 24, 2012 at 05:02:08PM -0700, Josiah Carlson wrote:
> 1. Whatever reactors are available, you need to be able to instantiate
> multiple of different types of reactors and multiple instances of the
> same type of reactor simultaneously (to support multiple threads
> handling different groups of reactors, or different reactors for
> different types of objects on certain platforms). While this allows
> for insanity in the worst-case, we're all consenting adults here, so
> shouldn't be limited by reactor singletons. There should be a default
> reactor class, which is defined on module/package import (use the
> "best" one for the platform).

i think that's already common. with asyncore, you can have different
maps (just one is installed globally as default). with the gtk main
loop, it's a little tricky (the gtk.main() function doesn't simply take
an argument), but the underlying glib can do that afaict.

> 2. The API must be simple. I am not sure that it can get easier than
> Idea #3 from:
> http://mail.python.org/pipermail/python-ideas/2012-May/015245.html

it's good that the necessities of call_later and call_every are
mentioned here, i'd have forgotten about them.


we've talked about many things we'd need in a python asynchronous
interface (not implementation), so what are the things we *don't* need?
(so we won't start building a framework like twisted). i'll start:

* high-level protocol handling (can be extra modules atop of it)
* ssl
* something like the twisted delayed framework (not sure about that, i
  guess the twisted people will have good reason to use it, but i don't
  see compelling reasons for such a thing in a minimal interface from my
  limited pov)
* explicit connection handling (retries, timeouts -- would be up to the
  user as well, eg urllib might want to set up a timeout and retries for
  asynchronous url requests)

best regards
chrysn

-- 
To use raw power is to make yourself infinitely vulnerable to greater powers.
  -- Bene Gesserit axiom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120926/44c24a49/attachment.pgp>

From dholth at gmail.com  Wed Sep 26 14:31:26 2012
From: dholth at gmail.com (Daniel Holth)
Date: Wed, 26 Sep 2012 08:31:26 -0400
Subject: [Python-ideas] bdist naming scheme (compatibility tags) PEP
In-Reply-To: <CACac1F9QhHdbr00c+vWonWnJVWXf6DOWePur+=UJk_q1dq1wYw@mail.gmail.com>
References: <CAG8k2+7BYcg5RRnUhjY6h7jKsZUCeYCMp-mUohPSSsh5kmaQFw@mail.gmail.com>
	<50236FD5.2060404@egenix.com>
	<CAG8k2+5Eygfk5MFx35OG98RhxcrVpDXC+0Ph9S+ecG4ahG5WWw@mail.gmail.com>
	<CAG8k2+59BaH_a4FcDy-H731b0g8Gk-9urWr4VvJ_bkQ3AZ-RMA@mail.gmail.com>
	<CADiSq7eU46twFdBV6+7NFbek9ddhX7RnJUvUcvZOgujthqL+vA@mail.gmail.com>
	<CACac1F9yD1FrSOj_dZUfx0iAzZQvixi1auL57=8jSvqHg6qDDA@mail.gmail.com>
	<CAG8k2+7n3HeSZcbuWQKpG5MxbqMmadsU14WvN5WVqmROrjCtHg@mail.gmail.com>
	<CACac1F9QhHdbr00c+vWonWnJVWXf6DOWePur+=UJk_q1dq1wYw@mail.gmail.com>
Message-ID: <CAG8k2+7f4SAoCuCxd3rC7nD8vG=5cnsRty8SrFNwm__eovQrGQ@mail.gmail.com>

> I think that this is fine, but the PEP needs to be explicit. If it's a
> user option, the PEP should say "installers should allow the user to
> specify the list of compatibility tags, and the default should be
> XXX". If it's static, the PEP should say what it is.
>
> Having different installers make different, incompatible assumptions,
> is unpleasant. At present, of course, the only 2 real contenders are
> the reference wheel implementation and pip. Others like
> distutils2/packaging may follow.

It might be easier to explain by defining a static list for each
version of Python and then say "and you can add previous versions to
the ordered set". Then for CPython 3.3, ignoring abi3, with pymalloc
giving the cp33m suffix, you could have only

cp33-cp33m-win32
cp33-none-win32
cp33-none-any
py33-none-any
py3-none-any

implementation - preferred abi - plat
implementation - none - plat
implementation - none - any
python major minor - none - any
python major - none - any

The rule for generating the last version's tags ignoring abi3 is that
you only keep the none-any tags:

cp32-none-any
py32-none-any
py3-none-any

appending the lists without duplicates you get

cp33-cp33m-win32
cp33-none-win32
cp33-none-any
py33-none-any
py3-none-any
cp32-none-any
py32-none-any

I'm not sure what to do with abi3 or whether to use the cp3 (major
only) implementation tag.


From p.f.moore at gmail.com  Wed Sep 26 16:23:02 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 26 Sep 2012 15:23:02 +0100
Subject: [Python-ideas] bdist naming scheme (compatibility tags) PEP
In-Reply-To: <CAG8k2+7f4SAoCuCxd3rC7nD8vG=5cnsRty8SrFNwm__eovQrGQ@mail.gmail.com>
References: <CAG8k2+7BYcg5RRnUhjY6h7jKsZUCeYCMp-mUohPSSsh5kmaQFw@mail.gmail.com>
	<50236FD5.2060404@egenix.com>
	<CAG8k2+5Eygfk5MFx35OG98RhxcrVpDXC+0Ph9S+ecG4ahG5WWw@mail.gmail.com>
	<CAG8k2+59BaH_a4FcDy-H731b0g8Gk-9urWr4VvJ_bkQ3AZ-RMA@mail.gmail.com>
	<CADiSq7eU46twFdBV6+7NFbek9ddhX7RnJUvUcvZOgujthqL+vA@mail.gmail.com>
	<CACac1F9yD1FrSOj_dZUfx0iAzZQvixi1auL57=8jSvqHg6qDDA@mail.gmail.com>
	<CAG8k2+7n3HeSZcbuWQKpG5MxbqMmadsU14WvN5WVqmROrjCtHg@mail.gmail.com>
	<CACac1F9QhHdbr00c+vWonWnJVWXf6DOWePur+=UJk_q1dq1wYw@mail.gmail.com>
	<CAG8k2+7f4SAoCuCxd3rC7nD8vG=5cnsRty8SrFNwm__eovQrGQ@mail.gmail.com>
Message-ID: <CACac1F9dU=yBas_urbP4exr2Gmj1uSXAr-_GEVXDvp_XRjY3mA@mail.gmail.com>

On 26 September 2012 13:31, Daniel Holth <dholth at gmail.com> wrote:
>> I think that this is fine, but the PEP needs to be explicit. If it's a
>> user option, the PEP should say "installers should allow the user to
>> specify the list of compatibility tags, and the default should be
>> XXX". If it's static, the PEP should say what it is.
>>
>> Having different installers make different, incompatible assumptions,
>> is unpleasant. At present, of course, the only 2 real contenders are
>> the reference wheel implementation and pip. Others like
>> distutils2/packaging may follow.
>
> It might be easier to explain by defining a static list for each
> version of Python and then say "and you can add previous versions to
> the ordered set". Then for CPython 3.3, ignoring abi3, with pymalloc
> giving the cp33m suffix, you could have only
>
> cp33-cp33m-win32
> cp33-none-win32
> cp33-none-any
> py33-none-any
> py3-none-any
>
> implementation - preferred abi - plat
> implementation - none - plat
> implementation - none - any
> python major minor - none - any
> python major - none - any
>
> The rule for generating the last version's tags ignoring abi3 is that
> you only keep the none-any tags:
>
> cp32-none-any
> py32-none-any
> py3-none-any
>
> appending the lists without duplicates you get
>
> cp33-cp33m-win32
> cp33-none-win32
> cp33-none-any
> py33-none-any
> py3-none-any
> cp32-none-any
> py32-none-any
>
> I'm not sure what to do with abi3 or whether to use the cp3 (major
> only) implementation tag.

Win32 is not a good example here. As far as I know (I've experimented
and read docs, but haven't analyzed the code), there is never a
declared ABI on Win32. In fact, Windows is pretty much trivially
simple:

cpXY-none-win32 (for distributions with C extensions)
pyXY-none-any (for pure-Python distributions)

In fact, those two are the only values the bdist_wheel format can
generate. Actually, for non-Windows, it's just as simple - bdist_wheel
can only generate

cpXY-ABI-PLAT (for distributions with C extensions)
pyXY-none-any (for pure-Python distributions)

ABI is the preferred ABI (the part of SOABI after the '-' from the
Python used to build) and PLAT is the platform. (So essentially,
Windows follows the standard pattern, but with an ABI of "none").

Eggs and wininst installers, if they used this convention, would be
the same. As would bdist_msi, as far as I know. So the question is,
what use case is there for anything more complicated than this? The
only possibilities I can see are:

1. The stable ABI. At the moment, I don't know how well that's
supported - I don't think the build tools detect whether code only
uses the stable ABI, so they assume the full ABI. Users could claim to
use the stable ABI by manual renaming. But without an agreed and
documented convention for the stable ABI, they can't do that, so I
think it's premature to worry about that case. It's easy enough to add
if needed (see below - it's just another ABI for installers to allow)

2. UCS2 vs UCS4. This is dead for Python 3.3+, so not worth
complicating the model for.

3. In theory, if a tool could create "fat" archives containing code
for multiple platforms/ABIs, then that might need something more
complex. But in the absence of any such tool, I'd call YAGNI on this.

4. Pure-python code which works on multiple versions of Python. This
is a real case, and needs to be considered. Code that is (presumed)
valid on all Python versions within the current major version can be
manually retagged as pyX. And code that supports Python 2 and 3 can be
retagged as py2.py3. To allow forward compatibility, installers should
allow the user to install pyXZ code on Python version X.Y when Z<Y.
But this should be a user option (possibly off by default) and an
exact match should always be preferred.

I'm not aware of any other cases that might matter here. The other
implementations may well add further use cases - for example, PyPy can
load (some) cPython code, I believe. But without details, let's wait
to hear from them rather than speculating.

The cases above where I suggest manual retagging may benefit from a UI
in the build tools to automatically change the tags, but that's a
quality of implementation issue. At a pinch, renaming a wheel file
would work fine (as long as you didn't lie by doing so!)

That covers the side of the proposal relating to how binary
distributions declare what they were built for.

As regards how installers should check whether packages are
compatible, it seems to me that the rules can be reasonably simple.

1. The installer maintains a spec of what tagsets the current Python
will support - that would be the exact implementation/version ("cpXY"
or similar), a list of supported ABIs in preference order, and the
current platform. The PEP should document how to get the list of
supported ABIs, for completeness.
2. An exact match wins every time. Where there are multiple ABIs, the
best match is based on the preference order supplied.
3. Non-exact matches can only occur for pure-Python packages (as
platform-specific ones declare an exact version/abi/platform as noted
above). Here, we ignore ABI and platform (they will always be
none-any) and work down the list pyXY, pyX, pyXZ (Z<Y, this only if
user allows it) in that order. Where a package declares multi-tags
(py2.py3 is the only likely case) break ties by taking the package
that specifies the fewest tags.

That should be it.

For me, the above summary (or something similar) needs to be in the
PEP, to provide a proper background if nothing else. What do people
think?

Paul.


From p.f.moore at gmail.com  Wed Sep 26 16:45:22 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 26 Sep 2012 15:45:22 +0100
Subject: [Python-ideas] bdist naming scheme (compatibility tags) PEP
In-Reply-To: <CADiSq7eU46twFdBV6+7NFbek9ddhX7RnJUvUcvZOgujthqL+vA@mail.gmail.com>
References: <CAG8k2+7BYcg5RRnUhjY6h7jKsZUCeYCMp-mUohPSSsh5kmaQFw@mail.gmail.com>
	<50236FD5.2060404@egenix.com>
	<CAG8k2+5Eygfk5MFx35OG98RhxcrVpDXC+0Ph9S+ecG4ahG5WWw@mail.gmail.com>
	<CAG8k2+59BaH_a4FcDy-H731b0g8Gk-9urWr4VvJ_bkQ3AZ-RMA@mail.gmail.com>
	<CADiSq7eU46twFdBV6+7NFbek9ddhX7RnJUvUcvZOgujthqL+vA@mail.gmail.com>
Message-ID: <CACac1F8oe73+95jND8gVP=Sw9zpSmfo8L=sfWMt=dQB_Zccyqg@mail.gmail.com>

On 9 September 2012 13:16, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Yep, those rules look sensible to me (and thanks for clarifying the
> intended semantics of the "py2" and "py3" version markers)

One (relatively minor) point: the Python tag isn't easily parseable.
To split the implementation and version bits, you can do tag[:2],
tag[2;} except for "Other Python implementations should use
sys.implementation.name". Or you could use tag[:-2], tag[-2:] except
for "py2". So you need to use a regex match to split off the trailing
digits, which is a bit excessive.

My current approach is the [:2], [2:] one, calling YAGNI on
implementations not covered by the 2-letter codes...

Paul.


From dholth at gmail.com  Wed Sep 26 16:53:03 2012
From: dholth at gmail.com (Daniel Holth)
Date: Wed, 26 Sep 2012 10:53:03 -0400
Subject: [Python-ideas] bdist naming scheme (compatibility tags) PEP
In-Reply-To: <CACac1F9dU=yBas_urbP4exr2Gmj1uSXAr-_GEVXDvp_XRjY3mA@mail.gmail.com>
References: <CAG8k2+7BYcg5RRnUhjY6h7jKsZUCeYCMp-mUohPSSsh5kmaQFw@mail.gmail.com>
	<50236FD5.2060404@egenix.com>
	<CAG8k2+5Eygfk5MFx35OG98RhxcrVpDXC+0Ph9S+ecG4ahG5WWw@mail.gmail.com>
	<CAG8k2+59BaH_a4FcDy-H731b0g8Gk-9urWr4VvJ_bkQ3AZ-RMA@mail.gmail.com>
	<CADiSq7eU46twFdBV6+7NFbek9ddhX7RnJUvUcvZOgujthqL+vA@mail.gmail.com>
	<CACac1F9yD1FrSOj_dZUfx0iAzZQvixi1auL57=8jSvqHg6qDDA@mail.gmail.com>
	<CAG8k2+7n3HeSZcbuWQKpG5MxbqMmadsU14WvN5WVqmROrjCtHg@mail.gmail.com>
	<CACac1F9QhHdbr00c+vWonWnJVWXf6DOWePur+=UJk_q1dq1wYw@mail.gmail.com>
	<CAG8k2+7f4SAoCuCxd3rC7nD8vG=5cnsRty8SrFNwm__eovQrGQ@mail.gmail.com>
	<CACac1F9dU=yBas_urbP4exr2Gmj1uSXAr-_GEVXDvp_XRjY3mA@mail.gmail.com>
Message-ID: <CAG8k2+6jmsRRAERC6bq6RWK8ahno2z9tMb_vZ99at+LaG03W1w@mail.gmail.com>

> Win32 is not a good example here. As far as I know (I've experimented
> and read docs, but haven't analyzed the code), there is never a
> declared ABI on Win32. In fact, Windows is pretty much trivially
> simple:
>
> cpXY-none-win32 (for distributions with C extensions)
> pyXY-none-any (for pure-Python distributions)
>
> In fact, those two are the only values the bdist_wheel format can
> generate. Actually, for non-Windows, it's just as simple - bdist_wheel
> can only generate

bdist_wheel is incomplete too. It should read from setup.cfg for
advice on the tags.

Does win32 have debug / pymalloc builds? That is why there is a cp33dm ABI.

On linux imp.get_suffixes() includes

('.cpython-33m.so', 'rb', 3), ('.abi3.so', 'rb', 3)

and the abi tag is just an abbreviation cp33m or abi3.

> cpXY-ABI-PLAT (for distributions with C extensions)
> pyXY-none-any (for pure-Python distributions)

These are the most important, and the ones bdist_wheel can (should)
generate without configuration.

> Eggs and wininst installers, if they used this convention, would be
> the same. As would bdist_msi, as far as I know. So the question is,
> what use case is there for anything more complicated than this? The
> only possibilities I can see are:
>
> 1. The stable ABI. At the moment, I don't know how well that's
> supported - I don't think the build tools detect whether code only
> uses the stable ABI, so they assume the full ABI. Users could claim to
> use the stable ABI by manual renaming. But without an agreed and
> documented convention for the stable ABI, they can't do that, so I
> think it's premature to worry about that case. It's easy enough to add
> if needed (see below - it's just another ABI for installers to allow)
>
> 2. UCS2 vs UCS4. This is dead for Python 3.3+, so not worth
> complicating the model for.

Python 2 continues to matter. I do not and can not use Python 3 commercially.

> 4. Pure-python code which works on multiple versions of Python. This
> is a real case, and needs to be considered. Code that is (presumed)
> valid on all Python versions within the current major version can be
> manually retagged as pyX. And code that supports Python 2 and 3 can be
> retagged as py2.py3. To allow forward compatibility, installers should
> allow the user to install pyXZ code on Python version X.Y when Z<Y.
> But this should be a user option (possibly off by default) and an
> exact match should always be preferred.
>
> I'm not aware of any other cases that might matter here. The other
> implementations may well add further use cases - for example, PyPy can
> load (some) cPython code, I believe. But without details, let's wait
> to hear from them rather than speculating.

PyPy has source compatibility for some CPython extensions, so it
counts as a different ABI.

Sometimes code uses ctypes or cffi instead of the CPython ABI (or even
includes an .exe that it calls with subprocess), there was some
discussion about using the 'none' abi in that case .

> 2. An exact match wins every time. Where there are multiple ABIs, the
> best match is based on the preference order supplied.

Just let an exact match be the only kind of match. Then there is no parsing.

The implementation tag is there because packages may have different
requirements based on the implementation and version based on if:
statements in setup.py. Maybe you use cp3 or py3 when you have added
conditional requirements a-la Requries-Dist: argparse; python_version
< 2.6 in PKG-INFO?


From p.f.moore at gmail.com  Wed Sep 26 17:16:40 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 26 Sep 2012 16:16:40 +0100
Subject: [Python-ideas] bdist naming scheme (compatibility tags) PEP
In-Reply-To: <CAG8k2+6jmsRRAERC6bq6RWK8ahno2z9tMb_vZ99at+LaG03W1w@mail.gmail.com>
References: <CAG8k2+7BYcg5RRnUhjY6h7jKsZUCeYCMp-mUohPSSsh5kmaQFw@mail.gmail.com>
	<50236FD5.2060404@egenix.com>
	<CAG8k2+5Eygfk5MFx35OG98RhxcrVpDXC+0Ph9S+ecG4ahG5WWw@mail.gmail.com>
	<CAG8k2+59BaH_a4FcDy-H731b0g8Gk-9urWr4VvJ_bkQ3AZ-RMA@mail.gmail.com>
	<CADiSq7eU46twFdBV6+7NFbek9ddhX7RnJUvUcvZOgujthqL+vA@mail.gmail.com>
	<CACac1F9yD1FrSOj_dZUfx0iAzZQvixi1auL57=8jSvqHg6qDDA@mail.gmail.com>
	<CAG8k2+7n3HeSZcbuWQKpG5MxbqMmadsU14WvN5WVqmROrjCtHg@mail.gmail.com>
	<CACac1F9QhHdbr00c+vWonWnJVWXf6DOWePur+=UJk_q1dq1wYw@mail.gmail.com>
	<CAG8k2+7f4SAoCuCxd3rC7nD8vG=5cnsRty8SrFNwm__eovQrGQ@mail.gmail.com>
	<CACac1F9dU=yBas_urbP4exr2Gmj1uSXAr-_GEVXDvp_XRjY3mA@mail.gmail.com>
	<CAG8k2+6jmsRRAERC6bq6RWK8ahno2z9tMb_vZ99at+LaG03W1w@mail.gmail.com>
Message-ID: <CACac1F-JqTOHCWUshxd6-SXkfVVaSCm2vGc75OMFCvyW=9YQnQ@mail.gmail.com>

On 26 September 2012 15:53, Daniel Holth <dholth at gmail.com> wrote:
> bdist_wheel is incomplete too. It should read from setup.cfg for
> advice on the tags.

I wasn't trying to imply that bdist_wheel was the reference, just that
it was the best example "in the wild" that exists at the moment. Using
setup.cfg to allow user configuration of the tags sounds reasonable.

> Does win32 have debug / pymalloc builds? That is why there is a cp33dm ABI.

debug yes. That's represented in the DLL names (a _d suffix). I'm not
sure about pymalloc. I don't know where the string "cp33dm" comes
from, this is why I think the valid values should be documented in the
PEP.

> On linux imp.get_suffixes() includes
>
> ('.cpython-33m.so', 'rb', 3), ('.abi3.so', 'rb', 3)
>
> and the abi tag is just an abbreviation cp33m or abi3.

On Windows, imp.get_suffixes shows:

>>> imp.get_suffixes()
[('.pyd', 'rb', 3), ('.py', 'U', 1), ('.pyw', 'U', 1), ('.pyc', 'rb', 2)]

I don't have a debug build to hand to check that, but Google tells me
that Martin von Loewis said:

if sys.executable.endswith("_d.exe"):
print "Debug version"

If relying on the executable name is too unsafe, you can also look
at imp.get_suffixes(), which includes "_d.pyd" in a debug build on
Windows.

>> 2. UCS2 vs UCS4. This is dead for Python 3.3+, so not worth
>> complicating the model for.
>
> Python 2 continues to matter. I do not and can not use Python 3 commercially.

I don't dispute this, but I'm not sure how the PEP should reflect
this. Regardless, if distinguishing UCS2 vs UCS4 matters, the PEP
should clarify how to do so.

>> 2. An exact match wins every time. Where there are multiple ABIs, the
>> best match is based on the preference order supplied.
>
> Just let an exact match be the only kind of match. Then there is no parsing.

I can see that argument, but to me it makes documenting (and
understanding!) what an implementation/installer is saying when it
lists the tags it will accept quite difficult. Maybe I'm just being
dense :-)

> The implementation tag is there because packages may have different
> requirements based on the implementation and version based on if:
> statements in setup.py. Maybe you use cp3 or py3 when you have added
> conditional requirements a-la Requries-Dist: argparse; python_version
> < 2.6 in PKG-INFO?

I'm sorry, that doesn't make any sense to me.

Paul


From dholth at gmail.com  Wed Sep 26 17:29:18 2012
From: dholth at gmail.com (Daniel Holth)
Date: Wed, 26 Sep 2012 11:29:18 -0400
Subject: [Python-ideas] bdist naming scheme (compatibility tags) PEP
In-Reply-To: <CACac1F-JqTOHCWUshxd6-SXkfVVaSCm2vGc75OMFCvyW=9YQnQ@mail.gmail.com>
References: <CAG8k2+7BYcg5RRnUhjY6h7jKsZUCeYCMp-mUohPSSsh5kmaQFw@mail.gmail.com>
	<50236FD5.2060404@egenix.com>
	<CAG8k2+5Eygfk5MFx35OG98RhxcrVpDXC+0Ph9S+ecG4ahG5WWw@mail.gmail.com>
	<CAG8k2+59BaH_a4FcDy-H731b0g8Gk-9urWr4VvJ_bkQ3AZ-RMA@mail.gmail.com>
	<CADiSq7eU46twFdBV6+7NFbek9ddhX7RnJUvUcvZOgujthqL+vA@mail.gmail.com>
	<CACac1F9yD1FrSOj_dZUfx0iAzZQvixi1auL57=8jSvqHg6qDDA@mail.gmail.com>
	<CAG8k2+7n3HeSZcbuWQKpG5MxbqMmadsU14WvN5WVqmROrjCtHg@mail.gmail.com>
	<CACac1F9QhHdbr00c+vWonWnJVWXf6DOWePur+=UJk_q1dq1wYw@mail.gmail.com>
	<CAG8k2+7f4SAoCuCxd3rC7nD8vG=5cnsRty8SrFNwm__eovQrGQ@mail.gmail.com>
	<CACac1F9dU=yBas_urbP4exr2Gmj1uSXAr-_GEVXDvp_XRjY3mA@mail.gmail.com>
	<CAG8k2+6jmsRRAERC6bq6RWK8ahno2z9tMb_vZ99at+LaG03W1w@mail.gmail.com>
	<CACac1F-JqTOHCWUshxd6-SXkfVVaSCm2vGc75OMFCvyW=9YQnQ@mail.gmail.com>
Message-ID: <CAG8k2+5hux-KXO7e6ppZVcLQo+U0JF56neBeRFLKxxsHfDz6Ow@mail.gmail.com>

On Wed, Sep 26, 2012 at 11:16 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> On 26 September 2012 15:53, Daniel Holth <dholth at gmail.com> wrote:

> I don't dispute this, but I'm not sure how the PEP should reflect
> this. Regardless, if distinguishing UCS2 vs UCS4 matters, the PEP
> should clarify how to do so.

ABIs ending with u use UCS4, and the dmu suffixes always appear in
that order. Should go into the pep.

py27dmu py27du py27mu cp33dm cp33d cp33m

>> Just let an exact match be the only kind of match. Then there is no parsing.
>
> I can see that argument, but to me it makes documenting (and
> understanding!) what an implementation/installer is saying when it
> lists the tags it will accept quite difficult. Maybe I'm just being
> dense :-)

Maybe we just need to attach a reference implementation to the PEP.

>> The implementation tag is there because packages may have different
>> requirements based on the implementation and version based on if:
>> statements in setup.py. Maybe you use cp3 or py3 when you have added
>> conditional requirements a-la Requries-Dist: argparse; python_version
>> < 2.6 in PKG-INFO?
>
> I'm sorry, that doesn't make any sense to me.

When you use the py2 or py3 tags, it would ideally also communicate a
promise "this code does not produce a different list of requirements
based on the build Python".


From josiah.carlson at gmail.com  Wed Sep 26 19:02:24 2012
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Wed, 26 Sep 2012 10:02:24 -0700
Subject: [Python-ideas] asyncore: included batteries don't fit
In-Reply-To: <20120926081718.GA20843@hephaistos.amsuess.com>
References: <CADcB0yhAys8hn7B1O9-nnEMR-GDNt3b13F26neBEx+6D-3ho+Q@mail.gmail.com>
	<CAFYqXL-rGN_2ZVpLbw575sCem8Ft9WAB1RNpZKexAhoMHKWL5g@mail.gmail.com>
	<20120926081718.GA20843@hephaistos.amsuess.com>
Message-ID: <CADcB0yi-7Pri-EeBoJxkHAgypANXa9AyeFuOmGhrQY3xYtFUtQ@mail.gmail.com>

On Wed, Sep 26, 2012 at 1:17 AM, chrysn <chrysn at fsfe.org> wrote:
> On Mon, Sep 24, 2012 at 03:31:37PM -0700, Giampaolo Rodol? wrote:
>> From a chronological standpoint I still think the best thing to do in order
>> to fix the "python async problem" once and for all is to first define and
>> possibly implement an "async WSGI interface" describing what a standard
>> async IO loop/reactor should look like (in terms of API) and how to
>> integrate with it, see:
>> http://mail.python.org/pipermail/python-ideas/2012-May/015223.html
>> http://mail.python.org/pipermail/python-ideas/2012-May/015235.html
>
> i wasn't aware that pep 3153 exists. given that, my original intention
> of this thread should be re-worded into "let's get pep3153 along!".

Go ahead and read PEP 3153, we will wait.

A careful reading of PEP 3153 will tell you that the intent is to make
a "light" version of Twisted built into Python. There isn't any
discussion as to *why* this is a good idea, it just lays out the plan
of action. Its ideas were gathered from the experience of the Twisted
folks.

Their experience is substantial, but in the intervening 1.5+ years
since Pycon 2011, only the barest of abstract interfaces has been
defined (https://github.com/lvh/async-pep/blob/master/async/abstract.py),
and no discussion has taken place as to forward migration of the
(fairly large) body of existing asyncore code.

> i'm not convinced by the api suggested in the first mail, as it sounds
> very unix centric (poll, read/write/error). i rather imagined leaving
> the details of the callbackable/mainloop interaction to be platform
> details. (a win32evtlog event source just couldn't possibly register
> with a select() based main loop). i'd prefer to keep the part that

Of course not, but then again no one would attempt to do as much. They
would use a WSAEvent reactor, because that's the only thing that it
would work with. That said, WSAEvent should arguably be the default on
Windows, so this issue shouldn't even come up there. Also, worrying
about platform-specific details like "what if someone uses a source
that is relatively uncommon on the platform" is a red-herring; get the
interface/api right, build it, and start using it.

To the point, Giampaolo already has a reactor that implements the
interface (more or less "idea #3" from his earlier message), and it's
been used in production (under staggering ftp(s) load). Even better,
it offers effectively transparent replacement of the existing asyncore
loop, and supports existing asyncore-derived classes. It is available:
https://code.google.com/p/pyftpdlib/source/browse/trunk/pyftpdlib/lib/ioloop.py

> registers with the a main loop concentrated to a very lowlevel common
> denominator. for unix, that'd mean that there is a basic callbackable
> for "things that receive events because they have a fileno". everything
> above that, eg the distinction whether a "w" event means that we can
> write() or that we must accept() could happen above that and wouldn't
> have to be concerned with the main loop integration any more.
>
> in case (pseudo)code gets the idea over better:
>
> class UnixFilehandle(object):
>     def __init__(self, fileno):
>         self._fileno = fileno
>
>     def register_with_main_loop(self, mainloop):
>         # it might happen that the main loop doesn't support unix
>         # filenos. tough luck, in that case -- the developer should
>         # select a more suitable main loop.
>         mainloop.register_unix_fileno(self._fileno, self)
>
>     def handle_r_event(self): raise NotImplementedError("Not configured to receive that sort of event")
>     # if you're sure you'd never receive any anyway, you can
>     # not-register them by setting them None in the subclass
>     handle_w_event = handle_e_event = handle_r_event
>
> class SocketServer(UnixFilehandle):
>     def __init__(self, socket):
>         self._socket = socket
>         UnixFilehandle.init(socket.fileno())
>
>     def handle_w_event(self):
>         self.handle_accept_event(self.socket.accept())
>
> other interfaces parallel to the file handle interface would, for
> example, handle unix signals. (built atop of that, like the
> accept-handling socket server, could be an that deals with child
> processes.) the interface for android might look different again,
> because there is no main loop and select never gets called by the
> application.

That is, incidentally, what Giampaolo has implemented already. I
encourage you to read the source I linked above.

>> From there the python stdlib *might* grow a new module implementing the
>> "async WSGI interface" (let's call it asyncore2) and some of the stdlib
>> batteries such as socketserver can possibly use it.
>>
>> In my mind this is the ideal long-term scenario but even managing to define
>> an "async WSGI interface" alone would be a big step forward.
>
> i'd welcome such an interface. if asyncore can then be retrofitted to
> accept that interface too w/o breaking compatibility, it'd be nice, but
> if not, it's asyncore2, then.

Easily done, because it's already been done ;)

>> Again, at this point in time what you're proposing looks too vague,
>> ambitious and premature to me.
>
> please don't get me wrong -- i'm not proposing anything for immediate
> action, i just want to start a thinking process towards a better
> integrated stdlib.

I am curious as to what you mean by "a better integrated stdlib". A
new interface that doesn't allow people to easily migrate from an
existing (and long-lived, though flawed) standard library is not
better integration. Better integration requires allowing previous
users to migrate, while encouraging new users to join in with any
later development. That's what Giampaolo's suggested interface offers
on the lowest level; something to handle file-handle reactors,
combined with a scheduler.

>From there; whether layers like Twisted are evolved, or more shallow
layers (like much existing asyncore-derived classes) is yet to be
determined by actual people using it.

> On Mon, Sep 24, 2012 at 05:02:08PM -0700, Josiah Carlson wrote:
>> 1. Whatever reactors are available, you need to be able to instantiate
>> multiple of different types of reactors and multiple instances of the
>> same type of reactor simultaneously (to support multiple threads
>> handling different groups of reactors, or different reactors for
>> different types of objects on certain platforms). While this allows
>> for insanity in the worst-case, we're all consenting adults here, so
>> shouldn't be limited by reactor singletons. There should be a default
>> reactor class, which is defined on module/package import (use the
>> "best" one for the platform).
>
> i think that's already common. with asyncore, you can have different
> maps (just one is installed globally as default). with the gtk main
> loop, it's a little tricky (the gtk.main() function doesn't simply take
> an argument), but the underlying glib can do that afaict.

Remember that a reactor isn't just a dictionary of file handles to do
stuff on, it's the thing that determines what underlying platform
mechanics will be used to multiplex across channels. But that level of
detail will be generally unused by most people, as most people will
only use one at a time. The point of offering multiple reactors is to
allow people to be flexible if they choose (or to pick from the
different reactors if they know that one is faster for their number of
expected handles).

>> 2. The API must be simple. I am not sure that it can get easier than
>> Idea #3 from:
>> http://mail.python.org/pipermail/python-ideas/2012-May/015245.html
>
> it's good that the necessities of call_later and call_every are
> mentioned here, i'd have forgotten about them.
>
> we've talked about many things we'd need in a python asynchronous
> interface (not implementation), so what are the things we *don't* need?
> (so we won't start building a framework like twisted). i'll start:
>
> * high-level protocol handling (can be extra modules atop of it)
> * ssl
> * something like the twisted delayed framework (not sure about that, i
>   guess the twisted people will have good reason to use it, but i don't
>   see compelling reasons for such a thing in a minimal interface from my
>   limited pov)
> * explicit connection handling (retries, timeouts -- would be up to the
>   user as well, eg urllib might want to set up a timeout and retries for
>   asynchronous url requests)

I disagree with the last 3. If you have an IO loop, more often than
not you want an opportunity to do something later in the same context.
This is commonly the case for bandwidth limiting, connection timeouts,
etc., which are otherwise *very* difficult to do at a higher level
(which are the reasons why schedulers are built into IO loops).
Further, SSL in async can be tricky to get right. Having the 20-line
SSL layer as an available class is a good idea, and will save people
time by not having them re-invent it (poorly or incorrectly) every
time.

Regards,
 - Josiah


From mikegraham at gmail.com  Wed Sep 26 20:51:22 2012
From: mikegraham at gmail.com (Mike Graham)
Date: Wed, 26 Sep 2012 14:51:22 -0400
Subject: [Python-ideas] Deprecate the round builtin
Message-ID: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>

The builtin round function is completely useless. I've never seen
anyone use it constructively. Usually people using it are new
programmers who are not comfortable with or aware of string
formatting. Sometimes people use it to poorly replicate functionality
that's implemented correctly in the decimal module.

Mike


From guido at python.org  Wed Sep 26 20:55:10 2012
From: guido at python.org (Guido van Rossum)
Date: Wed, 26 Sep 2012 11:55:10 -0700
Subject: [Python-ideas] Deprecate the round builtin
In-Reply-To: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
Message-ID: <CAP7+vJLep1WAvtJC0kg5Hp2DQ3aA_pCv0fqKyNwoRpqW0Bf++w@mail.gmail.com>

On Wed, Sep 26, 2012 at 11:51 AM, Mike Graham <mikegraham at gmail.com> wrote:
> The builtin round function is completely useless. I've never seen
> anyone use it constructively. Usually people using it are new
> programmers who are not comfortable with or aware of string
> formatting. Sometimes people use it to poorly replicate functionality
> that's implemented correctly in the decimal module.

You're probably right. It has ancient ancestry: it was one of the
functions I copied directly from ABC. (It's actually more useful now
that floats are printed with minimal digits.)

But given the pain of removing a builtin, is it worth it? maybe we can
just document the better ways of accomplishing its tasks?

-- 
--Guido van Rossum (python.org/~guido)


From dickinsm at gmail.com  Wed Sep 26 21:22:19 2012
From: dickinsm at gmail.com (Mark Dickinson)
Date: Wed, 26 Sep 2012 20:22:19 +0100
Subject: [Python-ideas] Deprecate the round builtin
In-Reply-To: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
Message-ID: <CAAu3qLVrda99RYi2qJpTTGcz9cSbAyb36AynynDpi9Kszj9cKg@mail.gmail.com>

On Wed, Sep 26, 2012 at 7:51 PM, Mike Graham <mikegraham at gmail.com> wrote:
> The builtin round function is completely useless. I've never seen
> anyone use it constructively

I disagree that it's *completely* useless:  the one-argument form
(take a float, return the closest integer) is a fundamental and useful
mathematics operation, just like floor and ceiling.  It would be crazy
to get rid of that.  I could live with it moving from builtins into
the math module, though.

Agreed that the two-argument form causes a lot of confusion, though.

--
Mark


From mal at egenix.com  Wed Sep 26 21:36:54 2012
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 26 Sep 2012 21:36:54 +0200
Subject: [Python-ideas] Deprecate the round builtin
In-Reply-To: <CAAu3qLVrda99RYi2qJpTTGcz9cSbAyb36AynynDpi9Kszj9cKg@mail.gmail.com>
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
	<CAAu3qLVrda99RYi2qJpTTGcz9cSbAyb36AynynDpi9Kszj9cKg@mail.gmail.com>
Message-ID: <50635956.4050409@egenix.com>

Mark Dickinson wrote:
> On Wed, Sep 26, 2012 at 7:51 PM, Mike Graham <mikegraham at gmail.com> wrote:
>> The builtin round function is completely useless. I've never seen
>> anyone use it constructively
> 
> I disagree that it's *completely* useless:  the one-argument form
> (take a float, return the closest integer) is a fundamental and useful
> mathematics operation, just like floor and ceiling.  It would be crazy
> to get rid of that.  I could live with it moving from builtins into
> the math module, though.
> 
> Agreed that the two-argument form causes a lot of confusion, though.

It's actually quite common in finance and time calculations to
round to the nearest say basis point, cent or say micro second
in calculations (rather than just string formatting). round()
is perfect for that and easy to use.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 26 2012)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2012-10-29: PyCon DE 2012, Leipzig, Germany ...            33 days to go
2012-10-23: Python Meeting Duesseldorf ...                 27 days to go
2012-09-26: Released mxODBC.Connect 2.0.1 ...     http://egenix.com/go34
2012-09-25: Released mxODBC 3.2.1 ...             http://egenix.com/go33


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From joshua.landau.ws at gmail.com  Wed Sep 26 21:47:09 2012
From: joshua.landau.ws at gmail.com (Joshua Landau)
Date: Wed, 26 Sep 2012 20:47:09 +0100
Subject: [Python-ideas] Deprecate the round builtin
In-Reply-To: <50635956.4050409@egenix.com>
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
	<CAAu3qLVrda99RYi2qJpTTGcz9cSbAyb36AynynDpi9Kszj9cKg@mail.gmail.com>
	<50635956.4050409@egenix.com>
Message-ID: <CAN1F8qUacBP4DXTwzTAb9sgZ2eWG7KgpN=uEm-PRJFNk=57+jg@mail.gmail.com>

On 26 September 2012 20:36, M.-A. Lemburg <mal at egenix.com> wrote:

> Mark Dickinson wrote:
> > On Wed, Sep 26, 2012 at 7:51 PM, Mike Graham <mikegraham at gmail.com>
> wrote:
> >> The builtin round function is completely useless. I've never seen
> >> anyone use it constructively
> >
> > I disagree that it's *completely* useless:  the one-argument form
> > (take a float, return the closest integer) is a fundamental and useful
> > mathematics operation, just like floor and ceiling.  It would be crazy
> > to get rid of that.  I could live with it moving from builtins into
> > the math module, though.
> >
> > Agreed that the two-argument form causes a lot of confusion, though.
>
> It's actually quite common in finance and time calculations to
> round to the nearest say basis point, cent or say micro second
> in calculations (rather than just string formatting). round()
> is perfect for that and easy to use.
>

Agreed. I've rounded before. It's useful.

Maybe not as useful as all() or whatever, but I don't see how it's
"redundant".
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120926/0c8bcd6c/attachment.html>

From dickinsm at gmail.com  Wed Sep 26 22:02:39 2012
From: dickinsm at gmail.com (Mark Dickinson)
Date: Wed, 26 Sep 2012 21:02:39 +0100
Subject: [Python-ideas] Deprecate the round builtin
In-Reply-To: <50635956.4050409@egenix.com>
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
	<CAAu3qLVrda99RYi2qJpTTGcz9cSbAyb36AynynDpi9Kszj9cKg@mail.gmail.com>
	<50635956.4050409@egenix.com>
Message-ID: <CAAu3qLWaqy=CRj19EKkaFOw2uFw3EqZbvLzRBNm82EdBe7dNtg@mail.gmail.com>

On Wed, Sep 26, 2012 at 8:36 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> It's actually quite common in finance and time calculations to
> round to the nearest say basis point, cent or say micro second
> in calculations

That's exactly where the problems creep in, though.  Naive users
expect rounding to give 'correct' results for decimal halfway cases,
and then are surprised when it doesn't.

>>> round(2.675, 2)
2.67

So you end up explaining again and again that computing binary
approximations to decimal rounds of binary approximations of decimal
halfway cases is a bad idea.

Mark


From mal at egenix.com  Wed Sep 26 22:17:36 2012
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 26 Sep 2012 22:17:36 +0200
Subject: [Python-ideas] Deprecate the round builtin
In-Reply-To: <CAAu3qLWaqy=CRj19EKkaFOw2uFw3EqZbvLzRBNm82EdBe7dNtg@mail.gmail.com>
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
	<CAAu3qLVrda99RYi2qJpTTGcz9cSbAyb36AynynDpi9Kszj9cKg@mail.gmail.com>
	<50635956.4050409@egenix.com>
	<CAAu3qLWaqy=CRj19EKkaFOw2uFw3EqZbvLzRBNm82EdBe7dNtg@mail.gmail.com>
Message-ID: <506362E0.3010405@egenix.com>

Mark Dickinson wrote:
> On Wed, Sep 26, 2012 at 8:36 PM, M.-A. Lemburg <mal at egenix.com> wrote:
>> It's actually quite common in finance and time calculations to
>> round to the nearest say basis point, cent or say micro second
>> in calculations
> 
> That's exactly where the problems creep in, though.  Naive users
> expect rounding to give 'correct' results for decimal halfway cases,
> and then are surprised when it doesn't.
> 
>>>> round(2.675, 2)
> 2.67
> 
> So you end up explaining again and again that computing binary
> approximations to decimal rounds of binary approximations of decimal
> halfway cases is a bad idea.

But that's the fault of round(), is it ? ;-) It's more one of
educating people of what to expect when working with floats.

Your example is a typical case that comes up when people enter
examples and wonder why they don't see the expected results.

In calculations, statistics, numeric, etc. such corner cases are
not all that common, so things are not as bad as they may appear
on first sight.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 26 2012)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2012-10-29: PyCon DE 2012, Leipzig, Germany ...            33 days to go
2012-10-23: Python Meeting Duesseldorf ...                 27 days to go
2012-09-26: Released mxODBC.Connect 2.0.1 ...     http://egenix.com/go34
2012-09-25: Released mxODBC 3.2.1 ...             http://egenix.com/go33


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From mikegraham at gmail.com  Wed Sep 26 22:36:28 2012
From: mikegraham at gmail.com (Mike Graham)
Date: Wed, 26 Sep 2012 16:36:28 -0400
Subject: [Python-ideas] Deprecate the round builtin
In-Reply-To: <50635956.4050409@egenix.com>
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
	<CAAu3qLVrda99RYi2qJpTTGcz9cSbAyb36AynynDpi9Kszj9cKg@mail.gmail.com>
	<50635956.4050409@egenix.com>
Message-ID: <CAEBZo3NHfvHzUQge2S0+RXjaZU4OsyjZL9_gJ2n5cT9ge3+YKQ@mail.gmail.com>

On Wed, Sep 26, 2012 at 3:36 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> It's actually quite common in finance and time calculations to
> round to the nearest say basis point, cent or say micro second
> in calculations (rather than just string formatting). round()
> is perfect for that and easy to use.

Rounding to the nearest cent is an important operation, but it's not
one that round really handles well. It certainly can't always round
you exactly to the nearest hundredth--it can round you there with some
inaccuracy when you're doing it to have an exact number. We have the
decimal module (or sometimes just plain int) to handle this in a much
more robust, correct way.

I'm not personally familiar with the cases where one would want to
round time like that for computations, but I can't help but suspect
that if you want a quantity in a number of microseconds, you'd be
wanting it to be exact as well.

round only appears to be useful. Using it to try to get something "in
cents" or similar is a strong code smell and probably a bug.

Mike


From jsbueno at python.org.br  Wed Sep 26 23:00:24 2012
From: jsbueno at python.org.br (Joao S. O. Bueno)
Date: Wed, 26 Sep 2012 18:00:24 -0300
Subject: [Python-ideas] Deprecate the round builtin
In-Reply-To: <CAEBZo3NHfvHzUQge2S0+RXjaZU4OsyjZL9_gJ2n5cT9ge3+YKQ@mail.gmail.com>
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
	<CAAu3qLVrda99RYi2qJpTTGcz9cSbAyb36AynynDpi9Kszj9cKg@mail.gmail.com>
	<50635956.4050409@egenix.com>
	<CAEBZo3NHfvHzUQge2S0+RXjaZU4OsyjZL9_gJ2n5cT9ge3+YKQ@mail.gmail.com>
Message-ID: <CAH0mxTS4PTmP3FPygtqBeF2W8uCVwhis7Nk3Jy1c0idKdqdyKQ@mail.gmail.com>

> On Wed, Sep 26, 2012 at 3:36 PM, M.-A. Lemburg <mal at egenix.com> wrote:
>> It's actually quite common in finance and time calculations to
>> round to the nearest say basis point, cent or say micro second
>> in calculations (rather than just string formatting). round()
>> is perfect for that and easy to use.
>

On the other hand, having round with a second parameter to return a Decimal
would be quite usefull. I can't see a way of changing the type returned
by the built-in round without crashing half the World --- but maybe it would
be possible to deprecate only the two-parameter form of the built-in round,
and add a decimal.round Decimal factory?

   js
  -><-


From dreamingforward at gmail.com  Wed Sep 26 23:15:52 2012
From: dreamingforward at gmail.com (Mark Adam)
Date: Wed, 26 Sep 2012 16:15:52 -0500
Subject: [Python-ideas] Deprecate the round builtin
In-Reply-To: <CAH0mxTS4PTmP3FPygtqBeF2W8uCVwhis7Nk3Jy1c0idKdqdyKQ@mail.gmail.com>
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
	<CAAu3qLVrda99RYi2qJpTTGcz9cSbAyb36AynynDpi9Kszj9cKg@mail.gmail.com>
	<50635956.4050409@egenix.com>
	<CAEBZo3NHfvHzUQge2S0+RXjaZU4OsyjZL9_gJ2n5cT9ge3+YKQ@mail.gmail.com>
	<CAH0mxTS4PTmP3FPygtqBeF2W8uCVwhis7Nk3Jy1c0idKdqdyKQ@mail.gmail.com>
Message-ID: <CAMjeLr8JjgTjzJYe0g741+LyuSZtk1q-MTcCWmBnNF=bah_tCA@mail.gmail.com>

> On the other hand, having round with a second parameter to return a Decimal
> would be quite usefull. I can't see a way of changing the type returned
> by the built-in round without crashing half the World --- but maybe it would
> be possible to deprecate only the two-parameter form of the built-in round,
> and add a decimal.round Decimal factory?

On second thought, the idea of "namespaces" within the interpreter
(like Tim Peters was suggesting I think in the doctest module) could
put some order to built-ins and globals within the interpreter.
round(), hex() and others could be in the namespace "Numbers" (Named
by their associated type), similarly for the other type-specific
builtins.

Create a keyword "expose" to dump a particular namespace into the
global scope when you don't want to type "Numbers.round(f)".

Thoughts?

markj


From phd at phdru.name  Wed Sep 26 23:21:27 2012
From: phd at phdru.name (Oleg Broytman)
Date: Thu, 27 Sep 2012 01:21:27 +0400
Subject: [Python-ideas] Deprecate the round builtin
In-Reply-To: <CAMjeLr8JjgTjzJYe0g741+LyuSZtk1q-MTcCWmBnNF=bah_tCA@mail.gmail.com>
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
	<CAAu3qLVrda99RYi2qJpTTGcz9cSbAyb36AynynDpi9Kszj9cKg@mail.gmail.com>
	<50635956.4050409@egenix.com>
	<CAEBZo3NHfvHzUQge2S0+RXjaZU4OsyjZL9_gJ2n5cT9ge3+YKQ@mail.gmail.com>
	<CAH0mxTS4PTmP3FPygtqBeF2W8uCVwhis7Nk3Jy1c0idKdqdyKQ@mail.gmail.com>
	<CAMjeLr8JjgTjzJYe0g741+LyuSZtk1q-MTcCWmBnNF=bah_tCA@mail.gmail.com>
Message-ID: <20120926212127.GA9680@iskra.aviel.ru>

On Wed, Sep 26, 2012 at 04:15:52PM -0500, Mark Adam <dreamingforward at gmail.com> wrote:
> On second thought, the idea of "namespaces" within the interpreter
> (like Tim Peters was suggesting I think in the doctest module) could
> put some order to built-ins and globals within the interpreter.
> round(), hex() and others could be in the namespace "Numbers" (Named
> by their associated type), similarly for the other type-specific
> builtins.
> 
> Create a keyword "expose" to dump a particular namespace into the
> global scope when you don't want to type "Numbers.round(f)".

   What is the difference between such namespaces and modules (which
*are* namespaces)?

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.


From dholth at gmail.com  Wed Sep 26 23:21:40 2012
From: dholth at gmail.com (Daniel Holth)
Date: Wed, 26 Sep 2012 17:21:40 -0400
Subject: [Python-ideas] Deprecate the round builtin
In-Reply-To: <CAMjeLr8JjgTjzJYe0g741+LyuSZtk1q-MTcCWmBnNF=bah_tCA@mail.gmail.com>
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
	<CAAu3qLVrda99RYi2qJpTTGcz9cSbAyb36AynynDpi9Kszj9cKg@mail.gmail.com>
	<50635956.4050409@egenix.com>
	<CAEBZo3NHfvHzUQge2S0+RXjaZU4OsyjZL9_gJ2n5cT9ge3+YKQ@mail.gmail.com>
	<CAH0mxTS4PTmP3FPygtqBeF2W8uCVwhis7Nk3Jy1c0idKdqdyKQ@mail.gmail.com>
	<CAMjeLr8JjgTjzJYe0g741+LyuSZtk1q-MTcCWmBnNF=bah_tCA@mail.gmail.com>
Message-ID: <CAG8k2+5ibcrvCV4RWa2uiNaRFSDbYmyiZFrQL9f3Mr9XjHZfuA@mail.gmail.com>

Normally deprecation means you keep it forever but don't mention it much in
the docs...
On Sep 26, 2012 5:16 PM, "Mark Adam" <dreamingforward at gmail.com> wrote:

> > On the other hand, having round with a second parameter to return a
> Decimal
> > would be quite usefull. I can't see a way of changing the type returned
> > by the built-in round without crashing half the World --- but maybe it
> would
> > be possible to deprecate only the two-parameter form of the built-in
> round,
> > and add a decimal.round Decimal factory?
>
> On second thought, the idea of "namespaces" within the interpreter
> (like Tim Peters was suggesting I think in the doctest module) could
> put some order to built-ins and globals within the interpreter.
> round(), hex() and others could be in the namespace "Numbers" (Named
> by their associated type), similarly for the other type-specific
> builtins.
>
> Create a keyword "expose" to dump a particular namespace into the
> global scope when you don't want to type "Numbers.round(f)".
>
> Thoughts?
>
> markj
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120926/77691ce0/attachment.html>

From greg.ewing at canterbury.ac.nz  Wed Sep 26 23:29:20 2012
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 27 Sep 2012 09:29:20 +1200
Subject: [Python-ideas] Deprecate the round builtin
In-Reply-To: <506362E0.3010405@egenix.com>
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
	<CAAu3qLVrda99RYi2qJpTTGcz9cSbAyb36AynynDpi9Kszj9cKg@mail.gmail.com>
	<50635956.4050409@egenix.com>
	<CAAu3qLWaqy=CRj19EKkaFOw2uFw3EqZbvLzRBNm82EdBe7dNtg@mail.gmail.com>
	<506362E0.3010405@egenix.com>
Message-ID: <506373B0.2010502@canterbury.ac.nz>

M.-A. Lemburg wrote:
> In calculations, statistics, numeric, etc. such corner cases are
> not all that common, so things are not as bad as they may appear
> on first sight.

If the corner cases don't matter, I'd say you don't
need to round your internal representations in the
first place. Rounding on output is sufficient and
probably preferable.

If they *do* matter -- such as when cents need to
add up exactly -- you're much better off using
Decimals. Two-argument round() is an attractive
nuisance in that case, IMO.

-- 
Greg



From dreamingforward at gmail.com  Wed Sep 26 23:30:33 2012
From: dreamingforward at gmail.com (Mark Adam)
Date: Wed, 26 Sep 2012 16:30:33 -0500
Subject: [Python-ideas] Deprecate the round builtin
In-Reply-To: <20120926212127.GA9680@iskra.aviel.ru>
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
	<CAAu3qLVrda99RYi2qJpTTGcz9cSbAyb36AynynDpi9Kszj9cKg@mail.gmail.com>
	<50635956.4050409@egenix.com>
	<CAEBZo3NHfvHzUQge2S0+RXjaZU4OsyjZL9_gJ2n5cT9ge3+YKQ@mail.gmail.com>
	<CAH0mxTS4PTmP3FPygtqBeF2W8uCVwhis7Nk3Jy1c0idKdqdyKQ@mail.gmail.com>
	<CAMjeLr8JjgTjzJYe0g741+LyuSZtk1q-MTcCWmBnNF=bah_tCA@mail.gmail.com>
	<20120926212127.GA9680@iskra.aviel.ru>
Message-ID: <CAMjeLr8VCXBdBjv7f5DXvqT+_ftfouhq4DAz=mo=4Gy_LaUbXw@mail.gmail.com>

On Wed, Sep 26, 2012 at 4:21 PM, Oleg Broytman <phd at phdru.name> wrote:
> On Wed, Sep 26, 2012 at 04:15:52PM -0500, Mark Adam <dreamingforward at gmail.com> wrote:
>> On second thought, the idea of "namespaces" within the interpreter
>> (like Tim Peters was suggesting I think in the doctest module) could
>> put some order to built-ins and globals within the interpreter.
>> round(), hex() and others could be in the namespace "Numbers" (Named
>> by their associated type), similarly for the other type-specific
>> builtins.
>>
>> Create a keyword "expose" to dump a particular namespace into the
>> global scope when you don't want to type "Numbers.round(f)".
>
>    What is the difference between such namespaces and modules (which
> *are* namespaces)?

Modules reside on file and are changeable, Namespaces solely in
memory, defined by the Interpreter itself.

mark


From amauryfa at gmail.com  Wed Sep 26 23:32:52 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 26 Sep 2012 23:32:52 +0200
Subject: [Python-ideas] Deprecate the round builtin
In-Reply-To: <506362E0.3010405@egenix.com>
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
	<CAAu3qLVrda99RYi2qJpTTGcz9cSbAyb36AynynDpi9Kszj9cKg@mail.gmail.com>
	<50635956.4050409@egenix.com>
	<CAAu3qLWaqy=CRj19EKkaFOw2uFw3EqZbvLzRBNm82EdBe7dNtg@mail.gmail.com>
	<506362E0.3010405@egenix.com>
Message-ID: <CAGmFidZ4xYsiWNcd3-xnV2sXgqDVwHUj1poks6EfEurvC5_-fg@mail.gmail.com>

2012/9/26 M.-A. Lemburg <mal at egenix.com>:
>>>>> round(2.675, 2)
>> 2.67
>>
>> So you end up explaining again and again that computing binary
>> approximations to decimal rounds of binary approximations of decimal
>> halfway cases is a bad idea.
>
> But that's the fault of round(), is it ? ;-) It's more one of
> educating people of what to expect when working with floats.

Could we change the round() behavior?
The second parameter is the number of decimal places,
so the operation could be defined in terms of decimal digits.

A similar change was already done for float.__str__.

-- 
Amaury Forgeot d'Arc


From phd at phdru.name  Wed Sep 26 23:51:30 2012
From: phd at phdru.name (Oleg Broytman)
Date: Thu, 27 Sep 2012 01:51:30 +0400
Subject: [Python-ideas] Deprecate the round builtin
In-Reply-To: <CAMjeLr8VCXBdBjv7f5DXvqT+_ftfouhq4DAz=mo=4Gy_LaUbXw@mail.gmail.com>
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
	<CAAu3qLVrda99RYi2qJpTTGcz9cSbAyb36AynynDpi9Kszj9cKg@mail.gmail.com>
	<50635956.4050409@egenix.com>
	<CAEBZo3NHfvHzUQge2S0+RXjaZU4OsyjZL9_gJ2n5cT9ge3+YKQ@mail.gmail.com>
	<CAH0mxTS4PTmP3FPygtqBeF2W8uCVwhis7Nk3Jy1c0idKdqdyKQ@mail.gmail.com>
	<CAMjeLr8JjgTjzJYe0g741+LyuSZtk1q-MTcCWmBnNF=bah_tCA@mail.gmail.com>
	<20120926212127.GA9680@iskra.aviel.ru>
	<CAMjeLr8VCXBdBjv7f5DXvqT+_ftfouhq4DAz=mo=4Gy_LaUbXw@mail.gmail.com>
Message-ID: <20120926215130.GA9810@iskra.aviel.ru>

On Wed, Sep 26, 2012 at 04:30:33PM -0500, Mark Adam <dreamingforward at gmail.com> wrote:
> On Wed, Sep 26, 2012 at 4:21 PM, Oleg Broytman <phd at phdru.name> wrote:
> > On Wed, Sep 26, 2012 at 04:15:52PM -0500, Mark Adam <dreamingforward at gmail.com> wrote:
> >> On second thought, the idea of "namespaces" within the interpreter
> >> (like Tim Peters was suggesting I think in the doctest module) could
> >> put some order to built-ins and globals within the interpreter.
> >> round(), hex() and others could be in the namespace "Numbers" (Named
> >> by their associated type), similarly for the other type-specific
> >> builtins.
> >>
> >> Create a keyword "expose" to dump a particular namespace into the
> >> global scope when you don't want to type "Numbers.round(f)".
> >
> >    What is the difference between such namespaces and modules (which
> > *are* namespaces)?
> 
> Modules reside on file and are changeable,

   Oh, really? What is the file for module sys? thread? zipimport?

> Namespaces solely in
> memory, defined by the Interpreter itself.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.


From python at mrabarnett.plus.com  Thu Sep 27 00:07:01 2012
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 26 Sep 2012 23:07:01 +0100
Subject: [Python-ideas] Deprecate the round builtin
In-Reply-To: <506362E0.3010405@egenix.com>
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
	<CAAu3qLVrda99RYi2qJpTTGcz9cSbAyb36AynynDpi9Kszj9cKg@mail.gmail.com>
	<50635956.4050409@egenix.com>
	<CAAu3qLWaqy=CRj19EKkaFOw2uFw3EqZbvLzRBNm82EdBe7dNtg@mail.gmail.com>
	<506362E0.3010405@egenix.com>
Message-ID: <50637C85.5060003@mrabarnett.plus.com>

On 2012-09-26 21:17, M.-A. Lemburg wrote:
> Mark Dickinson wrote:
>> On Wed, Sep 26, 2012 at 8:36 PM, M.-A. Lemburg <mal at egenix.com> wrote:
>>> It's actually quite common in finance and time calculations to
>>> round to the nearest say basis point, cent or say micro second
>>> in calculations
>>
>> That's exactly where the problems creep in, though.  Naive users
>> expect rounding to give 'correct' results for decimal halfway cases,
>> and then are surprised when it doesn't.
>>
>>>>> round(2.675, 2)
>> 2.67
>>
>> So you end up explaining again and again that computing binary
>> approximations to decimal rounds of binary approximations of decimal
>> halfway cases is a bad idea.
>
> But that's the fault of round(), is it ? ;-) It's more one of
> educating people of what to expect when working with floats.
>
> Your example is a typical case that comes up when people enter
> examples and wonder why they don't see the expected results.
>
> In calculations, statistics, numeric, etc. such corner cases are
> not all that common, so things are not as bad as they may appear
> on first sight.
>
If we're going to move 'round' into 'math' because it sometimes gives
results which puzzle naive users, shouldn't we do the same to 'float'?
:-)


From alexander.belopolsky at gmail.com  Thu Sep 27 00:14:34 2012
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Wed, 26 Sep 2012 18:14:34 -0400
Subject: [Python-ideas] Deprecate the round builtin
In-Reply-To: <50637C85.5060003@mrabarnett.plus.com>
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
	<CAAu3qLVrda99RYi2qJpTTGcz9cSbAyb36AynynDpi9Kszj9cKg@mail.gmail.com>
	<50635956.4050409@egenix.com>
	<CAAu3qLWaqy=CRj19EKkaFOw2uFw3EqZbvLzRBNm82EdBe7dNtg@mail.gmail.com>
	<506362E0.3010405@egenix.com>
	<50637C85.5060003@mrabarnett.plus.com>
Message-ID: <22AB0A86-59B4-436C-92F5-A3CB93C9B7A7@gmail.com>





On Sep 26, 2012, at 6:07 PM, MRAB <python at mrabarnett.plus.com> wrote:

> If we're going to move 'round' into 'math' because it sometimes gives
> results which puzzle naive users, shouldn't we do the same to 'float'?
> :-)

+1

In Python 4, type(1.5) should be Decimal, not float. 


From steve at pearwood.info  Thu Sep 27 01:07:40 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 27 Sep 2012 09:07:40 +1000
Subject: [Python-ideas] Deprecate the round builtin
In-Reply-To: <CAAu3qLWaqy=CRj19EKkaFOw2uFw3EqZbvLzRBNm82EdBe7dNtg@mail.gmail.com>
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
	<CAAu3qLVrda99RYi2qJpTTGcz9cSbAyb36AynynDpi9Kszj9cKg@mail.gmail.com>
	<50635956.4050409@egenix.com>
	<CAAu3qLWaqy=CRj19EKkaFOw2uFw3EqZbvLzRBNm82EdBe7dNtg@mail.gmail.com>
Message-ID: <50638ABC.4030303@pearwood.info>

On 27/09/12 06:02, Mark Dickinson wrote:
> On Wed, Sep 26, 2012 at 8:36 PM, M.-A. Lemburg<mal at egenix.com>  wrote:
>> It's actually quite common in finance and time calculations to
>> round to the nearest say basis point, cent or say micro second
>> in calculations
>
> That's exactly where the problems creep in, though.  Naive users
> expect rounding to give 'correct' results for decimal halfway cases,
> and then are surprised when it doesn't.
>
>>>> round(2.675, 2)
> 2.67
>
> So you end up explaining again and again that computing binary
> approximations to decimal rounds of binary approximations of decimal
> halfway cases is a bad idea.

Pretty much *everything* about binary floats is surprising to people
who expect decimal semantics. Unless we're going to make Decimal the
default floating point type, and shift binary floats to a module, I
don't see any way around that, and a particularly don't see any reason
to single round() out as more confusing than any of the other float
gotchas.

Speaking from my experience on the tutor@ and python-list at python.org
mailing lists, I believe that problems with round are vanishingly rare,
probably an order of magnitude fewer than "why doesn't Python add my
two floats correctly?" type questions.



-- 
Steven


From raymond.hettinger at gmail.com  Thu Sep 27 01:47:57 2012
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Wed, 26 Sep 2012 19:47:57 -0400
Subject: [Python-ideas] Deprecate the round builtin
In-Reply-To: <50638ABC.4030303@pearwood.info>
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
	<CAAu3qLVrda99RYi2qJpTTGcz9cSbAyb36AynynDpi9Kszj9cKg@mail.gmail.com>
	<50635956.4050409@egenix.com>
	<CAAu3qLWaqy=CRj19EKkaFOw2uFw3EqZbvLzRBNm82EdBe7dNtg@mail.gmail.com>
	<50638ABC.4030303@pearwood.info>
Message-ID: <7AF30372-37AD-4CB9-8709-0119C9EA138D@gmail.com>


On Sep 26, 2012, at 7:07 PM, Steven D'Aprano <steve at pearwood.info> wrote:

> Pretty much *everything* about binary floats is surprising to people
> who expect decimal semantics. Unless we're going to make Decimal the
> default floating point type, and shift binary floats to a module, I
> don't see any way around that, and a particularly don't see any reason
> to single round() out as more confusing than any of the other float
> gotchas.
> 
> Speaking from my experience on the tutor@ and python-list at python.org
> mailing lists, I believe that problems with round are vanishingly rare,
> probably an order of magnitude fewer than "why doesn't Python add my
> two floats correctly?" type questions.

My experience teaching and consulting is matches your experience on tutor.
I agree with your suggestion to leave round() untouched.


Raymond
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120926/3f89899c/attachment.html>

From janssen at parc.com  Thu Sep 27 02:32:54 2012
From: janssen at parc.com (Bill Janssen)
Date: Wed, 26 Sep 2012 17:32:54 -0700
Subject: [Python-ideas] Deprecate the round builtin
In-Reply-To: <50637C85.5060003@mrabarnett.plus.com>
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
	<CAAu3qLVrda99RYi2qJpTTGcz9cSbAyb36AynynDpi9Kszj9cKg@mail.gmail.com>
	<50635956.4050409@egenix.com>
	<CAAu3qLWaqy=CRj19EKkaFOw2uFw3EqZbvLzRBNm82EdBe7dNtg@mail.gmail.com>
	<506362E0.3010405@egenix.com>
	<50637C85.5060003@mrabarnett.plus.com>
Message-ID: <16654.1348705974@parc.com>

MRAB <python at mrabarnett.plus.com> wrote:

> If we're going to move 'round' into 'math' because it sometimes gives
> results which puzzle naive users, shouldn't we do the same to 'float'?
> :-)

Ah, but float is a type, not a function.

Bill


From ben+python at benfinney.id.au  Thu Sep 27 02:45:43 2012
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 27 Sep 2012 10:45:43 +1000
Subject: [Python-ideas] Deprecate the round builtin
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
	<CAAu3qLVrda99RYi2qJpTTGcz9cSbAyb36AynynDpi9Kszj9cKg@mail.gmail.com>
	<50635956.4050409@egenix.com>
	<CAAu3qLWaqy=CRj19EKkaFOw2uFw3EqZbvLzRBNm82EdBe7dNtg@mail.gmail.com>
	<506362E0.3010405@egenix.com>
Message-ID: <7wipb0e1jc.fsf@benfinney.id.au>

"M.-A. Lemburg" <mal at egenix.com> writes:

> Mark Dickinson wrote:
> >>>> round(2.675, 2)
> > 2.67
> > 
> > So you end up explaining again and again that computing binary
> > approximations to decimal rounds of binary approximations of decimal
> > halfway cases is a bad idea.
>
> But that's the fault of round(), is it ? ;-) It's more one of
> educating people of what to expect when working with floats.

It's the fault of the two-parameter ?round?, yes. It is an attractive
nuisance that appears to promise one thing but doesn't deliver, and
makes it easy to do the wrong thing by mistake.

Where feasible, and where it doesn't unreasonably restrict
functionality, IMO Python should make it difficult to do the wrong thing
by mistake.

> Your example is a typical case that comes up when people enter
> examples and wonder why they don't see the expected results.

I think this is an argument that deprecating, and eventually removing,
the two-parameter form of ?round?, would be helpful overall.

-- 
 \        ?Consider the daffodil. And while you're doing that, I'll be |
  `\              over here, looking through your stuff.? ?Jack Handey |
_o__)                                                                  |
Ben Finney



From ironfroggy at gmail.com  Thu Sep 27 03:14:32 2012
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Wed, 26 Sep 2012 21:14:32 -0400
Subject: [Python-ideas] Deprecate the round builtin
In-Reply-To: <CAP7+vJLep1WAvtJC0kg5Hp2DQ3aA_pCv0fqKyNwoRpqW0Bf++w@mail.gmail.com>
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
	<CAP7+vJLep1WAvtJC0kg5Hp2DQ3aA_pCv0fqKyNwoRpqW0Bf++w@mail.gmail.com>
Message-ID: <CAGaVwhT=FKPHncb_e8NCp3vtJEXOssP=h90JHqyN5_=6G18Vmg@mail.gmail.com>

On Wed, Sep 26, 2012 at 2:55 PM, Guido van Rossum <guido at python.org> wrote:
> On Wed, Sep 26, 2012 at 11:51 AM, Mike Graham <mikegraham at gmail.com> wrote:
>> The builtin round function is completely useless. I've never seen
>> anyone use it constructively. Usually people using it are new
>> programmers who are not comfortable with or aware of string
>> formatting. Sometimes people use it to poorly replicate functionality
>> that's implemented correctly in the decimal module.
>
> You're probably right. It has ancient ancestry: it was one of the
> functions I copied directly from ABC. (It's actually more useful now
> that floats are printed with minimal digits.)
>
> But given the pain of removing a builtin, is it worth it? maybe we can
> just document the better ways of accomplishing its tasks?

I think it is reasonable with the right justifications, deprecation period, and
obvious migration path.

Also, I'd be completely in support of dropping round() and agree it gets misused
and leads to too much confusion. We should promote the right ways, and some
times to show the right path you need to lock another door and throw
away the key.

> --
> --Guido van Rossum (python.org/~guido)
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas



-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://techblog.ironfroggy.com/
Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy


From steve at pearwood.info  Thu Sep 27 03:46:52 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 27 Sep 2012 11:46:52 +1000
Subject: [Python-ideas] Deprecate the round builtin
In-Reply-To: <16654.1348705974@parc.com>
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
	<CAAu3qLVrda99RYi2qJpTTGcz9cSbAyb36AynynDpi9Kszj9cKg@mail.gmail.com>
	<50635956.4050409@egenix.com>
	<CAAu3qLWaqy=CRj19EKkaFOw2uFw3EqZbvLzRBNm82EdBe7dNtg@mail.gmail.com>
	<506362E0.3010405@egenix.com>
	<50637C85.5060003@mrabarnett.plus.com> <16654.1348705974@parc.com>
Message-ID: <5063B00C.8080207@pearwood.info>

On 27/09/12 10:32, Bill Janssen wrote:
> MRAB<python at mrabarnett.plus.com>  wrote:
>
>> If we're going to move 'round' into 'math' because it sometimes gives
>> results which puzzle naive users, shouldn't we do the same to 'float'?
>> :-)
>
> Ah, but float is a type, not a function.

What difference does that make? Floats sometimes give results which puzzle
naive users.

py> L = [1/10]*10
py> print(L)
[0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1]
py> sum(L)  # should be 1.0
0.9999999999999999

"Why doesn't Python add up correctly?"

And so we have to explain that the float 0.1 is not actually 0.1 because
you can't represent 0.1 as a finite binary fraction, due to some fairly
subtle mathematics that goes right past most people. The float 0.1 is
actually a tiny bit *larger* than the decimal 0.1, but when you add ten of
them together, you end up with a number that it a tiny bit *smaller* than
the expected result.

Don't you just love binary floating point?

Matthew used a smiley there, but I think there is a very strong case for
making the default floating point numeric type Decimal rather than float.
Decimals behave more like people expect, and they interact better with
other numeric types (ints and Fractions) than floats.

Obviously this will be a big change, almost certainly requiring a PEP and
a long lead time, e.g. Decimal could become a built-in in Python 3.4, then
in 3.5 you could do "from __future__ import decimal_floats", and in 3.6 it
could be the standard behaviour, or perhaps a runtime flag to switch between
binary and decimal floats.

I hate to think how much work would be involved.



-- 
Steven


From python at mrabarnett.plus.com  Thu Sep 27 04:11:06 2012
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 27 Sep 2012 03:11:06 +0100
Subject: [Python-ideas] Deprecate the round builtin
In-Reply-To: <16654.1348705974@parc.com>
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
	<CAAu3qLVrda99RYi2qJpTTGcz9cSbAyb36AynynDpi9Kszj9cKg@mail.gmail.com>
	<50635956.4050409@egenix.com>
	<CAAu3qLWaqy=CRj19EKkaFOw2uFw3EqZbvLzRBNm82EdBe7dNtg@mail.gmail.com>
	<506362E0.3010405@egenix.com>
	<50637C85.5060003@mrabarnett.plus.com> <16654.1348705974@parc.com>
Message-ID: <5063B5BA.90300@mrabarnett.plus.com>

On 2012-09-27 01:32, Bill Janssen wrote:
> MRAB <python at mrabarnett.plus.com> wrote:
>
>> If we're going to move 'round' into 'math' because it sometimes gives
>> results which puzzle naive users, shouldn't we do the same to 'float'?
>> :-)
>
> Ah, but float is a type, not a function.
>
They are both (first-class) objects, aren't they? Why treat them
differently? Surely such discrimination is unPythonic? :-)


From steve at pearwood.info  Thu Sep 27 04:15:41 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 27 Sep 2012 12:15:41 +1000
Subject: [Python-ideas] Deprecate the round builtin
In-Reply-To: <CAGaVwhT=FKPHncb_e8NCp3vtJEXOssP=h90JHqyN5_=6G18Vmg@mail.gmail.com>
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
	<CAP7+vJLep1WAvtJC0kg5Hp2DQ3aA_pCv0fqKyNwoRpqW0Bf++w@mail.gmail.com>
	<CAGaVwhT=FKPHncb_e8NCp3vtJEXOssP=h90JHqyN5_=6G18Vmg@mail.gmail.com>
Message-ID: <5063B6CD.4030405@pearwood.info>

On 27/09/12 11:14, Calvin Spealman wrote:
> On Wed, Sep 26, 2012 at 2:55 PM, Guido van Rossum<guido at python.org>  wrote:
>> On Wed, Sep 26, 2012 at 11:51 AM, Mike Graham<mikegraham at gmail.com>  wrote:
>>> The builtin round function is completely useless. I've never seen
>>> anyone use it constructively. Usually people using it are new
>>> programmers who are not comfortable with or aware of string
>>> formatting. Sometimes people use it to poorly replicate functionality
>>> that's implemented correctly in the decimal module.
>>
>> You're probably right. It has ancient ancestry: it was one of the
>> functions I copied directly from ABC. (It's actually more useful now
>> that floats are printed with minimal digits.)
>>
>> But given the pain of removing a builtin, is it worth it? maybe we can
>> just document the better ways of accomplishing its tasks?
>
> I think it is reasonable with the right justifications, deprecation period, and
> obvious migration path.
>
> Also, I'd be completely in support of dropping round() and agree it gets misused
> and leads to too much confusion. We should promote the right ways, and some
> times to show the right path you need to lock another door and throw
> away the key.


I don't believe that round gets misused and causes confusion to any significant
degree. That belief is based on many years experience on two high-traffic mailing
lists with many beginners. I've seen plenty of examples of beginners confused that
Python can't add floats correctly, but if I've ever seen somebody confused by round,
it was so uncommon and so long ago I've forgotten it.

It seems to me that this proposal is based on a purely theoretical fear that some
people manage to escape being confused by the far more obvious floating point
gotchas[1] but can't understand round. There are much more common, and bigger,
surprises with binary floats than round, and it seems to me that depreciating it
just disrupts those who do use it for no real benefit.

So -1 on such a change.

I could be convinced to change that to a -0.5 if round were moved to the math module
exactly as is, including the second argument. But even that is a disruption for no
meaningful benefit.



[1] I managed to surprise myself the other week when I "discovered" what was obvious
in hindsight, that for sufficiently large values all finite floats are even integer
values. I had known the "integer value" part, but was surprised by the "even" part.
I shouldn't have been.


-- 
Steven


From mikegraham at gmail.com  Thu Sep 27 06:10:34 2012
From: mikegraham at gmail.com (Mike Graham)
Date: Thu, 27 Sep 2012 00:10:34 -0400
Subject: [Python-ideas] Deprecate the round builtin
In-Reply-To: <5063B6CD.4030405@pearwood.info>
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
	<CAP7+vJLep1WAvtJC0kg5Hp2DQ3aA_pCv0fqKyNwoRpqW0Bf++w@mail.gmail.com>
	<CAGaVwhT=FKPHncb_e8NCp3vtJEXOssP=h90JHqyN5_=6G18Vmg@mail.gmail.com>
	<5063B6CD.4030405@pearwood.info>
Message-ID: <CAEBZo3PFY1BKYw_L5H8nLLUEsgEDfNLUSy4et+L0c6Q-q3Vt0Q@mail.gmail.com>

On Wed, Sep 26, 2012 at 10:15 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> I don't believe that round gets misused and causes confusion to any
> significant
> degree. That belief is based on many years experience on two high-traffic
> mailing
> lists with many beginners. I've seen plenty of examples of beginners
> confused that
> Python can't add floats correctly, but if I've ever seen somebody confused
> by round,
> it was so uncommon and so long ago I've forgotten it.
>
> It seems to me that this proposal is based on a purely theoretical fear that
> some
> people manage to escape being confused by the far more obvious floating
> point
> gotchas[1] but can't understand round. There are much more common, and
> bigger,
> surprises with binary floats than round, and it seems to me that
> depreciating it
> just disrupts those who do use it for no real benefit.

I don't have a "purely theoretical fear" regarding round; I simply
have had a different experience and analysis than you. I have long
been an active member of the official Python IRC channel and other
Python IRC channels and of various other support communities (I beat
StackOverflow before they added the new levels :) ). I have seen many,
many people have questions about round and they almost always don't
want to be using it. Most often, people really want to be using string
formatting, but there are all sorts of . I posted here today after yet
another person came into #python with a round question where they
didn't want round.

My problem with round isn't that new people don't understand it--it's
that there's nothing worth understanding. (With floats, the situation
merely requires education then people can use floats well. With round,
when you learn you stop using it.) round(x, n) for n>0 is quite simply
not sane code. rounding is an exact operation and it does not make any
sense to do base 10 rounding on base 2 numbers in this way. round for
n <= 0 is not so criminal, but it sometimes-inconveniently returns a
float and it's trivially written shorter/as-easily for n=0 and seldom
actually needed for n<0.

Mike


From greg.ewing at canterbury.ac.nz  Thu Sep 27 12:23:20 2012
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 27 Sep 2012 22:23:20 +1200
Subject: [Python-ideas] Deprecate the round builtin
In-Reply-To: <CAMjeLr8VCXBdBjv7f5DXvqT+_ftfouhq4DAz=mo=4Gy_LaUbXw@mail.gmail.com>
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
	<CAAu3qLVrda99RYi2qJpTTGcz9cSbAyb36AynynDpi9Kszj9cKg@mail.gmail.com>
	<50635956.4050409@egenix.com>
	<CAEBZo3NHfvHzUQge2S0+RXjaZU4OsyjZL9_gJ2n5cT9ge3+YKQ@mail.gmail.com>
	<CAH0mxTS4PTmP3FPygtqBeF2W8uCVwhis7Nk3Jy1c0idKdqdyKQ@mail.gmail.com>
	<CAMjeLr8JjgTjzJYe0g741+LyuSZtk1q-MTcCWmBnNF=bah_tCA@mail.gmail.com>
	<20120926212127.GA9680@iskra.aviel.ru>
	<CAMjeLr8VCXBdBjv7f5DXvqT+_ftfouhq4DAz=mo=4Gy_LaUbXw@mail.gmail.com>
Message-ID: <50642918.1060804@canterbury.ac.nz>

Mark Adam wrote:
> Modules reside on file and are changeable, Namespaces solely in
> memory, defined by the Interpreter itself.

Presumably they would be implemented as module objects,
created automatically at interpreter startup instead of
being loaded from a file.

In which case "built-in module" might be a better term
for them. And their names should start with lower case.

Also you wouldn't need new syntax to get names out of
them, just the existing import machinery:

   from numbers import *

-- 
Greg


From jimjjewett at gmail.com  Thu Sep 27 21:40:27 2012
From: jimjjewett at gmail.com (Jim Jewett)
Date: Thu, 27 Sep 2012 15:40:27 -0400
Subject: [Python-ideas] file API
In-Reply-To: <CAMjeLr8P3GeZUwKf8H+AxjCiA0TtEELpDhmxYBuSSHs88JuNcA@mail.gmail.com>
References: <CAMjeLr8P3GeZUwKf8H+AxjCiA0TtEELpDhmxYBuSSHs88JuNcA@mail.gmail.com>
Message-ID: <CA+OGgf4pK5GUfefB2FU4hxubp7YY-NT38BO8ZfjxP59SA8Vxhg@mail.gmail.com>

On 9/24/12, Mark Adam <dreamingforward at gmail.com> wrote:
> For some time now, I've wanted to suggest a better abstraction for the
> <file> type in Python.  It currently uses an antiquated, low-level
> C-style interface for moving around in a file, with methods like
> tell() and seek().

I agree, but I'm not sure the improvement can be *enough* of an
improvement to justify the cost of change.

>>>> file.pos = x0ae1       #move file pointer to an absolute address
>>>> file.pos += 1             #increment the file pointer one byte

For text files, I would expect it to be a character count rather than
a byte count.  So this particular proposal might end up adding as much
confusion as it hopes to remove.

-jJ


From guido at python.org  Thu Sep 27 22:00:34 2012
From: guido at python.org (Guido van Rossum)
Date: Thu, 27 Sep 2012 13:00:34 -0700
Subject: [Python-ideas] file API
In-Reply-To: <CA+OGgf4pK5GUfefB2FU4hxubp7YY-NT38BO8ZfjxP59SA8Vxhg@mail.gmail.com>
References: <CAMjeLr8P3GeZUwKf8H+AxjCiA0TtEELpDhmxYBuSSHs88JuNcA@mail.gmail.com>
	<CA+OGgf4pK5GUfefB2FU4hxubp7YY-NT38BO8ZfjxP59SA8Vxhg@mail.gmail.com>
Message-ID: <CAP7+vJ+OD0f_4EYbEW1515aVnX2nLCuVVtBreJ_ba89ufMrcag@mail.gmail.com>

Also you can't express lseek()'s "relative to end of file" mode using
the proposed API. -1 on the whole thing.

On Thu, Sep 27, 2012 at 12:40 PM, Jim Jewett <jimjjewett at gmail.com> wrote:
> On 9/24/12, Mark Adam <dreamingforward at gmail.com> wrote:
>> For some time now, I've wanted to suggest a better abstraction for the
>> <file> type in Python.  It currently uses an antiquated, low-level
>> C-style interface for moving around in a file, with methods like
>> tell() and seek().
>
> I agree, but I'm not sure the improvement can be *enough* of an
> improvement to justify the cost of change.
>
>>>>> file.pos = x0ae1       #move file pointer to an absolute address
>>>>> file.pos += 1             #increment the file pointer one byte
>
> For text files, I would expect it to be a character count rather than
> a byte count.  So this particular proposal might end up adding as much
> confusion as it hopes to remove.
>
> -jJ
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas



-- 
--Guido van Rossum (python.org/~guido)


From python at mrabarnett.plus.com  Thu Sep 27 22:07:55 2012
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 27 Sep 2012 21:07:55 +0100
Subject: [Python-ideas] file API
In-Reply-To: <CA+OGgf4pK5GUfefB2FU4hxubp7YY-NT38BO8ZfjxP59SA8Vxhg@mail.gmail.com>
References: <CAMjeLr8P3GeZUwKf8H+AxjCiA0TtEELpDhmxYBuSSHs88JuNcA@mail.gmail.com>
	<CA+OGgf4pK5GUfefB2FU4hxubp7YY-NT38BO8ZfjxP59SA8Vxhg@mail.gmail.com>
Message-ID: <5064B21B.7050406@mrabarnett.plus.com>

On 2012-09-27 20:40, Jim Jewett wrote:
> On 9/24/12, Mark Adam <dreamingforward at gmail.com> wrote:
>> For some time now, I've wanted to suggest a better abstraction for the
>> <file> type in Python.  It currently uses an antiquated, low-level
>> C-style interface for moving around in a file, with methods like
>> tell() and seek().
>
> I agree, but I'm not sure the improvement can be *enough* of an
> improvement to justify the cost of change.
>
>>>>> file.pos = x0ae1       #move file pointer to an absolute address
>>>>> file.pos += 1             #increment the file pointer one byte
>
> For text files, I would expect it to be a character count rather than
> a byte count.  So this particular proposal might end up adding as much
> confusion as it hopes to remove.
>
In the talk about how to seek to the end of the file with file.pos, it
was suggested that negative positions and None could be used.

I wonder whether they could be used with seek. For example:

file.seek(-10)  # Seek 10 bytes from the end.
file.seek(None) # Seek to the end.



From pjenvey at underboss.org  Thu Sep 27 22:59:08 2012
From: pjenvey at underboss.org (Philip Jenvey)
Date: Thu, 27 Sep 2012 13:59:08 -0700
Subject: [Python-ideas] file API
In-Reply-To: <5064B21B.7050406@mrabarnett.plus.com>
References: <CAMjeLr8P3GeZUwKf8H+AxjCiA0TtEELpDhmxYBuSSHs88JuNcA@mail.gmail.com>
	<CA+OGgf4pK5GUfefB2FU4hxubp7YY-NT38BO8ZfjxP59SA8Vxhg@mail.gmail.com>
	<5064B21B.7050406@mrabarnett.plus.com>
Message-ID: <C105F587-8C85-4F10-80E6-2084B6C16DDA@underboss.org>


On Sep 27, 2012, at 1:07 PM, MRAB wrote:

> On 2012-09-27 20:40, Jim Jewett wrote:
>> On 9/24/12, Mark Adam <dreamingforward at gmail.com> wrote:
>>> For some time now, I've wanted to suggest a better abstraction for the
>>> <file> type in Python.  It currently uses an antiquated, low-level
>>> C-style interface for moving around in a file, with methods like
>>> tell() and seek().
>> 
>> I agree, but I'm not sure the improvement can be *enough* of an
>> improvement to justify the cost of change.
>> 
>>>>>> file.pos = x0ae1       #move file pointer to an absolute address
>>>>>> file.pos += 1             #increment the file pointer one byte
>> 
>> For text files, I would expect it to be a character count rather than
>> a byte count.  So this particular proposal might end up adding as much
>> confusion as it hopes to remove.
>> 
> In the talk about how to seek to the end of the file with file.pos, it
> was suggested that negative positions and None could be used.
> 
> I wonder whether they could be used with seek. For example:
> 
> file.seek(-10)  # Seek 10 bytes from the end.
> file.seek(None) # Seek to the end.

file.seek(0, os.SEEK_END) is a lot clearer than file.seek(None).

--
Philip Jenvey



From jeanpierreda at gmail.com  Thu Sep 27 23:39:30 2012
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Thu, 27 Sep 2012 17:39:30 -0400
Subject: [Python-ideas] file API
In-Reply-To: <C105F587-8C85-4F10-80E6-2084B6C16DDA@underboss.org>
References: <CAMjeLr8P3GeZUwKf8H+AxjCiA0TtEELpDhmxYBuSSHs88JuNcA@mail.gmail.com>
	<CA+OGgf4pK5GUfefB2FU4hxubp7YY-NT38BO8ZfjxP59SA8Vxhg@mail.gmail.com>
	<5064B21B.7050406@mrabarnett.plus.com>
	<C105F587-8C85-4F10-80E6-2084B6C16DDA@underboss.org>
Message-ID: <CABicbJ+s_6ORiNn4ZOJDKjiEE=gJZKcM=hkiMT3qjGMUYdnyfg@mail.gmail.com>

On Thu, Sep 27, 2012 at 4:59 PM, Philip Jenvey <pjenvey at underboss.org> wrote:
> file.seek(0, os.SEEK_END) is a lot clearer than file.seek(None).

Could always introduce new constants. file.seek(os.END_OF_FILE)

-- Devin


From ironfroggy at gmail.com  Thu Sep 27 23:45:11 2012
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Thu, 27 Sep 2012 17:45:11 -0400
Subject: [Python-ideas] file API
In-Reply-To: <CAP7+vJ+OD0f_4EYbEW1515aVnX2nLCuVVtBreJ_ba89ufMrcag@mail.gmail.com>
References: <CAMjeLr8P3GeZUwKf8H+AxjCiA0TtEELpDhmxYBuSSHs88JuNcA@mail.gmail.com>
	<CA+OGgf4pK5GUfefB2FU4hxubp7YY-NT38BO8ZfjxP59SA8Vxhg@mail.gmail.com>
	<CAP7+vJ+OD0f_4EYbEW1515aVnX2nLCuVVtBreJ_ba89ufMrcag@mail.gmail.com>
Message-ID: <CAGaVwhQ6e3kHkQABowX20daEZMcyzsPzJg+dqHk+fP5cRVU6tQ@mail.gmail.com>

On Thu, Sep 27, 2012 at 4:00 PM, Guido van Rossum <guido at python.org> wrote:
> Also you can't express lseek()'s "relative to end of file" mode using
> the proposed API. -1 on the whole thing.

You could use negative indexes, which is consistent with subscript and
slice interfaces. I still don't know that this is a good idea, but I'm
just saying.

If someone wants a more sequence-like interface to files, they should use mmap

> On Thu, Sep 27, 2012 at 12:40 PM, Jim Jewett <jimjjewett at gmail.com> wrote:
>> On 9/24/12, Mark Adam <dreamingforward at gmail.com> wrote:
>>> For some time now, I've wanted to suggest a better abstraction for the
>>> <file> type in Python.  It currently uses an antiquated, low-level
>>> C-style interface for moving around in a file, with methods like
>>> tell() and seek().
>>
>> I agree, but I'm not sure the improvement can be *enough* of an
>> improvement to justify the cost of change.
>>
>>>>>> file.pos = x0ae1       #move file pointer to an absolute address
>>>>>> file.pos += 1             #increment the file pointer one byte
>>
>> For text files, I would expect it to be a character count rather than
>> a byte count.  So this particular proposal might end up adding as much
>> confusion as it hopes to remove.
>>
>> -jJ
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> http://mail.python.org/mailman/listinfo/python-ideas
>
>
>
> --
> --Guido van Rossum (python.org/~guido)
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas



-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://techblog.ironfroggy.com/
Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy


From steve at pearwood.info  Fri Sep 28 01:56:35 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 28 Sep 2012 09:56:35 +1000
Subject: [Python-ideas] file API
In-Reply-To: <CAP7+vJ+OD0f_4EYbEW1515aVnX2nLCuVVtBreJ_ba89ufMrcag@mail.gmail.com>
References: <CAMjeLr8P3GeZUwKf8H+AxjCiA0TtEELpDhmxYBuSSHs88JuNcA@mail.gmail.com>
	<CA+OGgf4pK5GUfefB2FU4hxubp7YY-NT38BO8ZfjxP59SA8Vxhg@mail.gmail.com>
	<CAP7+vJ+OD0f_4EYbEW1515aVnX2nLCuVVtBreJ_ba89ufMrcag@mail.gmail.com>
Message-ID: <5064E7B3.4020407@pearwood.info>

On 28/09/12 06:00, Guido van Rossum wrote:
> Also you can't express lseek()'s "relative to end of file" mode using
> the proposed API. -1 on the whole thing.

For what it's worth, there was extensive discussion on comp.lang.python that
eventually decided that while you could express all the various invocations
of seek using file.pos, at best you save two characters of typing and the
whole thing isn't worth the change.

http://mail.python.org/pipermail/python-list/2012-September/thread.html#631501

Personally, I think the proposal has died a natural death, but if anyone
wants to resuscitate it, I encourage them to read the above thread before
doing so.


-- 
Steven


From greg.ewing at canterbury.ac.nz  Fri Sep 28 04:07:25 2012
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 28 Sep 2012 14:07:25 +1200
Subject: [Python-ideas] file API
In-Reply-To: <CA+OGgf4pK5GUfefB2FU4hxubp7YY-NT38BO8ZfjxP59SA8Vxhg@mail.gmail.com>
References: <CAMjeLr8P3GeZUwKf8H+AxjCiA0TtEELpDhmxYBuSSHs88JuNcA@mail.gmail.com>
	<CA+OGgf4pK5GUfefB2FU4hxubp7YY-NT38BO8ZfjxP59SA8Vxhg@mail.gmail.com>
Message-ID: <5065065D.1000801@canterbury.ac.nz>

Jim Jewett wrote:

> For text files, I would expect it to be a character count rather than
> a byte count.  So this particular proposal might end up adding as much
> confusion as it hopes to remove.

I'm given to understand that the file positions used by
the C standard library are supposed to be treated as
opaque tokens -- you're not guaranteed to be able to
perform arithmetic on them.

-- 
Greg


From python at mrabarnett.plus.com  Fri Sep 28 04:43:15 2012
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 28 Sep 2012 03:43:15 +0100
Subject: [Python-ideas] file API
In-Reply-To: <5065065D.1000801@canterbury.ac.nz>
References: <CAMjeLr8P3GeZUwKf8H+AxjCiA0TtEELpDhmxYBuSSHs88JuNcA@mail.gmail.com>
	<CA+OGgf4pK5GUfefB2FU4hxubp7YY-NT38BO8ZfjxP59SA8Vxhg@mail.gmail.com>
	<5065065D.1000801@canterbury.ac.nz>
Message-ID: <50650EC3.6020103@mrabarnett.plus.com>

On 2012-09-28 03:07, Greg Ewing wrote:
> Jim Jewett wrote:
>
>> For text files, I would expect it to be a character count rather than
>> a byte count.  So this particular proposal might end up adding as much
>> confusion as it hopes to remove.
>
> I'm given to understand that the file positions used by
> the C standard library are supposed to be treated as
> opaque tokens -- you're not guaranteed to be able to
> perform arithmetic on them.
>
Yet you're allowed to do relative seeks? Does that mean that the file
position basically works with some undefined units (bytes, characters,
whatever)?


From amauryfa at gmail.com  Fri Sep 28 13:39:32 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Fri, 28 Sep 2012 13:39:32 +0200
Subject: [Python-ideas] file API
In-Reply-To: <50650EC3.6020103@mrabarnett.plus.com>
References: <CAMjeLr8P3GeZUwKf8H+AxjCiA0TtEELpDhmxYBuSSHs88JuNcA@mail.gmail.com>
	<CA+OGgf4pK5GUfefB2FU4hxubp7YY-NT38BO8ZfjxP59SA8Vxhg@mail.gmail.com>
	<5065065D.1000801@canterbury.ac.nz>
	<50650EC3.6020103@mrabarnett.plus.com>
Message-ID: <CAGmFidb5r=nGq+sadUAsVnfhgveed1VRfB9uezbn4Rp5ehxqOA@mail.gmail.com>

2012/9/28 MRAB <python at mrabarnett.plus.com>:
> On 2012-09-28 03:07, Greg Ewing wrote:
>>
>> Jim Jewett wrote:
>>
>>> For text files, I would expect it to be a character count rather than
>>> a byte count.  So this particular proposal might end up adding as much
>>> confusion as it hopes to remove.
>>
>>
>> I'm given to understand that the file positions used by
>> the C standard library are supposed to be treated as
>> opaque tokens -- you're not guaranteed to be able to
>> perform arithmetic on them.
>>
> Yet you're allowed to do relative seeks? Does that mean that the file
> position basically works with some undefined units (bytes, characters,
> whatever)?

See the documentation: http://docs.python.org/library/io.html#io.TextIOBase.seek
With text streams, SEEK_CUR and SEEK_END only accept offset=0 (i.e. no
move, or go to EOF)
and SEEK_SET accepts a "cookie" which was returned a previous tell().

This cookie will often look like the absolute file position, but it
also has to contain
the codec status, which will be nontrivial for variable-length encodings.

-- 
Amaury Forgeot d'Arc


From greg at krypto.org  Sat Sep 29 21:51:47 2012
From: greg at krypto.org (Gregory P. Smith)
Date: Sat, 29 Sep 2012 12:51:47 -0700
Subject: [Python-ideas] make decimal the default non-integer instead of
	float?
Message-ID: <CAGE7PNKQpVXixWq_5xSQHHDBzc_f4Cw109PMvye4ePZVA=X3+A@mail.gmail.com>

-cc: python-dev
+cc: python-ideas

On Sat, Sep 29, 2012 at 11:39 AM, Chris Angelico <rosuav at gmail.com> wrote:

> On Sun, Sep 30, 2012 at 4:26 AM, Brett Cannon <brett at python.org> wrote:
> > Does this mean we want to re-open the discussion about decimal constants?
> > Last time this came up I think we decided that we wanted to wait for
> > cdecimal (which is obviously here) and work out how to handle contexts,
> the
> > syntax, etc.
>
> Just to throw a crazy idea out: How bad a change would it be to make
> decimal actually the default?
>
> (Caveat: I've not worked with decimal/cdecimal to any real extent and
> don't know its limitations etc.)
>

Painful for existing code, unittests and extension modules.  Definitely
python-ideas territory (thread moved there with an appropriate subject).

I'm not surprised at all that a decimal type can be "fast" in an
interpreted language due to the already dominant interpreter overhead.

I wish all spreadsheets had used decimals from day one rather than binary
floating point (blame Lotus?). Think of the trouble that would have saved
the world.

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120929/b91d156f/attachment.html>

From ironfroggy at gmail.com  Sat Sep 29 22:34:53 2012
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Sat, 29 Sep 2012 16:34:53 -0400
Subject: [Python-ideas] make decimal the default non-integer instead of
	float?
In-Reply-To: <CAGE7PNKQpVXixWq_5xSQHHDBzc_f4Cw109PMvye4ePZVA=X3+A@mail.gmail.com>
References: <CAGE7PNKQpVXixWq_5xSQHHDBzc_f4Cw109PMvye4ePZVA=X3+A@mail.gmail.com>
Message-ID: <CAGaVwhQuBdSFXDgVHW6+hLhF_Bb3TGWXQU66hiP846KeENBLpw@mail.gmail.com>

I like the idea a lot, but I recognize it will get a lot pushback. I
think learning Integer -> Decimal -> Float is a lot more natural than
learning Integer -> Float -> Decimal. The Float type represents a
specific hardware accelleration with data-loss tradeoffs, and the use
should be explicit. I think that as someone learns, the limitations of
Decimals will make a lot more sense than those of Floats.

+1

On Sat, Sep 29, 2012 at 3:51 PM, Gregory P. Smith <greg at krypto.org> wrote:
> -cc: python-dev
> +cc: python-ideas
>
> On Sat, Sep 29, 2012 at 11:39 AM, Chris Angelico <rosuav at gmail.com> wrote:
>>
>> On Sun, Sep 30, 2012 at 4:26 AM, Brett Cannon <brett at python.org> wrote:
>> > Does this mean we want to re-open the discussion about decimal
>> > constants?
>> > Last time this came up I think we decided that we wanted to wait for
>> > cdecimal (which is obviously here) and work out how to handle contexts,
>> > the
>> > syntax, etc.
>>
>> Just to throw a crazy idea out: How bad a change would it be to make
>> decimal actually the default?
>>
>> (Caveat: I've not worked with decimal/cdecimal to any real extent and
>> don't know its limitations etc.)
>
>
> Painful for existing code, unittests and extension modules.  Definitely
> python-ideas territory (thread moved there with an appropriate subject).
>
> I'm not surprised at all that a decimal type can be "fast" in an interpreted
> language due to the already dominant interpreter overhead.
>
> I wish all spreadsheets had used decimals from day one rather than binary
> floating point (blame Lotus?). Think of the trouble that would have saved
> the world.
>
> -gps
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://techblog.ironfroggy.com/
Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy


From guido at python.org  Sat Sep 29 23:06:44 2012
From: guido at python.org (Guido van Rossum)
Date: Sat, 29 Sep 2012 14:06:44 -0700
Subject: [Python-ideas] make decimal the default non-integer instead of
	float?
In-Reply-To: <CAGaVwhQuBdSFXDgVHW6+hLhF_Bb3TGWXQU66hiP846KeENBLpw@mail.gmail.com>
References: <CAGE7PNKQpVXixWq_5xSQHHDBzc_f4Cw109PMvye4ePZVA=X3+A@mail.gmail.com>
	<CAGaVwhQuBdSFXDgVHW6+hLhF_Bb3TGWXQU66hiP846KeENBLpw@mail.gmail.com>
Message-ID: <CAP7+vJLbd_A3cdyn=9DZxOjDKjUaM38md7hOWfiM_avsNeDUpQ@mail.gmail.com>

On Sat, Sep 29, 2012 at 1:34 PM, Calvin Spealman <ironfroggy at gmail.com> wrote:
> I like the idea a lot, but I recognize it will get a lot pushback. I
> think learning Integer -> Decimal -> Float is a lot more natural than
> learning Integer -> Float -> Decimal. The Float type represents a
> specific hardware accelleration with data-loss tradeoffs, and the use
> should be explicit. I think that as someone learns, the limitations of
> Decimals will make a lot more sense than those of Floats.

Hm. Remember decimals have data loss too: they can't represent 1/3 any
more accurately than floats can, and like floats they are limited to a
certain number of digits after which they begin dropping precision
even if the result *can* be represented exactly. It's just that they
can represent 1/5 exactly, which happens to be culturally important to
humans, and that the number of digits at which loss of precision
happens is configurable. (And the API to configure it may actually
make it more complex to learn.)

-- 
--Guido van Rossum (python.org/~guido)


From stefan_ml at behnel.de  Sun Sep 30 07:30:32 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 30 Sep 2012 07:30:32 +0200
Subject: [Python-ideas] make decimal the default non-integer instead of
	float?
In-Reply-To: <CAP7+vJLbd_A3cdyn=9DZxOjDKjUaM38md7hOWfiM_avsNeDUpQ@mail.gmail.com>
References: <CAGE7PNKQpVXixWq_5xSQHHDBzc_f4Cw109PMvye4ePZVA=X3+A@mail.gmail.com>
	<CAGaVwhQuBdSFXDgVHW6+hLhF_Bb3TGWXQU66hiP846KeENBLpw@mail.gmail.com>
	<CAP7+vJLbd_A3cdyn=9DZxOjDKjUaM38md7hOWfiM_avsNeDUpQ@mail.gmail.com>
Message-ID: <k48ldn$bl7$1@ger.gmane.org>

Guido van Rossum, 29.09.2012 23:06:
> On Sat, Sep 29, 2012 at 1:34 PM, Calvin Spealman wrote:
>> I like the idea a lot, but I recognize it will get a lot pushback. I
>> think learning Integer -> Decimal -> Float is a lot more natural than
>> learning Integer -> Float -> Decimal. The Float type represents a
>> specific hardware accelleration with data-loss tradeoffs, and the use
>> should be explicit. I think that as someone learns, the limitations of
>> Decimals will make a lot more sense than those of Floats.
> 
> Hm. Remember decimals have data loss too: they can't represent 1/3 any
> more accurately than floats can

That would be "fractions" territory. Given that all three have their own
area where they shine and a large enough area where they really don't, I
can't see a strong enough argument for making any of the three "the default".

Also note that current float is a very C friendly thing, whereas the other
two are far from it.

Stefan


anecdotal PS: I recently got caught in a discussion about the impressive
ugliness of decimals in Java, the unsuitability of float in a financial
context and the lack of a better alternative in the programming languages
that most people use in that area. I got a couple of people surprised by
the fact that Python has fractions right in its standard library.




From oscar.j.benjamin at gmail.com  Sun Sep 30 16:00:23 2012
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Sun, 30 Sep 2012 15:00:23 +0100
Subject: [Python-ideas] Visually confusable unicode characters in identifiers
Message-ID: <CAHVvXxTt1OpcJSKCOqcDWTz94Ud0oAGHo_L4-pXjMjOHwd_4NQ@mail.gmail.com>

Having just discovered that PEP 3131 [1] enables me to use greek letters to
represent variables in equations, it was pointed out to me that it also
allows
visually confusable characters in identifiers [2].

When I previously read the PEP I thought that the normalisation process
resolved these issues but now I see that the PEP leaves it as an open
problem.
I also previously thought that the PEP would be irrelevant if I was using
ascii-only code but now I can see that if a GREEK CAPITAL LETTER ALPHA can
sneak into my code (just like those pesky tab characters) I could still
have a
visually undetectable bug.

An example to show how an issue could arise:

"""
#!/usr/bin/env python3

code = '''
{0} = 123
{1} = 456
print('"{0}" == "{1}":', "{0}" == "{1}")
print('{0} == {1}:', {0} == {1})
'''

def test_identifier(identifier1, identifier2):
    exec(code.format(identifier1, identifier2))

test_identifier('\u212b', '\u00c5') # Different Angstrom code points
test_identifier('A', '\u0391') # LATIN/GREEK CAPITAL A/ALPHA
"""

When I run this I get:

$ ./test.py
"?" == "?": False
? == ?: True
"A" == "?": False
A == ?: False

Is the proposal mentioned in the PEP (to use something based on Unicode
Technical Standard #39 [3]) something that might be implemented at any
point?

Oscar

References:
[1] http://www.python.org/dev/peps/pep-3131/#open-issues
[2] http://article.gmane.org/gmane.comp.python.tutor/78116
[3] http://unicode.org/reports/tr39/#Confusable_Detection
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120930/1ae6cc8d/attachment.html>

From steve at pearwood.info  Sun Sep 30 17:10:18 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 01 Oct 2012 01:10:18 +1000
Subject: [Python-ideas] Visually confusable unicode characters in
	identifiers
In-Reply-To: <CAHVvXxTt1OpcJSKCOqcDWTz94Ud0oAGHo_L4-pXjMjOHwd_4NQ@mail.gmail.com>
References: <CAHVvXxTt1OpcJSKCOqcDWTz94Ud0oAGHo_L4-pXjMjOHwd_4NQ@mail.gmail.com>
Message-ID: <506860DA.7060905@pearwood.info>

On 01/10/12 00:00, Oscar Benjamin wrote:
> Having just discovered that PEP 3131 [1] enables me to use greek letters to
> represent variables in equations, it was pointed out to me that it also
> allows visually confusable characters in identifiers [2].

You don't need PEP 3131 to have visually confusable identifiers.

MyObject = My0bject = "many fonts use the same glyph for O and 0"

rn = m = 23  # try reading this in Ariel with a small font size

x += l


I don't think it's up to Python to protect you from arbitrarily poor choices
in identifiers and typefaces, or against obfuscated code (whether deliberately
so or by accident). Use of confusable identifiers is a code-quality issue,
little different from any other code-quality issue:

class myfunction:
     def __init__(a, b, c, d, e, f, g, h, i, j, k, l):
         a.b = b-e+k*h
         a.a = i + 1j*j
         a.l = ll + l1 + l
         a.somebodytoldmeishouldusemoredesccriptivevaraiblenames = g+d
         a.somebodytoldmeishouldusemoredesccribtivevaraiblenames = c+f

You surely wouldn't expect Python to protect you from ignorant or obnoxious
programmers who wrote code like that. I likewise don't think Python should
protect you from programmers who do things like this:

py> A = 42
py> ? = 23
py> A == ?
False

Besides, just because you and I can't distinguish A from ? in my editor,
using one particular choice of font, doesn't mean that the author or his
intended audience (Greek programmers perhaps?) can't distinguish them, using
their editor and a more suitable typeface. The two characters are distinct
using Courier or Lucinda Typewriter, to mention only two.


> Is the proposal mentioned in the PEP (to use something based on Unicode
> Technical Standard #39 [3]) something that might be implemented at any
> point?

> [3] http://unicode.org/reports/tr39/#Confusable_Detection

I would welcome "confusable detection" in the standard library, possibly a
string method "skeleton" or some other interface to the Confusables file,
perhaps in unicodedata. And I would encourage code checkers like PyFlakes,
PyLint, PyChecker to check for confusable identifiers. But I do not believe
that this should be built into the Python language itself.



-- 
Steven


From storchaka at gmail.com  Sun Sep 30 18:35:39 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Sun, 30 Sep 2012 19:35:39 +0300
Subject: [Python-ideas] make decimal the default non-integer instead of
	float?
In-Reply-To: <k48ldn$bl7$1@ger.gmane.org>
References: <CAGE7PNKQpVXixWq_5xSQHHDBzc_f4Cw109PMvye4ePZVA=X3+A@mail.gmail.com>
	<CAGaVwhQuBdSFXDgVHW6+hLhF_Bb3TGWXQU66hiP846KeENBLpw@mail.gmail.com>
	<CAP7+vJLbd_A3cdyn=9DZxOjDKjUaM38md7hOWfiM_avsNeDUpQ@mail.gmail.com>
	<k48ldn$bl7$1@ger.gmane.org>
Message-ID: <k4a29h$7rr$1@ger.gmane.org>

Instructive story about fractions: 
http://python-history.blogspot.com/2009/03/problem-with-integer-division.html 
.

Let's not fall into the same trap.



From greg at krypto.org  Sun Sep 30 23:38:33 2012
From: greg at krypto.org (Gregory P. Smith)
Date: Sun, 30 Sep 2012 14:38:33 -0700
Subject: [Python-ideas] Deprecate the round builtin
In-Reply-To: <50642918.1060804@canterbury.ac.nz>
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
	<CAAu3qLVrda99RYi2qJpTTGcz9cSbAyb36AynynDpi9Kszj9cKg@mail.gmail.com>
	<50635956.4050409@egenix.com>
	<CAEBZo3NHfvHzUQge2S0+RXjaZU4OsyjZL9_gJ2n5cT9ge3+YKQ@mail.gmail.com>
	<CAH0mxTS4PTmP3FPygtqBeF2W8uCVwhis7Nk3Jy1c0idKdqdyKQ@mail.gmail.com>
	<CAMjeLr8JjgTjzJYe0g741+LyuSZtk1q-MTcCWmBnNF=bah_tCA@mail.gmail.com>
	<20120926212127.GA9680@iskra.aviel.ru>
	<CAMjeLr8VCXBdBjv7f5DXvqT+_ftfouhq4DAz=mo=4Gy_LaUbXw@mail.gmail.com>
	<50642918.1060804@canterbury.ac.nz>
Message-ID: <CAGE7PNJ0BBmbr78a51rYADwq123rsyxJyKR8Xax0j6RQqz4gkg@mail.gmail.com>

Why suggest adding new round-like functions to the math module rather than
defining a new round method on all numerical objects?

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120930/69cf3246/attachment.html>

From joshua.landau.ws at gmail.com  Sun Sep 30 23:48:21 2012
From: joshua.landau.ws at gmail.com (Joshua Landau)
Date: Sun, 30 Sep 2012 22:48:21 +0100
Subject: [Python-ideas] Deprecate the round builtin
In-Reply-To: <CAGE7PNJ0BBmbr78a51rYADwq123rsyxJyKR8Xax0j6RQqz4gkg@mail.gmail.com>
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
	<CAAu3qLVrda99RYi2qJpTTGcz9cSbAyb36AynynDpi9Kszj9cKg@mail.gmail.com>
	<50635956.4050409@egenix.com>
	<CAEBZo3NHfvHzUQge2S0+RXjaZU4OsyjZL9_gJ2n5cT9ge3+YKQ@mail.gmail.com>
	<CAH0mxTS4PTmP3FPygtqBeF2W8uCVwhis7Nk3Jy1c0idKdqdyKQ@mail.gmail.com>
	<CAMjeLr8JjgTjzJYe0g741+LyuSZtk1q-MTcCWmBnNF=bah_tCA@mail.gmail.com>
	<20120926212127.GA9680@iskra.aviel.ru>
	<CAMjeLr8VCXBdBjv7f5DXvqT+_ftfouhq4DAz=mo=4Gy_LaUbXw@mail.gmail.com>
	<50642918.1060804@canterbury.ac.nz>
	<CAGE7PNJ0BBmbr78a51rYADwq123rsyxJyKR8Xax0j6RQqz4gkg@mail.gmail.com>
Message-ID: <CAN1F8qWsA55OKQbnFfggtCF0_4zbqjjNh59zyxOPR2yJTm80_A@mail.gmail.com>

On 30 September 2012 22:38, Gregory P. Smith <greg at krypto.org> wrote:

> Why suggest adding new round-like functions to the math module rather than
> defining a new round method on all numerical objects?


 >>> round("")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: type str doesn't define __round__ method

It seems it is done like that already ;)
This seems like a problem for the proposal, though: we can't have it in the
math library if it's a method!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120930/df082114/attachment.html>

From joshua.landau.ws at gmail.com  Sun Sep 30 23:51:30 2012
From: joshua.landau.ws at gmail.com (Joshua Landau)
Date: Sun, 30 Sep 2012 22:51:30 +0100
Subject: [Python-ideas] Deprecate the round builtin
In-Reply-To: <CAN1F8qWsA55OKQbnFfggtCF0_4zbqjjNh59zyxOPR2yJTm80_A@mail.gmail.com>
References: <CAEBZo3MW9LUKmAjpR53X2dHSz54jvOpv1vZodZRJaHPokyx4sQ@mail.gmail.com>
	<CAAu3qLVrda99RYi2qJpTTGcz9cSbAyb36AynynDpi9Kszj9cKg@mail.gmail.com>
	<50635956.4050409@egenix.com>
	<CAEBZo3NHfvHzUQge2S0+RXjaZU4OsyjZL9_gJ2n5cT9ge3+YKQ@mail.gmail.com>
	<CAH0mxTS4PTmP3FPygtqBeF2W8uCVwhis7Nk3Jy1c0idKdqdyKQ@mail.gmail.com>
	<CAMjeLr8JjgTjzJYe0g741+LyuSZtk1q-MTcCWmBnNF=bah_tCA@mail.gmail.com>
	<20120926212127.GA9680@iskra.aviel.ru>
	<CAMjeLr8VCXBdBjv7f5DXvqT+_ftfouhq4DAz=mo=4Gy_LaUbXw@mail.gmail.com>
	<50642918.1060804@canterbury.ac.nz>
	<CAGE7PNJ0BBmbr78a51rYADwq123rsyxJyKR8Xax0j6RQqz4gkg@mail.gmail.com>
	<CAN1F8qWsA55OKQbnFfggtCF0_4zbqjjNh59zyxOPR2yJTm80_A@mail.gmail.com>
Message-ID: <CAN1F8qUhN1V=TLNR+h-0sd_Uisoy_qgynhMjwnExJAHF9c5g+Q@mail.gmail.com>

On 30 September 2012 22:48, Joshua Landau <joshua.landau.ws at gmail.com>wrote:

> This seems like a problem for the proposal, though: we can't have it in
> the math library if it's a method!
>

Now I think about it: yeah, it can be. We just coerce to float/decimal
first. *sigh*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120930/c0ea04c0/attachment.html>