From tlesher at gmail.com  Thu Jul  9 17:15:06 2009
From: tlesher at gmail.com (Tim Lesher)
Date: Thu, 9 Jul 2009 11:15:06 -0400
Subject: [Python-ideas] Signed eggs?
Message-ID: <9613db600907090815n4d2308c3kac84f19b720193b8@mail.gmail.com>

In all the current discussion on python-dev about improving eggs and
setuptools in general, I don't think I've seen anything regarding
digitally signed eggs or verifiable egg distribution.  Google doesn't
seem to turn anything up, either.

Has anyone put any thought into this?

-- 
Tim Lesher <tlesher at gmail.com>


From stefan_ml at behnel.de  Fri Jul 10 06:57:15 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 10 Jul 2009 06:57:15 +0200
Subject: [Python-ideas] Signed eggs?
In-Reply-To: <9613db600907090815n4d2308c3kac84f19b720193b8@mail.gmail.com>
References: <9613db600907090815n4d2308c3kac84f19b720193b8@mail.gmail.com>
Message-ID: <h36hnb$itc$1@ger.gmane.org>

Tim Lesher wrote:
> In all the current discussion on python-dev about improving eggs and
> setuptools in general, I don't think I've seen anything regarding
> digitally signed eggs or verifiable egg distribution.  Google doesn't
> seem to turn anything up, either.
> 
> Has anyone put any thought into this?

Well, you can sign all stuff that you upload to PyPI. It usually doesn't
get verified on installation, though.

Stefan



From p.f.moore at gmail.com  Fri Jul 10 12:18:16 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 10 Jul 2009 11:18:16 +0100
Subject: [Python-ideas] Signed eggs?
In-Reply-To: <h36hnb$itc$1@ger.gmane.org>
References: <9613db600907090815n4d2308c3kac84f19b720193b8@mail.gmail.com>
	<h36hnb$itc$1@ger.gmane.org>
Message-ID: <79990c6b0907100318g2697189aw44ba312543342a9f@mail.gmail.com>

2009/7/10 Stefan Behnel <stefan_ml at behnel.de>:
> Tim Lesher wrote:
>> In all the current discussion on python-dev about improving eggs and
>> setuptools in general, I don't think I've seen anything regarding
>> digitally signed eggs or verifiable egg distribution. ?Google doesn't
>> seem to turn anything up, either.
>>
>> Has anyone put any thought into this?
>
> Well, you can sign all stuff that you upload to PyPI. It usually doesn't
> get verified on installation, though.

And you could write a PEP 302 installer to load & verify signed eggs.
Nothing new here, other than no-one has wanted to do it so far.

BTW, eggs and setuptools are a 3rd party package - there's nothing
about them in core Python. The discussions on python-dev are about
enhancing *distutils* - ironically, in a way that possibly reduces the
need for setuptools - and not about setptools. Setuptools isn't
appropriate for python-dev (the distutils SIG mailing list hosts
discussions about setuptools if you want to raise the subject there).

Paul.


From tlesher at gmail.com  Fri Jul 10 15:31:14 2009
From: tlesher at gmail.com (Tim Lesher)
Date: Fri, 10 Jul 2009 09:31:14 -0400
Subject: [Python-ideas] Signed eggs?
In-Reply-To: <79990c6b0907100318g2697189aw44ba312543342a9f@mail.gmail.com>
References: <9613db600907090815n4d2308c3kac84f19b720193b8@mail.gmail.com>
	<h36hnb$itc$1@ger.gmane.org>
	<79990c6b0907100318g2697189aw44ba312543342a9f@mail.gmail.com>
Message-ID: <9613db600907100631r2023f09bx2df9124682401363@mail.gmail.com>

On Fri, Jul 10, 2009 at 06:18, Paul Moore<p.f.moore at gmail.com> wrote:
> 2009/7/10 Stefan Behnel <stefan_ml at behnel.de>:
>> Tim Lesher wrote:
>>> In all the current discussion on python-dev about improving eggs and
>>> setuptools in general, I don't think I've seen anything regarding
>>> digitally signed eggs or verifiable egg distribution. ?Google doesn't
>>> seem to turn anything up, either.
>>>
>>> Has anyone put any thought into this?
>>
>> Well, you can sign all stuff that you upload to PyPI. It usually doesn't
>> get verified on installation, though.
>
> And you could write a PEP 302 installer to load & verify signed eggs.
> Nothing new here, other than no-one has wanted to do it so far.

Right--that's part of what I'm going to be doing for a current work project.

The rest is "where to store the signature" and "what inputs should
feed the signature calculation" and "how to verify the egg *without*
trying to import it".

If there were any past efforts (even failed ones) to do so, I was
curious to learn from those experiences.  Sounds like it's a green
field, though.

> BTW, eggs and setuptools are a 3rd party package - there's nothing
> about them in core Python.

Correct--I misspoke. While eggs are probably the implementation
technique I'll be looking at, I was interested in any other attempts
in the past.
-- 
Tim Lesher <tlesher at gmail.com>


From p.f.moore at gmail.com  Fri Jul 10 15:45:57 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 10 Jul 2009 14:45:57 +0100
Subject: [Python-ideas] Signed eggs?
In-Reply-To: <9613db600907100631r2023f09bx2df9124682401363@mail.gmail.com>
References: <9613db600907090815n4d2308c3kac84f19b720193b8@mail.gmail.com>
	<h36hnb$itc$1@ger.gmane.org>
	<79990c6b0907100318g2697189aw44ba312543342a9f@mail.gmail.com>
	<9613db600907100631r2023f09bx2df9124682401363@mail.gmail.com>
Message-ID: <79990c6b0907100645o6db60797vf2bb4bd12c1648a3@mail.gmail.com>

2009/7/10 Tim Lesher <tlesher at gmail.com>:
>> And you could write a PEP 302 installer to load & verify signed eggs.
>> Nothing new here, other than no-one has wanted to do it so far.
>
> Right--that's part of what I'm going to be doing for a current work project.
>
> The rest is "where to store the signature" and "what inputs should
> feed the signature calculation" and "how to verify the egg *without*
> trying to import it".
>
> If there were any past efforts (even failed ones) to do so, I was
> curious to learn from those experiences. ?Sounds like it's a green
> field, though.
>
>> BTW, eggs and setuptools are a 3rd party package - there's nothing
>> about them in core Python.
>
> Correct--I misspoke. While eggs are probably the implementation
> technique I'll be looking at, I was interested in any other attempts
> in the past.

I've not done anything like this, so I can't help much. But one thing
that might be worth considering, if you don't mind a new format, would
be a SQLite database for holding the modules. Advantages over zip
files (which is what eggs use) are that you can add extra table
columns, for things like signatures, and that it's read-write, so you
could generate .pyc "files" on the fly rather than relying on
pregeneration like eggs do. The big disadvantage is that you'd be
inventing a new format (although you could write a utility to extract
the files from an egg and load them into a sqlite file, so you'd be
able to reuse existing eggs to some extent.

Paul.


From chrisperkins99 at gmail.com  Sun Jul 12 22:11:20 2009
From: chrisperkins99 at gmail.com (Chris Perkins)
Date: Sun, 12 Jul 2009 13:11:20 -0700 (PDT)
Subject: [Python-ideas] Proposal for function expressions
Message-ID: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>

I have a proposal for a language feature - a limited form of function-
definition expression, similar (superficially) to Ruby's blocks.

The big problem with making "def" an expression is the indentation of
the function body. You can't just embed an indented block of code into
the middle of an expression. The solution I propose (inspired by Ruby)
is to allow a block of code to be appended to the end of an expression
in certain, limited circumstances. The block is simply syntactic sugar
for a local anonymous function definition which is passed as an
argument to the function call that it follows.

First, a simple example to give the broad strokes:
foo() do:
    BODY

is equivalent to:

def ANON():
    BODY
foo(ANON)

where the name ANON is a placeholder for illustration - it is never
actually bound in the local namespace.

Specifically, here is what I propose:

* A new syntactic construct, the "block", which consists of:
  * the keyword "do" (alternative: "def", repurposed)
  * an optional argument list
  * a colon
  * an indented block
* A block is only allowed immediately following a call (or indexing
expression? see below), on the same line as the closing right brace
* The block defines an anonymous local function which is passed as an
argument to the call that it follows.
* The block is passed as the last argument to the function by default,
but this can be changed by putting a placeholder, the character "&",
into the parameter list of function. This is best illustrated by
example:

# 1) In the following, foo recieves TWO arguments: 23 and a callable,
in that order.
foo(23) do:
    pass

# 2) Here, foo also recieves TWO arguments: a callable and 23, in that
order.
foo(&, 23) do:
    pass

Why "&"? No particular reason - just because Ruby uses "&" for
something similar (but not the same).
Why do we need the "&" feature?  Consider this example:

map(&, my_list) do(item):
    return do_something_to(item)

This also works for keyword arguments:

foo(a=3, b=&, c=7) do():
    whatever()


To make this syntax work, we need several restrictions on where a
block is allowed. Intuitively, the rules are:
* If the line following "do" would normally be indented, then a block
is not allowed.
* If the line following the "do" would be one on which leading
indentation is insignificant, a block is not allowed.

To clarify, the first rule means that this is not allowed:
if foo() do:
    # are we in the body of the block, or of the if?

The second rule means that this is not allowed:
bar(23, foo() do:
    body_of_block()
    ) # closing brace of call to bar


Here are some properties of blocks that may not be immediately
obvious:
* Blocks are a feature of the call site, and do not affect function
definitions. In other words, there is no such thing as a "function
expecting a block", as there is in Ruby. From the callee's point of
view, he has simply been passed a callable as an argument.
* Blocks can be used with any existing callable that expects to be
passed a function as one of its arguments.


OK, let's move on to the fun part: Motivating Examples.

###################
# Networking
dfr = twisted.whatever(...)
dfr.addCallback() do(result):
    handle(result)
dfr.addErrback() do(err):
    handle_err(err)


###################
# GUI (for some hypothetical library)
b = my_widget.add_button('Go')
b.on('click') do(evt):
    if evt.ctrl_key:
        do_something()
    else:
        do_other_stuff()


###################
# Routing HTTP requests
map.match('/') do(req):
    return render_home(req)
map.match('/about') do(req):
    return render_about(req)
map.match(/(?P<controller>\w+)\/$/) do(req, **kw):
    return handlers[kw['controller']](req)


###################
# Threads
thread.start_new() do:
    do_some_work()


###################
# Reduce the temptation to cram too much into a lambda.

# Sort a list of filenames that look like this:
#   Xyz-1.1.3.tgz, acbd-4.7.tgz, Xyz-1.2.5.tgz, ...
my_list.sort(key=&, reverse=True) do(item):
    name, _ = os.path.splitext(item)
    root, _, version = name.partition('-')
    parts = version.split('.')
    return (root.lower(),) + tuple(map(int, parts))

stuff = filter(&, my_list) do(item):
    # some code here...
    return pred(item)

stuff = re.sub(rx, &, s) do(m):
	t = m.group(0)
    if cond(t): return t.upper()
    else if othercond(t): return t.lower()
    else: return ''


###################
# Faking switch
switch(expr) do(case):
    case(int) do:
        stuff()
    case(str, bytes) do:
        other_stuff()

switch(value) do(case):
    case[0:10] do(val):
        handle_small(val)
    case[10:100] do(val):
        handle_medium(val)
    case.default() do(val):
        handle_big(val)


###################
# Overloaded functions (adapted from PEP 3124)
@overload
def flatten(when):
    when(basestring) do(ob):
        yield ob
    when(Iterable) do(ob):
        for o in ob:
            for ob in flatten(o):
                yield ob
    when.otherwise() do(ob):
        yield ob

# later:
flatten.when(file) do(f):
    for line in f:
        yield line


###################
# Sort-of a "with(lock)" block, but running the body in a thread.
pool.run_with(my_lock) do:
    # lock is held here
    work()
other_work() # in parallel

###################

Well, that should give you the general idea.

Some final random points:
* As proposed, blocks are a new kind of beast in Python - an indented
block that is part of an expression, rather than being a statement. I
think that by constraining a block to be the last thing in an
expression, this works out OK, but I may be missing something
* Since writing this, I have started to lean towards "def" as the
keyword, rather than "do". I used "do" in the examples mostly to make
them look like Ruby, but I suspect that making Python "look like Ruby"
is not a design goal for most people :)
* Terminology: "block"? "def expression"? "anonymous def"? "lambda++"?
* Idea: Could we relax the rules for where a block is allowed,
removing the constraint that it follows a call or indexing expression?
Then we could do, for example, this:
  def f():
    return def(args):
        pass
* Would a proliferation of nameless functions be a nightmare for
debugging?


Implementation is here: http://bitbucket.org/grammati/python-trunk/
Patch is here: http://bugs.python.org/issue6469
To be honest, I'm not entirely sure if the patch is in the right
format - it comes from hg, not svn.


Chris Perkins


From tjreedy at udel.edu  Mon Jul 13 02:12:00 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 12 Jul 2009 20:12:00 -0400
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
Message-ID: <h3du4e$l12$1@ger.gmane.org>

Chris Perkins wrote:
> I have a proposal for a language feature - a limited form of function-
> definition expression, similar (superficially) to Ruby's blocks.
> 
> The big problem with making "def" an expression is the indentation of
> the function body. You can't just embed an indented block of code into
> the middle of an expression. The solution I propose (inspired by Ruby)
> is to allow a block of code to be appended to the end of an expression
> in certain, limited circumstances. The block is simply syntactic sugar
> for a local anonymous function definition which is passed as an
> argument to the function call that it follows.
> 
> First, a simple example to give the broad strokes:
> foo() do:
>     BODY
> 
> is equivalent to:
> 
> def ANON():
>     BODY
> foo(ANON)

Add del anon and the equivalence is almost exact.
Call all one-off functions '_' and there is no need to ever delete the 
name and you have

def _(): body
foo(_)

which is nearly identical to your proposed foo(&).

In any case, ideas similar to this have been proposed and discussed ad 
nausem and rejected. Guide decided for Python 3 to neiher delete 
function expressions, which he seriously considered, nor to expand them. 
Feel free to peruse the archives and PEPs.

Terry Jan Reedy



From jimjjewett at gmail.com  Mon Jul 13 02:31:46 2009
From: jimjjewett at gmail.com (Jim Jewett)
Date: Sun, 12 Jul 2009 20:31:46 -0400
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
Message-ID: <fb6fbf560907121731u2578ecd1o739fa0b78d1a69ab@mail.gmail.com>

On Sun, Jul 12, 2009 at 4:11 PM, Chris Perkins<chrisperkins99 at gmail.com> wrote:
> First, a simple example to give the broad strokes:
> foo() do:
> ? ?BODY

> is equivalent to:

> def ANON():
> ? ?BODY
> foo(ANON)

So the named and anoymous functions don't share scope in any way?

Then what is the advantage?  Is putting the call ahead of the def that
valuable for making the code clear?

> Implementation is here: http://bitbucket.org/grammati/python-trunk/
> Patch is here: http://bugs.python.org/issue6469
> To be honest, I'm not entirely sure if the patch is in the right
> format - it comes from hg, not svn.

If using hg for patches is a problem, then the conversion from svn to
hg is in for some rough sledding.

-jJ


From chrisperkins99 at gmail.com  Mon Jul 13 12:51:27 2009
From: chrisperkins99 at gmail.com (Chris Perkins)
Date: Mon, 13 Jul 2009 06:51:27 -0400
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <fb6fbf560907121731u2578ecd1o739fa0b78d1a69ab@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<fb6fbf560907121731u2578ecd1o739fa0b78d1a69ab@mail.gmail.com>
Message-ID: <184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>

On Sun, Jul 12, 2009 at 8:31 PM, Jim Jewett<jimjjewett at gmail.com> wrote:
> So the named and anoymous functions don't share scope in any way?
>
> Then what is the advantage? ?Is putting the call ahead of the def that
> valuable for making the code clear?

Yes, exactly - or at least, I think it is. I have found that putting
the call before the def adds a surprising amount readability. I came
to this conclusion from doing primarily JavaScript for the last couple
of years - there, you have a choice of whether to predefine a local
function, or to put one inline. eg:

  var callback = function(result) {
    // 10 lines of code...
  };
  make_ajax_call(url, callback);

vs.

  make_ajax_call(url, function(){
    // 10 lines of code...
  });

I have come to hate the former style, and find myself frequently
refactoring it into the latter, because the main thrust of what this
code does is summed up by "make_ajax_call(url, ..)", so that is what I
want to see first - not way down at the end, like an afterthought.

In other words, I think code that puts things in the opposite order
from the way you think of them is harder to read.

Another example: if I'm thinking that what I want to do is "substitute
some stuff in a string", then I want to start by typing/reading
"re.sub(...)", and not "def some_made_up_name(...):"

re.sub(pat, &, s) do(m):
    # several lines of code

vs.

def temp(m):
    # several lines of code
re.sub(pat, temp, s)

The latter relegates the key line of code to the very last line,
making it harder to see at a glance what it does.

So yes, the point of this really is just to allow you to write code
"in the right order".

I guess I should have made all this clearer in my original email :)


Chris Perkins


From zuo at chopin.edu.pl  Mon Jul 13 13:40:37 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Mon, 13 Jul 2009 13:40:37 +0200
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<fb6fbf560907121731u2578ecd1o739fa0b78d1a69ab@mail.gmail.com>
	<184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>
Message-ID: <op.uwz71idbfvx12t@jerzozwiesz.home.aster.pl>

Dear Pythonistas,

It's my first post in the list, so let me introduce myself. My name is Jan
Kaliszewski, I am a composer (studied in Frederic Chopin Academy of Music
in Warsaw) and a programmer (currently working in Record System company on
Anakonda -- ERP system being programmed in Python).

Comming to the matter...

13-07-2009 Chris Perkins <chrisperkins99 at gmail.com>:

> In other words, I think code that puts things in the opposite order
> from the way you think of them is harder to read.
>
> Another example: if I'm thinking that what I want to do is "substitute
> some stuff in a string", then I want to start by typing/reading
> "re.sub(...)", and not "def some_made_up_name(...):"
>
> re.sub(pat, &, s) do(m):
>     # several lines of code
>
> vs.
>
> def temp(m):
>     # several lines of code
> re.sub(pat, temp, s)
>
> The latter relegates the key line of code to the very last line,
> making it harder to see at a glance what it does.
>
> So yes, the point of this really is just to allow you to write code
> "in the right order".

I like the idea of '&' (or another character/word in that role) -- but
maybe it'd be enough to extend an existing feature: decorators with it,
and to add possibility to use that special character/word as a function
name after 'def'?

E.g.:

###################
# Networking

dfr = twisted.whatever(...)

# @dfr.addCallback(&) could be equivalent
@dfr.addCallback
def &(result):
      handle(result)

@dfr.addErrback
def &(err):
      handle_err(err)

###################
# GUI (for some hypothetical library)

@my_widget.add_button('Go').on('click', &)
def &(evt):
      if evt.ctrl_key:
          do_something()
      else:
          do_other_stuff()

# or, if we want to keep the result:

@my_widget.add_button('Go').on('click', &)
def button_clicked(evt):
      if evt.ctrl_key:
          do_something()
      else:
          do_other_stuff()

###################
# Threads

@thread.start_new:
def &():
       do_some_work()


Best regards,
Jan Kaliszewski


From phd at phd.pp.ru  Mon Jul 13 13:13:02 2009
From: phd at phd.pp.ru (Oleg Broytmann)
Date: Mon, 13 Jul 2009 15:13:02 +0400
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<fb6fbf560907121731u2578ecd1o739fa0b78d1a69ab@mail.gmail.com>
	<184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>
Message-ID: <20090713111301.GA3107@phd.pp.ru>

On Mon, Jul 13, 2009 at 06:51:27AM -0400, Chris Perkins wrote:
>   var callback = function(result) {
>     // 10 lines of code...
>   };
>   make_ajax_call(url, callback);
> 
> vs.
> 
>   make_ajax_call(url, function(){
>     // 10 lines of code...
>   });

   For me, the latter is unreadable (it's hard to see where is the inner
block, and where is the outer; where are the make_ajax_call's arguments and
where are the function's arguments and body) even if the function is 1-2
line(s) long; if the function's body is longer the style becomes completely
unacceptable for my not so sharp eyes and brain.

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.


From grosser.meister.morti at gmx.net  Mon Jul 13 14:10:17 2009
From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=)
Date: Mon, 13 Jul 2009 14:10:17 +0200
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <op.uwz71idbfvx12t@jerzozwiesz.home.aster.pl>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>	<fb6fbf560907121731u2578ecd1o739fa0b78d1a69ab@mail.gmail.com>	<184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>
	<op.uwz71idbfvx12t@jerzozwiesz.home.aster.pl>
Message-ID: <4A5B2429.70908@gmx.net>

This seems to be a nice idea.
+0.5

	-panzi

On 07/13/2009 01:40 PM, Jan Kaliszewski wrote:
> Dear Pythonistas,
>
> It's my first post in the list, so let me introduce myself. My name is Jan
> Kaliszewski, I am a composer (studied in Frederic Chopin Academy of Music
> in Warsaw) and a programmer (currently working in Record System company on
> Anakonda -- ERP system being programmed in Python).
>
> Comming to the matter...
>
> 13-07-2009 Chris Perkins <chrisperkins99 at gmail.com>:
>
>> In other words, I think code that puts things in the opposite order
>> from the way you think of them is harder to read.
>>
>> Another example: if I'm thinking that what I want to do is "substitute
>> some stuff in a string", then I want to start by typing/reading
>> "re.sub(...)", and not "def some_made_up_name(...):"
>>
>> re.sub(pat, &, s) do(m):
>>     # several lines of code
>>
>> vs.
>>
>> def temp(m):
>>     # several lines of code
>> re.sub(pat, temp, s)
>>
>> The latter relegates the key line of code to the very last line,
>> making it harder to see at a glance what it does.
>>
>> So yes, the point of this really is just to allow you to write code
>> "in the right order".
>
> I like the idea of '&' (or another character/word in that role) -- but
> maybe it'd be enough to extend an existing feature: decorators with it,
> and to add possibility to use that special character/word as a function
> name after 'def'?
>
> E.g.:
>
> ###################
> # Networking
>
> dfr = twisted.whatever(...)
>
> # @dfr.addCallback(&) could be equivalent
> @dfr.addCallback
> def &(result):
>      handle(result)
>
> @dfr.addErrback
> def &(err):
>      handle_err(err)
>
> ###################
> # GUI (for some hypothetical library)
>
> @my_widget.add_button('Go').on('click', &)
> def &(evt):
>      if evt.ctrl_key:
>          do_something()
>      else:
>          do_other_stuff()
>
> # or, if we want to keep the result:
>
> @my_widget.add_button('Go').on('click', &)
> def button_clicked(evt):
>      if evt.ctrl_key:
>          do_something()
>      else:
>          do_other_stuff()
>
> ###################
> # Threads
>
> @thread.start_new:
> def &():
>       do_some_work()
>
>
> Best regards,
> Jan Kaliszewski


From ncoghlan at gmail.com  Mon Jul 13 14:43:47 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 13 Jul 2009 22:43:47 +1000
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <h3du4e$l12$1@ger.gmane.org>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<h3du4e$l12$1@ger.gmane.org>
Message-ID: <4A5B2C03.4050506@gmail.com>

Terry Reedy wrote:
> Chris Perkins wrote:
>> First, a simple example to give the broad strokes:
>> foo() do:
>>     BODY
>>
>> is equivalent to:
>>
>> def ANON():
>>     BODY
>> foo(ANON)
> 
> Add del anon and the equivalence is almost exact.
> Call all one-off functions '_' and there is no need to ever delete the
> name and you have
> 
> def _(): body
> foo(_)
> 
> which is nearly identical to your proposed foo(&).

You can even use decorator notation to move the call before the function
body:

@foo
def thunk():
  BODY

(Note that most code block ideas haven't actually made it to the PEP
stage - the only one along those lines that I can see in PEP 0 is PEP
359's "make" statement, which had more to do with metaclasses than code
blocks. That said, I recall assorted code block based ideas being thrown
around in the PEP 340/342/343/346 discussions that eventually lead to
the introduction of PEP 343's with statement and PEP 342's generator
enhancements)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From dstanek at dstanek.com  Mon Jul 13 17:19:35 2009
From: dstanek at dstanek.com (David Stanek)
Date: Mon, 13 Jul 2009 11:19:35 -0400
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<fb6fbf560907121731u2578ecd1o739fa0b78d1a69ab@mail.gmail.com>
	<184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>
Message-ID: <de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>

On Mon, Jul 13, 2009 at 6:51 AM, Chris Perkins<chrisperkins99 at gmail.com> wrote:
>
> ?var callback = function(result) {
> ? ?// 10 lines of code...
> ?};
> ?make_ajax_call(url, callback);
>
> vs.
>
> ?make_ajax_call(url, function(){
> ? ?// 10 lines of code...
> ?});
>

This is common in Javascript, but still makes me cry. I would rather
see something like this:

  function main() {
      make_ajax_call(url, ajax_call);
  }

  function ajax_call() {
    // 10 lines of code
  }

This way ajax_call can be in a different file and can be used in other
code without duplication.

> I have come to hate the former style, and find myself frequently
> refactoring it into the latter, because the main thrust of what this
> code does is summed up by "make_ajax_call(url, ..)", so that is what I
> want to see first - not way down at the end, like an afterthought.

I think you are refactoring in reverse.

-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek


From anfedorov at gmail.com  Mon Jul 13 18:36:08 2009
From: anfedorov at gmail.com (Andrey Fedorov)
Date: Mon, 13 Jul 2009 12:36:08 -0400
Subject: [Python-ideas] Decorations
Message-ID: <7659cab30907130936g4f20cd08he08a34943a3d0e5@mail.gmail.com>

Hi there,

It's always bothered me that decorators require a lot of "cruft" to work.
Consider an example, from PEP-318
<http://www.python.org/dev/peps/pep-0318/>(Decorators for Functions
and Methods):

def returns(rtype):
    def check_returns(f):
        def new_f(*args, **kwds):
            result = f(*args, **kwds)
            assert isinstance(result, rtype), \
                   "return value %r does not match %s" % (result,rtype)
            return result
        new_f.func_name = f.func_name
        return new_f
    return check_returns

It seems to me everything but line 5 "assert..." is cruft. In my mind, it
makes sense to call that line the "decoration" (part of the decorator).
Since it's a decoration that's executed after the decorated function, it
makes sense to call it a post-decoration, one that may be more nicely
defined as:

@postdecoration
def returns(result, rtype):
    assert isinstance(result, rtype), "return value %r does not match %s" %
(result, rtype)

Then, `returns' can be used as in PEP-318 (but without the cruft). I've
defined similar `decoration', and `predecoration' decorators. I use the
latter for my Django views to accept POST parameters as arguments:

def filter_dict(d, fltr):
    return dict( (str(x.lower()), d[x]) for x in d if x.lower() in fltr )

@predecoration
def takesPOST(f, args):
    if len(args) == 1:
        fargs = getargspec(f).args
        get = args[0].POST
        return filter_dict(get, fargs[1:])

This way, I can define a view login(reqest, user, password) which can be
called simply from another view, without having to construct a special
`request' object.

An initial shot at the definitions is here: http://gist.github.com/146235

The design decisions I'm least confident about is the argument specs. In the
`predecoration' example, the argument `args' is a special name that
specifies the arguments passed into the decoratee, `kwargs' would be the
keyword arcgs passed into the decorateee, dec_args and dec_params being the
arguments passed into the *decorator*. This doesn't seem like an optimal
solution. The problem I'm trying to solve is "moving from a decoration which
takes no parameters to a decoration which takes parameters". For example, it
was trivial to refactor takesPOST to group POST parameters by regex passed
into the decorator. Still, I'm open to questions, ideas, opinions, etc.

What do you guys this?

Cheers,
Andrey
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090713/6fb0d831/attachment.html>

From daniel at stutzbachenterprises.com  Mon Jul 13 18:48:02 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Mon, 13 Jul 2009 11:48:02 -0500
Subject: [Python-ideas] Decorations
In-Reply-To: <7659cab30907130936g4f20cd08he08a34943a3d0e5@mail.gmail.com>
References: <7659cab30907130936g4f20cd08he08a34943a3d0e5@mail.gmail.com>
Message-ID: <eae285400907130948v2b15b580x70aa529764b8cc23@mail.gmail.com>

On Mon, Jul 13, 2009 at 11:36 AM, Andrey Fedorov <anfedorov at gmail.com>wrote:

> It seems to me everything but line 5 "assert..." is cruft. In my mind, it
> makes sense to call that line the "decoration" (part of the decorator).
> Since it's a decoration that's executed after the decorated function, it
> makes sense to call it a post-decoration, one that may be more nicely
> defined as:
>
> @postdecoration
> def returns(result, rtype):
>     assert isinstance(result, rtype), "return value %r does not match %s"
> % (result, rtype)
>
>
You might be interested in the decorator module, which similarly tries to
reduce the amount of boiler-plate code needed to build a decorator:

http://pypi.python.org/pypi/decorator

(though it does not address exactly the same use case that you are
addressing)

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090713/4e3f28c8/attachment.html>

From anfedorov at gmail.com  Mon Jul 13 19:39:23 2009
From: anfedorov at gmail.com (Andrey Fedorov)
Date: Mon, 13 Jul 2009 13:39:23 -0400
Subject: [Python-ideas] Decorations
In-Reply-To: <eae285400907130948v2b15b580x70aa529764b8cc23@mail.gmail.com>
References: <7659cab30907130936g4f20cd08he08a34943a3d0e5@mail.gmail.com> 
	<eae285400907130948v2b15b580x70aa529764b8cc23@mail.gmail.com>
Message-ID: <7659cab30907131039w42d39765h2acb912d58c86a02@mail.gmail.com>

Thanks for the pointer. One thing the decorator library doesn't solve is
abstracting the boiler of decorators that take parameters (and making it
simple to switch between "parametrized" decorators, "naked" decorators, and
"mixed" decorators). Although it's certainly worth giving some thought to
and re-considering some of my internals.

Cheers,
Andrey

On Mon, Jul 13, 2009 at 12:48 PM, Daniel Stutzbach <
daniel at stutzbachenterprises.com> wrote:

> On Mon, Jul 13, 2009 at 11:36 AM, Andrey Fedorov <anfedorov at gmail.com>wrote:
>
>> It seems to me everything but line 5 "assert..." is cruft. In my mind, it
>> makes sense to call that line the "decoration" (part of the decorator).
>> Since it's a decoration that's executed after the decorated function, it
>> makes sense to call it a post-decoration, one that may be more nicely
>> defined as:
>>
>> @postdecoration
>> def returns(result, rtype):
>>     assert isinstance(result, rtype), "return value %r does not match %s"
>> % (result, rtype)
>>
>>
> You might be interested in the decorator module, which similarly tries to
> reduce the amount of boiler-plate code needed to build a decorator:
>
> http://pypi.python.org/pypi/decorator
>
> (though it does not address exactly the same use case that you are
> addressing)
>
> --
> Daniel Stutzbach, Ph.D.
> President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090713/8bdf31ea/attachment.html>

From Scott.Daniels at Acm.Org  Mon Jul 13 20:55:49 2009
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Mon, 13 Jul 2009 11:55:49 -0700
Subject: [Python-ideas] Decorations
In-Reply-To: <7659cab30907131039w42d39765h2acb912d58c86a02@mail.gmail.com>
References: <7659cab30907130936g4f20cd08he08a34943a3d0e5@mail.gmail.com>
	<eae285400907130948v2b15b580x70aa529764b8cc23@mail.gmail.com>
	<7659cab30907131039w42d39765h2acb912d58c86a02@mail.gmail.com>
Message-ID: <h3fvhm$34k$1@ger.gmane.org>

Andrey Fedorov (should have written so as to avoid top-posting):
 > Daniel Stutzbach wrote:
 >
 >> Andrey Fedorov wrote:
 >>
 >>> ... It seems to me everything but line 5 "assert..." is cruft...
 >>> it makes sense to call that line the "decoration"....
 >>
 >> You might be interested in the decorator module, which similarly
 >>  tries toreduce the amount of boiler-plate code needed to build
 >> a decorator:
 >>     http://pypi.python.org/pypi/decorator
 >> (though it does not address exactly the same use case that you
 >> are addressing)
 >
> Thanks for the pointer. One thing the decorator library doesn't solve is
> abstracting the boiler of decorators that take parameters (and making it
> simple to switch between "parametrized" decorators, "naked" decorators, and
> "mixed" decorators). Although it's certainly worth giving some thought to
> and re-considering some of my internals.


Also consider using in functools.partial to convert your "parametrized"
(or "mixed" decorators, I don't quite get your distinction) to "naked"
decorators.

--Scott David Daniels
Scott.Daniels at Acm.Org



From anfedorov at gmail.com  Tue Jul 14 00:29:55 2009
From: anfedorov at gmail.com (Andrey Fedorov)
Date: Mon, 13 Jul 2009 18:29:55 -0400
Subject: [Python-ideas] Decorations
In-Reply-To: <h3fvhm$34k$1@ger.gmane.org>
References: <7659cab30907130936g4f20cd08he08a34943a3d0e5@mail.gmail.com> 
	<eae285400907130948v2b15b580x70aa529764b8cc23@mail.gmail.com> 
	<7659cab30907131039w42d39765h2acb912d58c86a02@mail.gmail.com> 
	<h3fvhm$34k$1@ger.gmane.org>
Message-ID: <7659cab30907131529s6db3a476geed6809d1d46047b@mail.gmail.com>

The terminology may very well be silly:

"naked" decorators don't expect parameters
"mixed" decorators should work both with parameters and without
"parametrized" decorators only work with parameters

Also consider using in functools.partial to convert your "parametrized"
> decorators.


Thanks, I'll try to incorporate that.

Cheers,
Andrey

On Mon, Jul 13, 2009 at 2:55 PM, Scott David Daniels
<Scott.Daniels at acm.org>wrote:

> Andrey Fedorov (should have written so as to avoid top-posting):
> > Daniel Stutzbach wrote:
> >
> >> Andrey Fedorov wrote:
> >>
> >>> ... It seems to me everything but line 5 "assert..." is cruft...
> >>> it makes sense to call that line the "decoration"....
> >>
> >> You might be interested in the decorator module, which similarly
> >>  tries toreduce the amount of boiler-plate code needed to build
> >> a decorator:
> >>     http://pypi.python.org/pypi/decorator
> >> (though it does not address exactly the same use case that you
> >> are addressing)
> >
>
>> Thanks for the pointer. One thing the decorator library doesn't solve is
>> abstracting the boiler of decorators that take parameters (and making it
>> simple to switch between "parametrized" decorators, "naked" decorators,
>> and
>> "mixed" decorators). Although it's certainly worth giving some thought to
>> and re-considering some of my internals.
>>
>
>
> Also consider using in functools.partial to convert your "parametrized"
> (or "mixed" decorators, I don't quite get your distinction) to "naked"
> decorators.
>
> --Scott David Daniels
> Scott.Daniels at Acm.Org
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090713/78a3075e/attachment.html>

From cmjohnson.mailinglist at gmail.com  Tue Jul 14 13:41:52 2009
From: cmjohnson.mailinglist at gmail.com (Carl Johnson)
Date: Tue, 14 Jul 2009 01:41:52 -1000
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<fb6fbf560907121731u2578ecd1o739fa0b78d1a69ab@mail.gmail.com>
	<184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>
	<de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>
Message-ID: <3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>

This idea seems to be almost identical to my thread from November,
"Proposal for Ruby-style anonymous block functions	(that don't kill
the indention)":
http://mail.python.org/pipermail/python-ideas/2008-November/002340.html

The real difference is that your proposal uses & (which is a bad idea,
since & already has a meaning: __and__) and in

foo() do:
   BODY

it implicitly injects the BODY function into the args of foo, which
violates Explicit Is Better Than Implicit.

I agree though that sometimes rather than doing

def spam():
   BODY
result = foo(spam)

you would rather do

result = foo(spam*) *To Be Defined on the next line, OK!
def spam():
   BODY

because it reads better. As I understand it, this was the reason the
decorator syntax was invented. Basically,

@classmethod
def foo():
    BODY

reads better than

def foo():
    BODY
foo = classmethod(foo)

so decorators are just syntatic sugar to improve readability.

So, I agree with the basic motivation of your proposal, but I think
the specifics of it aren't as good as my old one (although, I *would*
think that, wouldn't I?) and from experience, I expect the BDFL to
come and give you -2 in a couple days if the thread keeps up?

In terms of refining my old proposal, it strikes me that ! isn't used
in Python currently. If we have to use a sigil of some sort, we could
try:

new_list = sorted(old_list, key=!keyfunc)
def keyfunc(item):
    normalize data?

or something to that effect with the general semantic being that !
means "here comes a variable name, I will define this variable on the
next line, please wait for it!"

2-cents-ly-yrs,

-- Carl Johnson


From chrisperkins99 at gmail.com  Tue Jul 14 16:21:56 2009
From: chrisperkins99 at gmail.com (Chris Perkins)
Date: Tue, 14 Jul 2009 10:21:56 -0400
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<fb6fbf560907121731u2578ecd1o739fa0b78d1a69ab@mail.gmail.com>
	<184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>
	<de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>
	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
Message-ID: <184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>

On Tue, Jul 14, 2009 at 7:41 AM, Carl
Johnson<cmjohnson.mailinglist at gmail.com> wrote:
> This idea seems to be almost identical to my thread from November,
> "Proposal for Ruby-style anonymous block functions ? ? ?(that don't kill
> the indention)":
> http://mail.python.org/pipermail/python-ideas/2008-November/002340.html

Cool! I had not seen that thread, but the fact that you and I have
proposed nearly the same thing independently increases the odds that
I'm not just nuts. :)

> The real difference is that your proposal uses & (which is a bad idea,
> since & already has a meaning: __and__) and in
>
> foo() do:
> ? BODY
>
> it implicitly injects the BODY function into the args of foo, which
> violates Explicit Is Better Than Implicit.

I considered the use of a "magic" placeholder character to be an
unfortunate but necessary evil - the alternative seemed to be to
restrict the block/def-expression to becoming the last argument to the
function call that it follows, and that was just too restrictive. I
have no particular attachment to "&" - but note that there is no
ambiguity with the bitwise and operator, as far as I can tell. Eg.
this works:

>>> def f(a): return a()
...
>>> 3 & f(&) do: return 5
1

(not that you would write real code like that...)

> I agree though that sometimes rather than doing
>
> def spam():
> ? BODY
> result = foo(spam)
>
> you would rather do
>
> result = foo(spam*) *To Be Defined on the next line, OK!
> def spam():
> ? BODY
>
> because it reads better. As I understand it, this was the reason the
> decorator syntax was invented. Basically,
>
> @classmethod
> def foo():
> ? ?BODY
>
> reads better than
>
> def foo():
> ? ?BODY
> foo = classmethod(foo)
>
> so decorators are just syntatic sugar to improve readability.
>
> So, I agree with the basic motivation of your proposal, but I think
> the specifics of it aren't as good as my old one (although, I *would*
> think that, wouldn't I?) and from experience, I expect the BDFL to
> come and give you -2 in a couple days if the thread keeps up?
>
> In terms of refining my old proposal, it strikes me that ! isn't used
> in Python currently. If we have to use a sigil of some sort, we could
> try:
>
> new_list = sorted(old_list, key=!keyfunc)
> def keyfunc(item):
> ? ?normalize data?
>
> or something to that effect with the general semantic being that !
> means "here comes a variable name, I will define this variable on the
> next line, please wait for it!"

I prefer not repeating the function name, I prefer putting the "do:"
(or "def:") on the same line (makes it clearer that the indented block
is effectively part of the expression that it follows), and I like the
fact that the function has no name - it tells you immediately that
"this function is only used within this one call - don't go looking
for uses of it later on, because they won't be there".

Other than that, it seems we are in perfect agreement on the feature.

It also seems that your earlier proposal met with the same rabid
ambivalence that mine has so far :)

A couple of interesting quotes from the discussion of your proposal:

Carl: "this proposal is not about adding power to Python, just making
things more readable"
Exactly!

Greg: "You could just allow any expression-statement to be followed by
a block, and sort out whether it makes sense later on."
That's exactly how I ended up implementing it.

Carl, if you still have any interest in this feature, you should try
out the patch - I think you'll be the first. :)  Also I'll be happy to
give you commit privileges to the hg repo where I did the
implementation, in case you want to try to come up with some middle
ground between my proposal and yours:
http://bitbucket.org/grammati/python-trunk/

Failing that, it looks like our proposals will die the same
not-so-slow death for lack of interest. Oh well.


Chris Perkins


From p.f.moore at gmail.com  Tue Jul 14 17:04:28 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 14 Jul 2009 16:04:28 +0100
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<fb6fbf560907121731u2578ecd1o739fa0b78d1a69ab@mail.gmail.com>
	<184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>
	<de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>
	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
Message-ID: <79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>

2009/7/14 Chris Perkins <chrisperkins99 at gmail.com>:
> Failing that, it looks like our proposals will die the same
> not-so-slow death for lack of interest. Oh well.

OK, so I'm not accused of "lack of interest"... :-)

First of all, I have never felt the need for a construct like this. So
it adds nothing but complexity to the language for me. But assuming
that in the future, I do find a need for this, or there's a large
group of other users who would find this useful, please can you
address the following:

Your

fn() do (...):
    stmt1
    stmt2

is equivalent to

@fn
def _(...):
    stmt1
    stmt2

Why is your proposal better (ignoring "I like it better", as
subjective preferences other than Guido's don't get the language
changed)?

For the case of function calls with arguments, why is

fn(a, b, c) do(...):
    stmt1
    stmt2

better than

@functools.partial(fn, a, b, c)
def _(...):
    stmt1
    stmt2

(note that I could make my version look a bit nicer by a "from
functools import partial", so you're still not allowed to argue
aesthetics :-))?

That leaves your general cases where you stick an "&" somewhere in the
call as a placeholder. Only a couple of your motivating examples used
that form, and to be honest I didn't find them very convincing. By the
time you're getting that complex, I really do think you're trying to
cram too much into one statement (and I know that's a subjective
statement, sorry :-))

The same applies to the "x = fn(...) do (...)" cases that you include
in the example, but don't explicitly state in your specification. I
don't see any objective reason to think your form is clearer - yes,
"put the assignment, which is the main thing, up front" is an argument
in your favour, but "don't put too many things together just to avoid
thinking of a name" is an equally valid against.

OK, I hope that helps. I'm still against the idea, but I hope this
gives you some idea why - it's not simply indifference.

A construct like this needs to bring something better than mere
improved readability to the language, in my view.

Paul.


From python at mrabarnett.plus.com  Tue Jul 14 17:26:27 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 14 Jul 2009 16:26:27 +0100
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>	<fb6fbf560907121731u2578ecd1o739fa0b78d1a69ab@mail.gmail.com>	<184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>	<de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
Message-ID: <4A5CA3A3.1040302@mrabarnett.plus.com>

Chris Perkins wrote:
> On Tue, Jul 14, 2009 at 7:41 AM, Carl
> Johnson<cmjohnson.mailinglist at gmail.com> wrote:
>> This idea seems to be almost identical to my thread from November,
>> "Proposal for Ruby-style anonymous block functions      (that don't kill
>> the indention)":
>> http://mail.python.org/pipermail/python-ideas/2008-November/002340.html
> 
> Cool! I had not seen that thread, but the fact that you and I have
> proposed nearly the same thing independently increases the odds that
> I'm not just nuts. :)
> 
>> The real difference is that your proposal uses & (which is a bad idea,
>> since & already has a meaning: __and__) and in
>>
>> foo() do:
>>   BODY
>>
>> it implicitly injects the BODY function into the args of foo, which
>> violates Explicit Is Better Than Implicit.
> 
> I considered the use of a "magic" placeholder character to be an
> unfortunate but necessary evil - the alternative seemed to be to
> restrict the block/def-expression to becoming the last argument to the
> function call that it follows, and that was just too restrictive. I
> have no particular attachment to "&" - but note that there is no
> ambiguity with the bitwise and operator, as far as I can tell. Eg.
> this works:
> 
>>>> def f(a): return a()
> ...
>>>> 3 & f(&) do: return 5
> 1
> 
> (not that you would write real code like that...)
> 
[snip]

Another possibility is to permit a local name (it's not an anonymous
function any more!) so that you can have multiple functions:

3 & f(foo) def foo:
     return 5

3 & f(foo, bar) def foo, bar:
     return 5
and:
     return 6

Anyway, how would you handle condition expressions, as used in 'if' and
'while' statements? /They/ also expect a terminating colon and an
indented block.



From chrisperkins99 at gmail.com  Tue Jul 14 18:31:25 2009
From: chrisperkins99 at gmail.com (Chris Perkins)
Date: Tue, 14 Jul 2009 12:31:25 -0400
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <4A5CA3A3.1040302@mrabarnett.plus.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<fb6fbf560907121731u2578ecd1o739fa0b78d1a69ab@mail.gmail.com>
	<184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>
	<de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>
	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
	<4A5CA3A3.1040302@mrabarnett.plus.com>
Message-ID: <184a9f5a0907140931p17f540dbu31672dc7eb0596f7@mail.gmail.com>

On Tue, Jul 14, 2009 at 11:26 AM, MRAB<python at mrabarnett.plus.com> wrote:
>
> Anyway, how would you handle condition expressions, as used in 'if' and
> 'while' statements? /They/ also expect a terminating colon and an
> indented block.

Python 2.7a0 (trunk, Jul 10 2009, 16:43:32) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> def f(a): return a()
...
>>> if f() do: return True:
  File "<stdin>", line 1
    if f() do: return True:
            ^
SyntaxError: invalid syntax
>>> while (f() do: return 1):
  File "<stdin>", line 1
    while (f() do: return 1):
                ^
SyntaxError: invalid syntax
>>>

Yeah, so the error message could be clearer :)  But I think I have
covered all the bases. I think. That's why I'm hoping at least a few
people will apply the patch and find the edge cases that I have
inevitably missed.

To answer your question more specifically: the modification to the
grammar allows a block following a "simple_stmt" only. Here's the
relevant bit:

stmt: simple_stmt | compound_stmt
# additional restrictions enforced by the interpreter - only certain
# types of small_stmt are allowed to have a block attached:
simple_stmt: small_stmt (';' small_stmt)* (([';'] NEWLINE) | block)
block: 'do' [parameters] ':' suite

The cases you are concerned about are "compount_stmt"s, so a block is
not allowed.

Chris Perkins


From chrisperkins99 at gmail.com  Tue Jul 14 18:56:38 2009
From: chrisperkins99 at gmail.com (Chris Perkins)
Date: Tue, 14 Jul 2009 12:56:38 -0400
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<fb6fbf560907121731u2578ecd1o739fa0b78d1a69ab@mail.gmail.com>
	<184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>
	<de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>
	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
Message-ID: <184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>

On Tue, Jul 14, 2009 at 11:04 AM, Paul Moore<p.f.moore at gmail.com> wrote:
> 2009/7/14 Chris Perkins <chrisperkins99 at gmail.com>:
>> Failing that, it looks like our proposals will die the same
>> not-so-slow death for lack of interest. Oh well.
>
> OK, so I'm not accused of "lack of interest"... :-)
>
> First of all, I have never felt the need for a construct like this. So
> it adds nothing but complexity to the language for me. But assuming
> that in the future, I do find a need for this, or there's a large
> group of other users who would find this useful, please can you
> address the following:
>
> Your
>
> fn() do (...):
> ? ?stmt1
> ? ?stmt2
>
> is equivalent to
>
> @fn
> def _(...):
> ? ?stmt1
> ? ?stmt2
>
> Why is your proposal better (ignoring "I like it better", as
> subjective preferences other than Guido's don't get the language
> changed)?

I agree that decorators can handle some of the use cases - I
anticipated that argument, and it is indeed a valid one. Nevertheless,
I think that's a bit a perversion of the the purpose of decorators.
And as you point out below, decorators are significantly more limited
in what they can do.  And besides that, "I like it better" :)

> For the case of function calls with arguments, why is
>
> fn(a, b, c) do(...):
> ? ?stmt1
> ? ?stmt2
>
> better than
>
> @functools.partial(fn, a, b, c)
> def _(...):
> ? ?stmt1
> ? ?stmt2
>
> (note that I could make my version look a bit nicer by a "from
> functools import partial", so you're still not allowed to argue
> aesthetics :-))?

Yes, again, that does work, but it's getting uglier (yes, I know,
subjectiveness again), and it forces the callable to be the last
argument, which is limiting.

> That leaves your general cases where you stick an "&" somewhere in the
> call as a placeholder. Only a couple of your motivating examples used
> that form, and to be honest I didn't find them very convincing. By the
> time you're getting that complex, I really do think you're trying to
> cram too much into one statement (and I know that's a subjective
> statement, sorry :-))

Indeed, like any language construct it could be used for evil. I
prefer to think of it as enabling the programmer to write clearer code
in some cases - trying to prevent him from writing unclear code is, in
my opinion, a lost cause.  Some programmers will find a way to write
unclear code no matter what tools you give them.

> The same applies to the "x = fn(...) do (...)" cases that you include
> in the example, but don't explicitly state in your specification. I
> don't see any objective reason to think your form is clearer - yes,
> "put the assignment, which is the main thing, up front" is an argument
> in your favour, but "don't put too many things together just to avoid
> thinking of a name" is an equally valid against.

For me, avoiding thinking of a name was never the primary motivation -
putting things in the "right order" is. But I do believe that there
are situations where the context that a block of code sits in tells
far more about its purpose that a name ever could - in those
circumstances, a forced function name can be useless at best, and
distracting clutter at worst. For example:

y = sorted(x, key=&) do(item):
    name = item.split('-')[1]
    return name.upper()

I think that forcing that two-line block of code to have a name would
serve no purpose - it's abundantly clear what it does. I also think
that the current state of affairs encourages poorer readability - I
mean, admit it, today you would probably write that snippet like this:

y = sorted(x, key=lambda item: item.split('-')[0].upper())

which to me loses clarity both by being crammed into one line, and by
not being able to create the local variable "name", which tells you
quite a bit about why the the item is being chopped up the way it is.

> OK, I hope that helps. I'm still against the idea, but I hope this
> gives you some idea why - it's not simply indifference.

It does help - thank you for your comments, Paul.

> A construct like this needs to bring something better than mere
> improved readability to the language, in my view.

Aha! You said "improved readability"! So I'm going to conveniently
forget everything else you said and write your name in the "pro"
column. :)

Seriously, I guess this is where we differ - I think that improved
readability _is_ a sufficient goal unto itself.  The only question is
whether other people (OK, one other person in particular) thinks it
improves readability, and if so, then by how much.


Chris Perkins


From p.f.moore at gmail.com  Tue Jul 14 21:16:01 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 14 Jul 2009 20:16:01 +0100
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<fb6fbf560907121731u2578ecd1o739fa0b78d1a69ab@mail.gmail.com>
	<184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>
	<de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>
	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
Message-ID: <79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>

2009/7/14 Chris Perkins <chrisperkins99 at gmail.com>:
> For me, avoiding thinking of a name was never the primary motivation -
> putting things in the "right order" is. But I do believe that there
> are situations where the context that a block of code sits in tells
> far more about its purpose that a name ever could - in those
> circumstances, a forced function name can be useless at best, and
> distracting clutter at worst. For example:
>
> y = sorted(x, key=&) do(item):
> ? ?name = item.split('-')[1]
> ? ?return name.upper()
>
> I think that forcing that two-line block of code to have a name would
> serve no purpose - it's abundantly clear what it does.

That's a very telling statement. To me, that code is utterly baffling.
(Seriously! Without analysing it line by line, it means nothing to
me). By contrast.

def uc_name(item):
    name = item.split('-')[1]
    return name.upper()

y = sorted(x, key=uc_name)

is a lot clearer to me. And it would be even better if I knew what was
going on with the splitting of items on dashes, and taking the second
bit, was all about. Then, I could give the function a more meaningful
name.

[[An admission here - when I first wrote that example, I spelled the
function name differently in the 2 places. So my "see how much better
my way is" example was wrong - in a way yours couldn't be! In my
defence, the error would have come up as soon as I ran the code, and I
could say that if I had thought of a better name it wouldn't have
happened - but nevertheless. In the spirit of friendly debate, I'll
give you that argument for your case free of charge :-)]]

My point is that *technically* I can see what the 2-line block of code
does - but without giving it a name, I don't have a clue what it's
*for*.

But I don't imagine we're ever going to agree on this. Let it stand
that (without implying any criticism - people differ) your idea of
readable code is confusing and difficult to follow for me. Hence my
distrust of readability arguments :-)

>> A construct like this needs to bring something better than mere
>> improved readability to the language, in my view.
>
> Aha! You said "improved readability"! So I'm going to conveniently
> forget everything else you said and write your name in the "pro"
> column. :)

Good try, but you don't win that easily :-)

I don't think this construct improves readability. Personally, I think
it *harms* readability (although I concede that like anything, it
could be used tastefully). What I was saying was that your *only*
argument was readability, and I don't think that is enough of an
argument in itself.

> Seriously, I guess this is where we differ - I think that improved
> readability _is_ a sufficient goal unto itself. ?The only question is
> whether other people (OK, one other person in particular) thinks it
> improves readability, and if so, then by how much.

It has to improve readability, uncontroversially, in enough cases to
justify the cost in terms extra of language complexity (both for
implementation, and definition/teaching).

To demonstrate the benefits, a good place to look is often the
standard library. If your proposed change can be demonstrated to
improve the code shipped with Python in the stdlib, in a number of
places, people will start to listen.

Also, apart from the basic cost of any change (the fundamental barrier
to entry, if you like) the cost of your change is higher because there
are obscure corner cases that aren't easy to explain - witness
questions here about interaction of this form with if statements, etc.
This is a unique construct in that it's the only indentation-sensitive
multi-line element of an expression. People don't have an intuition
about how it "should" work, because there's nothing similar to
extrapolate from. Ultimately, that's why this idea (in one form or
another) has been around for so long and never been accepted - nobody
has ever managed to come up with an idea that prompted the "oh, yes,
obviously!" reaction in people needed to confirm that it's the "right"
solution.

I'd like to think that one day, someone will come up with the ideal
answer which will finally address all these "extended lambda"/"code
block" needs. But I don't think it's happened yet.

Sorry :-)
Paul.


From steve at pearwood.info  Wed Jul 15 02:10:40 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 15 Jul 2009 10:10:40 +1000
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
Message-ID: <200907151010.40770.steve@pearwood.info>

On Wed, 15 Jul 2009 02:56:38 am Chris Perkins wrote:

> For me, avoiding thinking of a name was never the primary motivation
> - putting things in the "right order" is. But I do believe that there
> are situations where the context that a block of code sits in tells
> far more about its purpose that a name ever could - in those
> circumstances, a forced function name can be useless at best, and
> distracting clutter at worst. For example:
>
> y = sorted(x, key=&) do(item):
>     name = item.split('-')[1]
>     return name.upper()
>
> I think that forcing that two-line block of code to have a name would
> serve no purpose - it's abundantly clear what it does. 

I'm afraid I'm with Paul Moore on this one -- it's not clear to me 
either.

The key=& seems like a mysterious Perlism -- what's the & operator doing 
in there? (Admittedly, it *might* be nice to be able to write things 
like reduce(&, alist) instead of reduce(operator.and_, alist). But 
that's another story.) "do(item):" confuses me, because it looks like 
the start of a do loop, but it isn't.

None of this is to say that I couldn't learn the proposed syntax, but in 
my opinion it doesn't mesh well with existing Python constructs.

Did I understand correctly that you prohibited using `if`, `while` and 
other block statements inside the do block? If so, then to my mind the 
proposal is useless -- it's neither an expression, like the body of 
lambda, nor a block, like the body of a function, but a freakish 
chimera.


> I also think 
> that the current state of affairs encourages poorer readability - I
> mean, admit it, today you would probably write that snippet like
> this:
>
> y = sorted(x, key=lambda item: item.split('-')[0].upper())
>
> which to me loses clarity both by being crammed into one line, and by
> not being able to create the local variable "name", which tells you
> quite a bit about why the the item is being chopped up the way it is.

If I were doing that operation more than once, I'd make a function:

def get_canonical_name(item):
    return item.split('-')[0].upper()

and I'd give it a docstring and doctests. The function 
get_canonical_name tells you *much* more about why the item is 
processed the way it is than a mere local variable "name".

You'll note that I wouldn't bother using a local -- this is short 
enough, and simple enough, that I don't need it. But if I only did it 
once, then I see nothing wrong with the lambda version. If you're 
worried about it being "crammed" into one line:

y = sorted(x,  # use the canonical name as the sort key
    key=lambda item: item.split('-')[0].upper()
    )

works for me.


> Seriously, I guess this is where we differ - I think that improved
> readability _is_ a sufficient goal unto itself.  The only question is
> whether other people (OK, one other person in particular) thinks it
> improves readability, and if so, then by how much.

Personally, the only time I've ever missed the ability to create 
multi-line lambdas was when I had a series of code blocks like this:

try:
    BLOCK
except exception, e:
    process_error(e)

where the BLOCK was different in each one, and I really wanted to factor 
out the common code into a function "process" and pass each BLOCK as an 
argument:

while condition:
    process(BLOCK1)
process(BLOCK2)
if something:
    process(BLOCK3)

sort of thing.

To my mind, multi-line lambdas are really only useful for the ability to 
factor out common code, not to avoid defining a function or spreading 
lambdas out over multiple lines.



-- 
Steven D'Aprano


From cmjohnson.mailinglist at gmail.com  Wed Jul 15 02:22:34 2009
From: cmjohnson.mailinglist at gmail.com (Carl Johnson)
Date: Tue, 14 Jul 2009 14:22:34 -1000
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<fb6fbf560907121731u2578ecd1o739fa0b78d1a69ab@mail.gmail.com>
	<184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>
	<de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>
	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>
Message-ID: <3bdda690907141722m77ad6d92u490c81a4046c730f@mail.gmail.com>

It may be possible to work out the grammar so that & is never
ambiguous between "insert block here" and "and these two things
together." I'd need to think about it some more to be sure. One
advantage of using & as a sigil is that it jives with what Rubyists
are used to. That said, I have to insist that what you've proposed as

f() do:
    BLOCK

be written instead as

f(&) do:
    BLOCK

Just magically inserting the block as the last argument to a call is
something I hate about Ruby, and I think their experience has shown
that it's a mistake, since it hurts flexibility. EIBTI, etc.

To those claiming this can all be done with decorators, see another
one of my old threads: "Allow Lambda Decorators"
http://mail.python.org/pipermail/python-ideas/2009-February/002787.html

It is true that instead of using

new_list = sorted(old_list, key=&) do item:
    name = item.split("-")[0]
    return name.lower()

one might do

@list_sorted(oldlist)
def new_list(item):
    name = item.split("-")[0]
    return name.lower()

with list_sorted defined as

def list_sorted(seq):
    def inner(f):
        return sorted(seq, key=f)
    return inner

But the problem with this is that the reader of your code will see the
"@" and think "here comes the definition of a callable." It's only
after a lot of thinking that the reader will realize that new_list is
a list, not a callable. So, in terms of readability, using decorators,
though possible, is probably a mistake.

-- Carl Johnson


From cmjohnson.mailinglist at gmail.com  Wed Jul 15 02:38:34 2009
From: cmjohnson.mailinglist at gmail.com (Carl Johnson)
Date: Tue, 14 Jul 2009 14:38:34 -1000
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <200907151010.40770.steve@pearwood.info>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<200907151010.40770.steve@pearwood.info>
Message-ID: <3bdda690907141738n23168c26u854d482afcb3bba0@mail.gmail.com>

Steven D'Aprano wrote:

> The key=& seems like a mysterious Perlism -- what's the & operator doing
> in there? (Admittedly, it *might* be nice to be able to write things
> like reduce(&, alist) instead of reduce(operator.and_, alist). But
> that's another story.) "do(item):" confuses me, because it looks like
> the start of a do loop, but it isn't.

The thing about this debate that confuses me is why was the @
decorator ever approved in the first place? It's line-noise, and
worse, it's line-noise with a meaning completely different from any
other language, ever. On top of that if you do help(), you get nothing
[Ed: in Python <2.6. Good job, whoever fixed this!], and Googling is
worthless. When I first learned Python, I was very confused by
decorators--in part because I didn't know "decorator" was the term to
Google for when trying to figure out the meaning of "@"!

And yet, @ was approved, in spite of its worthlessness (it literally
adds no power to the language, just sugar) and strangeness because of
the perceived benefits to readability. And in my opinion, there are
similar cases here, where readability is improved by having the
definition of a function come after the expression to which the
function will be passed. Yet, there's a lot of resistance to this, and
I'm not entirely sure why. If it's just a problem with using "&" and
"do" other line-noise and keywords can be proposed to substitute for
it. $ has no meaning in Python, for example. If it's the lack of
function names and docstrings, there's no reason these
lambda-like-thingies can't have support for names and docstrings added
(as in my old proposal, for example). But I feel like there's some
deeper reason people don't think this is a readability win. What is
it?

-- Carl


From mwm-keyword-python.b4bdba at mired.org  Wed Jul 15 03:13:45 2009
From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer)
Date: Tue, 14 Jul 2009 21:13:45 -0400
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <3bdda690907141738n23168c26u854d482afcb3bba0@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<200907151010.40770.steve@pearwood.info>
	<3bdda690907141738n23168c26u854d482afcb3bba0@mail.gmail.com>
Message-ID: <20090714211345.284c9c0e@bhuda.mired.org>

On Tue, 14 Jul 2009 14:38:34 -1000
Carl Johnson <cmjohnson.mailinglist at gmail.com> wrote:

> Steven D'Aprano wrote:
> 
> > The key=& seems like a mysterious Perlism -- what's the & operator doing
> > in there? (Admittedly, it *might* be nice to be able to write things
> > like reduce(&, alist) instead of reduce(operator.and_, alist). But
> > that's another story.) "do(item):" confuses me, because it looks like
> > the start of a do loop, but it isn't.
> 
> The thing about this debate that confuses me is why was the @
> decorator ever approved in the first place? It's line-noise, and
> worse, it's line-noise with a meaning completely different from any
> other language, ever.

You just stumbled on it. What other language, ever, had something like
decorators? It's a magic syntax that performs magic on the following
function definition. An unusual symbol in an unusual location is a clue
that something unusual is going on.

For a more thorough discussion of the reason for the adopted syntax, read
the PEP: http://www.python.org/dev/peps/pep-0318/.

    <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


From jimjjewett at gmail.com  Wed Jul 15 04:26:42 2009
From: jimjjewett at gmail.com (Jim Jewett)
Date: Tue, 14 Jul 2009 22:26:42 -0400
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <3bdda690907141738n23168c26u854d482afcb3bba0@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<200907151010.40770.steve@pearwood.info>
	<3bdda690907141738n23168c26u854d482afcb3bba0@mail.gmail.com>
Message-ID: <fb6fbf560907141926x484b90f4h849cc3e04b783e9f@mail.gmail.com>

On 7/14/09, Carl Johnson <cmjohnson.mailinglist at gmail.com> wrote:
> Steven D'Aprano wrote:

> The thing about this debate that confuses me is why was the @
> decorator ever approved in the first place?

It wasn't, for quite a while.  I believe it was proposed for 2.2.  I'm
sure it was proposed for 2.3.  Applying it to classes as well as
functions was in the working patch -- but waited until another version
just in case.

> And yet, @ was approved, in spite of its worthlessness (it literally
> adds no power to the language, just sugar) and strangeness because of
> the perceived benefits to readability.

I have a bit of an unusual perspective; I ended up writing some drafts
of the PEP, but not because I wanted decorators.  Rather, I wanted to
avoid a really bad syntax.  (@ wasn't my favorite, but it is way
better than several of the earlier proposals -- particularly in terms
of what it doesn't assume about the future.)

Almost any syntax is a big improvement *for*the*relevant*use*cases*.
The concern is what it would do to python in general, and which future
alternatives it would cut off.

> And in my opinion, there are
> similar cases here, where readability is improved by having the
> definition of a function come after the expression to which the
> function will be passed.

Some of the examples looked like the order was an improvement.

But are there huge areas where this is needed all over the place?
(Extension frameworks such as Cocoa were mentioned for decorators.)

Is it useful across several types of programming?  (I see it as
obvious for callbacks, but maybe not so clearcut elsewhere.)

What is the cost of not doing it?  With decorators, that cost was
often delaying the wrapper for 20-30 lines, and then having to repeat
the function name three times, and then using a code style that does
confuse some people.

In this case, I think the cost of not doing it is more like "Think up
a name for the 2-3 line function, and then move the definition up
ahead of the usage."  Moving the definition is annoying, but not so
horrible with small functions, and there is no need to repeat names.
(I suppose you could argue that there is one repetition, if the
function will never be used elsewhere.)

What would the proposed syntax cost?  In the case of @, that cost was
reduced to burning a punctuation character, and explaining that
functions can be annotated -- and the annotation is more than a
comment.

For the currently proposed syntax, it confuses several issues with
suites (blocks).  It sounds like the parser can probably handle it,
but it isn't as clear that humans can.  It might well preclude later
block extensions, including more powerful versions, such as thunks or
macros.  (And it still burns a punctuation character anyhow.)

> Yet, there's a lot of resistance to this, and
> I'm not entirely sure why. If it's just a problem with using "&" and
> "do" other line-noise and keywords can be proposed to substitute for
> it.

Yes.  Or "using:", of course.  The catch is that those are all so
generic that this is only one of a zillion possible meanings -- and
for most people, not the most obvious.  A more specific keyword might
have a better chance.  (Not a *good* chance -- there is quite a high
bar for keywords -- but still *better*.)

> $ has no meaning in Python, for example.

I think it starts to with string templates.
(And I think "!" is used by some popular extensions, such as scipy,
and I think @ interfered with the leo editor, and ...)

I'm not saying it is ambiguous to the parser, but the universe of
unused ascii-punctuation isn't really all that huge.

> If it's the lack of
> function names and docstrings, there's no reason these
> lambda-like-thingies can't have support for names and docstrings added
> (as in my old proposal, for example).

That wouldn't help unless people actually used the names and
docstrings -- which again brings up the question about why existing
use of functions as first class objects isn't enough.

> But I feel like there's some
> deeper reason people don't think this is a readability win. What is
> it?

Cost-benefit.  It is harder to see the costs, because they're spread
out thinly all over the language, while the benefits are concentrated.
 But those costs are important.

In this case, it looks like the benefits are smaller than for
decorators, and the costs (at least with the proposed syntax) are
higher.  And decorators were not a shoe-in.

-jJ


From stephen at xemacs.org  Wed Jul 15 05:36:36 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 15 Jul 2009 12:36:36 +0900
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <3bdda690907141738n23168c26u854d482afcb3bba0@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<200907151010.40770.steve@pearwood.info>
	<3bdda690907141738n23168c26u854d482afcb3bba0@mail.gmail.com>
Message-ID: <874otek46z.fsf@uwakimon.sk.tsukuba.ac.jp>

Carl Johnson writes:

 > The thing about this debate that confuses me is why was the @
 > decorator ever approved in the first place?

Decorators serve the same kind of purpose that define-syntax does in
Scheme, and are similarly hygenic (which is a technical term in Scheme
that more or less means "the rope is too short to get all the way
around your neck").  They are *not* worthless to me; they allow
boilerplate to be given a name, and to be applied to transform the
behavior of a function at definition time rather than invocation time.
By giving them a syntactic implementation, it becomes possible and
encouraged to associate them with the declaration of a function,
rather than placing them at some arbitrary point later in the file.
(I can just see somebody inventing a style where all the decorators
are defined and applied at the very end of the file, for example.)

Agreed, that said, "@" doesn't evoke any association with the concept
of decoration, but that's a problem of documentation.  So I guess you
had to be there to understand it....

 > And yet, @ was approved, in spite of its worthlessness (it literally
 > adds no power to the language, just sugar)

But that is true of *all* operators.  Lisp does just fine with no
operators at all.

 > and strangeness because of the perceived benefits to readability.
 > And in my opinion, there are similar cases here, where
 > readability is improved by having the definition of a function come
 > after the expression to which the function will be passed.

Right.  That's your opinion, and as you've seen the majority of those
posting to this thread disagree that the examples given are more
readable.  Even more distressing, all though he hasn't posted, GvR
almost certainly would be included in them.

On the other hand, most Pythonistas agree that

@compute_some_boilerplate(variant)
def foo ():
    # 20 lines of code

is more readable than

def foo ():
    # 20 lines of code
foo = compute_some_boilerplate(foo, variant)

 > But I feel like there's some deeper reason people don't think this
 > is a readability win. What is it?

Some people feel that giving names to a cognitive chunk is an
essential part of the process of chunking.  They are made
uncomfortable, and often confused, by code which is named only by its
own source (aka "blocks").  Others don't feel that way.  The former
oppose this kind of proposal strongly, the latter support it more or
less actively.

Guido apparently is in the former camp.  But I think the deciding
factor most likely is that it is the opinion of the BDFL that the harm
done by allowing blocks to the cognition of the "please give it a
name" group is much greater than the convenience to the "I don't need
a name when I've got the code in front of me" group.


From george.sakkis at gmail.com  Wed Jul 15 05:50:29 2009
From: george.sakkis at gmail.com (George Sakkis)
Date: Tue, 14 Jul 2009 23:50:29 -0400
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<fb6fbf560907121731u2578ecd1o739fa0b78d1a69ab@mail.gmail.com>
	<184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>
	<de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>
	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>
Message-ID: <91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>

On Tue, Jul 14, 2009 at 3:16 PM, Paul Moore<p.f.moore at gmail.com> wrote:

> 2009/7/14 Chris Perkins <chrisperkins99 at gmail.com>:
>>
>> y = sorted(x, key=&) do(item):
>> ? ?name = item.split('-')[1]
>> ? ?return name.upper()
>>
>> I think that forcing that two-line block of code to have a name would
>> serve no purpose - it's abundantly clear what it does.
>
> That's a very telling statement. To me, that code is utterly baffling.
> (Seriously! Without analysing it line by line, it means nothing to
> me).

Seconded, there's too much going on ('&', what's "do", a function ?,
what's "item" ?, etc.). FWIW the only readable suggestion for
multiline lambdas with almost zero learning curve I have seen is the
one implemented in Boo, anonymous def and regular indentation:

y = sorted(x, key=def (item):
                               name = item.split('-')[1]
                               return name.upper()
               )

There must be somewhere posted the reason this was rejected but my
google-fu is failing me.

George

[1] http://boo.codehaus.org/Closures


From pyideas at rebertia.com  Wed Jul 15 05:55:47 2009
From: pyideas at rebertia.com (Chris Rebert)
Date: Tue, 14 Jul 2009 20:55:47 -0700
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<fb6fbf560907121731u2578ecd1o739fa0b78d1a69ab@mail.gmail.com>
	<184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>
	<de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>
	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>
	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>
Message-ID: <50697b2c0907142055g5f984dd8qb7da19730838f4c@mail.gmail.com>

On Tue, Jul 14, 2009 at 8:50 PM, George Sakkis<george.sakkis at gmail.com> wrote:
> On Tue, Jul 14, 2009 at 3:16 PM, Paul Moore<p.f.moore at gmail.com> wrote:
>
>> 2009/7/14 Chris Perkins <chrisperkins99 at gmail.com>:
>>>
>>> y = sorted(x, key=&) do(item):
>>> ? ?name = item.split('-')[1]
>>> ? ?return name.upper()
>>>
>>> I think that forcing that two-line block of code to have a name would
>>> serve no purpose - it's abundantly clear what it does.
>>
>> That's a very telling statement. To me, that code is utterly baffling.
>> (Seriously! Without analysing it line by line, it means nothing to
>> me).
>
> Seconded, there's too much going on ('&', what's "do", a function ?,
> what's "item" ?, etc.). FWIW the only readable suggestion for
> multiline lambdas with almost zero learning curve I have seen is the
> one implemented in Boo, anonymous def and regular indentation:
>
> y = sorted(x, key=def (item):
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? name = item.split('-')[1]
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? return name.upper()
> ? ? ? ? ? ? ? )
>
> There must be somewhere posted the reason this was rejected but my
> google-fu is failing me.

IIRC, it screws with the essential statement-expression dichotomy of Python:
http://unlimitednovelty.com/2009/03/indentation-sensitivity-post-mortem.html

Cheers,
Chris
--
http://blog.rebertia.com


From mwm-keyword-python.b4bdba at mired.org  Wed Jul 15 06:01:30 2009
From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer)
Date: Wed, 15 Jul 2009 00:01:30 -0400
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<fb6fbf560907121731u2578ecd1o739fa0b78d1a69ab@mail.gmail.com>
	<184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>
	<de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>
	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>
	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>
Message-ID: <20090715000130.1db9b653@bhuda.mired.org>

On Tue, 14 Jul 2009 23:50:29 -0400
George Sakkis <george.sakkis at gmail.com> wrote:
> Seconded, there's too much going on ('&', what's "do", a function ?,
> what's "item" ?, etc.). FWIW the only readable suggestion for
> multiline lambdas with almost zero learning curve I have seen is the
> one implemented in Boo, anonymous def and regular indentation:
> 
> y = sorted(x, key=def (item):
>                                name = item.split('-')[1]
>                                return name.upper()
>                )
> 
> There must be somewhere posted the reason this was rejected but my
> google-fu is failing me.

It gets ugly when carried to - well, extremes is the word I want to
use, but is wanting two or three functional arguments inlined really
"extreme"? Likewise, what happens if you want to inline a functional
argument in an already inlined function?

The current proposal seems to avoid that - by limiting the number of
function parameters to 1, which is a pretty serious shortcoming.

Pretty much all such proposals fail on one of these two issues.

       <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


From george.sakkis at gmail.com  Wed Jul 15 06:54:10 2009
From: george.sakkis at gmail.com (George Sakkis)
Date: Wed, 15 Jul 2009 00:54:10 -0400
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <50697b2c0907142055g5f984dd8qb7da19730838f4c@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>
	<de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>
	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>
	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>
	<50697b2c0907142055g5f984dd8qb7da19730838f4c@mail.gmail.com>
Message-ID: <91ad5bf80907142154r6a4e6a57l482c3ca4e5083009@mail.gmail.com>

On Tue, Jul 14, 2009 at 11:55 PM, Chris Rebert<pyideas at rebertia.com> wrote:
> On Tue, Jul 14, 2009 at 8:50 PM, George Sakkis<george.sakkis at gmail.com> wrote:
>> On Tue, Jul 14, 2009 at 3:16 PM, Paul Moore<p.f.moore at gmail.com> wrote:
>>
>>> 2009/7/14 Chris Perkins <chrisperkins99 at gmail.com>:
>>>>
>>>> y = sorted(x, key=&) do(item):
>>>> ? ?name = item.split('-')[1]
>>>> ? ?return name.upper()
>>>>
>>>> I think that forcing that two-line block of code to have a name would
>>>> serve no purpose - it's abundantly clear what it does.
>>>
>>> That's a very telling statement. To me, that code is utterly baffling.
>>> (Seriously! Without analysing it line by line, it means nothing to
>>> me).
>>
>> Seconded, there's too much going on ('&', what's "do", a function ?,
>> what's "item" ?, etc.). FWIW the only readable suggestion for
>> multiline lambdas with almost zero learning curve I have seen is the
>> one implemented in Boo, anonymous def and regular indentation:
>>
>> y = sorted(x, key=def (item):
>> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? name = item.split('-')[1]
>> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? return name.upper()
>> ? ? ? ? ? ? ? )
>>
>> There must be somewhere posted the reason this was rejected but my
>> google-fu is failing me.
>
> IIRC, it screws with the essential statement-expression dichotomy of Python:
> http://unlimitednovelty.com/2009/03/indentation-sensitivity-post-mortem.html

Thanks, this was it. The important part is Guido's quote "any solution
that embeds an indentation-based block in the middle of an expression
is unacceptable", so it's basically a design judgement, not an
implementation constraint.

George


From cmjohnson.mailinglist at gmail.com  Wed Jul 15 07:51:07 2009
From: cmjohnson.mailinglist at gmail.com (Carl Johnson)
Date: Tue, 14 Jul 2009 19:51:07 -1000
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<fb6fbf560907121731u2578ecd1o739fa0b78d1a69ab@mail.gmail.com>
	<184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>
	<de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>
	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>
	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>
Message-ID: <3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>

George Sakkis wrote:

> Seconded, there's too much going on ('&', what's "do", a function ?,
> what's "item" ?, etc.). FWIW the only readable suggestion for
> multiline lambdas with almost zero learning curve I have seen is the
> one implemented in Boo, anonymous def and regular indentation:
>
> y = sorted(x, key=def (item):
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? name = item.split('-')[1]
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? return name.upper()
> ? ? ? ? ? ? ? )

FWIW, I find that completely ugly and non-Python-esque (unpythonic
being perhaps too strong of criticism). :-D

Clearly this is an issue which has a lot of good arguments on both
sides, and since the final metric has to be "readability of code" it's
going to hard to make a knock-down argument for one side or the other.
I'm fine with the status quo for now: in Guido's gut we trust.

My suggestion for future would be language extenders is that they try
to solve a broader problem than just the "how do I make callbacks more
convenient" problem. Just solving that one problem by itself seems
unlikely to make it into the language, since it is pure "sugar" and
Pythoneers have a well justified wariness toward anonymous functions
(why not just name them and be done with it?). Perhaps someone should
take a look at the original "with" proposal, or some of the various
"do" and "block" proposals. One idea I think about sometimes is having
a more convenient way to do something along the lines of
metaclass.__prepare__ but for functions?

-- Carl Johnson


From greg.ewing at canterbury.ac.nz  Wed Jul 15 09:55:15 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 15 Jul 2009 19:55:15 +1200
Subject: [Python-ideas] Where-statement (Proposal for function expressions)
In-Reply-To: <3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<fb6fbf560907121731u2578ecd1o739fa0b78d1a69ab@mail.gmail.com>
	<184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>
	<de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>
	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>
	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>
	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>
Message-ID: <4A5D8B63.4040509@canterbury.ac.nz>

Carl Johnson wrote:

> My suggestion for future would be language extenders is that they try
> to solve a broader problem than just the "how do I make callbacks more
> convenient" problem.

One such extension might be a "where" block. Applied
to the current problem:

   foo(f) where:
     def f(x):
       ...

Benefits include:

* It's pretty much self-explanatory, as it builds on
   the existing syntax for defining functions, and
   "where" is used in a similar way in mathematics and
   some existing programming languages.

* You get to give the function a meaningful name if
   you want.

* It's not restricted to a single function argument:

   def foo(f, g) where:
     def f(x):
       ...
     def g(y):
       ...

* It needn't be restricted to functions:

   foo(x, y) where:
     x = a * 42
     y = b / 17

Drawbacks include:

* Execution happens out of order, although that's an
   inherent feature of the original proposal as well.
   There are precedents in the language already, such
   as list comprehensions and conditional expressions.

* Allowing arbitrary statements in the block could possibly
   be considered detrimental to code readability. To mitigate
   that, the block contents could perhaps be restricted to
   binding constructs only.

-- 
Greg


From p.f.moore at gmail.com  Wed Jul 15 10:27:11 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 15 Jul 2009 09:27:11 +0100
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>
	<de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>
	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>
	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>
	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>
Message-ID: <79990c6b0907150127l1ee15976p8f0f17ec8834a113@mail.gmail.com>

2009/7/15 Carl Johnson <cmjohnson.mailinglist at gmail.com>:
> My suggestion for future would be language extenders is that they try
> to solve a broader problem than just the "how do I make callbacks more
> convenient" problem. Just solving that one problem by itself seems
> unlikely to make it into the language, since it is pure "sugar" and
> Pythoneers have a well justified wariness toward anonymous functions
> (why not just name them and be done with it?). Perhaps someone should
> take a look at the original "with" proposal, or some of the various
> "do" and "block" proposals. One idea I think about sometimes is having
> a more convenient way to do something along the lines of
> metaclass.__prepare__ but for functions?

I agree entirely. There's certainly *something* here worthy of
consideration, otherwise it wouldn't keep coming up. But there's
probably a more general construct (yes, I know, "fully general
anonymous functions" - maybe there's something *else*...?) lurking
underneath all of the proposals, and it's worth waiting until someone
discovers it.

The important principle that George quoted:

2009/7/15 George Sakkis <george.sakkis at gmail.com>:
> Thanks, this was it. The important part is Guido's quote "any solution
> that embeds an indentation-based block in the middle of an expression
> is unacceptable", so it's basically a design judgement, not an
> implementation constraint.

is key here - keep blocks and expressions distinct. Future proposals
should keep that in mind.

Maybe there's value in having a "language design principles" PEP where
pronouncements like this could be recorded, for the benefit of people
proposing new features?

Paul.


From cmjohnson.mailinglist at gmail.com  Wed Jul 15 11:59:39 2009
From: cmjohnson.mailinglist at gmail.com (Carl Johnson)
Date: Tue, 14 Jul 2009 23:59:39 -1000
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <4A5D8B63.4040509@canterbury.ac.nz>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>
	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>
	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>
	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>
	<4A5D8B63.4040509@canterbury.ac.nz>
Message-ID: <3bdda690907150259q314ded3eoa4b8c30823261e39@mail.gmail.com>

Greg Ewing wrote:

Nitpicking:

> * It's not restricted to a single function argument:
>
> ?def foo(f, g) where:
> ? ?def f(x):
> ? ? ?...
> ? ?def g(y):
> ? ? ?...
>

Surely, you meant

foo(f, g) where:
   def f(x):
     ...
   def g(y):
     ...

Ending a statement that ends in a ":" normally with a "where" is too
ambiguous and ought to be prohibited.

I need to think about this proposal some more, but it would help solve
the issue brought up on the list recently that [f(x) for f in fs if
f(x)] is inefficient and [i for i in (f(x) for f in fs) if i] is ugly.

new_list = [fx for f in fs if fx] where:
    fx = f(x)

The "where" on the end of the clause does catch your eye and make you
think "OK, be aware, something is undefined on this line but will be
defined on the next."

OTOH, in

new_list = [i for i in xs
              for j in ys if fx] where:
    fx = f(x)

or

new_list = [i for i in [j for j in ys if fx?] if fx?] where:
    fx = f(x)

Does the "where" get re-used with each loop of the inner list
comprehension or just the outer one?

What about

total = sum(count for item in l) where:
    count = item.size if item.size > 0 else 1

It seems like for that case, we definitely want the "where" to apply
every time the generator expression loops though, but? Is that
feasible?

Hmm.

-- Carl Johnson


From p.f.moore at gmail.com  Wed Jul 15 12:25:04 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 15 Jul 2009 11:25:04 +0100
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <3bdda690907150259q314ded3eoa4b8c30823261e39@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>
	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>
	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>
	<4A5D8B63.4040509@canterbury.ac.nz>
	<3bdda690907150259q314ded3eoa4b8c30823261e39@mail.gmail.com>
Message-ID: <79990c6b0907150325k6c648994id7cee7edec745a65@mail.gmail.com>

2009/7/15 Carl Johnson <cmjohnson.mailinglist at gmail.com>:
> It seems like for that case, we definitely want the "where" to apply
> every time the generator expression loops though, but? Is that
> feasible?

To make the form follow the intent, the where block would need to be
inside the [...] - which clearly violates the "no indented block
structure within expressions" design rule.

My initial instinct was that the original form violated that rule, too
- but if you interpret the where block as a statement postfix, you're
(almost) OK. The (almost) bit is because the where block has to be
limited to simple (single-line) statements.

But with those restrictions, the semantics become

stmt where:
   a
   b
   c

=>

a
b
c
stmt

with the proviso that variables defined in a,b,c are only available
within a, b, c and stmt.

But of course that immediately prompts the question:

x = 1
y = x where:
    x = 2
print x

What does that print?

x = []
y = 1 where:
    x.append(1)
print x

What about that?

x = []
y = x
z = 1 where:
  x.append(1)
print x, y, x is y

And what about that???

And before someone points out Greg's proposal to limit the construct
to "binding statements", please define (clearly) what is a "binding
statement". And then explain how that helps if I replace x.append(1)
with _ = x.append(1) in the above...

I sort of like the idea, but it needs much better defined syntax and
semantics if it's going to work.

Paul.


From clockworksaint at gmail.com  Wed Jul 15 12:45:05 2009
From: clockworksaint at gmail.com (Weeble)
Date: Wed, 15 Jul 2009 12:45:05 +0200
Subject: [Python-ideas] Where-statement (Proposal for function expressions)
Message-ID: <13e3f9930907150345i22d1ba7chd4d5e97f3469f5e@mail.gmail.com>

> Date: Wed, 15 Jul 2009 19:55:15 +1200
> From: Greg Ewing <greg.ewing at canterbury.ac.nz>
> To: python-ideas at python.org
> Subject: [Python-ideas] Where-statement (Proposal for function
> ? ? ? ?expressions)
> Message-ID: <4A5D8B63.4040509 at canterbury.ac.nz>
> Content-Type: text/plain; charset=UTF-8; format=flowed
>
> Carl Johnson wrote:
>
>> My suggestion for future would be language extenders is that they try
>> to solve a broader problem than just the "how do I make callbacks more
>> convenient" problem.
>
> One such extension might be a "where" block. Applied
> to the current problem:
>
> ? foo(f) where:
> ? ? def f(x):
> ? ? ? ...

I was just thinking of something similar:

in:
    foo(f,a)
let:
    def f(x):
        ...
    a = ...

(Keywords open to debate.) It's a little verbose, but I dislike it
much less than all the proposals that introduce special sigils or do
weird things to the syntax of the code. Things I like about it:
there's a clear warning (the "in:") right at the start that the
statement (or block?) is special and relies on code that comes after;
the syntax isn't complex; both lets you provide a name and prevents it
leaking into places you don't want it; and the contents of both the
"in:" block and the "let:" block are normal Python. I'd expect the
behaviour to be something like this: execute the "let:" block in its
own nested scope, then execute the "in:" block in the regular local
scope but with the symbols defined by the "let:" block available. Does
that make some sort of sense?

Most of the things I like are achieved by Greg's "where:" block, the
only thing I don't like about it is tacking a keyword onto the end of
a statement, which feels a bit awkward to me. That may just be
aesthetics. I certainly like the "where:" idea a lot better than the
other proposals. It seems to achieve the most utility with the least
confusing syntax.


From chrisperkins99 at gmail.com  Wed Jul 15 13:27:08 2009
From: chrisperkins99 at gmail.com (Chris Perkins)
Date: Wed, 15 Jul 2009 07:27:08 -0400
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <79990c6b0907150127l1ee15976p8f0f17ec8834a113@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>
	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>
	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>
	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>
	<79990c6b0907150127l1ee15976p8f0f17ec8834a113@mail.gmail.com>
Message-ID: <184a9f5a0907150427x38e01f45o31de86ce3931bd96@mail.gmail.com>

On Wed, Jul 15, 2009 at 4:27 AM, Paul Moore<p.f.moore at gmail.com> wrote:
>
> The important principle that George quoted:
>
> 2009/7/15 George Sakkis <george.sakkis at gmail.com>:
>> Thanks, this was it. The important part is Guido's quote "any solution
>> that embeds an indentation-based block in the middle of an expression
>> is unacceptable", so it's basically a design judgement, not an
>> implementation constraint.
>
> is key here - keep blocks and expressions distinct. Future proposals
> should keep that in mind.

I think you may be misinterpreting - note Guido's use of the word
"middle".  That is precisely why I proposed allowing blocks only at
the _end_ of expressions, and not in the middle.  OTOH, I may just be
projecting the interpretation of that statement that suits me best. :)

To answer some of the other points that have been raised:

Steven:
>Did I understand correctly that you prohibited using `if`, `while` and
>other block statements inside the do block?
No, you misunderstood. What you are thinking of is the fact that the
test expression in an if block, or a while block, cannot have an
appended block. A block is really just syntactic sugar for a def - it
lacks only a name, and the ability to be decorated. It can have a
docstring, it can be a generator, etc.

Steven:
>Personally, the only time I've ever missed the ability to create
>multi-line lambdas was when I had a series of code blocks like this:
>
>try:
>   BLOCK
>except exception, e:
>   process_error(e)
>
>where the BLOCK was different in each one, and I really wanted to factor
>out the common code into a function "process" and pass each BLOCK as an
>argument:
>
>while condition:
>   process(BLOCK1)
>process(BLOCK2)
>if something:
>   process(BLOCK3)
>
>sort of thing.

Do you mean something like this?
def safely(body):
    try:
        body()
    except exception, e:
        process_error(e)

while condition:
    safely() do:
        BLOCK1
safely() do:
    BLOCK2
if something:
    safely() do:
        BLOCK3

I don't claim that a pattern like that will work in all cases, because
the body of the blocks have their own local scope. If you need to set
a variable in the enclosing scope, it might not do what you want (but
we do have "nonlocal" now, so that helps).

Carl:
>That said, I have to insist that what you've proposed as
>
>f() do:
>   BLOCK
>
>be written instead as
>
>f(&) do:
>   BLOCK
>
>Just magically inserting the block as the last argument to a call is
>something I hate about Ruby

Sure, that's OK with me. I also expected the argument that the empty
parentheses after "do" should not be optional, for symmetry with def,
and that's fine with me to. I was just trying to keep the syntax as
"lightweight" as possible for the simplest cases.

Stephen:
>Some people feel that giving names to a cognitive chunk is an
>essential part of the process of chunking.  They are made
>uncomfortable, and often confused, by code which is named only by its
>own source (aka "blocks").  Others don't feel that way.  The former
>oppose this kind of proposal strongly, the latter support it more or
>less actively.

Agreed, and I do have a slight nagging worry that blocks could be
abused, in that people could use them in cases where a named function
would be better (and I agree that there will always be many such
cases). But then I remember that "we're all adults here", and that in
practice people can just as easily, and often do, give useless names
to small functions, and then I don't worry about it so much anymore.

Carl:
>If it's the lack of function names and docstrings

Python 2.7a0 (trunk, Jul 12 2009, 07:54:12) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> def f(b): return b.__doc__
...
>>> f() do:
...     "Look, ma, I'm a docstring!"
...
"Look, ma, I'm a docstring!"
>>>

Overall, I think we're at the point where most of us agree that this
is a very subjective matter, and it comes down to taste. Guido has a
well-recorded history of saying (paraphrasing) "just write a def with
a name and quit whining!", so I suppose the odds for this proposal are
not great. But I have certainly enjoyed the discussion, so thanks to
everyone who has participated.

Chris Perkins


From daniel at stutzbachenterprises.com  Wed Jul 15 14:02:10 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Wed, 15 Jul 2009 07:02:10 -0500
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <3bdda690907150259q314ded3eoa4b8c30823261e39@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>
	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>
	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>
	<4A5D8B63.4040509@canterbury.ac.nz>
	<3bdda690907150259q314ded3eoa4b8c30823261e39@mail.gmail.com>
Message-ID: <eae285400907150502j5e2a289cy8b1a70ababe46f04@mail.gmail.com>

On Wed, Jul 15, 2009 at 4:59 AM, Carl Johnson <
cmjohnson.mailinglist at gmail.com> wrote:

> total = sum(count for item in l) where:
>    count = item.size if item.size > 0 else 1
>
> It seems like for that case, we definitely want the "where" to apply
> every time the generator expression loops though, but? Is that
> feasible?
>

Unlikely, but this is:

total = sum(f(item) for item in lst) where:
    def f(item):
        return item.size if item.size > 0 else 1

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090715/3dc179ff/attachment.html>

From daniel at stutzbachenterprises.com  Wed Jul 15 14:49:36 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Wed, 15 Jul 2009 07:49:36 -0500
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <4A5D8B63.4040509@canterbury.ac.nz>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>
	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>
	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>
	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>
	<4A5D8B63.4040509@canterbury.ac.nz>
Message-ID: <eae285400907150549k429f1aeha8ab1b85f581eb7@mail.gmail.com>

On Wed, Jul 15, 2009 at 2:55 AM, Greg Ewing <greg.ewing at canterbury.ac.nz>wrote:

> One such extension might be a "where" block. Applied
> to the current problem:
>
>  foo(f) where:
>    def f(x):
>      ...
>

I like this proposal much more than any of the previous proposals.  It has
some of the flavor of Lisp's "let", but puts the important line first
instead of at the end.  In a nutshell, it says "define these symbols for
this one line only".  On the downside, for the common case of wanting to
define a single function, the body of the function must be indented twice.

I suggest the following grammar and meaning, which would be referred to by
the assignment, yield, return, and expression statements:

where_expression ::= expression_list "where" ":" suite | expression_list


It evaluates as follows.

[return|yield|x=] expression_list where:
   suite

is roughly equivalent to:

def where_expression():
    suite
    return expression_list
[return|yield|x=] expression_list()


How about the following as additional syntactic sugar for the common
one-function case?

x = blah(f) where def f(item):
   body_of_f

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090715/cb93c799/attachment.html>

From gerald.britton at gmail.com  Wed Jul 15 15:38:07 2009
From: gerald.britton at gmail.com (Gerald Britton)
Date: Wed, 15 Jul 2009 09:38:07 -0400
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <eae285400907150549k429f1aeha8ab1b85f581eb7@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com> 
	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com> 
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com> 
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com> 
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com> 
	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com> 
	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com> 
	<4A5D8B63.4040509@canterbury.ac.nz>
	<eae285400907150549k429f1aeha8ab1b85f581eb7@mail.gmail.com>
Message-ID: <5d1a32000907150638j65131770gf26df83dbadc5af2@mail.gmail.com>

Why not just use the Haskell approach?

foo(x,y) = myfunc(bar) where:
           myfunc, bar = f(x), g(y) where:
              f,g = func1, func2

assuming func1 and func2 were previously defined, otherwise another
"where" clause could follow. That way, introducing the "where" clause
effectively causes foo to be defined as a function.  Applying that to
the previous example:

l = [i for i in myiterator if fi] where:
           f = f(i)

if "myiterator" was previously defined, or even:

 l = [i for i in myiterator if f(i)] where:
           myiterator, f = (i for i in xrange(10)), bool

if "myiterator" was not previously defined.

On Wed, Jul 15, 2009 at 8:49 AM, Daniel
Stutzbach<daniel at stutzbachenterprises.com> wrote:
> On Wed, Jul 15, 2009 at 2:55 AM, Greg Ewing <greg.ewing at canterbury.ac.nz>
> wrote:
>>
>> One such extension might be a "where" block. Applied
>> to the current problem:
>>
>> ?foo(f) where:
>> ? ?def f(x):
>> ? ? ?...
>
> I like this proposal much more than any of the previous proposals.? It has
> some of the flavor of Lisp's "let", but puts the important line first
> instead of at the end.? In a nutshell, it says "define these symbols for
> this one line only".? On the downside, for the common case of wanting to
> define a single function, the body of the function must be indented twice.
>
> I suggest the following grammar and meaning, which would be referred to by
> the assignment, yield, return, and expression statements:
>
> where_expression ::= expression_list "where" ":" suite | expression_list
>
>
> It evaluates as follows.
>
> [return|yield|x=] expression_list where:
> ?? suite
>
> is roughly equivalent to:
>
> def where_expression():
> ??? suite
> ??? return expression_list
> [return|yield|x=] expression_list()
>
>
> How about the following as additional syntactic sugar for the common
> one-function case?
>
> x = blah(f) where def f(item):
> ?? body_of_f
>
> --
> Daniel Stutzbach, Ph.D.
> President, Stutzbach Enterprises, LLC
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
>



-- 
Gerald Britton


From ncoghlan at gmail.com  Wed Jul 15 15:55:03 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 15 Jul 2009 23:55:03 +1000
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>	<fb6fbf560907121731u2578ecd1o739fa0b78d1a69ab@mail.gmail.com>	<184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>	<de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
Message-ID: <4A5DDFB7.8030800@gmail.com>

Paul Moore wrote:
> (note that I could make my version look a bit nicer by a "from
> functools import partial", so you're still not allowed to argue
> aesthetics :-))?

Make that a "from functools import partial as do" and you get:

@do(fn)
def _(...):
    stmt1
    stmt2

@do(fn, a, b, c)
def _(...):
    stmt1
    stmt2

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From aahz at pythoncraft.com  Wed Jul 15 17:30:12 2009
From: aahz at pythoncraft.com (Aahz)
Date: Wed, 15 Jul 2009 08:30:12 -0700
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <79990c6b0907150127l1ee15976p8f0f17ec8834a113@mail.gmail.com>
References: <184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>
	<de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>
	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>
	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>
	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>
	<79990c6b0907150127l1ee15976p8f0f17ec8834a113@mail.gmail.com>
Message-ID: <20090715153012.GB21135@panix.com>

On Wed, Jul 15, 2009, Paul Moore wrote:
>
> Maybe there's value in having a "language design principles" PEP where
> pronouncements like this could be recorded, for the benefit of people
> proposing new features?

Sure, although PEP 20 seems like an obvious spot.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"If you think it's expensive to hire a professional to do the job, wait
until you hire an amateur."  --Red Adair


From zuo at chopin.edu.pl  Wed Jul 15 19:21:19 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Wed, 15 Jul 2009 19:21:19 +0200
Subject: [Python-ideas] Where-statement (Proposal for function
 expressions)
In-Reply-To: <79990c6b0907150325k6c648994id7cee7edec745a65@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>
	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>
	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>
	<4A5D8B63.4040509@canterbury.ac.nz>
	<3bdda690907150259q314ded3eoa4b8c30823261e39@mail.gmail.com>
	<79990c6b0907150325k6c648994id7cee7edec745a65@mail.gmail.com>
Message-ID: <op.uw4c5pu3fvx12t@jerzozwiesz.home.aster.pl>

Hello,

15-07-2009, 09:55 Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> One such extension might be a "where" block. Applied
> to the current problem:
>
>    foo(f) where:
>      def f(x):
>        ...

At first glance, +1 from me.

15-07-2009, 12:25 Paul Moore <p.f.moore at gmail.com> wrote:

> inside the [...] - which clearly violates the "no indented block
> structure within expressions" design rule.
>
> My initial instinct was that the original form violated that rule, too
> - but if you interpret the where block as a statement postfix, you're
> (almost) OK. The (almost) bit is because the where block has to be
> limited to simple (single-line) statements.

> stmt where:
>    a
>    b
>    c
> =>
>
> a
> b
> c
> stmt
>
> with the proviso that variables defined in a,b,c are only available
> within a, b, c and stmt.

I don't like the direction of limiting content of the block to set of  
singular simple statements. I suppose where-block should simply create a  
separate nested scope.

So...

> x = 1
> y = x where:
>     x = 2
> print x
>
> What does that print?

I tkink it should print 1 -- because, if where-block creates a separate  
nested scope, its local variables shouldn't leak.

But there is another question: what is the value of y: 1 or 2? Or, more  
genarally: should be the line ended with 'where' included *in* that nested  
scope or not? I suppose it should, so the final value of y should be 2.

Another example:

a = 7
b = 8
x = 'X'
z = 'zzzzz'

foo(a, b, c("It's..."), x) where:
     # a = input() in Python 3.x :)
     a = raw_input(z)  # -> zzzzz
     try:
         a = int(a)
     except ValueError:
         a = 0
         def b(bar):
             'very complex function'
             print(a, bar)
     else:
         def b(bar):
             'also very complex function'
             print(bar, a, bar)

     def c(*args, **kwargs):
         'also also very complex function'

print(a, b)  # -> 7 8


Best regards,

-- 
Jan Kaliszewski <zuo at chopin.edu.pl>


From zuo at chopin.edu.pl  Wed Jul 15 19:38:10 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Wed, 15 Jul 2009 19:38:10 +0200
Subject: [Python-ideas] Where-statement (Proposal for function
 expressions)
In-Reply-To: <op.uw4c5pu3fvx12t@jerzozwiesz.home.aster.pl>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>
	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>
	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>
	<4A5D8B63.4040509@canterbury.ac.nz>
	<3bdda690907150259q314ded3eoa4b8c30823261e39@mail.gmail.com>
	<79990c6b0907150325k6c648994id7cee7edec745a65@mail.gmail.com>
	<op.uw4c5pu3fvx12t@jerzozwiesz.home.aster.pl>
Message-ID: <op.uw4dxwlefvx12t@jerzozwiesz.home.aster.pl>

15-07-2009 o 19:21 Jan Kaliszewski <zuo at chopin.edu.pl> wrote:

> I don't like the direction of limiting content of the block to set of  
> singular simple statements. I suppose where-block should simply create a  
> separate nested scope.

...Because IMHO an indented block suggests that the content is a sequence  
of statements creating a piece of code without unnatural limitations (like  
in class or function body).

*j

-- 
Jan Kaliszewski <zuo at chopin.edu.pl>


From jkwon.work at gmail.com  Wed Jul 15 21:36:22 2009
From: jkwon.work at gmail.com (Jae Kwon)
Date: Wed, 15 Jul 2009 12:36:22 -0700
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <4A5D8B63.4040509@canterbury.ac.nz>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<fb6fbf560907121731u2578ecd1o739fa0b78d1a69ab@mail.gmail.com>
	<184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>
	<de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>
	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>
	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>
	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>
	<4A5D8B63.4040509@canterbury.ac.nz>
Message-ID: <2BD5F2FB-2ACA-4203-A24F-BBE4E1AAE58F@gmail.com>

i like defining things prior to running them. One issue, however, is  
namespace management. i'd like temporary names to be discarded  
automatically.

where:
     def foo(x,y,z):
         return x+y+z
do:
     bar = foo(1,2,3)

print foo(1,2,3) // foo is undefined!
print bar // 6

you could create a local namespace w/ just the 'where' block.

where:
     x = 1
     y = x

print x // x is undefined!


Perhaps we could bind the variables and pass them around.

where my_locals:
     x = 1
     y = 2

print my_locals() // {'x': 1, 'y': 2}
print x // x is undefined!

  - Jae

On Jul 15, 2009, at 12:55 AM, Greg Ewing wrote:

> Carl Johnson wrote:
>
>> My suggestion for future would be language extenders is that they try
>> to solve a broader problem than just the "how do I make callbacks  
>> more
>> convenient" problem.
>
> One such extension might be a "where" block. Applied
> to the current problem:
>
>  foo(f) where:
>    def f(x):
>      ...
>
> Benefits include:
>
> * It's pretty much self-explanatory, as it builds on
>  the existing syntax for defining functions, and
>  "where" is used in a similar way in mathematics and
>  some existing programming languages.
>
> * You get to give the function a meaningful name if
>  you want.
>
> * It's not restricted to a single function argument:
>
>  def foo(f, g) where:
>    def f(x):
>      ...
>    def g(y):
>      ...
>
> * It needn't be restricted to functions:
>
>  foo(x, y) where:
>    x = a * 42
>    y = b / 17
>
> Drawbacks include:
>
> * Execution happens out of order, although that's an
>  inherent feature of the original proposal as well.
>  There are precedents in the language already, such
>  as list comprehensions and conditional expressions.
>
> * Allowing arbitrary statements in the block could possibly
>  be considered detrimental to code readability. To mitigate
>  that, the block contents could perhaps be restricted to
>  binding constructs only.
>
> -- 
> Greg
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas



From pyideas at rebertia.com  Wed Jul 15 23:16:35 2009
From: pyideas at rebertia.com (Chris Rebert)
Date: Wed, 15 Jul 2009 14:16:35 -0700
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <20090715153012.GB21135@panix.com>
References: <184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>
	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>
	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>
	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>
	<79990c6b0907150127l1ee15976p8f0f17ec8834a113@mail.gmail.com>
	<20090715153012.GB21135@panix.com>
Message-ID: <50697b2c0907151416l2fc82807wd2a6e64e9ec36926@mail.gmail.com>

On Wed, Jul 15, 2009 at 8:30 AM, Aahz<aahz at pythoncraft.com> wrote:
> On Wed, Jul 15, 2009, Paul Moore wrote:
>>
>> Maybe there's value in having a "language design principles" PEP where
>> pronouncements like this could be recorded, for the benefit of people
>> proposing new features?
>
> Sure, although PEP 20 seems like an obvious spot.

Also, PEP 3099, although it's more about prohibitions than principles:
http://www.python.org/dev/peps/pep-3099/

Cheers,
Chris
-- 
http://blog.rebertia.com


From Scott.Daniels at Acm.Org  Thu Jul 16 00:43:28 2009
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Wed, 15 Jul 2009 15:43:28 -0700
Subject: [Python-ideas] Proposal for function expressions
In-Reply-To: <4A5DDFB7.8030800@gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>	<fb6fbf560907121731u2578ecd1o739fa0b78d1a69ab@mail.gmail.com>	<184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>	<de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<4A5DDFB7.8030800@gmail.com>
Message-ID: <h3llka$qvg$1@ger.gmane.org>

Nick Coghlan wrote:
...
> from functools import partial as do
> 
> @do(fn)
^^^ The moral equivalent of add_callback(lambda x: function(x)) ^^^
> def _(...):
>     stmt1
>     ...

--Scott David Daniels
Scott.Daniels at Acm.Org



From zuo at chopin.edu.pl  Thu Jul 16 00:50:06 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Thu, 16 Jul 2009 00:50:06 +0200
Subject: [Python-ideas] Immemorial desire for "do-while"-like construction
Message-ID: <op.uw4sdsxffvx12t@jerzozwiesz.home.aster.pl>

Hello,

The issue has been coming back repeatedly:

How we could elegantly and pythonic'ly avoid repeating ourselves if
we need the control flow structure that e.g. in PASCAL has the form:

repeat
...
until CONDITION

In Python we must use while-loop in not very DRY (Don't Repeat
Yourself) manner:

SOME
ACTIONS
HERE
...
while CONDITION:
       THE SAME
       ACTIONS
       AGAIN
       ...


(See also:
http://www.python.org/dev/peps/pep-0315/
http://mail.python.org/pipermail/python-dev/2006-February/060718.html
http://preview.tinyurl.com/nodgwt
)

Maybe here the simplest approach can lead us to the solution? I.e.:

repeat while CONDITION:
       SOME
       ACTIONS
       HERE
       ...

In human language it'd mean: do ACTIONS once unconditionally, then
*repeat* them while the CONDITION is true.

Additional advantage is that probably for many people associate
'repeat' with such construction (from experience with other languages).

What do you think about the idea?

-- 
Jan Kaliszewski <zuo at chopin.edu.pl>


From pyideas at rebertia.com  Thu Jul 16 00:54:07 2009
From: pyideas at rebertia.com (Chris Rebert)
Date: Wed, 15 Jul 2009 15:54:07 -0700
Subject: [Python-ideas] Immemorial desire for "do-while"-like
	construction
In-Reply-To: <op.uw4sdsxffvx12t@jerzozwiesz.home.aster.pl>
References: <op.uw4sdsxffvx12t@jerzozwiesz.home.aster.pl>
Message-ID: <50697b2c0907151554p2175d631s5d983b734d4164d3@mail.gmail.com>

On Wed, Jul 15, 2009 at 3:50 PM, Jan Kaliszewski<zuo at chopin.edu.pl> wrote:
> Hello,
>
> The issue has been coming back repeatedly:
>
> How we could elegantly and pythonic'ly avoid repeating ourselves if
> we need the control flow structure that e.g. in PASCAL has the form:
>
> repeat
> ...
> until CONDITION
>
> In Python we must use while-loop in not very DRY (Don't Repeat
> Yourself) manner:
>
> SOME
> ACTIONS
> HERE
> ...
> while CONDITION:
> ? ? ?THE SAME
> ? ? ?ACTIONS
> ? ? ?AGAIN
> ? ? ?...

You can fix that by just writing it as:

while True:
    SOME
    ACTIONS
    HERE
    if not CONDITION: break

Cheers,
Chris
-- 
http://blog.rebertia.com


From cmjohnson.mailinglist at gmail.com  Thu Jul 16 01:12:17 2009
From: cmjohnson.mailinglist at gmail.com (Carl Johnson)
Date: Wed, 15 Jul 2009 13:12:17 -1000
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>
	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>
	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>
	<4A5D8B63.4040509@canterbury.ac.nz>
	<eae285400907150549k429f1aeha8ab1b85f581eb7@mail.gmail.com>
	<5d1a32000907150638j65131770gf26df83dbadc5af2@mail.gmail.com>
	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>
Message-ID: <3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>

Daniel Stutzbach wrote:

> Unlikely, but this is:
>
> total =3D sum(f(item) for item in lst) where:
>     def f(item):
>         return item.size if item.size > 0 else 1

Yes, that seems much more sensible than trying to repeat the where's
inside of list comp or gen exp.

Daniel Stutzbach wrote:

> How about the following as additional syntactic sugar for the common one-=
function case?
>
> x =3D blah(f) where def f(item):
>    body_of_f

I like it.

Gerald Britton wrote:

> Why not just use the Haskell approach?
>
> foo(x,y) =3D myfunc(bar) where:
> =A0 =A0 =A0 =A0 =A0 myfunc, bar =3D f(x), g(y) where:
> =A0 =A0 =A0 =A0 =A0 =A0 =A0f,g =3D func1, func2

Because I've been looking at that for a couple minutes and still have
no idea what it is supposed to mean. :-( List comprehensions and
pattern matching are the only Haskell features that make any sense to
me.

Jan Kaliszewski wrote:

> I don't like the direction of limiting content of the block to set of sin=
gular simple statements. I suppose where-block should simply create a separ=
ate nested scope.

Yes. It strikes me that one of the problems with Ruby is that there
are different scoping rules for blocks, lambda, methods, procs, and
whatever else=85 It would be much easier for users if "where" blocks had
the exact same scoping rules as functions. Daniel Stutzbach's
suggested equivalence strikes me as exactly correct:

[return|yield|x=3D] expression_list where:
   suite

is roughly equivalent to:

def where_expression():
    suite
    return expression_list
[return|yield|x=3D] where_expression()


-- Carl Johnson


From python at mrabarnett.plus.com  Thu Jul 16 01:13:58 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 16 Jul 2009 00:13:58 +0100
Subject: [Python-ideas] Immemorial desire for
	"do-while"-like	construction
In-Reply-To: <50697b2c0907151554p2175d631s5d983b734d4164d3@mail.gmail.com>
References: <op.uw4sdsxffvx12t@jerzozwiesz.home.aster.pl>
	<50697b2c0907151554p2175d631s5d983b734d4164d3@mail.gmail.com>
Message-ID: <4A5E62B6.90408@mrabarnett.plus.com>

Chris Rebert wrote:
> On Wed, Jul 15, 2009 at 3:50 PM, Jan Kaliszewski<zuo at chopin.edu.pl> wrote:
>> Hello,
>>
>> The issue has been coming back repeatedly:
>>
>> How we could elegantly and pythonic'ly avoid repeating ourselves if
>> we need the control flow structure that e.g. in PASCAL has the form:
>>
>> repeat
>> ...
>> until CONDITION
>>
>> In Python we must use while-loop in not very DRY (Don't Repeat
>> Yourself) manner:
>>
>> SOME
>> ACTIONS
>> HERE
>> ...
>> while CONDITION:
>>      THE SAME
>>      ACTIONS
>>      AGAIN
>>      ...
> 
> You can fix that by just writing it as:
> 
> while True:
>     SOME
>     ACTIONS
>     HERE
>     if not CONDITION: break
> 
If we were going to add 'repeat' then it could be:

     repeat:
         SOME
         ACTIONS
         HERE
     while CONDITION

If 'while' starts a loop it ends with a colon and the body of the loop
follows either on the same line or indented on the next lines. If
'while' ends a 'repeat' loop it doesn't end with a colon and there isn't
anything following on the same line or indented on the next lines.



From steve at pearwood.info  Thu Jul 16 01:59:39 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 16 Jul 2009 09:59:39 +1000
Subject: [Python-ideas] Immemorial desire for "do-while"-like
	construction
In-Reply-To: <op.uw4sdsxffvx12t@jerzozwiesz.home.aster.pl>
References: <op.uw4sdsxffvx12t@jerzozwiesz.home.aster.pl>
Message-ID: <200907160959.40022.steve@pearwood.info>

On Thu, 16 Jul 2009 08:50:06 am Jan Kaliszewski wrote:
> Hello,
>
> The issue has been coming back repeatedly:

And not very long ago at that.

Please have a read of this thread:

http://mail.python.org/pipermail/python-ideas/2009-April/004306.html

and (if I may be so immodest to link to my own post) especially this one 
where I discuss various possible ways of spelling the loop:

http://mail.python.org/pipermail/python-ideas/2009-April/004326.html


> Maybe here the simplest approach can lead us to the solution? I.e.:
>
> repeat while CONDITION:
>        SOME
>        ACTIONS
>        HERE
>        ...

(1) I dislike that it puts the test at the top of the loop even though 
it is not tested until the bottom.

(2) I dislike that it has two keywords back-to-back.

(3) I prefer to swap the sense of the test. Rather than end the loop 
when CONDITION becomes false, I'd prefer the Pascal semantics of ending 
the loop when CONDITION becomes true. Using your proposed syntax, that 
would become:

repeat until CONDITION:
    block

Now we can discard the redundant double keyword and just write:

until CONDITION:
    block

which is still funny because the test is at the top of the block 
although it's not tested until the bottom.


> In human language it'd mean: do ACTIONS once unconditionally, then
> *repeat* them while the CONDITION is true.
>
> Additional advantage is that probably for many people associate
> 'repeat' with such construction (from experience with other
> languages).

Other people associate 'do' with such a construction.


I suspect we are doomed to keep re-visiting this question until 
Python5K, because (1) there are so many ways of spelling a do...until 
loop and the differences are (almost entirely) just aesthetic; and (2) 
there is no overwhelming advantage to do...until compared to a while 
loop with an explicit break.



-- 
Steven D'Aprano


From zuo at chopin.edu.pl  Thu Jul 16 02:16:08 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Thu, 16 Jul 2009 02:16:08 +0200
Subject: [Python-ideas] Immemorial desire for "do-while"-like
	construction
In-Reply-To: <50697b2c0907151554p2175d631s5d983b734d4164d3@mail.gmail.com>
References: <op.uw4sdsxffvx12t@jerzozwiesz.home.aster.pl>
	<50697b2c0907151554p2175d631s5d983b734d4164d3@mail.gmail.com>
Message-ID: <op.uw4wc6nkfvx12t@jerzozwiesz.home.aster.pl>

16-07-2009 o 00:54 Chris Rebert <pyideas at rebertia.com> wrote:

> You can fix that by just writing it as:
>
> while True:
>     SOME
>     ACTIONS
>     HERE
>     if not CONDITION: break

Yeah, but it's not the same :) because eyes must look for the actual loop
condition somewhere-within-the-loop (after all, "while True" is common
idiom with large field of usage, not only in such situations...).

Another solution I invented for myself is to use object which once
evaluates to True, then always to False, e.g.:

   from itertools import chain, repeat
class FirstTrue(object):
        def __init__(self):
            self._iter = chain([True], repeat(False))
        def __nonzero__(self):  # __bool__ for Py3k
            return next(self._iter)

Usage:

first = FirstTrue()
while first or CONDITION:
      SOME
      ACTIONS
      HERE

or, if we want to *evaluate* CONDITION also at first time:

first = FirstTrue()
while CONDITION or first:
      SOME
      ACTIONS
      HERE

Wouldn't be nice to have such factory as built-in or in itertools?

Best regards,
*j

-- 
Jan Kaliszewski <zuo at chopin.edu.pl>


From josiah.carlson at gmail.com  Thu Jul 16 02:32:01 2009
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Wed, 15 Jul 2009 17:32:01 -0700
Subject: [Python-ideas] Immemorial desire for "do-while"-like
	construction
In-Reply-To: <op.uw4wc6nkfvx12t@jerzozwiesz.home.aster.pl>
References: <op.uw4sdsxffvx12t@jerzozwiesz.home.aster.pl>
	<50697b2c0907151554p2175d631s5d983b734d4164d3@mail.gmail.com>
	<op.uw4wc6nkfvx12t@jerzozwiesz.home.aster.pl>
Message-ID: <e6511dbf0907151732p28defd52m8ec468a4b7cfc722@mail.gmail.com>

On Wed, Jul 15, 2009 at 5:16 PM, Jan Kaliszewski<zuo at chopin.edu.pl> wrote:
> 16-07-2009 o 00:54 Chris Rebert <pyideas at rebertia.com> wrote:
>
>> You can fix that by just writing it as:
>>
>> while True:
>> ? ?SOME
>> ? ?ACTIONS
>> ? ?HERE
>> ? ?if not CONDITION: break
>
> Yeah, but it's not the same :) because eyes must look for the actual loop
> condition somewhere-within-the-loop (after all, "while True" is common
> idiom with large field of usage, not only in such situations...).
>
> Another solution I invented for myself is to use object which once
> evaluates to True, then always to False, e.g.:
>
> ?from itertools import chain, repeat
> class FirstTrue(object):
> ? ? ? def __init__(self):
> ? ? ? ? ? self._iter = chain([True], repeat(False))
> ? ? ? def __nonzero__(self): ?# __bool__ for Py3k
> ? ? ? ? ? return next(self._iter)
>
> Usage:
>
> first = FirstTrue()
> while first or CONDITION:
> ? ? SOME
> ? ? ACTIONS
> ? ? HERE
>
> or, if we want to *evaluate* CONDITION also at first time:
>
> first = FirstTrue()
> while CONDITION or first:
> ? ? SOME
> ? ? ACTIONS
> ? ? HERE

I usually do:
FIRST = 1
while CONDITION or FIRST:
    FIRST = 0
    SOME
    ACTIONS
    HERE

I use 1/0 instead of True/False because older Pythons loaded numeric
constants faster than the names True/False (is that still the case?)

 - Josiah

>
> Wouldn't be nice to have such factory as built-in or in itertools?
>
> Best regards,
> *j
>
> --
> Jan Kaliszewski <zuo at chopin.edu.pl>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>


From zuo at chopin.edu.pl  Thu Jul 16 02:37:38 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Thu, 16 Jul 2009 02:37:38 +0200
Subject: [Python-ideas] Immemorial desire for "do-while"-like
	construction
In-Reply-To: <op.uw4wc6nkfvx12t@jerzozwiesz.home.aster.pl>
References: <op.uw4sdsxffvx12t@jerzozwiesz.home.aster.pl>
	<50697b2c0907151554p2175d631s5d983b734d4164d3@mail.gmail.com>
	<op.uw4wc6nkfvx12t@jerzozwiesz.home.aster.pl>
Message-ID: <op.uw4xc0o2fvx12t@jerzozwiesz.home.aster.pl>

As Xavier noticed, my second example is buggy -- pls forget about it...

> or, if we want to *evaluate* CONDITION also at first time:
>
> first = FirstTrue()
> while CONDITION or first:
>       SOME
>       ACTIONS
>       HERE

It means it's time for me to go to bed :) [02:36]

Cheers,
*j


From cmjohnson.mailinglist at gmail.com  Thu Jul 16 02:47:10 2009
From: cmjohnson.mailinglist at gmail.com (Carl Johnson)
Date: Wed, 15 Jul 2009 14:47:10 -1000
Subject: [Python-ideas] Immemorial desire for "do-while"-like
	construction
In-Reply-To: <op.uw4wc6nkfvx12t@jerzozwiesz.home.aster.pl>
References: <op.uw4sdsxffvx12t@jerzozwiesz.home.aster.pl>
	<50697b2c0907151554p2175d631s5d983b734d4164d3@mail.gmail.com>
	<op.uw4wc6nkfvx12t@jerzozwiesz.home.aster.pl>
Message-ID: <3bdda690907151747k44f6b53eh114f307ab5f8e03@mail.gmail.com>

On Wed, Jul 15, 2009 at 2:16 PM, Jan Kaliszewski<zuo at chopin.edu.pl> wrote:
> 16-07-2009 o 00:54 Chris Rebert <pyideas at rebertia.com> wrote:
>
>> You can fix that by just writing it as:
>>
>> while True:
>> ? ?SOME
>> ? ?ACTIONS
>> ? ?HERE
>> ? ?if not CONDITION: break
>
> Yeah, but it's not the same :) because eyes must look for the actual loop
> condition somewhere-within-the-loop (after all, "while True" is common
> idiom with large field of usage, not only in such situations...).

I think we can just piggyback off of Python's existing optional
support for braces, BEGIN, END, etc.:

while True: #DO WHILE COND {
    something
# }
    if cond: break

:-D

My favorite proposal out of the last discussion of do-while was to
make "while:" mean "while True:". Combined with Python's optional
support for braces, it sounds like a winner to me! :-D

Optionally-yrs,

-- Carl Johnson


From gerald.britton at gmail.com  Thu Jul 16 03:51:03 2009
From: gerald.britton at gmail.com (Gerald Britton)
Date: Wed, 15 Jul 2009 21:51:03 -0400
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com> 
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com> 
	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com> 
	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com> 
	<4A5D8B63.4040509@canterbury.ac.nz>
	<eae285400907150549k429f1aeha8ab1b85f581eb7@mail.gmail.com> 
	<5d1a32000907150638j65131770gf26df83dbadc5af2@mail.gmail.com> 
	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com> 
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
Message-ID: <5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>

foo(x,y) = myfunc(bar) where:
    myfunc = lambda x:f(x)
    bar = lambda y:g (y)
    where:
        f = lambda x:getattr(x, 'func')
        g = lambda y:str(y)

would translate to:

def foo(x,y):
    return getattr(x, 'func')str(y)

Basically Haskell reads like math:

circle_area = pi * radius**2 where pi is 3.14159 and radius is the
radius of the circle

You unpack it as you go.  You could do the same this way:

f = lambda x:getattr(x,'func')
g = lambda y:str(y)

bar = lambda y: g(y)
myfunc = lambda x: f(x)

foo(x,y) = lambda x,y: myfunc(x)(bar(y))

though this is just a toy example to give the general flavor.


On Wed, Jul 15, 2009 at 7:12 PM, Carl
Johnson<cmjohnson.mailinglist at gmail.com> wrote:
> Daniel Stutzbach wrote:
>
>> Unlikely, but this is:
>>
>> total =3D sum(f(item) for item in lst) where:
>> ? ? def f(item):
>> ? ? ? ? return item.size if item.size > 0 else 1
>
> Yes, that seems much more sensible than trying to repeat the where's
> inside of list comp or gen exp.
>
> Daniel Stutzbach wrote:
>
>> How about the following as additional syntactic sugar for the common one-=
> function case?
>>
>> x =3D blah(f) where def f(item):
>> ? ?body_of_f
>
> I like it.
>
> Gerald Britton wrote:
>
>> Why not just use the Haskell approach?
>>
>> foo(x,y) =3D myfunc(bar) where:
>> =A0 =A0 =A0 =A0 =A0 myfunc, bar =3D f(x), g(y) where:
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0f,g =3D func1, func2
>
> Because I've been looking at that for a couple minutes and still have
> no idea what it is supposed to mean. :-( List comprehensions and
> pattern matching are the only Haskell features that make any sense to
> me.
>
> Jan Kaliszewski wrote:
>
>> I don't like the direction of limiting content of the block to set of sin=
> gular simple statements. I suppose where-block should simply create a separ=
> ate nested scope.
>
> Yes. It strikes me that one of the problems with Ruby is that there
> are different scoping rules for blocks, lambda, methods, procs, and
> whatever else=85 It would be much easier for users if "where" blocks had
> the exact same scoping rules as functions. Daniel Stutzbach's
> suggested equivalence strikes me as exactly correct:
>
> [return|yield|x=3D] expression_list where:
> ? suite
>
> is roughly equivalent to:
>
> def where_expression():
> ? ?suite
> ? ?return expression_list
> [return|yield|x=3D] where_expression()
>
>
> -- Carl Johnson
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
Gerald Britton


From greg.ewing at canterbury.ac.nz  Thu Jul 16 09:28:20 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 16 Jul 2009 19:28:20 +1200
Subject: [Python-ideas] Where-statement (Proposal for
	function	expressions)
In-Reply-To: <3bdda690907150259q314ded3eoa4b8c30823261e39@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>
	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>
	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>
	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>
	<4A5D8B63.4040509@canterbury.ac.nz>
	<3bdda690907150259q314ded3eoa4b8c30823261e39@mail.gmail.com>
Message-ID: <4A5ED694.1000504@canterbury.ac.nz>

Carl Johnson wrote:

> Surely, you meant
> 
> foo(f, g) where:
>    def f(x):
>      ...
>    def g(y):
>      ...

Yes, that's what I meant.

> new_list = [fx for f in fs if fx] where:
>     fx = f(x)

No, there would have to be a separate, analogous
extension to list comprehensions:

   new_list = [fx for f in fs if fx where fx = f(x)]

or maybe

   new_list = [fx where fx = f(x) for f in fs if fx]

-- 
Greg


From piet at cs.uu.nl  Thu Jul 16 11:46:19 2009
From: piet at cs.uu.nl (Piet van Oostrum)
Date: Thu, 16 Jul 2009 11:46:19 +0200
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>
	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>
	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>
	<4A5D8B63.4040509@canterbury.ac.nz>
	<eae285400907150549k429f1aeha8ab1b85f581eb7@mail.gmail.com>
	<5d1a32000907150638j65131770gf26df83dbadc5af2@mail.gmail.com>
	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
Message-ID: <m2vdltufis.fsf@cs.uu.nl>

>>>>> Gerald Britton <gerald.britton at gmail.com> (GB) wrote:

>GB> foo(x,y) = myfunc(bar) where:
>GB>     myfunc = lambda x:f(x)
>GB>     bar = lambda y:g (y)
>GB>     where:
>GB>         f = lambda x:getattr(x, 'func')
>GB>         g = lambda y:str(y)

>GB> would translate to:

>GB> def foo(x,y):
>GB>     return getattr(x, 'func')str(y)

I don't think so. In the original code x and y after the where are only
used as bound variables in the lambda's so they are not the x and y of
the foo def. You could replace all of them by p and q without changing
the meaning.

If you replace all lambda x: fun(x) by the equivalent fun then you get
this:

def foo(x,y):
    return getattr(str, 'func')

x and y are not used.

>GB> Basically Haskell reads like math:

>GB> circle_area = pi * radius**2 where pi is 3.14159 and radius is the
>GB> radius of the circle

>GB> You unpack it as you go.  You could do the same this way:

>GB> f = lambda x:getattr(x,'func')
>GB> g = lambda y:str(y)

>GB> bar = lambda y: g(y)
>GB> myfunc = lambda x: f(x)

>GB> foo(x,y) = lambda x,y: myfunc(x)(bar(y))

Huh?
How do you come at this step? IMHO myfunc(bar) is not the same as 
lambda x,y: myfunc(x)(bar(y)).
It would be the same as (lambda x: myfunc(x))(lambda y: bar(y))

-- 
Piet van Oostrum <piet at cs.uu.nl>
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: piet at vanoostrum.org



From steve at pearwood.info  Thu Jul 16 13:26:03 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 16 Jul 2009 21:26:03 +1000
Subject: [Python-ideas] Immemorial desire for "do-while"-like
	construction
In-Reply-To: <e6511dbf0907151732p28defd52m8ec468a4b7cfc722@mail.gmail.com>
References: <op.uw4sdsxffvx12t@jerzozwiesz.home.aster.pl>
	<op.uw4wc6nkfvx12t@jerzozwiesz.home.aster.pl>
	<e6511dbf0907151732p28defd52m8ec468a4b7cfc722@mail.gmail.com>
Message-ID: <200907162126.03776.steve@pearwood.info>

On Thu, 16 Jul 2009 10:32:01 am Josiah Carlson wrote:
> I use 1/0 instead of True/False because older Pythons loaded numeric
> constants faster than the names True/False (is that still the case?)

From Python 2.6.1:

>>> import dis
>>> dis.dis(compile("while 1: break", "", "exec"))
  1           0 SETUP_LOOP               4 (to 7)
        >>    3 BREAK_LOOP
              4 JUMP_ABSOLUTE            3
        >>    7 LOAD_CONST               0 (None)
             10 RETURN_VALUE
>>> dis.dis(compile("while True: break", "", "exec"))
  1           0 SETUP_LOOP              13 (to 16)
        >>    3 LOAD_NAME                0 (True)
              6 JUMP_IF_FALSE            5 (to 14)
              9 POP_TOP
             10 BREAK_LOOP
             11 JUMP_ABSOLUTE            3
        >>   14 POP_TOP
             15 POP_BLOCK
        >>   16 LOAD_CONST               0 (None)
             19 RETURN_VALUE


True is a constant in Python 3.0.


-- 
Steven D'Aprano


From ncoghlan at gmail.com  Thu Jul 16 13:49:19 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 16 Jul 2009 21:49:19 +1000
Subject: [Python-ideas] Immemorial desire for
	"do-while"-like	construction
In-Reply-To: <200907160959.40022.steve@pearwood.info>
References: <op.uw4sdsxffvx12t@jerzozwiesz.home.aster.pl>
	<200907160959.40022.steve@pearwood.info>
Message-ID: <4A5F13BF.3060908@gmail.com>

Steven D'Aprano wrote:
> On Thu, 16 Jul 2009 08:50:06 am Jan Kaliszewski wrote:
>> Hello,
>>
>> The issue has been coming back repeatedly:
> 
> And not very long ago at that.

So often that there's even a PEP* that documents the pros and cons of
different ideas and points out that a viable syntactic challenger to the
"while True with inner break" approach has yet to be found.

The current idiom does have the downside of potentially hiding the
termination condition, but it otherwise handles generalised looping very
nicely. (Putting the break on a separate line also helps a great deal
with the visibility of the termination condition)

Cheers,
Nick.

* http://www.python.org/dev/peps/pep-0315/

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From gerald.britton at gmail.com  Thu Jul 16 15:39:01 2009
From: gerald.britton at gmail.com (Gerald Britton)
Date: Thu, 16 Jul 2009 09:39:01 -0400
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <m2vdltufis.fsf@cs.uu.nl>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com> 
	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com> 
	<4A5D8B63.4040509@canterbury.ac.nz>
	<eae285400907150549k429f1aeha8ab1b85f581eb7@mail.gmail.com> 
	<5d1a32000907150638j65131770gf26df83dbadc5af2@mail.gmail.com> 
	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com> 
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com> 
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com> 
	<m2vdltufis.fsf@cs.uu.nl>
Message-ID: <5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>

> Huh?
> How do you come at this step? IMHO myfunc(bar) is not the same as
> lambda x,y: myfunc(x)(bar(y)).
> It would be the same as (lambda x: myfunc(x))(lambda y: bar(y))

Possibly, I wrote it up in a hurry.  Perhaps this one is clearer:


foo = lambda (x,y): myfunc(bar) where:
     myfunc = getattr(x, 'func')
     bar = str(y)

Should be the same as:

foo = lambda (x,y): getattr(x, 'func')(str(y))

The "where" clause is just syntactic sugar but can make some things
easier to follow.  Even in the toy example above it is tough to
(visually) parse:

    getattr(x, 'func')(str(y))

and some feel that breaking it down bit by bit is easier.  So the
example could be written:

def foo(x,y):
    myfunc = getattr(x, 'func')
    bar = str(y)
    return myfunc(bar)

It really comes down to a matter of taste.  Should the programmer be
forced to define things in execution order (like the def foo, above)
or should there be freedom to define things when you absolutely need
to (the example with the "where" clause)?

Another example:

  area(x) = pi * r ** 2 where:
        import math
        pi = math.pi
        r = circle.get_radius()
        where:
             circle = Circle(object)
             where:
                    class Circle(radius):
                          def __init__(self, radius):
                                self.radius = radius
                          def get_radius(self):
                                return self.radius

I know, it starts to look absurd after a while.  However, I would use
it if available to make long expressions more readable.  For example,
say I have a database class with many functions with descriptive names
that are a bit long.  Let's set it up this way:

class MyDataBase(DB):
     def __init__(self, path):
         self.path = path
     ...
     def get_records_with_key(self, key):
         rec = db.get_by_key(key)
         while rec:
             yield rec
             rec = db.get_next_by_key(key)

Now, in my code I might have a snippet:

    employees = MyDataBase('employees.db')
    ....
    smiths = [record for record in db.get_records_with_key('smith')]

If I has a "where" clause, I'd prefer to write this:

    smiths = [record for record in stuff] where:
                      stuff = db.get_records_with_key('smith')

Of course there are other ways to write this in standard Python, but I
don't think that's the point of this thread.  I think that adding a
"where" clause would make it possible to write deep list
comprehensions and other long expressions without lots of parentheses.
 I suspect that I'm not alone in messing up either the placement or
number of parentheses or both.

Here's another one. Remember that a generator expression sometimes
needs parentheses and sometimes not?  For example:

from itertools import chain
c = chain(i for i in [2,3,5,7,11])   # no parentheses around the generator
d = chain( (i for i in [2,3,5]) , (j for j in [7,11] )  # needs parentheses

It's easy to mess up the extra parentheses (I sure do (and did, above!))

This might be nicer:

d = chain(g1, g2) where:
         g1 = (i for i in [2,3,5])
         g1 = (j for j in [7,11])

Even though there are just as many parentheses, they're broken out and
easier to spot.  Sure, you could define g1 and g2 ahead of time, but
if the "where" clause creates a new namespace like a function def in a
function, then g1 and g2 are just placeholders in the expression and
don't pollute the variable namespace of the calling context.

Anyway, I don't feel strongly either way.  I see the benefits of
"where" clauses and I'd use them if they were available, but I can
certainly do without.


From chrisperkins99 at gmail.com  Thu Jul 16 16:51:54 2009
From: chrisperkins99 at gmail.com (Chris Perkins)
Date: Thu, 16 Jul 2009 10:51:54 -0400
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>
	<4A5D8B63.4040509@canterbury.ac.nz>
	<eae285400907150549k429f1aeha8ab1b85f581eb7@mail.gmail.com>
	<5d1a32000907150638j65131770gf26df83dbadc5af2@mail.gmail.com>
	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
Message-ID: <184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>

On Thu, Jul 16, 2009 at 9:39 AM, Gerald Britton<gerald.britton at gmail.com> wrote:
>
> d = chain(g1, g2) where:
>         g1 = (i for i in [2,3,5])
>         g1 = (j for j in [7,11])
>
> Even though there are just as many parentheses, they're broken out and
> easier to spot.  Sure, you could define g1 and g2 ahead of time, but
> if the "where" clause creates a new namespace like a function def in a
> function, then g1 and g2 are just placeholders in the expression and
> don't pollute the variable namespace of the calling context.

This idea is really growing on me.

Here's another example - how do you calculate sample variance? You're
probably thinking "Um, well, I think it's the square root of the sum
of the squares minus the square of the sums". Great, that's a high-ish
level, intuitive description of the algorithm. The precise details of
how to go about calculating that are not important yet. So write it
that way:

def variance(data):
    return sqrt(sum_of_squares - square_of_sums) where:
        ... elided ...

There - we've written the core of the algorithm at the same level of
abstraction as we think about it. The code matches our mental model.
Only then do we go about filling in the boring details.

def variance(data):
    return sqrt(sum_of_squares - square_of_sums) where:
        def sqrt(v): return v ** 0.5
        sum_of_squares = sum(v**2 for v in data)
        square_of_sums = sum(data) ** 2

The advantage is that now readers of the code, having developed some
intuition about what "where blocks" mean, can see at a glance that the
code at the first level of indentation is the important stuff, and the
indented block is the implementation details; the stuff that's at a
lower level of abstraction.

I think this construct would be a great way to enable SLAP - the Same
Level of Abstraction Principle (which I'm embarrasingly in love with).
Traditional SLAP says that all the code in a function should be at the
same level of abstraction - code at a lower level of abstraction
should be split out into other functions or methods. A refinement
could could state that code at a lower level of abstraction should be
either moved to another function, or indented in a where block.

> Anyway, I don't feel strongly either way.  I see the benefits of
> "where" clauses and I'd use them if they were available, but I can
> certainly do without.

I would use them too, and I can't do without. I'm going to hold my
breath until we get them. ;)

As for the precise specification, I see only a few issues.

As a first approximation, assume that this code:

[LHS OP] EXPR where:
    BODY

is expanded into the equivalent of this:

def TEMP():
    BODY
    return EXPR
[LHS OP] TEMP()

That seems like the most obvious way to get a nested scope to run the body in.

Problem 1) return
Since there will be an implicit return injected into the body of TEMP,
what happens if the user puts an explicit return in the body of the
block:
    x = foo(y) where:
        return 23
expands to
    def TEMP():
        return 23
        return foo(y)
    x = TEMP()
So return should be disallowed.

Problem 2) break/continue
    for a in range(b):
        x = foo(y) where:
            break #or continue
expands to:
    for a in range(b):
        def TEMP():
            break # or continue
            return foo(y)
        x = TEMP()
Which is an error. However:
    x = foo(y) where:
        y = []
        for i in range(n):
            if cond(i):
                break
            y.append(i)
must be allowed (right?). So we can't make a blanket restriction on
break/continue within the body.

Problem 3) yield
    x = foo(y) where:
        yield 3
expands to:
    def TEMP():
        yield 3
        return foo(y)
    x = TEMP()
which is an error. So yield must be disallowed.

I can't think of any other constructs that would need special
handling, and I'm sure the issues I have listed are solvable.


Chris Perkins


From p.f.moore at gmail.com  Thu Jul 16 17:06:13 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 16 Jul 2009 16:06:13 +0100
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<4A5D8B63.4040509@canterbury.ac.nz>
	<eae285400907150549k429f1aeha8ab1b85f581eb7@mail.gmail.com>
	<5d1a32000907150638j65131770gf26df83dbadc5af2@mail.gmail.com>
	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
Message-ID: <79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>

2009/7/16 Chris Perkins <chrisperkins99 at gmail.com>:
> As for the precise specification, I see only a few issues.
>
> As a first approximation, assume that this code:
>
> [LHS OP] EXPR where:
> ? ?BODY
>
> is expanded into the equivalent of this:
>
> def TEMP():
> ? ?BODY
> ? ?return EXPR
> [LHS OP] TEMP()

So where constructs are only allowed for code of the form [LHS OP] EXPR?

What does that correspond to in the grammar? (If it doesn't correspond
to something at the moment, you'll need to extend the grammar to
define a "thing that can have a where clause" production...)

You've also disallowed

x[i] = 12 where:
    i = some_complicated_expression()

Was that deliberate? If so, could you explain why now, so that it's on
record for the legions of people who will ask after it's implemented?
:-)

Paul.


From chrisperkins99 at gmail.com  Thu Jul 16 17:21:10 2009
From: chrisperkins99 at gmail.com (Chris Perkins)
Date: Thu, 16 Jul 2009 11:21:10 -0400
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<eae285400907150549k429f1aeha8ab1b85f581eb7@mail.gmail.com>
	<5d1a32000907150638j65131770gf26df83dbadc5af2@mail.gmail.com>
	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
Message-ID: <184a9f5a0907160821q79780f12vd6b9a65e870bffd4@mail.gmail.com>

On Thu, Jul 16, 2009 at 11:06 AM, Paul Moore<p.f.moore at gmail.com> wrote:
>
> So where constructs are only allowed for code of the form [LHS OP] EXPR?
>
> What does that correspond to in the grammar? (If it doesn't correspond
> to something at the moment, you'll need to extend the grammar to
> define a "thing that can have a where clause" production...)

Without having thought too hard about it, I would allow a where block
after a "simple_stmt" - same thing I did with my "do block" proposal.
That means you could have a where after any of these:

small_stmt: (expr_stmt | print_stmt  | del_stmt | pass_stmt | flow_stmt |
             import_stmt | global_stmt | exec_stmt | assert_stmt)

So by EXPR above, I guess I was thinking of any "small_stmt"
production. Of course, it makes no sense for some of them, like
pass_stmt, global_stmt, etc, but I don't see any harm in allowing the
block to be there - I don't think I could have any effect though.

> You've also disallowed
>
> x[i] = 12 where:
> ? ?i = some_complicated_expression()
>
> Was that deliberate? If so, could you explain why now, so that it's on
> record for the legions of people who will ask after it's implemented?
> :-)

Well, my original answer was "no, I just didnt' think of that'.  But
now that you mention it, I think in the case of an "augassign"
production, it makes sense to have the left hand side evaluated in the
current scope (not in the 'where block" scope).  So in your examle,
you would either get a NameError for i, the code would do somthing
that you didn't expect.


Chris Perkins


From daniel at stutzbachenterprises.com  Thu Jul 16 17:37:34 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Thu, 16 Jul 2009 10:37:34 -0500
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<eae285400907150549k429f1aeha8ab1b85f581eb7@mail.gmail.com>
	<5d1a32000907150638j65131770gf26df83dbadc5af2@mail.gmail.com>
	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
Message-ID: <eae285400907160837v7f483df9t3d757cd46e2429af@mail.gmail.com>

On Thu, Jul 16, 2009 at 10:06 AM, Paul Moore <p.f.moore at gmail.com> wrote:

> x[i] = 12 where:
>    i = some_complicated_expression()
>
> Was that deliberate? If so, could you explain why now, so that it's on
> record for the legions of people who will ask after it's implemented?
> :-)


Because only the right-hand side is in the where-block's scope.  If the
left-hand side were in the where-block's scope, then:

x = 0
x = i where:
   i = 5
print x

Would print "0" not "5".

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090716/881cf968/attachment.html>

From george.sakkis at gmail.com  Thu Jul 16 18:05:19 2009
From: george.sakkis at gmail.com (George Sakkis)
Date: Thu, 16 Jul 2009 12:05:19 -0400
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<4A5D8B63.4040509@canterbury.ac.nz>
	<eae285400907150549k429f1aeha8ab1b85f581eb7@mail.gmail.com>
	<5d1a32000907150638j65131770gf26df83dbadc5af2@mail.gmail.com>
	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
Message-ID: <91ad5bf80907160905ta896d60x335ec53c8545f44e@mail.gmail.com>

On Thu, Jul 16, 2009 at 10:51 AM, Chris Perkins<chrisperkins99 at gmail.com> wrote:

> This idea is really growing on me.
>
> Here's another example - how do you calculate sample variance? You're
> probably thinking "Um, well, I think it's the square root of the sum
> of the squares minus the square of the sums". Great, that's a high-ish
> level, intuitive description of the algorithm. The precise details of
> how to go about calculating that are not important yet. So write it
> that way:
>
> def variance(data):
> ? ?return sqrt(sum_of_squares - square_of_sums) where:
> ? ? ? ?... elided ...
>
> There - we've written the core of the algorithm at the same level of
> abstraction as we think about it. The code matches our mental model.
> Only then do we go about filling in the boring details.
>
> def variance(data):
> ? ?return sqrt(sum_of_squares - square_of_sums) where:
> ? ? ? ?def sqrt(v): return v ** 0.5
> ? ? ? ?sum_of_squares = sum(v**2 for v in data)
> ? ? ? ?square_of_sums = sum(data) ** 2
>
> The advantage is that now readers of the code, having developed some
> intuition about what "where blocks" mean, can see at a glance that the
> code at the first level of indentation is the important stuff, and the
> indented block is the implementation details; the stuff that's at a
> lower level of abstraction.

I realize the appeal of this but I see two issues with it:

- What you call an advantage can be a disadvantage for many
programmers used to the standard bottom-up way most procedural and
object-oriented languages support. For better or for worse, thinking
sequentially feels more natural than in terms of top-down levels of
abstractions. Unless one is (or gets) used to this style, "where"
blocks would be the code equivalent of top-posting; it doesn't make
sense unless you read it backwards.

- TIMTOWTDI. In any case, bottom-up won't go away so for any
non-trivial piece of code one has to choose between two equivalent
ways of expressing it. Think of Perl's "STATEMENT (if|unless) (EXPR)".

I'm not saying that these are necessarily show stoppers but they
should be addressed to see whether the benefits are worth the added
complexity.

George


From chrisperkins99 at gmail.com  Thu Jul 16 18:16:21 2009
From: chrisperkins99 at gmail.com (Chris Perkins)
Date: Thu, 16 Jul 2009 12:16:21 -0400
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <91ad5bf80907160905ta896d60x335ec53c8545f44e@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<eae285400907150549k429f1aeha8ab1b85f581eb7@mail.gmail.com>
	<5d1a32000907150638j65131770gf26df83dbadc5af2@mail.gmail.com>
	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<91ad5bf80907160905ta896d60x335ec53c8545f44e@mail.gmail.com>
Message-ID: <184a9f5a0907160916t20bfc344pf4ca424c061342b6@mail.gmail.com>

On Thu, Jul 16, 2009 at 12:05 PM, George Sakkis<george.sakkis at gmail.com> wrote:
> - What you call an advantage can be a disadvantage for many
> programmers used to the standard bottom-up way most procedural and
> object-oriented languages support. For better or for worse, thinking
> sequentially feels more natural than in terms of top-down levels of
> abstractions. Unless one is (or gets) used to this style, "where"
> blocks would be the code equivalent of top-posting; it doesn't make
> sense unless you read it backwards.

That's interesting - I see it as exactly the opposite. For me,
bottom-up approach appears backwards, and I get annoyed by having to
read backwards.  It clearly depends on what you're used to, I guess.

> - TIMTOWTDI. In any case, bottom-up won't go away so for any
> non-trivial piece of code one has to choose between two equivalent
> ways of expressing it. Think of Perl's "STATEMENT (if|unless) (EXPR)".
>
> I'm not saying that these are necessarily show stoppers but they
> should be addressed to see whether the benefits are worth the added
> complexity.

Yes, I'm very interested to see what proportion of people find the
top-down style more readable.

Chris Perkins


From zuo at chopin.edu.pl  Thu Jul 16 18:51:15 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Thu, 16 Jul 2009 18:51:15 +0200
Subject: [Python-ideas] Where-statement (Proposal for function
 expressions)
In-Reply-To: <eae285400907160837v7f483df9t3d757cd46e2429af@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<eae285400907150549k429f1aeha8ab1b85f581eb7@mail.gmail.com>
	<5d1a32000907150638j65131770gf26df83dbadc5af2@mail.gmail.com>
	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<eae285400907160837v7f483df9t3d757cd46e2429af@mail.gmail.com>
Message-ID: <op.uw56fmtffvx12t@jerzozwiesz.home.aster.pl>

16-07-2009, 17:37 Daniel Stutzbach <daniel at stutzbachenterprises.com> wrote:

> On Thu, Jul 16, 2009 at 10:06 AM, Paul Moore <p.f.moore at gmail.com> wrote:
>
>> x[i] = 12 where:
>>    i = some_complicated_expression()
>>
>> Was that deliberate? If so, could you explain why now, so that it's on
>> record for the legions of people who will ask after it's implemented?
>> :-)
>
> Because only the right-hand side is in the where-block's scope.

Yeah. But I believe it should be possible using global/nonlocal keywords,
e.g.:

x[i] = 12 where:
    global i
    i = some_complicated_expression()

I think that many rules that apply for functions should also apply for
where-blocks. (But obviously not all of them, e.g. return, yield and such
stuff don't make sense)

16-07-2009, 18:16 Chris Perkins <chrisperkins99 at gmail.com> wrote:

> Yes, I'm very interested to see what proportion of people find the
> top-down style more readable.

I think even for the same person it can strongly depend on particular
application. It'll be good for some things, and useless for other.
But it isn't a strong argument against the feature. IMHO it'll be
useful in many situations.

15-07-2009, 14:49 Daniel Stutzbach <daniel at stutzbachenterprises.com> wrote:

> How about the following as additional syntactic sugar for the common
> one-function case?
> x = blah(f) where def f(item):
>    body_of_f

+1 from me also.

Cheers,

-- 
Jan Kaliszewski <zuo at chopin.edu.pl>


From chrisperkins99 at gmail.com  Thu Jul 16 19:02:20 2009
From: chrisperkins99 at gmail.com (Chris Perkins)
Date: Thu, 16 Jul 2009 13:02:20 -0400
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <op.uw56fmtffvx12t@jerzozwiesz.home.aster.pl>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<eae285400907160837v7f483df9t3d757cd46e2429af@mail.gmail.com>
	<op.uw56fmtffvx12t@jerzozwiesz.home.aster.pl>
Message-ID: <184a9f5a0907161002l31d7e6f2vf10e8b63b0a90473@mail.gmail.com>

On Thu, Jul 16, 2009 at 12:51 PM, Jan Kaliszewski<zuo at chopin.edu.pl> wrote:
>> How about the following as additional syntactic sugar for the common
>> one-function case?
>> x = blah(f) where def f(item):
>> ? body_of_f
>
> +1 from me also.
>

I'm not sold on that one yet, only because someone is inevitably going
to ask why they can't create a singleton-ish sort of thing like this:

thing = Thing() where class Thing(object):
    def __init__(...): etc

Or maybe that should be allowed too?


Chris Perkins


From steve at pearwood.info  Thu Jul 16 19:13:18 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 17 Jul 2009 03:13:18 +1000
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
Message-ID: <200907170313.19012.steve@pearwood.info>

On Fri, 17 Jul 2009 12:51:54 am Chris Perkins wrote:

> Here's another example - how do you calculate sample variance? You're
> probably thinking "Um, well, I think it's the square root of the sum
> of the squares minus the square of the sums".  Great, that's a  
> high-ish level, intuitive description of the algorithm. The precise
> details of how to go about calculating that are not important yet. So
> write it that way:
>
> def variance(data):
>     return sqrt(sum_of_squares - square_of_sums) where:
>         ... elided ...

That would actually be the standard deviation, not the variance.

> There - we've written the core of the algorithm at the same level of
> abstraction as we think about it. The code matches our mental model.
> Only then do we go about filling in the boring details.
> 
> def variance(data):
> ? ? return sqrt(sum_of_squares - square_of_sums) where:
> ? ? ? ? def sqrt(v): return v ** 0.5
> ? ? ? ? sum_of_squares = sum(v**2 for v in data)
> ? ? ? ? square_of_sums = sum(data) ** 2


I don't know what you've calculated, but it's not the variance, or even 
the standard deviation. The variance is actually the mean of the 
squares of the deviations from the mean of the data. Written another 
way, that's the mean of the squares of the data minus the square of the 
mean. But putting all that aside, let's pretend that the formula you 
gave is correct. It's certainly calculating *something* -- let's 
pretend it's the variance.


I'm now going to try to convince you that this entire approach is 
actually harmful and should be avoided. You said:

"...sum of the squares minus the square of the sums".

That's all very well, but what squares? What sums? You have 
under-specified what the function should do. Is it squares of the first 
fifty prime numbers? Sums of odd-positioned data? You can't just 
pretend that this is an implementation detail -- you have to face up to 
what the function actually does at the design phase.

If you had, you would have noted that the so-called "variance" is the 
sum of the squares *of the data*, minus the square of the sums *of the 
data*. The sums-and-squares are functions of data, not expressions or 
constants, which suggests writing them as functions.

This gives us:

def variance(data):  # for some definition of "variance"
    return sqrt(sum_of_squares(data) - square_of_sums(data)) where:
        ...

This naturally suggests that they are (or could be) reusable functions 
that belong in the module namespace, not repeated inside each function 
that uses them:

def sqrt(v):
    return v**0.5

def sum_of_squares(data):
    return sum(v**2 for v in data)

def square_of_sums(data):
    return sum(data)**2

These can now easily be documented and tested, which is a major win.

Having done this, the "where" statement is redundant:

def variance(data):
    return sqrt(sum_of_squares(data) - square_of_sums(data))


Not just redundant, but actively harmful: it encourages the coder to 
duplicate common code inside functions instead of factoring it out into 
a single external function. This not only violates Don't Repeat 
Yourself, but it makes it harder to test and document the code.



Now of course it's possible to inappropriately inline common code inside 
functions without "where". For instance, I might have written:

def variance(data):
    def sqrt(v):
        return v**0.5
    def sum_of_squares(data):
        return sum(v**2 for v in data)
    def square_of_sums(data):
        return sum(data)**2
    return sqrt(sum_of_squares(data) - square_of_sums(data))

which is a micro-pessimation (not only do I lose the opportunity to 
re-use and test the internal functions, but I also pay the micro-cost 
of re-creating them every time I call the function).

Or:

def variance(data):
    sum_of_squares = sum(v**2 for v in data)
    square_of_sums = sum(data)**2
    return (sum_of_squares - square_of_sums)**0.5


In either case, writing the sums-and-squares before the return suggests 
that these are common code best written as re-usable functions. 
But "where" changes the focus of the coder away from factoring code 
*out* of functions into placing code *inside* internal blocks. It 
encourages the coder to think of sum_of_squares as a private 
implementation detail instead of a re-usable component. And that is, I 
believe, harmful.



-- 
Steven D'Aprano


From zuo at chopin.edu.pl  Thu Jul 16 19:16:50 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Thu, 16 Jul 2009 19:16:50 +0200
Subject: [Python-ideas] Where-statement (Proposal for function
 expressions)
In-Reply-To: <4A5D8B63.4040509@canterbury.ac.nz>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<fb6fbf560907121731u2578ecd1o739fa0b78d1a69ab@mail.gmail.com>
	<184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>
	<de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>
	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>
	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>
	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>
	<4A5D8B63.4040509@canterbury.ac.nz>
Message-ID: <op.uw57mcv8fvx12t@jerzozwiesz.home.aster.pl>

16-07-2009, 13:49 Nick Coghlan <ncoghlan at gmail.com> wrote,
(in thread 'Immemorial desire for "do-while"-likeconstruction'):

> Steven D'Aprano wrote:
>> On Thu, 16 Jul 2009 08:50:06 am Jan Kaliszewski wrote:
>>> Hello,
>>>
>>> The issue has been coming back repeatedly:
>>
>> And not very long ago at that.
>
> So often that there's even a PEP* that documents the pros and cons of
> different ideas and points out that a viable syntactic challenger to the
> "while True with inner break" approach has yet to be found.

Maybe the where-statement could be a candidate? Consider an example:

while foo(x) and bar() where:
     foo = SOMETHING
     def bar():
         SOMETHING ELSE
do:
     THE
     LOOP
     BODY

or maybe better:

while foo(x) and bar():
     where:
         foo = SOMETHING
         def bar():
             SOMETHING ELSE
     THE
     LOOP
     BODY

and Daniel's idea of one-def-shortcut also could apply:

while setup(x < y + 9 or bzzz):
     where def setup(cond):
         SETUP
         ACTIONS
         return cond
     THE
     LOOP
     BODY


And what about for-loop?

for i in smartgen():
     where def smartgen():
         SOME ACTIONS
         yield SOMETHING
     THE
     LOOP
     BODY


Cheers,

-- 
Jan Kaliszewski <zuo at chopin.edu.pl>


From zuo at chopin.edu.pl  Thu Jul 16 19:38:23 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Thu, 16 Jul 2009 19:38:23 +0200
Subject: [Python-ideas] Where-statement (Proposal for function
 expressions)
In-Reply-To: <200907170313.19012.steve@pearwood.info>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<200907170313.19012.steve@pearwood.info>
Message-ID: <op.uw58l9avfvx12t@jerzozwiesz.home.aster.pl>

16-07-2009, 19:13 Steven D'Aprano <steve at pearwood.info> wrote:

> Not just redundant, but actively harmful: it encourages the coder to
> duplicate common code inside functions instead of factoring it out into
> a single external function.

But note that, if it's necessary, refactoring such code moving
appropriate parts e.g. into global scope could be done very easily.

> Now of course it's possible to inappropriately inline common code
> inside functions without "where".

Every language feature can be misused. More powerful feature -- larger
possibilities of misusing. But language is not for substituting
programmer's sense.

Cheers,

-- 
Jan Kaliszewski <zuo at chopin.edu.pl>


From zuo at chopin.edu.pl  Thu Jul 16 19:57:55 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Thu, 16 Jul 2009 19:57:55 +0200
Subject: [Python-ideas] Where-statement (Proposal for function
 expressions)
In-Reply-To: <op.uw57mcv8fvx12t@jerzozwiesz.home.aster.pl>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<fb6fbf560907121731u2578ecd1o739fa0b78d1a69ab@mail.gmail.com>
	<184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>
	<de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>
	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>
	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>
	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>
	<4A5D8B63.4040509@canterbury.ac.nz>
	<op.uw57mcv8fvx12t@jerzozwiesz.home.aster.pl>
Message-ID: <op.uw59itbcfvx12t@jerzozwiesz.home.aster.pl>

> or maybe better:
>
> while foo(x) and bar():
>      where:
>          foo = SOMETHING
>          def bar():
>              SOMETHING ELSE
>      THE
>      LOOP
>      BODY

Another possible variant:

while foo(x) and bar()    # no colon
where:
         foo = SOMETHING
         def bar():
             SOMETHING ELSE
do:
         THE
         LOOP
         BODY

But the former has such an advantage that where-block content is
indented deeper than loop body -- what reflects that it (where-block)
creates nested scope (contrary to loop body).

-- 
Jan Kaliszewski <zuo at chopin.edu.pl>


From chrisperkins99 at gmail.com  Thu Jul 16 19:58:21 2009
From: chrisperkins99 at gmail.com (Chris Perkins)
Date: Thu, 16 Jul 2009 13:58:21 -0400
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <200907170313.19012.steve@pearwood.info>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<200907170313.19012.steve@pearwood.info>
Message-ID: <184a9f5a0907161058v32f0f6dey6e5838333c5998bd@mail.gmail.com>

On Thu, Jul 16, 2009 at 1:13 PM, Steven D'Aprano<steve at pearwood.info> wrote:
> On Fri, 17 Jul 2009 12:51:54 am Chris Perkins wrote:
>
>> def variance(data):
>> ? ? return sqrt(sum_of_squares - square_of_sums) where:
>> ? ? ? ? ... elided ...
>
> That would actually be the standard deviation, not the variance.

Hence my use of "Um, I think it's..." :)
Yeah, I couldn't remember if this was right, but as you say below,
it's not the point of the example, so I didn't bother to look it up.

> I'm now going to try to convince you that this entire approach is
> actually harmful and should be avoided. You said:

[ .. snip me being almost - but not quite - convinced ...]

>
> def variance(data):
> ? ?def sqrt(v):
> ? ? ? ?return v**0.5
> ? ?def sum_of_squares(data):
> ? ? ? ?return sum(v**2 for v in data)
> ? ?def square_of_sums(data):
> ? ? ? ?return sum(data)**2
> ? ?return sqrt(sum_of_squares(data) - square_of_sums(data))

This is the case that I was trying to present an alternative to. The
fact is, people _do_ write code like this. Sometimes pulling the local
stuff out into reusable functions is the right thing to do, but I
don't think it always is.

> In either case, writing the sums-and-squares before the return suggests
> that these are common code best written as re-usable functions.
> But "where" changes the focus of the coder away from factoring code
> *out* of functions into placing code *inside* internal blocks. It
> encourages the coder to think of sum_of_squares as a private
> implementation detail instead of a re-usable component. And that is, I
> believe, harmful.

But surely you agree that _sometimes_ there is code that really is an
implementation detail - not everything is reusable, or at least not
usefully reusable. Taking your argument to the extreme, every function
would be only one line of code. I prefer to trust programmers to make
the best use of the tools available to create good code.

In other words:

Jan Kaliszewski:
>Every language feature can be misused. More powerful feature -- larger
>possibilities of misusing. But language is not for substituting
>programmer's sense.

Yeah, ditto to what he said.


Chris Perkins


From python at mrabarnett.plus.com  Thu Jul 16 20:20:27 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 16 Jul 2009 19:20:27 +0100
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <op.uw56fmtffvx12t@jerzozwiesz.home.aster.pl>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>	<eae285400907150549k429f1aeha8ab1b85f581eb7@mail.gmail.com>	<5d1a32000907150638j65131770gf26df83dbadc5af2@mail.gmail.com>	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>	<m2vdltufis.fsf@cs.uu.nl>	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>	<eae285400907160837v7f483df9t3d757cd46e2429af@mail.gmail.com>
	<op.uw56fmtffvx12t@jerzozwiesz.home.aster.pl>
Message-ID: <4A5F6F6B.7030604@mrabarnett.plus.com>

Jan Kaliszewski wrote:
> 16-07-2009, 17:37 Daniel Stutzbach <daniel at stutzbachenterprises.com> wrote:
> 
>> On Thu, Jul 16, 2009 at 10:06 AM, Paul Moore <p.f.moore at gmail.com> wrote:
>>
>>> x[i] = 12 where:
>>>    i = some_complicated_expression()
>>>
>>> Was that deliberate? If so, could you explain why now, so that it's on
>>> record for the legions of people who will ask after it's implemented?
>>> :-)
>>
>> Because only the right-hand side is in the where-block's scope.
> 
> Yeah. But I believe it should be possible using global/nonlocal keywords,
> e.g.:
> 
> x[i] = 12 where:
>    global i
>    i = some_complicated_expression()
> 
> I think that many rules that apply for functions should also apply for
> where-blocks. (But obviously not all of them, e.g. return, yield and such
> stuff don't make sense)
> 
> 16-07-2009, 18:16 Chris Perkins <chrisperkins99 at gmail.com> wrote:
> 
>> Yes, I'm very interested to see what proportion of people find the
>> top-down style more readable.
> 
> I think even for the same person it can strongly depend on particular
> application. It'll be good for some things, and useless for other.
> But it isn't a strong argument against the feature. IMHO it'll be
> useful in many situations.
> 
> 15-07-2009, 14:49 Daniel Stutzbach <daniel at stutzbachenterprises.com> wrote:
> 
>> How about the following as additional syntactic sugar for the common
>> one-function case?
>> x = blah(f) where def f(item):
>>    body_of_f
> 
> +1 from me also.
> 
[Apologies for the word 'assignment' in what follows. :-)]

When defining a function, any name that's the target of an assignment
will default to being local; other names will be non-local.

In a 'where' construct the rule could be that any name that's the
target of an assignment before the 'where' will be non-local; other
names will default to local if they are the target of an assignment
within the 'where' construct, but non-local otherwise.


For example:

     bar = "outside"
     foo = bar where:
         bar = "inside"

'foo' is non-local. There's an assignment to 'bar' with the 'where' 
construct, so 'bar' is local in the statement and hides the non-local 'bar'.

Therefore 'foo' becomes bound to "inside".

More clearly:

     bar = "outside"
     local:
         nonlocal foo
         bar = "inside"
         foo = bar


Another example:

     x[i] = 12 where:
         i = some_complicated_expression()

'x' is non-local because there's no assignment to 'x' within the 'where'
construct'. 'i' is local because there is an assignment to 'i' within
the 'where' construct'.

More clearly:

     local:
         i = some_complicated_expression()
         x[i] = 12


From tjreedy at udel.edu  Thu Jul 16 20:27:49 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 16 Jul 2009 14:27:49 -0400
Subject: [Python-ideas] Immemorial desire for "do-while"-like
	construction
In-Reply-To: <200907162126.03776.steve@pearwood.info>
References: <op.uw4sdsxffvx12t@jerzozwiesz.home.aster.pl>	<op.uw4wc6nkfvx12t@jerzozwiesz.home.aster.pl>	<e6511dbf0907151732p28defd52m8ec468a4b7cfc722@mail.gmail.com>
	<200907162126.03776.steve@pearwood.info>
Message-ID: <h3nrf5$mv1$1@ger.gmane.org>

Steven D'Aprano wrote:
> On Thu, 16 Jul 2009 10:32:01 am Josiah Carlson wrote:
>> I use 1/0 instead of True/False because older Pythons loaded numeric
>> constants faster than the names True/False (is that still the case?)
> 
>>From Python 2.6.1:
> 
>>>> import dis
>>>> dis.dis(compile("while 1: break", "", "exec"))
>   1           0 SETUP_LOOP               4 (to 7)
>         >>    3 BREAK_LOOP
>               4 JUMP_ABSOLUTE            3
>         >>    7 LOAD_CONST               0 (None)
>              10 RETURN_VALUE
>>>> dis.dis(compile("while True: break", "", "exec"))
>   1           0 SETUP_LOOP              13 (to 16)
>         >>    3 LOAD_NAME                0 (True)
>               6 JUMP_IF_FALSE            5 (to 14)
>               9 POP_TOP
>              10 BREAK_LOOP
>              11 JUMP_ABSOLUTE            3
>         >>   14 POP_TOP
>              15 POP_BLOCK
>         >>   16 LOAD_CONST               0 (None)
>              19 RETURN_VALUE
> 
> 
> True is a constant in Python 3.0.

In 3.1
 >>> dis.dis(compile("while True: break", "", "exec"))
   1           0 SETUP_LOOP               4 (to 7)
         >>    3 BREAK_LOOP
               4 JUMP_ABSOLUTE            3
         >>    7 LOAD_CONST               0 (None)
              10 RETURN_VALUE

Same as while 1

tjr



From tjreedy at udel.edu  Thu Jul 16 20:35:40 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 16 Jul 2009 14:35:40 -0400
Subject: [Python-ideas] Augment dis.dis to autocompile strings
Message-ID: <h3nrtu$oog$1@ger.gmane.org>

Currently
"dis.dis(x=None)
Disassemble the x object. x can denote either a module, a class, a 
method, a function, or a code object. "

The behavior depends on the class of x, so dis already has an internal 
switch.

Currently, to see the assembly equivalent of CPython bytecode for a 
small snippet, one must either wrap it in a function or *remember*
the exact idiom: compile("while True: break", "", "exec")

Proposal: if the input is a string, do the above (do what I mean)
instead of raising
TypeError: don't know how to disassemble str objects

Any negatives before I submit a feature request?

Terry Jan Reedy



From p.f.moore at gmail.com  Thu Jul 16 20:51:43 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 16 Jul 2009 19:51:43 +0100
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <184a9f5a0907160821q79780f12vd6b9a65e870bffd4@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<5d1a32000907150638j65131770gf26df83dbadc5af2@mail.gmail.com>
	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<184a9f5a0907160821q79780f12vd6b9a65e870bffd4@mail.gmail.com>
Message-ID: <79990c6b0907161151x967e8f8ncf0d3d54344177f3@mail.gmail.com>

2009/7/16 Chris Perkins <chrisperkins99 at gmail.com>:
>> You've also disallowed
>>
>> x[i] = 12 where:
>> ? ?i = some_complicated_expression()
>>
>> Was that deliberate? If so, could you explain why now, so that it's on
>> record for the legions of people who will ask after it's implemented?
>> :-)
>
> Well, my original answer was "no, I just didnt' think of that'. ?But
> now that you mention it, I think in the case of an "augassign"
> production, it makes sense to have the left hand side evaluated in the
> current scope (not in the 'where block" scope). ?So in your examle,
> you would either get a NameError for i, the code would do somthing
> that you didn't expect.

And yet the user's intent is obviously to use the i defined inside the
where (that's how functional languages work, so it's hardly an
unrealistic intent).

I'd argue that having this example use a previously defined value of i, when

x = i where:
    i = some_complicated_expression()

uses the i defined in the where clause, is a disaster waiting to happen.

On another point that came up -

i = 5
print i # 5
x[i] = 12 where:
    global i
    i = 3
print i # 3

seems to me to be so unexpected and non-obvious that it makes the
whole construct suspect. Yes, I know that the semantics, as given in
terms of a rewrite, define what happens, but it seems to me that it
goes counter to intuition.

Thinking about it, the issue is that you're making where the *only*
example in Python (that i can think of) of a non-defining construct
which creates a new scope. Currently, scopes are created by the class
and def statements. Both of these define objects (classes and
functions, respectively). The where clause as you've stated it defines
a scope, but no object. That doesn't make it ill-defined, but it does
make it (very, in my view) non-intuitive.

Paul.


From daniel at stutzbachenterprises.com  Thu Jul 16 21:26:08 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Thu, 16 Jul 2009 14:26:08 -0500
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <4A5F6F6B.7030604@mrabarnett.plus.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<eae285400907160837v7f483df9t3d757cd46e2429af@mail.gmail.com>
	<op.uw56fmtffvx12t@jerzozwiesz.home.aster.pl>
	<4A5F6F6B.7030604@mrabarnett.plus.com>
Message-ID: <eae285400907161226g6ce12ccbn82f1369067e3f5e9@mail.gmail.com>

On Thu, Jul 16, 2009 at 1:20 PM, MRAB <python at mrabarnett.plus.com> wrote:

>     bar = "outside"
>    local:
>        nonlocal foo
>        bar = "inside"
>        foo = bar
>

"nonlocal" requires that "foo" is already defined in an enclosing scope.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090716/7d194168/attachment.html>

From cmjohnson.mailinglist at gmail.com  Thu Jul 16 21:48:30 2009
From: cmjohnson.mailinglist at gmail.com (Carl Johnson)
Date: Thu, 16 Jul 2009 09:48:30 -1000
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<4A5D8B63.4040509@canterbury.ac.nz>
	<eae285400907150549k429f1aeha8ab1b85f581eb7@mail.gmail.com>
	<5d1a32000907150638j65131770gf26df83dbadc5af2@mail.gmail.com>
	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
Message-ID: <3bdda690907161248p481123au574e0b0ce171c114@mail.gmail.com>

Chris Perkins wrote:

> Which is an error. However:
> ? ?x = foo(y) where:
> ? ? ? ?y = []
> ? ? ? ?for i in range(n):
> ? ? ? ? ? ?if cond(i):
> ? ? ? ? ? ? ? ?break
> ? ? ? ? ? ?y.append(i)
> must be allowed (right?). So we can't make a blanket restriction on
> break/continue within the body.

Same with return and yield if there's a def inside the where. (Seems
obvious, but you never mentioned it.)

-- Carl Johnson


From zuo at chopin.edu.pl  Thu Jul 16 22:08:46 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Thu, 16 Jul 2009 22:08:46 +0200
Subject: [Python-ideas] Where-statement (Proposal for function
 expressions)
In-Reply-To: <79990c6b0907161151x967e8f8ncf0d3d54344177f3@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<5d1a32000907150638j65131770gf26df83dbadc5af2@mail.gmail.com>
	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<184a9f5a0907160821q79780f12vd6b9a65e870bffd4@mail.gmail.com>
	<79990c6b0907161151x967e8f8ncf0d3d54344177f3@mail.gmail.com>
Message-ID: <op.uw6fkwd9fvx12t@jerzozwiesz.home.aster.pl>

16-07-2009, 20:51 Paul Moore <p.f.moore at gmail.com> wrote:

>>> x[i] = 12 where:
>>> ? ?i = some_complicated_expression()
[snip]
> And yet the user's intent is obviously to use the i defined inside the
> where (that's how functional languages work, so it's hardly an
> unrealistic intent).
[snip]
> Yes, I know that the semantics, as given in
> terms of a rewrite, define what happens, but it seems to me that it
> goes counter to intuition.
>
> Thinking about it, the issue is that you're making where the *only*
> example in Python (that i can think of) of a non-defining construct
> which creates a new scope. Currently, scopes are created by the class
> and def statements. Both of these define objects (classes and
> functions, respectively). The where clause as you've stated it defines
> a scope, but no object. That doesn't make it ill-defined, but it does
> make it (very, in my view) non-intuitive.

Yeah, It is an issue :-/

I can imagine some solutions based on name overriding rules, e.g.:

* in the "target line" only those names are treated as part of nested
where-block scope, which are targets of assignment within the where-block
(as MRAB proposes, if I understood well), or

* in the "target line" only those names are treated as part of nested
where-block scope, which are not targets of assignment within the target
line.

But it seems to me obscure at first sight, and I don't know if
implementation wouldn't be problematic...

Maybe the solution is to make 'where' creating an object, somehow
similarly to the way the 'class' statement works? Some examples:

x[somename.i] = 12 where somename:
     i = some_complicated_expression()

def variance(data):  # for some definition of "variance"
     return var.sqrt(var.sum_of_squares(data)
                     - var.square_of_sums(data)) where var:
         def sqrt...
         def sum_of_squares...
         def square_of_sums...

foo(numbers.x, numbers.y) where numbers:
     x = a * 42
     y = b / 17

a = {}
a[fn.key] = bar(fn.f, fn.g) where fn:
     key = ...
     def f(x):
       ...
     def g(y):
       ...


Another possibility is to allow using "anonymus name":

x[.index] = 12 where:
     index = some_complicated_expression()

foo(.x, .y) where:
     x = a * 42
     y = b / 17


Cheers,

zuo


From daniel at stutzbachenterprises.com  Thu Jul 16 22:30:52 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Thu, 16 Jul 2009 15:30:52 -0500
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <79990c6b0907161151x967e8f8ncf0d3d54344177f3@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<184a9f5a0907160821q79780f12vd6b9a65e870bffd4@mail.gmail.com>
	<79990c6b0907161151x967e8f8ncf0d3d54344177f3@mail.gmail.com>
Message-ID: <eae285400907161330l5f81b1d7g53b54f1004ebfc49@mail.gmail.com>

On Thu, Jul 16, 2009 at 1:51 PM, Paul Moore <p.f.moore at gmail.com> wrote:

> Thinking about it, the issue is that you're making where the *only*
> example in Python (that i can think of) of a non-defining construct
> which creates a new scope.


I think that's why the "where" clause is being proposed: because there's no
construct to create a new scope without defining a new class or function.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090716/48a995aa/attachment.html>

From ncoghlan at gmail.com  Thu Jul 16 23:23:26 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 17 Jul 2009 07:23:26 +1000
Subject: [Python-ideas] Augment dis.dis to autocompile strings
In-Reply-To: <h3nrtu$oog$1@ger.gmane.org>
References: <h3nrtu$oog$1@ger.gmane.org>
Message-ID: <4A5F9A4E.8030106@gmail.com>

Terry Reedy wrote:
> Currently
> "dis.dis(x=None)
> Disassemble the x object. x can denote either a module, a class, a
> method, a function, or a code object. "
> 
> The behavior depends on the class of x, so dis already has an internal
> switch.
> 
> Currently, to see the assembly equivalent of CPython bytecode for a
> small snippet, one must either wrap it in a function or *remember*
> the exact idiom: compile("while True: break", "", "exec")
> 
> Proposal: if the input is a string, do the above (do what I mean)
> instead of raising
> TypeError: don't know how to disassemble str objects
> 
> Any negatives before I submit a feature request?

Not that I can see. +1.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From p.f.moore at gmail.com  Thu Jul 16 23:25:32 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 16 Jul 2009 22:25:32 +0100
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <eae285400907161330l5f81b1d7g53b54f1004ebfc49@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<184a9f5a0907160821q79780f12vd6b9a65e870bffd4@mail.gmail.com>
	<79990c6b0907161151x967e8f8ncf0d3d54344177f3@mail.gmail.com>
	<eae285400907161330l5f81b1d7g53b54f1004ebfc49@mail.gmail.com>
Message-ID: <79990c6b0907161425j4005b05bh6f2801f959207910@mail.gmail.com>

2009/7/16 Daniel Stutzbach <daniel at stutzbachenterprises.com>:
> On Thu, Jul 16, 2009 at 1:51 PM, Paul Moore <p.f.moore at gmail.com> wrote:
>>
>> Thinking about it, the issue is that you're making where the *only*
>> example in Python (that i can think of) of a non-defining construct
>> which creates a new scope.
>
> I think that's why the "where" clause is being proposed: because there's no
> construct to create a new scope without defining a new class or function.

That's a very interesting perspective. Maybe then it would make more
sense to simply bite the bullet and define a scope-creating statement
-

scope:
  i = a_complicated_expression()
  x[i] = 15

Without a way of having some values "escape" the scope, this may be a
bit limited, though. And it's almost certainly not going to appeal to
the people who want to see the expression first, and the subordinate
names defined afterwards. But that's a different underlying concept -
a new construct to execute code in something other than top-to-bottom
order (and one I have less interest in, personally).

Paul.


From python at mrabarnett.plus.com  Thu Jul 16 23:54:50 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 16 Jul 2009 22:54:50 +0100
Subject: [Python-ideas] Where-statement (Proposal for
	function	expressions)
In-Reply-To: <79990c6b0907161425j4005b05bh6f2801f959207910@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>	<m2vdltufis.fsf@cs.uu.nl>	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>	<184a9f5a0907160821q79780f12vd6b9a65e870bffd4@mail.gmail.com>	<79990c6b0907161151x967e8f8ncf0d3d54344177f3@mail.gmail.com>	<eae285400907161330l5f81b1d7g53b54f1004ebfc49@mail.gmail.com>
	<79990c6b0907161425j4005b05bh6f2801f959207910@mail.gmail.com>
Message-ID: <4A5FA1AA.7000509@mrabarnett.plus.com>

Paul Moore wrote:
> 2009/7/16 Daniel Stutzbach <daniel at stutzbachenterprises.com>:
>> On Thu, Jul 16, 2009 at 1:51 PM, Paul Moore <p.f.moore at gmail.com> wrote:
>>> Thinking about it, the issue is that you're making where the *only*
>>> example in Python (that i can think of) of a non-defining construct
>>> which creates a new scope.
>> I think that's why the "where" clause is being proposed: because there's no
>> construct to create a new scope without defining a new class or function.
> 
> That's a very interesting perspective. Maybe then it would make more
> sense to simply bite the bullet and define a scope-creating statement
> -
> 
> scope:
>   i = a_complicated_expression()
>   x[i] = 15
> 
> Without a way of having some values "escape" the scope, this may be a
> bit limited, though. And it's almost certainly not going to appeal to
> the people who want to see the expression first, and the subordinate
> names defined afterwards. But that's a different underlying concept -
> a new construct to execute code in something other than top-to-bottom
> order (and one I have less interest in, personally).
> 

Perhaps:

     scope except foo:
         ...

:-)


From mwm-keyword-python.b4bdba at mired.org  Fri Jul 17 00:51:12 2009
From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer)
Date: Thu, 16 Jul 2009 18:51:12 -0400
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>
	<4A5D8B63.4040509@canterbury.ac.nz>
	<eae285400907150549k429f1aeha8ab1b85f581eb7@mail.gmail.com>
	<5d1a32000907150638j65131770gf26df83dbadc5af2@mail.gmail.com>
	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
Message-ID: <20090716185112.2c7d1a60@bhuda.mired.org>

[ Maybe a little late, but nobody else seems to have noticed this...]

On Thu, 16 Jul 2009 10:51:54 -0400
Chris Perkins <chrisperkins99 at gmail.com> wrote:
> As a first approximation, assume that this code:

This seems reasonable, but there are a couple of questions. Key words:
"first approximation".

> [LHS OP] EXPR where:
>     BODY
> 
> is expanded into the equivalent of this:
> 
> def TEMP():
>     BODY
>     return EXPR
> [LHS OP] TEMP()
> 
> Problem 1) return
> Since there will be an implicit return injected into the body of TEMP,
> what happens if the user puts an explicit return in the body of the
> block:
>     x = foo(y) where:
>         return 23
> expands to
>     def TEMP():
>         return 23
>         return foo(y)
>     x = TEMP()
> So return should be disallowed.
. [ ...]
> Problem 3) yield
>     x = foo(y) where:
>         yield 3
> expands to:
>     def TEMP():
>         yield 3
>         return foo(y)
>     x = TEMP()
> which is an error. So yield must be disallowed.

I disagree with both these conclusions. Clearly, they are paired, as
they serve a similar purpose, but I don't see that they need to be
disallowed at all. That they cause problems with your first attempt at
describing an implementation certainly isn't reason to disallow them
all by itself.

On the other hand, using a where block to create some temporary names
doesn't in any way prevent me from discovering in the middle of
calculating their values that I have a value to return (or yield) from
the enclosing function. This would pretty clearly be a wart should it
be so.

That said - if where can't be implemented in such a way as to allow a
return or yield from inside a where, I don't believe that's sufficient
reason to reject where out of hand. But we shouldn't start by assuming
we have to break it.

   <mike

-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


From zuo at chopin.edu.pl  Fri Jul 17 02:04:05 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Fri, 17 Jul 2009 02:04:05 +0200
Subject: [Python-ideas] Where-statement (Proposal for function
 expressions)
In-Reply-To: <20090716185112.2c7d1a60@bhuda.mired.org>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>
	<4A5D8B63.4040509@canterbury.ac.nz>
	<eae285400907150549k429f1aeha8ab1b85f581eb7@mail.gmail.com>
	<5d1a32000907150638j65131770gf26df83dbadc5af2@mail.gmail.com>
	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<20090716185112.2c7d1a60@bhuda.mired.org>
Message-ID: <op.uw6qg3s7fvx12t@jerzozwiesz.home.aster.pl>

17-07-2009, 00:51 Mike Meyer <mwm-keyword-python.b4bdba at mired.org> wrote:

> That they cause problems with your first attempt at
> describing an implementation certainly isn't reason to disallow them
> all by itself.

But, it isn't a problem of implementation. I don't see any reasonable
*semantics* of return or yield within where-block in proposed form.
return/yield statements seem to be completely unnecessary in this context.
(Of course, I don't say about return/yield statements within functions
defined within where-block).

-- 
Jan Kaliszewski <zuo at chopin.edu.pl>


From chrisperkins99 at gmail.com  Fri Jul 17 02:26:53 2009
From: chrisperkins99 at gmail.com (Chris Perkins)
Date: Thu, 16 Jul 2009 20:26:53 -0400
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <79990c6b0907161151x967e8f8ncf0d3d54344177f3@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<184a9f5a0907160821q79780f12vd6b9a65e870bffd4@mail.gmail.com>
	<79990c6b0907161151x967e8f8ncf0d3d54344177f3@mail.gmail.com>
Message-ID: <184a9f5a0907161726n27b4f55fn79aa1ff0974e0f30@mail.gmail.com>

On Thu, Jul 16, 2009 at 2:51 PM, Paul Moore<p.f.moore at gmail.com> wrote:
> 2009/7/16 Chris Perkins <chrisperkins99 at gmail.com>:
>>> You've also disallowed
>>>
>>> x[i] = 12 where:
>>> ? ?i = some_complicated_expression()
>
> And yet the user's intent is obviously to use the i defined inside the
> where (that's how functional languages work, so it's hardly an
> unrealistic intent).

I don't follow this argument - in what way is that how functional
languages work?

I see that code as analogous to the following Clojure, for example:

(def x (into-array [1 2]))
(def i 0)
(aset x i (let [i 1] 99))

You wouldn't expect x to now be [1 99], rather than [99 2], would you?

> I'd argue that having this example use a previously defined value of i, when
>
> x = i where:
> ? ?i = some_complicated_expression()
>
> uses the i defined in the where clause, is a disaster waiting to happen.

I don't really see the problem with it. If you told people that the
where block only applies to the right-hand-side of an assignment, and
that the left-hand-side is always evaluated outside the scope of the
where block, I doubt there would be much confusion.

> On another point that came up -
>
> i = 5
> print i # 5
> x[i] = 12 where:
> ? ?global i
> ? ?i = 3
> print i # 3
>
> seems to me to be so unexpected and non-obvious that it makes the
> whole construct suspect. Yes, I know that the semantics, as given in
> terms of a rewrite, define what happens, but it seems to me that it
> goes counter to intuition.

Again, I don't think that's a big deal. One solution is to disallow
global statements (if we're already disallowing stuff like break and
return, why not). Another is just "don't do that".

> Thinking about it, the issue is that you're making where the *only*
> example in Python (that i can think of) of a non-defining construct
> which creates a new scope. Currently, scopes are created by the class
> and def statements. Both of these define objects (classes and
> functions, respectively). The where clause as you've stated it defines
> a scope, but no object. That doesn't make it ill-defined, but it does
> make it (very, in my view) non-intuitive.

"Namespaces are one honking great idea -- let's do more of those!"

And everyone knows, you can't argue with the Zen. ;)


Chris Perkins


From python at mrabarnett.plus.com  Fri Jul 17 02:28:35 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 17 Jul 2009 01:28:35 +0100
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <op.uw6qg3s7fvx12t@jerzozwiesz.home.aster.pl>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>	<4A5D8B63.4040509@canterbury.ac.nz>	<eae285400907150549k429f1aeha8ab1b85f581eb7@mail.gmail.com>	<5d1a32000907150638j65131770gf26df83dbadc5af2@mail.gmail.com>	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>	<m2vdltufis.fsf@cs.uu.nl>	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>	<20090716185112.2c7d1a60@bhuda.mired.org>
	<op.uw6qg3s7fvx12t@jerzozwiesz.home.aster.pl>
Message-ID: <4A5FC5B3.3040704@mrabarnett.plus.com>

Jan Kaliszewski wrote:
> 17-07-2009, 00:51 Mike Meyer <mwm-keyword-python.b4bdba at mired.org> wrote:
> 
>> That they cause problems with your first attempt at
>> describing an implementation certainly isn't reason to disallow them
>> all by itself.
> 
> But, it isn't a problem of implementation. I don't see any reasonable
> *semantics* of return or yield within where-block in proposed form.
> return/yield statements seem to be completely unnecessary in this context.
> (Of course, I don't say about return/yield statements within functions
> defined within where-block).
> 
I think that a return, yield, break or continue in a where-block would
in a sense be no different to one in a normal suite, except, of course,
that there happens to be a local namespace.

foo = bar where:
     if condition:
         return
     bar = "bar"

would be equivalent to:

local:
     nonlocal foo
     if condition:
         return
     bar = "bar"
     foo = bar

(I accept that 'nonlocal' might not be the right keyword, but you get
the idea.)


From jimjjewett at gmail.com  Fri Jul 17 03:05:10 2009
From: jimjjewett at gmail.com (Jim Jewett)
Date: Thu, 16 Jul 2009 21:05:10 -0400
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <op.uw6qg3s7fvx12t@jerzozwiesz.home.aster.pl>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<5d1a32000907150638j65131770gf26df83dbadc5af2@mail.gmail.com> 
	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com> 
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com> 
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com> 
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com> 
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com> 
	<20090716185112.2c7d1a60@bhuda.mired.org>
	<op.uw6qg3s7fvx12t@jerzozwiesz.home.aster.pl>
Message-ID: <fb6fbf560907161805s53454d28keec9fda2d973eb59@mail.gmail.com>

On Thu, Jul 16, 2009 at 8:04 PM, Jan Kaliszewski<zuo at chopin.edu.pl> wrote:

> ... I don't see any reasonable *semantics* of
> return or yield within where-block in proposed form.

next_batch = myfilter(candidates, failed) where:
    failed=[]
    for x in this_batch:
        if magical(x):
            # We actually found one!!!
            return x
        # more failures, of course.
        # bias the next batch to a different section
        # of the search space.
        failed.append(x)

I leave to others the judgment on whether or not any function
containing code is already too long...

alternatively,

provide(foo(item)) where:
    start_to_generate_an_item()
    if flag():
        # Are there reasons to return instead of throwing?
        raise HigherPriority
    item=finish_generation()

Again, I'm not certain that the style should be encouraged, but
neither I am sure it shouldn't be.

-jJ


From greg.ewing at canterbury.ac.nz  Fri Jul 17 04:02:34 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 17 Jul 2009 14:02:34 +1200
Subject: [Python-ideas] Where-statement (Proposal for
	function	expressions)
In-Reply-To: <eae285400907160837v7f483df9t3d757cd46e2429af@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<eae285400907150549k429f1aeha8ab1b85f581eb7@mail.gmail.com>
	<5d1a32000907150638j65131770gf26df83dbadc5af2@mail.gmail.com>
	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<eae285400907160837v7f483df9t3d757cd46e2429af@mail.gmail.com>
Message-ID: <4A5FDBBA.6@canterbury.ac.nz>

Daniel Stutzbach wrote:

> If the
> left-hand side were in the where-block's scope, then:
> 
> x = 0
> x = i where:
>    i = 5
> print x
> 
> Would print "0" not "5".

My intuitive expectations are different for assignment
to a bare name vs. assignment to an item or attribute.
In

   x = i where:
     i = 5

I would expect x to be bound in the current scope,
not the where-block scope. But I would expect

   x[i] = 5 where:
     x = y
     i = 7

to be equivalent to

   y[7] = 5

i.e. both x and i are *evaluated* in the where-block
scope.

I'm not sure where this leaves us with respect to
augmented assignment, though. If you follow it to
its logical conclusion, then

   x += i where:
     i = 5

should be equivalent to

   x = x.__iadd__(i) where:
     i = 5

and then the evaluation and rebinding of 'x' would
happen in different scopes!

-- 
Greg


From dangyogi at gmail.com  Fri Jul 17 04:19:59 2009
From: dangyogi at gmail.com (Bruce Frederiksen)
Date: Thu, 16 Jul 2009 22:19:59 -0400
Subject: [Python-ideas] Where-statement (Proposal for
	function	expressions)
In-Reply-To: <20090716185112.2c7d1a60@bhuda.mired.org>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>	<4A5D8B63.4040509@canterbury.ac.nz>	<eae285400907150549k429f1aeha8ab1b85f581eb7@mail.gmail.com>	<5d1a32000907150638j65131770gf26df83dbadc5af2@mail.gmail.com>	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>	<m2vdltufis.fsf@cs.uu.nl>	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<20090716185112.2c7d1a60@bhuda.mired.org>
Message-ID: <4A5FDFCF.5070202@gmail.com>

Mike Meyer wrote:
> I disagree with both these conclusions. Clearly, they are paired, as
> they serve a similar purpose, but I don't see that they need to be
> disallowed at all. That they cause problems with your first attempt at
> describing an implementation certainly isn't reason to disallow them
> all by itself.
>   
Let me toss out an alternate implementation.

statement where:
     body

creates a new temporary local context and evaluates the body in it.  Any 
variables assigned to within the body are set in this temporary context 
(as you'd expect).  Variables in the outer context are still visible to 
the body.

Now the difference:

Then the statement is run within this temporary local context, but with 
a change in that now all variables assigned to bypass the temporary 
context (perhaps unbinding the variable there, if it's set, so that it 
no longer hides the outer value?) and are set in the outer context.  But 
variables within the temporary context are still visible to the 
statement.  This is a new use of contexts for Python.

Once the statement is finished executing, the temporary context is 
discarded.

Thus:

i = 2
x[27] = 2
x[i] = 4 where:
    i = 27
print(x[27]) # 4
print(i) # 2

has access to the temporary i (set to 27).

While:

x = 1
x = i where:
    i = 27
print(x)  # 27

assigns 27 to x in the outer context.

Thus, the where clause applies to the whole statement, without limiting 
the statement's ability to bind variables in the outer context.

-Bruce


From mwm-keyword-python.b4bdba at mired.org  Fri Jul 17 04:25:41 2009
From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer)
Date: Thu, 16 Jul 2009 22:25:41 -0400
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <4A5FDBBA.6@canterbury.ac.nz>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<eae285400907150549k429f1aeha8ab1b85f581eb7@mail.gmail.com>
	<5d1a32000907150638j65131770gf26df83dbadc5af2@mail.gmail.com>
	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<eae285400907160837v7f483df9t3d757cd46e2429af@mail.gmail.com>
	<4A5FDBBA.6@canterbury.ac.nz>
Message-ID: <20090716222541.03ee1ac6@bhuda.mired.org>

On Fri, 17 Jul 2009 14:02:34 +1200
Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> Daniel Stutzbach wrote:
> 
> > If the
> > left-hand side were in the where-block's scope, then:
> > 
> > x = 0
> > x = i where:
> >    i = 5
> > print x
> > 
> > Would print "0" not "5".
> 
> My intuitive expectations are different for assignment
> to a bare name vs. assignment to an item or attribute.
> In
> 
>    x = i where:
>      i = 5
> 
> I would expect x to be bound in the current scope,
> not the where-block scope. But I would expect
> 
>    x[i] = 5 where:
>      x = y
>      i = 7
> 
> to be equivalent to
> 
>    y[7] = 5
> 
> i.e. both x and i are *evaluated* in the where-block
> scope.
> 
> I'm not sure where this leaves us with respect to
> augmented assignment, though. If you follow it to
> its logical conclusion, then
> 
>    x += i where:
>      i = 5
> 
> should be equivalent to
> 
>    x = x.__iadd__(i) where:
>      i = 5
> 
> and then the evaluation and rebinding of 'x' would
> happen in different scopes!

That evaluating an expression to the right of an augmented assignment
might return something different than what you would bind that
expression to has always been an issue with augmented assignments. I
believe this particular problem belongs has more to do with them than
with the proposed where statement.

I think the intuitive behavior doesn't involve position relative to
the assignment, but binding vs. non-bindings. What you *expect* is
that expressions left of the where will be evaluated in the inner
scope (created by the where), but that any bindings that happen there
will happen in the outer scope (before the where).

That makes what you want to happen happen - *most* of the time:

i = 3
x += i where:
   i = 5

increments x by 5.

i = 'a'
x[i] = 'b' where:
   i = 'c'

sets x['c'] to 'b'.

Things get really ugly when the bound variables start appearing in
both scopes combined with augmented assignments:

x = 3
x += 4 where:
    x = 5
print x

What should this print?

     <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


From steve at pearwood.info  Fri Jul 17 04:32:21 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 17 Jul 2009 12:32:21 +1000
Subject: [Python-ideas] Augment dis.dis to autocompile strings
In-Reply-To: <h3nrtu$oog$1@ger.gmane.org>
References: <h3nrtu$oog$1@ger.gmane.org>
Message-ID: <200907171232.21702.steve@pearwood.info>

On Fri, 17 Jul 2009 04:35:40 am Terry Reedy wrote:

> Proposal: if the input is a string, do the above (do what I mean)
> instead of raising
> TypeError: don't know how to disassemble str objects
>
> Any negatives before I submit a feature request?

+1 on the idea, but how do you determine which of the following are 
required?

compile('x+1', '', 'eval')
compile('x = x+1', '', 'single')
compile('while x < 42: x += 1', '', 'exec')


Or do you just assume 'exec'?


-- 
Steven D'Aprano


From greg.ewing at canterbury.ac.nz  Fri Jul 17 04:38:41 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 17 Jul 2009 14:38:41 +1200
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <op.uw57mcv8fvx12t@jerzozwiesz.home.aster.pl>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<fb6fbf560907121731u2578ecd1o739fa0b78d1a69ab@mail.gmail.com>
	<184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>
	<de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>
	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>
	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>
	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>
	<4A5D8B63.4040509@canterbury.ac.nz>
	<op.uw57mcv8fvx12t@jerzozwiesz.home.aster.pl>
Message-ID: <4A5FE431.1070407@canterbury.ac.nz>

Jan Kaliszewski wrote:

> while foo(x) and bar() where:
>     foo = SOMETHING
>     def bar():
>         SOMETHING ELSE
> do:
>     THE
>     LOOP
>     BODY

That's kind of clever, although it does require
modifying the while-loop syntax, and if you're
doing that, there's not a lot of gain over
introducing a dedicated loop-and-a-half syntax.

Also, once we're allowed 'where' clauses on
while statements, people are going to want them
on other kinds of statements as well:

   if a == b where:
     a = foo()
     b = blarg()
   then:
     ...

   class Ham(Spam) where:
     class Spam:
       ...
   is:
     ...

   def f(x = default) where:
     default = whatever
   as:
     ...

   try:
     ...
   except eels where:
     eels = something_bad()
   do:
     ...

Where do we stop?

> while foo(x) and bar():
>     where:
>         foo = SOMETHING
>         def bar():
>             SOMETHING ELSE
>     THE
>     LOOP
>     BODY

I think I liked the first version better. There's
something jarring about the unindent half way
through without any keyword to mark the boundary.
Also the bare 'where' relating to the previous line
suggests some weird grammatical gymnastics going
on.

-- 
Greg


From greg.ewing at canterbury.ac.nz  Fri Jul 17 04:48:31 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 17 Jul 2009 14:48:31 +1200
Subject: [Python-ideas] Where-statement (Proposal for
	function	expressions)
In-Reply-To: <79990c6b0907161151x967e8f8ncf0d3d54344177f3@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<5d1a32000907150638j65131770gf26df83dbadc5af2@mail.gmail.com>
	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<184a9f5a0907160821q79780f12vd6b9a65e870bffd4@mail.gmail.com>
	<79990c6b0907161151x967e8f8ncf0d3d54344177f3@mail.gmail.com>
Message-ID: <4A5FE67F.3080408@canterbury.ac.nz>

Paul Moore wrote:

> Thinking about it, the issue is that you're making where the *only*
> example in Python (that i can think of) of a non-defining construct
> which creates a new scope.

Not quite true any more, since LCs and generator
expressions now use a local scope for their
iteration variables.

However in this case, it might just be less
confusing overall to not have a new scope for
the where-block. It's not necessary in order
to get the main intended benefit.

-- 
Greg


From greg.ewing at canterbury.ac.nz  Fri Jul 17 04:54:35 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 17 Jul 2009 14:54:35 +1200
Subject: [Python-ideas] Where-statement (Proposal for
	function	expressions)
In-Reply-To: <eae285400907161330l5f81b1d7g53b54f1004ebfc49@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<184a9f5a0907160821q79780f12vd6b9a65e870bffd4@mail.gmail.com>
	<79990c6b0907161151x967e8f8ncf0d3d54344177f3@mail.gmail.com>
	<eae285400907161330l5f81b1d7g53b54f1004ebfc49@mail.gmail.com>
Message-ID: <4A5FE7EB.4020307@canterbury.ac.nz>

Daniel Stutzbach wrote:

> I think that's why the "where" clause is being proposed: because there's no
> construct to create a new scope without defining a new class or function.

No, the reason is to provide a way to put
implementation details in a convenient place
after where they're used, but not too far
away.

-- 
Greg


From steve at pearwood.info  Fri Jul 17 04:57:11 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 17 Jul 2009 12:57:11 +1000
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <184a9f5a0907161058v32f0f6dey6e5838333c5998bd@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<200907170313.19012.steve@pearwood.info>
	<184a9f5a0907161058v32f0f6dey6e5838333c5998bd@mail.gmail.com>
Message-ID: <200907171257.11440.steve@pearwood.info>

On Fri, 17 Jul 2009 03:58:21 am Chris Perkins wrote:

> > In either case, writing the sums-and-squares before the return
> > suggests that these are common code best written as re-usable
> > functions. But "where" changes the focus of the coder away from
> > factoring code *out* of functions into placing code *inside*
> > internal blocks. It encourages the coder to think of sum_of_squares
> > as a private implementation detail instead of a re-usable
> > component. And that is, I believe, harmful.
>
> But surely you agree that _sometimes_ there is code that really is an
> implementation detail - not everything is reusable, or at least not
> usefully reusable. 

Of course. What your suggestion gives us, though, is a second way of 
including multi-line functions:

def parrot():
    x = 1
    y = 2
    return x + y


versus


def parrot():
    return x + y where:
        x = 1
        y = 2

I'm not convinced that allowing the second is either necessary or 
beneficial.


> Taking your argument to the extreme, every 
> function would be only one line of code. I prefer to trust
> programmers to make the best use of the tools available to create
> good code.

The Python community takes the attitude that we're all consenting adults 
here, and that if you want to shoot yourself in the foot, you should be 
allowed to. And that's fine. But it's only fine because Python, by and 
large, is a safe language, one with very few jagged edges, which 
encourages good practices rather than bad.

Which brings me to this:

> In other words:
>
> Jan Kaliszewski:
> >Every language feature can be misused. More powerful feature --
> > larger possibilities of misusing. But language is not for
> > substituting programmer's sense.
>
> Yeah, ditto to what he said.

Some language features are begging to be misused, and although I've 
occasionally wanted a way to define code blocks on the fly outside of a 
function, I'm leaning towards the conclusion that this suggestion is a 
feature where the abuses outweigh the benefits.



-- 
Steven D'Aprano


From zuo at chopin.edu.pl  Fri Jul 17 10:09:20 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Fri, 17 Jul 2009 10:09:20 +0200
Subject: [Python-ideas] Where-statement (Proposal for function
 expressions)
In-Reply-To: <20090716222541.03ee1ac6@bhuda.mired.org>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<eae285400907150549k429f1aeha8ab1b85f581eb7@mail.gmail.com>
	<5d1a32000907150638j65131770gf26df83dbadc5af2@mail.gmail.com>
	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<eae285400907160837v7f483df9t3d757cd46e2429af@mail.gmail.com>
	<4A5FDBBA.6@canterbury.ac.nz> <20090716222541.03ee1ac6@bhuda.mired.org>
Message-ID: <op.uw7cxuzjfvx12t@jerzozwiesz.home.aster.pl>

17-07-2009, 04:25
Mike Meyer <mwm-keyword-python.b4bdba at mired.org> wrote:

> On Fri, 17 Jul 2009 14:02:34 +1200
> Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>
>> Daniel Stutzbach wrote:
>>
>> > If the
>> > left-hand side were in the where-block's scope, then:
>> >
>> > x = 0
>> > x = i where:
>> >    i = 5
>> > print x
>> >
>> > Would print "0" not "5".
>>
>> My intuitive expectations are different for assignment
>> to a bare name vs. assignment to an item or attribute.
>> In
>>
>>    x = i where:
>>      i = 5
>>
>> I would expect x to be bound in the current scope,
>> not the where-block scope. But I would expect
>>
>>    x[i] = 5 where:
>>      x = y
>>      i = 7
>>
>> to be equivalent to
>>
>>    y[7] = 5
>>
>> i.e. both x and i are *evaluated* in the where-block
>> scope.
>>
>> I'm not sure where this leaves us with respect to
>> augmented assignment, though. If you follow it to
>> its logical conclusion, then
>>
>>    x += i where:
>>      i = 5
>>
>> should be equivalent to
>>
>>    x = x.__iadd__(i) where:
>>      i = 5
>>
>> and then the evaluation and rebinding of 'x' would
>> happen in different scopes!
[snip]
> I think the intuitive behavior doesn't involve position relative to
> the assignment, but binding vs. non-bindings. What you *expect* is
> that expressions left of the where will be evaluated in the inner
> scope (created by the where), but that any bindings that happen there
> will happen in the outer scope (before the where).
>
> That makes what you want to happen happen - *most* of the time:
>
> i = 3
> x += i where:
>    i = 5
>
> increments x by 5.
>
> i = 'a'
> x[i] = 'b' where:
>    i = 'c'
>
> sets x['c'] to 'b'.
>
> Things get really ugly when the bound variables start appearing in
> both scopes combined with augmented assignments:
>
> x = 3
> x += 4 where:
>     x = 5
> print x
>
> What should this print?

I think, a good rule could be that:

All *expressions* of the target line are evaluated within the
where-block scope; and -- moreover -- those names (variables) that
are *bound* in the target line, are "leaked" to the outer scope.

[target line == the line with where statement]

Then the former example code should print 9.

-- 
Jan Kaliszewski <zuo at chopin.edu.pl>


From daniel at stutzbachenterprises.com  Fri Jul 17 10:20:08 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Fri, 17 Jul 2009 03:20:08 -0500
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <4A5FDBBA.6@canterbury.ac.nz>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<eae285400907160837v7f483df9t3d757cd46e2429af@mail.gmail.com>
	<4A5FDBBA.6@canterbury.ac.nz>
Message-ID: <eae285400907170120u5932eef2obaa667ccd87c30b7@mail.gmail.com>

On Thu, Jul 16, 2009 at 9:02 PM, Greg Ewing <greg.ewing at canterbury.ac.nz>wrote:

> I would expect x to be bound in the current scope,
> not the where-block scope. But I would expect
>
>  x[i] = 5 where:
>    x = y
>    i = 7
>
> to be equivalent to
>
>  y[7] = 5
>

How about if attempts to re-bind variables in outer scopes should throw an
exception?  Much like they already do in this example:

>>> x = 5
>>> def foo():
...   blah = x
...   x = 6
...
>>> foo()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in foo
UnboundLocalError: local variable 'x' referenced before assignment

(rebinding is allowed if "global" or "nonlocal" is explicitly used, of
course)

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090717/134bf994/attachment.html>

From dstanek at dstanek.com  Fri Jul 17 13:50:55 2009
From: dstanek at dstanek.com (David Stanek)
Date: Fri, 17 Jul 2009 07:50:55 -0400
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <eae285400907170120u5932eef2obaa667ccd87c30b7@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<eae285400907160837v7f483df9t3d757cd46e2429af@mail.gmail.com>
	<4A5FDBBA.6@canterbury.ac.nz>
	<eae285400907170120u5932eef2obaa667ccd87c30b7@mail.gmail.com>
Message-ID: <de32cc030907170450x2f40671bj35c6322142012cb2@mail.gmail.com>

On Fri, Jul 17, 2009 at 4:20 AM, Daniel
Stutzbach<daniel at stutzbachenterprises.com> wrote:
> On Thu, Jul 16, 2009 at 9:02 PM, Greg Ewing <greg.ewing at canterbury.ac.nz>
> wrote:
>>
>> I would expect x to be bound in the current scope,
>> not the where-block scope. But I would expect
>>
>> ?x[i] = 5 where:
>> ? ?x = y
>> ? ?i = 7
>>
>> to be equivalent to
>>
>> ?y[7] = 5
>
> How about if attempts to re-bind variables in outer scopes should throw an
> exception?? Much like they already do in this example:
>
>>>> x = 5
>>>> def foo():
> ...?? blah = x
> ...?? x = 6
> ...
>>>> foo()
> Traceback (most recent call last):
> ? File "<stdin>", line 1, in <module>
> ? File "<stdin>", line 2, in foo
> UnboundLocalError: local variable 'x' referenced before assignment
>
> (rebinding is allowed if "global" or "nonlocal" is explicitly used, of
> course)
>

This thread is a little confusing to me. I am failing to see how the where
block makes anything clearer. The fact that a new scope is created and names
are available during a singe line of execution scares me.

I also worry that this will encourage code duplication. Any substantial logic
should be rolling into a function not into a where block.

-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek


From ben+python at benfinney.id.au  Fri Jul 17 14:12:40 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 17 Jul 2009 22:12:40 +1000
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<eae285400907160837v7f483df9t3d757cd46e2429af@mail.gmail.com>
	<4A5FDBBA.6@canterbury.ac.nz>
	<eae285400907170120u5932eef2obaa667ccd87c30b7@mail.gmail.com>
	<de32cc030907170450x2f40671bj35c6322142012cb2@mail.gmail.com>
Message-ID: <87my73fqyv.fsf@benfinney.id.au>

David Stanek <dstanek at dstanek.com> writes:

> This thread is a little confusing to me. I am failing to see how the
> where block makes anything clearer.

+1. I haven't found that any of the examples improve the clarity of the
code over using a named function.

-- 
 \        ?If I melt dry ice, can I swim without getting wet?? ?Steven |
  `\                                                            Wright |
_o__)                                                                  |
Ben Finney



From python at mrabarnett.plus.com  Fri Jul 17 15:06:14 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 17 Jul 2009 14:06:14 +0100
Subject: [Python-ideas] Where-statement (Proposal for
	function	expressions)
In-Reply-To: <20090716222541.03ee1ac6@bhuda.mired.org>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>	<eae285400907150549k429f1aeha8ab1b85f581eb7@mail.gmail.com>	<5d1a32000907150638j65131770gf26df83dbadc5af2@mail.gmail.com>	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>	<m2vdltufis.fsf@cs.uu.nl>	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>	<eae285400907160837v7f483df9t3d757cd46e2429af@mail.gmail.com>	<4A5FDBBA.6@canterbury.ac.nz>
	<20090716222541.03ee1ac6@bhuda.mired.org>
Message-ID: <4A607746.4000208@mrabarnett.plus.com>

Mike Meyer wrote:
> On Fri, 17 Jul 2009 14:02:34 +1200
> Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> 
>> Daniel Stutzbach wrote:
>>
>>> If the
>>> left-hand side were in the where-block's scope, then:
>>>
>>> x = 0
>>> x = i where:
>>>    i = 5
>>> print x
>>>
>>> Would print "0" not "5".
>> My intuitive expectations are different for assignment
>> to a bare name vs. assignment to an item or attribute.
>> In
>>
>>    x = i where:
>>      i = 5
>>
>> I would expect x to be bound in the current scope,
>> not the where-block scope. But I would expect
>>
>>    x[i] = 5 where:
>>      x = y
>>      i = 7
>>
>> to be equivalent to
>>
>>    y[7] = 5
>>
>> i.e. both x and i are *evaluated* in the where-block
>> scope.
>>
>> I'm not sure where this leaves us with respect to
>> augmented assignment, though. If you follow it to
>> its logical conclusion, then
>>
>>    x += i where:
>>      i = 5
>>
>> should be equivalent to
>>
>>    x = x.__iadd__(i) where:
>>      i = 5
>>
>> and then the evaluation and rebinding of 'x' would
>> happen in different scopes!
> 
> That evaluating an expression to the right of an augmented assignment
> might return something different than what you would bind that
> expression to has always been an issue with augmented assignments. I
> believe this particular problem belongs has more to do with them than
> with the proposed where statement.
> 
> I think the intuitive behavior doesn't involve position relative to
> the assignment, but binding vs. non-bindings. What you *expect* is
> that expressions left of the where will be evaluated in the inner
> scope (created by the where), but that any bindings that happen there
> will happen in the outer scope (before the where).
> 
> That makes what you want to happen happen - *most* of the time:
> 
> i = 3
> x += i where:
>    i = 5
> 
> increments x by 5.
> 
> i = 'a'
> x[i] = 'b' where:
>    i = 'c'
> 
> sets x['c'] to 'b'.
> 
> Things get really ugly when the bound variables start appearing in
> both scopes combined with augmented assignments:
> 
> x = 3
> x += 4 where:
>     x = 5
> print x
> 
> What should this print?
> 
The target line assigns to x, therefore x is not local. It should print 9.

x = 3
local:
     nonlocal x
     x = 5
     x += 4 # The target line.


From python at mrabarnett.plus.com  Fri Jul 17 15:09:23 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 17 Jul 2009 14:09:23 +0100
Subject: [Python-ideas] Where-statement (Proposal for
	function	expressions)
In-Reply-To: <4A5FE431.1070407@canterbury.ac.nz>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>	<fb6fbf560907121731u2578ecd1o739fa0b78d1a69ab@mail.gmail.com>	<184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>	<de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>	<4A5D8B63.4040509@canterbury.ac.nz>	<op.uw57mcv8fvx12t@jerzozwiesz.home.aster.pl>
	<4A5FE431.1070407@canterbury.ac.nz>
Message-ID: <4A607803.2030801@mrabarnett.plus.com>

Greg Ewing wrote:
> Jan Kaliszewski wrote:
> 
>> while foo(x) and bar() where:
>>     foo = SOMETHING
>>     def bar():
>>         SOMETHING ELSE
>> do:
>>     THE
>>     LOOP
>>     BODY
> 
> That's kind of clever, although it does require
> modifying the while-loop syntax, and if you're
> doing that, there's not a lot of gain over
> introducing a dedicated loop-and-a-half syntax.
> 
> Also, once we're allowed 'where' clauses on
> while statements, people are going to want them
> on other kinds of statements as well:
> 
>   if a == b where:
>     a = foo()
>     b = blarg()
>   then:
>     ...
> 
>   class Ham(Spam) where:
>     class Spam:
>       ...
>   is:
>     ...
> 
>   def f(x = default) where:
>     default = whatever
>   as:
>     ...
> 
>   try:
>     ...
>   except eels where:
>     eels = something_bad()
>   do:
>     ...
> 
> Where do we stop?
> 
>> while foo(x) and bar():
>>     where:
>>         foo = SOMETHING
>>         def bar():
>>             SOMETHING ELSE
>>     THE
>>     LOOP
>>     BODY
> 
> I think I liked the first version better. There's
> something jarring about the unindent half way
> through without any keyword to mark the boundary.
> Also the bare 'where' relating to the previous line
> suggests some weird grammatical gymnastics going
> on.
> 
"do" might be a sufficiently generic word to cover all the cases.

     while ... where:
         ...
     do:
         ...


     if a == b where:
         ...
     do:
         ...

etc.


From p.f.moore at gmail.com  Fri Jul 17 15:23:29 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 17 Jul 2009 14:23:29 +0100
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <184a9f5a0907161726n27b4f55fn79aa1ff0974e0f30@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<184a9f5a0907160821q79780f12vd6b9a65e870bffd4@mail.gmail.com>
	<79990c6b0907161151x967e8f8ncf0d3d54344177f3@mail.gmail.com>
	<184a9f5a0907161726n27b4f55fn79aa1ff0974e0f30@mail.gmail.com>
Message-ID: <79990c6b0907170623y2fec30e4h2a44a300f301adf7@mail.gmail.com>

2009/7/17 Chris Perkins <chrisperkins99 at gmail.com>:
> On Thu, Jul 16, 2009 at 2:51 PM, Paul Moore<p.f.moore at gmail.com> wrote:
>> 2009/7/16 Chris Perkins <chrisperkins99 at gmail.com>:
>>>> You've also disallowed
>>>>
>>>> x[i] = 12 where:
>>>> ? ?i = some_complicated_expression()
>>
>> And yet the user's intent is obviously to use the i defined inside the
>> where (that's how functional languages work, so it's hardly an
>> unrealistic intent).
>
> I don't follow this argument - in what way is that how functional
> languages work?

Sorry, I should have said "Haskell, and some other functional
languages I vaguely recall working similarly" :-)

In Haskell, the expressions given names in the where clause are used
throughout the expression to the left of the where. Of course
immutability and referential transparency makes that a lot less
controversial...

> I don't really see the problem with it. If you told people that the
> where block only applies to the right-hand-side of an assignment, and
> that the left-hand-side is always evaluated outside the scope of the
> where block, I doubt there would be much confusion.

That somewhat assumes that a where block is only used for assignments
(or more accurately, that there's a special exception for the left
side of an assignment not being included). That's at best a wart.

> Again, I don't think that's a big deal. One solution is to disallow
> global statements (if we're already disallowing stuff like break and
> return, why not). Another is just "don't do that".

More warts. At some point, the weight of the various exceptions make
the basic idea not worth it.

> "Namespaces are one honking great idea -- let's do more of those!"
>
> And everyone knows, you can't argue with the Zen. ;)

"If the implementation is hard to explain, it's a bad idea."
"Simple is better than complex."

But the Zen can have arguments with itself :-)

Paul.


From gerald.britton at gmail.com  Fri Jul 17 15:25:15 2009
From: gerald.britton at gmail.com (Gerald Britton)
Date: Fri, 17 Jul 2009 09:25:15 -0400
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <4A607803.2030801@mrabarnett.plus.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com> 
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com> 
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com> 
	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com> 
	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com> 
	<4A5D8B63.4040509@canterbury.ac.nz>
	<op.uw57mcv8fvx12t@jerzozwiesz.home.aster.pl> 
	<4A5FE431.1070407@canterbury.ac.nz>
	<4A607803.2030801@mrabarnett.plus.com>
Message-ID: <5d1a32000907170625v5ff9dd2dmddbf83e17f183e71@mail.gmail.com>

Sometimes I use helper functions to make things easier to read.
However, I don't like unnecessary function calls since they are
expensive in Python compared to other languages. Still I might do
this:

def foo(x):
  def bar(y):
      ...
      return ... #something depending on y
  ...
  return bar(z)

If I had a "where" clause I could write:

def foo(x):
   ...
   return y where:
        #compute y


Using "where" would have the advantage of avoiding a function call,
though I suppose it wouldn't save (much) stack space since it needs a
new scope anyway.  I see the "where" clause as a useful way to
abstract a section of code to keep the high level in view, then break
it down as you go.


From p.f.moore at gmail.com  Fri Jul 17 15:36:10 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 17 Jul 2009 14:36:10 +0100
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <5d1a32000907170625v5ff9dd2dmddbf83e17f183e71@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>
	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>
	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>
	<4A5D8B63.4040509@canterbury.ac.nz>
	<op.uw57mcv8fvx12t@jerzozwiesz.home.aster.pl>
	<4A5FE431.1070407@canterbury.ac.nz>
	<4A607803.2030801@mrabarnett.plus.com>
	<5d1a32000907170625v5ff9dd2dmddbf83e17f183e71@mail.gmail.com>
Message-ID: <79990c6b0907170636p355dce83i328b9ca200d8f069@mail.gmail.com>

2009/7/17 Gerald Britton <gerald.britton at gmail.com>:
> Using "where" would have the advantage of avoiding a function call,
> though I suppose it wouldn't save (much) stack space since it needs a
> new scope anyway. ?I see the "where" clause as a useful way to
> abstract a section of code to keep the high level in view, then break
> it down as you go.

You're assuming that the implementation of where is faster than a
function call. Given that the defined semantics are in terms of
defining then calling a temporary function, this may well be an
invalid assumption. (And if it isn't, then can the tricks which get
used to speed up the where cause please be used to speed up function
calls in general, too - as that would be far more generally useful!
:-))

Paul.


From steve at pearwood.info  Fri Jul 17 16:24:30 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 18 Jul 2009 00:24:30 +1000
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <5d1a32000907170625v5ff9dd2dmddbf83e17f183e71@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<4A607803.2030801@mrabarnett.plus.com>
	<5d1a32000907170625v5ff9dd2dmddbf83e17f183e71@mail.gmail.com>
Message-ID: <200907180024.31209.steve@pearwood.info>

On Fri, 17 Jul 2009 11:25:15 pm Gerald Britton wrote:
> Sometimes I use helper functions to make things easier to read.
> However, I don't like unnecessary function calls since they are
> expensive in Python compared to other languages. 

That's only a problem if:

(1) Your code actually is too slow; and

(2) You have profiled your code and discovered that the biggest 
contributor to your code's slowness is the cost of calling functions, 
not the execution time of the functions.

Unless both of these cases hold, then I suggest you are guilty of 
premature optimization. Possibly even pessimation -- given the 
complexities of CPU caches, memory, swap, etc, it's possible that your 
code could be *slower*, not faster, due to your "optimization".

People's intuitions about what's fast and what's slow in Python code are 
often completely at odds with reality. Unless you actually have a 
problem with slow code that needs solving, my advice is to not worry 
about it. Use good algorithms, use tried and tested fast data 
structures, and worry about micro-optimizations when you *actually* 
need them.



-- 
Steven D'Aprano


From stephen at xemacs.org  Fri Jul 17 16:31:04 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 17 Jul 2009 23:31:04 +0900
Subject: [Python-ideas] Where-statement (Proposal for
	function	expressions)
In-Reply-To: <79990c6b0907170623y2fec30e4h2a44a300f301adf7@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<184a9f5a0907160821q79780f12vd6b9a65e870bffd4@mail.gmail.com>
	<79990c6b0907161151x967e8f8ncf0d3d54344177f3@mail.gmail.com>
	<184a9f5a0907161726n27b4f55fn79aa1ff0974e0f30@mail.gmail.com>
	<79990c6b0907170623y2fec30e4h2a44a300f301adf7@mail.gmail.com>
Message-ID: <873a8vidp3.fsf@uwakimon.sk.tsukuba.ac.jp>

Paul Moore writes:

 > But the Zen can have arguments with itself :-)

If it didn't, it wouldn't be the Zen.



From arnodel at googlemail.com  Fri Jul 17 18:35:52 2009
From: arnodel at googlemail.com (Arnaud Delobelle)
Date: Fri, 17 Jul 2009 17:35:52 +0100
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <4A5FDBBA.6@canterbury.ac.nz>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<eae285400907150549k429f1aeha8ab1b85f581eb7@mail.gmail.com>
	<5d1a32000907150638j65131770gf26df83dbadc5af2@mail.gmail.com>
	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<eae285400907160837v7f483df9t3d757cd46e2429af@mail.gmail.com>
	<4A5FDBBA.6@canterbury.ac.nz>
Message-ID: <4F6074ED-E0C6-4899-999F-8AD18F1CA13E@googlemail.com>


On 17 Jul 2009, at 03:02, Greg Ewing wrote:
>
>  [...] I would expect
>
>  x[i] = 5 where:
>    x = y
>    i = 7
>
> to be equivalent to
>
>  y[7] = 5
>
> i.e. both x and i are *evaluated* in the where-block
> scope.
>

So would I, given that the statement
	
	x[i] = 5

is the really the same as the expression

	x.__setitem__(i, 5)

> I'm not sure where this leaves us with respect to
> augmented assignment, though. If you follow it to
> its logical conclusion, then
>
>  x += i where:
>    i = 5
>
> should be equivalent to
>
>  x = x.__iadd__(i) where:
>    i = 5
>
> and then the evaluation and rebinding of 'x' would
> happen in different scopes!

Indeed.  But why not simply evaluate the whole statement in the where  
scope, with all names not bound in the where scope declared as nonlocal?

i.e

	x[i], y = foo(a), b where:
   		i = 7
		def foo(x):
			return bar(x, x)

would be equivalent to:

	def _():
		nonlocal x, y, b
		i = 7
		def foo(x):
			return bar(x, x)
		x[i], y = foo(a), b
	_()

-- 
Arnaud

		 


From tjreedy at udel.edu  Fri Jul 17 21:07:04 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 17 Jul 2009 15:07:04 -0400
Subject: [Python-ideas] Augment dis.dis to autocompile strings
In-Reply-To: <200907171232.21702.steve@pearwood.info>
References: <h3nrtu$oog$1@ger.gmane.org>
	<200907171232.21702.steve@pearwood.info>
Message-ID: <h3qi4p$52s$1@ger.gmane.org>

Steven D'Aprano wrote:
> On Fri, 17 Jul 2009 04:35:40 am Terry Reedy wrote:
> 
>> Proposal: if the input is a string, do the above (do what I mean)
>> instead of raising
>> TypeError: don't know how to disassemble str objects
>>
>> Any negatives before I submit a feature request?
> 
> +1 on the idea, but how do you determine which of the following are 
> required?
> 
> compile('x+1', '', 'eval')
> compile('x = x+1', '', 'single')
> compile('while x < 42: x += 1', '', 'exec')
> 
> 
> Or do you just assume 'exec'?

Good question. Let us see what difference it makes in 3.1

 >>> dis(compile('x+1','', 'eval'))
   1           0 LOAD_NAME                0 (x)
               3 LOAD_CONST               0 (1)
               6 BINARY_ADD
               7 RETURN_VALUE

 >>> dis(compile('x+1', '', 'exec'))
   1           0 LOAD_NAME                0 (x)
               3 LOAD_CONST               0 (1)
               6 BINARY_ADD
               7 POP_TOP
               8 LOAD_CONST               1 (None)
              11 RETURN_VALUE

Adds an extra POP_TOP and LOAD_CONST None

 >>> dis(compile('x = x+1', '', 'single'))
   1           0 LOAD_NAME                0 (x)
               3 LOAD_CONST               0 (1)
               6 BINARY_ADD
               7 STORE_NAME               0 (x)
              10 LOAD_CONST               1 (None)
              13 RETURN_VALUE

 >>> dis(compile('x = x+1', '', 'exec'))
   1           0 LOAD_NAME                0 (x)
               3 LOAD_CONST               0 (1)
               6 BINARY_ADD
               7 STORE_NAME               0 (x)
              10 LOAD_CONST               1 (None)
              13 RETURN_VALUE

No difference

I decided to submit this to the tracker.
http://bugs.python.org/issue6507

Terry Jan Reedy




From tjreedy at udel.edu  Fri Jul 17 21:09:43 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 17 Jul 2009 15:09:43 -0400
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <87my73fqyv.fsf@benfinney.id.au>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>	<m2vdltufis.fsf@cs.uu.nl>	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>	<eae285400907160837v7f483df9t3d757cd46e2429af@mail.gmail.com>	<4A5FDBBA.6@canterbury.ac.nz>	<eae285400907170120u5932eef2obaa667ccd87c30b7@mail.gmail.com>	<de32cc030907170450x2f40671bj35c6322142012cb2@mail.gmail.com>
	<87my73fqyv.fsf@benfinney.id.au>
Message-ID: <h3qi9o$52s$2@ger.gmane.org>

Ben Finney wrote:
> David Stanek <dstanek at dstanek.com> writes:
> 
>> This thread is a little confusing to me. I am failing to see how the
>> where block makes anything clearer.
> 
> +1. I haven't found that any of the examples improve the clarity of the
> code over using a named function.

I presume you mean -1 on the proposal, as am I (strongly).




From gerald.britton at gmail.com  Fri Jul 17 22:28:40 2009
From: gerald.britton at gmail.com (Gerald Britton)
Date: Fri, 17 Jul 2009 16:28:40 -0400
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <200907180024.31209.steve@pearwood.info>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<4A607803.2030801@mrabarnett.plus.com>
	<5d1a32000907170625v5ff9dd2dmddbf83e17f183e71@mail.gmail.com> 
	<200907180024.31209.steve@pearwood.info>
Message-ID: <5d1a32000907171328y6da57c7j30b5bfe154e90f32@mail.gmail.com>

I often time my code and often find that removing function calls
"where" possible actually makes a measurable difference.  This is
especially the case with little getters and setters in class defs.

Anyway, I like the "where" idea not because of real or imagined
performance gains, but because of its cleanness when expressing
problems.  A big use for me would be in list comprehensions.  In one
project I work on, I see things like:

for i in [item in self.someobject.get_generator() if self.important_test(item)]

and other really long object references.

which I would like to write:

mylist =  [item in f if g(item)] where:
    f = self.someobject.get_generator()
    g = self.important_test


To my eyes, the first is harder to read than the second one.  Of
course I can do this:

    f = self.someobject.get_generator()
    g = self.important_test
    mylist =  [item in f if g(item)]:


but then "f" and "g" pollute the calling context's namespace.

Anyway, I don't have a strong preference, just a "nice to have" feeling.

On Fri, Jul 17, 2009 at 10:24 AM, Steven D'Aprano<steve at pearwood.info> wrote:
> On Fri, 17 Jul 2009 11:25:15 pm Gerald Britton wrote:
>> Sometimes I use helper functions to make things easier to read.
>> However, I don't like unnecessary function calls since they are
>> expensive in Python compared to other languages.
>
> That's only a problem if:
>
> (1) Your code actually is too slow; and
>
> (2) You have profiled your code and discovered that the biggest
> contributor to your code's slowness is the cost of calling functions,
> not the execution time of the functions.
>
> Unless both of these cases hold, then I suggest you are guilty of
> premature optimization. Possibly even pessimation -- given the
> complexities of CPU caches, memory, swap, etc, it's possible that your
> code could be *slower*, not faster, due to your "optimization".
>
> People's intuitions about what's fast and what's slow in Python code are
> often completely at odds with reality. Unless you actually have a
> problem with slow code that needs solving, my advice is to not worry
> about it. Use good algorithms, use tried and tested fast data
> structures, and worry about micro-optimizations when you *actually*
> need them.
>
>
>
> --
> Steven D'Aprano
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
Gerald Britton


From chrisperkins99 at gmail.com  Fri Jul 17 23:13:23 2009
From: chrisperkins99 at gmail.com (Chris Perkins)
Date: Fri, 17 Jul 2009 17:13:23 -0400
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <79990c6b0907170623y2fec30e4h2a44a300f301adf7@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<184a9f5a0907160821q79780f12vd6b9a65e870bffd4@mail.gmail.com>
	<79990c6b0907161151x967e8f8ncf0d3d54344177f3@mail.gmail.com>
	<184a9f5a0907161726n27b4f55fn79aa1ff0974e0f30@mail.gmail.com>
	<79990c6b0907170623y2fec30e4h2a44a300f301adf7@mail.gmail.com>
Message-ID: <184a9f5a0907171413g221702f6pc883cf8cf0364e23@mail.gmail.com>

On Fri, Jul 17, 2009 at 9:23 AM, Paul Moore<p.f.moore at gmail.com> wrote:
>
>> I don't really see the problem with it. If you told people that the
>> where block only applies to the right-hand-side of an assignment, and
>> that the left-hand-side is always evaluated outside the scope of the
>> where block, I doubt there would be much confusion.
>
> That somewhat assumes that a where block is only used for assignments
> (or more accurately, that there's a special exception for the left
> side of an assignment not being included). That's at best a wart.

The "special case for assignments" is what I was assuming.  In fact, I
liked this proposal partly because the semantics seemed so obvious
that you could guess them and be right - but looking through this
thread, I see that the semantics are obvious to different people in
different ways.

For example, I see that many people seem to expect:

i = 1
x[i] = 3 where:
    i = 2

to mean "x[2] = 3", which I would find utterly surprising.

>> Again, I don't think that's a big deal. One solution is to disallow
>> global statements (if we're already disallowing stuff like break and
>> return, why not). Another is just "don't do that".
>
> More warts. At some point, the weight of the various exceptions make
> the basic idea not worth it.

That does worry me, I'll admit - if the code that you can legally
write in a where block is only a subset of Python, then, well...
that's just ugly.

Nevertheless, I find several of the examples to have such a
readability advantage - similar to the way that list comprehensions
are immediately and wonderfully more readable than building a list in
a for loop - that I'm still in favor of the idea.


Chris Perkins


From tjreedy at udel.edu  Sat Jul 18 00:12:48 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 17 Jul 2009 18:12:48 -0400
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <5d1a32000907171328y6da57c7j30b5bfe154e90f32@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>	<4A607803.2030801@mrabarnett.plus.com>	<5d1a32000907170625v5ff9dd2dmddbf83e17f183e71@mail.gmail.com>
	<200907180024.31209.steve@pearwood.info>
	<5d1a32000907171328y6da57c7j30b5bfe154e90f32@mail.gmail.com>
Message-ID: <h3qt12$3n6$1@ger.gmane.org>


> To my eyes, the first is harder to read than the second one.  Of
> course I can do this:
> 
>     f = self.someobject.get_generator()
>     g = self.important_test
>     mylist =  [item in f if g(item)]:
> 
> but then "f" and "g" pollute the calling context's namespace.

So what? Seriously, what harm does a temporary association do? 
Especially when one can easily 'clean up' with 'del f,g' if it actually 
make a functional difference or if one just wants to cater to an obsession.

In particular, why is this reason to *PERMANENTLYT pollute the language 
with an otherwise useless new feature?

Unlike namespace associations, deleting language features is really 
difficult.

Terry Jan Reedy



From zuo at chopin.edu.pl  Sat Jul 18 00:40:16 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Sat, 18 Jul 2009 00:40:16 +0200
Subject: [Python-ideas] Where-statement (Proposal for function
 expressions)
In-Reply-To: <5d1a32000907171328y6da57c7j30b5bfe154e90f32@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<4A607803.2030801@mrabarnett.plus.com>
	<5d1a32000907170625v5ff9dd2dmddbf83e17f183e71@mail.gmail.com>
	<200907180024.31209.steve@pearwood.info>
	<5d1a32000907171328y6da57c7j30b5bfe154e90f32@mail.gmail.com>
Message-ID: <op.uw8g9et5fvx12t@jerzozwiesz.home.aster.pl>

17-07-2009 o 14:12 Ben Finney <ben+python at benfinney.id.au> wrote:

> David Stanek <dstanek at dstanek.com> writes:
>
>> This thread is a little confusing to me. I am failing to see how the
>> where block makes anything clearer.
>
> +1. I haven't found that any of the examples improve the clarity of the
> code over using a named function.

And what about that:

class SomeClass:
      ...

      def summarize_proust(self, objects, args, method_name)
          "Summarize Proust in 15 seconds"

          return map(action, objects, args, alts) where:

              def action(obj, arg, alt):
                  result = getattr(item, method_name)(arg) or alt
                  obj.set_last_result(result)
                  return result

              alts = map(weirdtrans, objects) where def weirdtrans(obj):
                  # some transformations not very
                  # useful in more general context


...is more logical and readable for me, than:

class SomeClass:
      ...

      def _weirdtrans(self, obj):
          # some transformations useful
          # only for somefilter method

      def summarize_proust(self, objects, args, method_name)
          "Summarize Proust in 15 seconds"

          def action(obj, arg, alt):
              result = getattr(item, method_name)(arg) or alt
              obj.set_last_result(result)
              return result

          alts = map(self._weirdtrans, objects)
          return map(action, objects, args, alts)


17-07-2009, 22:28 Gerald Britton <gerald.britton at gmail.com> wrote:

> which I would like to write:
>
> mylist =  [item in f if g(item)] where:

Obviously you ment: "[item for item if g(item)] where:", didn't you?
In that particular case you could write: filter(g, f).
Anyway still I'd prefer:

mylist = filter(g, f) where:
      f = self.someobject.get_generator()
      g = self.important_test

Than:

f = self.someobject.get_generator()
g = self.important_test
mylist = filter(g, f) where:

> then "f" and "g" pollute the calling context's namespace.

+1

Generally one of main applications of 'where' would be using it
for functions what use function(s) as argument(s) (map, filter
et consortes...).

After all it's something like lambda, but:
* more beautiful (IMHO),
* not limited to expression.

Regards,

-- 
Jan Kaliszewski <zuo at chopin.edu.pl>


From mwm-keyword-python.b4bdba at mired.org  Sat Jul 18 00:47:19 2009
From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer)
Date: Fri, 17 Jul 2009 18:47:19 -0400
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <op.uw6qg3s7fvx12t@jerzozwiesz.home.aster.pl>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>
	<4A5D8B63.4040509@canterbury.ac.nz>
	<eae285400907150549k429f1aeha8ab1b85f581eb7@mail.gmail.com>
	<5d1a32000907150638j65131770gf26df83dbadc5af2@mail.gmail.com>
	<3bdda690907151611u6da07958oe92baf0c727c45b9@mail.gmail.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<20090716185112.2c7d1a60@bhuda.mired.org>
	<op.uw6qg3s7fvx12t@jerzozwiesz.home.aster.pl>
Message-ID: <20090717184719.3553f1a3@bhuda.mired.org>

On Fri, 17 Jul 2009 02:04:05 +0200
"Jan Kaliszewski" <zuo at chopin.edu.pl> wrote:

> 17-07-2009, 00:51 Mike Meyer <mwm-keyword-python.b4bdba at mired.org> wrote:
> 
> > That they cause problems with your first attempt at
> > describing an implementation certainly isn't reason to disallow them
> > all by itself.
> 
> But, it isn't a problem of implementation. I don't see any reasonable
> *semantics* of return or yield within where-block in proposed form.
> return/yield statements seem to be completely unnecessary in this context.
> (Of course, I don't say about return/yield statements within functions
> defined within where-block).

Seems obvious to me:

def foo(x):
    bar = foundbar where:
        if x is None:
	   return -1
  	foundbar = x * 2
    return bar

The return will return -1 from foo if it's invoked; the assignment to
bar (and foundbar, for that matter) will never happen.

    <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


From ncoghlan at gmail.com  Sat Jul 18 00:52:21 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 18 Jul 2009 08:52:21 +1000
Subject: [Python-ideas] Augment dis.dis to autocompile strings
In-Reply-To: <200907171232.21702.steve@pearwood.info>
References: <h3nrtu$oog$1@ger.gmane.org>
	<200907171232.21702.steve@pearwood.info>
Message-ID: <4A6100A5.8000008@gmail.com>

Steven D'Aprano wrote:
> On Fri, 17 Jul 2009 04:35:40 am Terry Reedy wrote:
> 
>> Proposal: if the input is a string, do the above (do what I mean)
>> instead of raising
>> TypeError: don't know how to disassemble str objects
>>
>> Any negatives before I submit a feature request?
> 
> +1 on the idea, but how do you determine which of the following are 
> required?
> 
> compile('x+1', '', 'eval')
> compile('x = x+1', '', 'single')
> compile('while x < 42: x += 1', '', 'exec')
> 
> 
> Or do you just assume 'exec'?

We could define it as trying the three in order (first 'eval', then
'single', then 'exec') moving on to the next option if it raises syntax
error:

from dis import dis
def dis_str(source):
  modes = ('eval', 'single', 'exec')
  for mode in modes:
    try:
      c = compile(source, '', mode)
      break
    except SyntaxError:
      if mode is modes[-1]:
        raise
  return dis(c)

>>> dis_str("x+1")
  1           0 LOAD_NAME                0 (x)
              3 LOAD_CONST               0 (1)
              6 BINARY_ADD
              7 RETURN_VALUE
>>> dis_str("x = x+1")
  1           0 LOAD_NAME                0 (x)
              3 LOAD_CONST               0 (1)
              6 BINARY_ADD
              7 STORE_NAME               0 (x)
             10 LOAD_CONST               1 (None)
             13 RETURN_VALUE
>>> dis_str("while x < 42: x += 1")
  1           0 SETUP_LOOP              26 (to 29)
        >>    3 LOAD_NAME                0 (x)
              6 LOAD_CONST               0 (42)
              9 COMPARE_OP               0 (<)
             12 POP_JUMP_IF_FALSE       28
             15 LOAD_NAME                0 (x)
             18 LOAD_CONST               1 (1)
             21 INPLACE_ADD
             22 STORE_NAME               0 (x)
             25 JUMP_ABSOLUTE            3
        >>   28 POP_BLOCK
        >>   29 LOAD_CONST               2 (None)
             32 RETURN_VALUE
>>> dis_str("bad syntax still goes boom")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 5, in dis_str
  File "", line 1
    bad syntax still goes boom
             ^
SyntaxError: invalid syntax

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From fuzzyman at gmail.com  Sat Jul 18 01:41:10 2009
From: fuzzyman at gmail.com (Michael)
Date: Sat, 18 Jul 2009 00:41:10 +0100
Subject: [Python-ideas] Make return inside a finally a SyntaxError
Message-ID: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>

Here are two examples of why allowing return inside a finally block is  
a bad idea:

def f():
   try:
     return 3
   finally:
     return 4

def f():
   try:
     raise Exception()
   finally:
     return 4


Michael Foord

--
http://www.ironpythoninaction.com


From collinw at gmail.com  Sat Jul 18 01:44:41 2009
From: collinw at gmail.com (Collin Winter)
Date: Fri, 17 Jul 2009 16:44:41 -0700
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
Message-ID: <43aa6ff70907171644t6f3b6b66i6db0f61751fa9bc1@mail.gmail.com>

On Fri, Jul 17, 2009 at 4:41 PM, Michael<fuzzyman at gmail.com> wrote:
> Here are two examples of why allowing return inside a finally block is a bad
> idea:
>
> def f():
> ?try:
> ? ?return 3
> ?finally:
> ? ?return 4
>
> def f():
> ?try:
> ? ?raise Exception()
> ?finally:
> ? ?return 4

Do you have real code that suffers from this problem? Is this a common
mistake for Python beginners?

Collin Winter


From ben+python at benfinney.id.au  Sat Jul 18 01:44:25 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 18 Jul 2009 09:44:25 +1000
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<4A607803.2030801@mrabarnett.plus.com>
	<5d1a32000907170625v5ff9dd2dmddbf83e17f183e71@mail.gmail.com>
	<200907180024.31209.steve@pearwood.info>
	<5d1a32000907171328y6da57c7j30b5bfe154e90f32@mail.gmail.com>
Message-ID: <87ocrieuxy.fsf@benfinney.id.au>

Gerald Britton <gerald.britton at gmail.com>
writes:

> Anyway, I like the "where" idea not because of real or imagined
> performance gains, but because of its cleanness when expressing
> problems.

(Speaking of cwclean expression, please don't top-post; instead, remove
irrelevant quoted material and respond beneath each remaining point
<URL:http://en.wikipedia.org/wiki/Posting_style#Inline_replying>.)

> A big use for me would be in list comprehensions. In one
> project I work on, I see things like:
> 
> for i in [item in self.someobject.get_generator() if self.important_test(item)]
> 
> and other really long object references.
> 
> which I would like to write:
> 
> mylist =  [item in f if g(item)] where:
>     f = self.someobject.get_generator()
>     g = self.important_test

I presume these two are supposed to be equivalent (and syntactically
valid), so I'll take it the first one should be something like::

    mylist = [item for item in self.someobject.get_generator() if self.important_test(item)]

> To my eyes, the first is harder to read than the second one.

That's largely attributable to the fact that you've got one alternative
all on a single line, and the other broken into more easily-readable
lines. I don't think the ?where? syntax is much help there.

I would write the single-statement version as::

    mylist = [
        item for item in self.someobject.get_generator()
        if self.important_test(item)]

which makes it far more readable. I argue that this does away with
pretty much any justification for your use case above.

-- 
 \          ?? a Microsoft Certified System Engineer is to information |
  `\     technology as a McDonalds Certified Food Specialist is to the |
_o__)                               culinary arts.? ?Michael Bacarella |
Ben Finney



From ben+python at benfinney.id.au  Sat Jul 18 01:52:22 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 18 Jul 2009 09:52:22 +1000
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<4A607803.2030801@mrabarnett.plus.com>
	<5d1a32000907170625v5ff9dd2dmddbf83e17f183e71@mail.gmail.com>
	<200907180024.31209.steve@pearwood.info>
	<5d1a32000907171328y6da57c7j30b5bfe154e90f32@mail.gmail.com>
	<op.uw8g9et5fvx12t@jerzozwiesz.home.aster.pl>
Message-ID: <87k526eukp.fsf@benfinney.id.au>

"Jan Kaliszewski" <zuo at chopin.edu.pl> writes:

> And what about that:
> 
> class SomeClass:
>      ...
> 
>      def summarize_proust(self, objects, args, method_name)
>          "Summarize Proust in 15 seconds"
> 
>          return map(action, objects, args, alts) where:
> 
>              def action(obj, arg, alt):
>                  result = getattr(item, method_name)(arg) or alt
>                  obj.set_last_result(result)
>                  return result
> 
>              alts = map(weirdtrans, objects) where def weirdtrans(obj):
>                  # some transformations not very
>                  # useful in more general context

You then make an invalid comparison of this versus making ?weirdtrans?
an unnecessary instance method. But the correct equivalent of your
example above is to make another local function::

    class SomeClass:
        # ?

        def summarize_proust(self, objects, args, method_name):
            """ Summarize Proust in 15 seconds. """

            def weirdtrans(obj):
                # some transformations not very
                # useful in more general context

            alts = map(weirdtrans, objects)

            def action(obj, arg, alt):
               result = getattr(item, method_name)(arg) or alt
               obj.set_last_result(result)
               return result

            return map(action, objects, args, alts)

That seems more straightforward to me. Certainly I don't find the one
with ?where? any clearer.

-- 
 \                   ?Holy unrefillable prescriptions, Batman!? ?Robin |
  `\                                                                   |
_o__)                                                                  |
Ben Finney



From ben+python at benfinney.id.au  Sat Jul 18 01:53:07 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 18 Jul 2009 09:53:07 +1000
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<eae285400907160837v7f483df9t3d757cd46e2429af@mail.gmail.com>
	<4A5FDBBA.6@canterbury.ac.nz>
	<eae285400907170120u5932eef2obaa667ccd87c30b7@mail.gmail.com>
	<de32cc030907170450x2f40671bj35c6322142012cb2@mail.gmail.com>
	<87my73fqyv.fsf@benfinney.id.au> <h3qi9o$52s$2@ger.gmane.org>
Message-ID: <87fxcueujg.fsf@benfinney.id.au>

Terry Reedy <tjreedy at udel.edu> writes:

> Ben Finney wrote:
> > David Stanek <dstanek at dstanek.com> writes:
> >
> >> This thread is a little confusing to me. I am failing to see how the
> >> where block makes anything clearer.
> >
> > +1. I haven't found that any of the examples improve the clarity of the
> > code over using a named function.
> 
> I presume you mean -1 on the proposal, as am I (strongly).

Yes, my support was for David's dissent :-)

-- 
 \       ?If you see an animal and you can't tell if it's a skunk or a |
  `\   cat, here's a good saying to help: ?Black and white, stinks all |
_o__)              right. Tabby-colored, likes a fella.?? ?Jack Handey |
Ben Finney



From fuzzyman at gmail.com  Sat Jul 18 02:00:30 2009
From: fuzzyman at gmail.com (Michael)
Date: Sat, 18 Jul 2009 01:00:30 +0100
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <43aa6ff70907171644t6f3b6b66i6db0f61751fa9bc1@mail.gmail.com>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
	<43aa6ff70907171644t6f3b6b66i6db0f61751fa9bc1@mail.gmail.com>
Message-ID: <528CB0C4-61AF-4449-8AEB-EEBADA999559@gmail.com>




--
http://www.ironpythoninaction.com

On 18 Jul 2009, at 00:44, Collin Winter <collinw at gmail.com> wrote:

> On Fri, Jul 17, 2009 at 4:41 PM, Michael<fuzzyman at gmail.com> wrote:
>> Here are two examples of why allowing return inside a finally block  
>> is a bad
>> idea:
>>
>> def f():
>>  try:
>>    return 3
>>  finally:
>>    return 4
>>
>> def f():
>>  try:
>>    raise Exception()
>>  finally:
>>    return 4
>
> Do you have real code that suffers from this problem? Is this a common
> mistake for Python beginners?
>
> Collin Winter


Not specifically but I think it is an unfortunate design decision that  
should be corrected.

Michael 


From curt at hagenlocher.org  Sat Jul 18 02:07:37 2009
From: curt at hagenlocher.org (Curt Hagenlocher)
Date: Fri, 17 Jul 2009 17:07:37 -0700
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
Message-ID: <d2155e360907171707v5c71dd80k970dca1da79f8d4e@mail.gmail.com>

On Fri, Jul 17, 2009 at 4:41 PM, Michael<fuzzyman at gmail.com> wrote:
> Here are two examples of why allowing return inside a finally block is a bad
> idea:

Wow. I agree. That was definitely not what I would have expected

--
Curt Hagenlocher
curt at hagenlocher.org


From collinw at gmail.com  Sat Jul 18 02:13:23 2009
From: collinw at gmail.com (Collin Winter)
Date: Fri, 17 Jul 2009 17:13:23 -0700
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <528CB0C4-61AF-4449-8AEB-EEBADA999559@gmail.com>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
	<43aa6ff70907171644t6f3b6b66i6db0f61751fa9bc1@mail.gmail.com>
	<528CB0C4-61AF-4449-8AEB-EEBADA999559@gmail.com>
Message-ID: <43aa6ff70907171713l4f6e6f05ya5892ca43c62eb5f@mail.gmail.com>

On Fri, Jul 17, 2009 at 5:00 PM, Michael<fuzzyman at gmail.com> wrote:
> On 18 Jul 2009, at 00:44, Collin Winter <collinw at gmail.com> wrote:
>
>> On Fri, Jul 17, 2009 at 4:41 PM, Michael<fuzzyman at gmail.com> wrote:
>>>
>>> Here are two examples of why allowing return inside a finally block is a
>>> bad
>>> idea:
>>>
>>> def f():
>>> ?try:
>>> ? return 3
>>> ?finally:
>>> ? return 4
>>
>> Do you have real code that suffers from this problem? Is this a common
>> mistake for Python beginners?
>
> Not specifically but I think it is an unfortunate design decision that
> should be corrected.

Do you believe

def f():
  try:
    raise OSError
  finally:
    raise KeyError

should be a SyntaxError, too?

Collin Winter


From fuzzyman at gmail.com  Sat Jul 18 02:17:37 2009
From: fuzzyman at gmail.com (Michael)
Date: Sat, 18 Jul 2009 01:17:37 +0100
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <43aa6ff70907171713l4f6e6f05ya5892ca43c62eb5f@mail.gmail.com>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
	<43aa6ff70907171644t6f3b6b66i6db0f61751fa9bc1@mail.gmail.com>
	<528CB0C4-61AF-4449-8AEB-EEBADA999559@gmail.com>
	<43aa6ff70907171713l4f6e6f05ya5892ca43c62eb5f@mail.gmail.com>
Message-ID: <0221A66B-A05F-4456-9A99-25750AAABF38@gmail.com>




--
http://www.ironpythoninaction.com

On 18 Jul 2009, at 01:13, Collin Winter <collinw at gmail.com> wrote:

> On Fri, Jul 17, 2009 at 5:00 PM, Michael<fuzzyman at gmail.com> wrote:
>> On 18 Jul 2009, at 00:44, Collin Winter <collinw at gmail.com> wrote:
>>
>>> On Fri, Jul 17, 2009 at 4:41 PM, Michael<fuzzyman at gmail.com> wrote:
>>>>
>>>> Here are two examples of why allowing return inside a finally  
>>>> block is a
>>>> bad
>>>> idea:
>>>>
>>>> def f():
>>>>  try:
>>>>   return 3
>>>>  finally:
>>>>   return 4
>>>
>>> Do you have real code that suffers from this problem? Is this a  
>>> common
>>> mistake for Python beginners?
>>
>> Not specifically but I think it is an unfortunate design decision  
>> that
>> should be corrected.
>
> Do you believe
>
> def f():
>  try:
>    raise OSError
>  finally:
>    raise KeyError
>
> should be a SyntaxError, too?


No, the semantics of exceptions raised in finally blocks are pretty  
clear.

Michael


>
> Collin Winter


From python at mrabarnett.plus.com  Sat Jul 18 02:18:36 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Sat, 18 Jul 2009 01:18:36 +0100
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
Message-ID: <4A6114DC.9090207@mrabarnett.plus.com>

Michael wrote:
> Here are two examples of why allowing return inside a finally block is a 
> bad idea:
> 
> def f():
>   try:
>     return 3
>   finally:
>     return 4
> 
> def f():
>   try:
>     raise Exception()
>   finally:
>     return 4
> 
I wouldn't call that a syntax error. Wouldn't you also have to forbid 
break, continue and, perhaps, yield?

Isn't Python for consenting adults, or something like that? :-)


From lists at cheimes.de  Sat Jul 18 02:31:13 2009
From: lists at cheimes.de (Christian Heimes)
Date: Sat, 18 Jul 2009 02:31:13 +0200
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
Message-ID: <h3r54h$l99$1@ger.gmane.org>

Michael schrieb:
> Here are two examples of why allowing return inside a finally block is a
> bad idea:
> 
> def f():
>   try:
>     return 3
>   finally:
>     return 4
> 
> def f():
>   try:
>     raise Exception()
>   finally:
>     return 4
> 

Woot! That's totally. It never even occurred to me to use return in the
finally block. +1 for the syntax error. Nice catch, Michael!

Christian

PS: I wasn't sure so I tested the functions. Both functions return 4.



From steve at pearwood.info  Sat Jul 18 02:39:56 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 18 Jul 2009 10:39:56 +1000
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <5d1a32000907171328y6da57c7j30b5bfe154e90f32@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<200907180024.31209.steve@pearwood.info>
	<5d1a32000907171328y6da57c7j30b5bfe154e90f32@mail.gmail.com>
Message-ID: <200907181039.57256.steve@pearwood.info>

On Sat, 18 Jul 2009 06:28:40 am Gerald Britton wrote:

> I often time my code and often find that removing function calls
> "where" possible actually makes a measurable difference.  This is
> especially the case with little getters and setters in class defs.

Perhaps you shouldn't be writing Java code with lots of getters and 
setters then *wink*

Seriously, I've very glad to hear you're measuring before optimising, 
but a "measurable difference" is not necessarily a significant 
difference. Anyway, we're not here to discuss the pros and cons of 
optimization, so back to the top on hand:

> Anyway, I like the "where" idea not because of real or imagined
> performance gains, but because of its cleanness when expressing
> problems.

I just don't see this cleanness.

It seems to me that the "where" construct makes writing code easier than 
reading code. It is a top-down construct: first you come up with some 
abstract expression:

element = w + x.y - f(z) where:

and then you need to think about how to implement the construct:

    w = 2
    x = thingy()
    z = 4

That roughly models the process of writing code in the first place, so I 
can see the attraction. But it doesn't model the process of *reading* 
code:

"Hmmm, element is equal to w + x.y - f(z), okay, but I haven't seen any 
of those names before, are they globals? Ah, there's a 'where' 
statement, better pop that expression into short-term memory while I 
read the block and find out what they are..."

In a real sense, the proposed 'where' block breaks the flow of reading 
code. Normally, reading a function proceeds in an orderly fashion from 
the start of the function to the end in (mostly) sequential order, a 
bottom-up process:

def parrot():
    x = 1
    y = 2
    z = 3
    result = x+y+z
    return result

A complication: if the function you are reading relies on globals, 
including functions, then you may need to jump out of the function to 
discover what it is. But if you already know what the function does, or 
can infer it from the name, then it doesn't disrupt the sequential 
reading.

But the where clause introduces *look-ahead* to the process:

def parrot():
    x = 1
    result = x+y+z where:  # look-ahead
        y = 2
        z = 3
    return result


This is not a heavy burden if you are limited to a few simple names, but 
as soon as you re-use pre-existing names in the 'where' scope, or 
increase its complexity, readability suffers greatly.

def harder():
    w = 5
    z = 2
    result = w + x.y - f(z) where:
        w = 2
        class K:
            y = 3
        x = K()
        def f(a):
            return a**2 + a
        z = 4
    return result


Look-ahead is simply harder on the reader than reading sequentially.

You can abuse any tool, write spaghetti code in any language, but some 
idioms encourage misuse, and in my opinion this is one of them.



> A big use for me would be in list comprehensions.  In one 
> project I work on, I see things like:
>
> for i in [item in self.someobject.get_generator() if
> self.important_test(item)]
>
> and other really long object references.

I don't see anything objectionable in that. It's a tad on the long side, 
but not excessively.


> which I would like to write:
>
> mylist =  [item in f if g(item)] where:
>     f = self.someobject.get_generator()
>     g = self.important_test

I don't think much of your naming convention. Surely g should be used 
for the generator, and f for the function, instead of the other way 
around?



> To my eyes, the first is harder to read than the second one.  Of
> course I can do this:
>
>     f = self.someobject.get_generator()
>     g = self.important_test
>     mylist =  [item in f if g(item)]:
>
>
> but then "f" and "g" pollute the calling context's namespace.

Holy cow! How many variable names do you have in a single function that 
you have to worry about that???

Honestly, I think this entire proposal is a pessimation: you're 
proposing to significantly complicate the language and negatively 
impact the readability of code in order to avoid polluting the 
namespace of a function?

Of course, then people will start worrying about "polluting the 
namespace" of the where-block, and start doing this:

result = x + y + z where:
    x = 1
    y = 2
    z = a*b - c where:
        a = 5
        b = 6
        c = d*e where:
            d = 3
            e = 4




-- 
Steven D'Aprano


From python at mrabarnett.plus.com  Sat Jul 18 02:41:11 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Sat, 18 Jul 2009 01:41:11 +0100
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <4A6114DC.9090207@mrabarnett.plus.com>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
	<4A6114DC.9090207@mrabarnett.plus.com>
Message-ID: <4A611A27.3020003@mrabarnett.plus.com>

MRAB wrote:
> Michael wrote:
>> Here are two examples of why allowing return inside a finally block is 
>> a bad idea:
>>
>> def f():
>>   try:
>>     return 3
>>   finally:
>>     return 4
>>
>> def f():
>>   try:
>>     raise Exception()
>>   finally:
>>     return 4
>>
> I wouldn't call that a syntax error. Wouldn't you also have to forbid 
> break, continue and, perhaps, yield?
> 
I shouldn't checked before posting... :-(


Test 1: break swallows the exception.

 >>> for n in range(3):
	try:
		print n
		raise IndexError
	finally:
		break

	
0


Test 2: continue raises a SyntaxError.

 >>> for n in range(2):
	try:
		print n
		raise IndexError
	finally:
		continue
	
SyntaxError: 'continue' not supported inside 'finally' clause


Test 3: yield propagates the exception on resuming.

 >>> def test():
	for n in range(3):
		try:
			print n
			raise IndexError
		finally:
			print "yielding", n
			yield n

			
 >>> t = test()
 >>> t.next()
0
yielding 0
0
 >>> t.next()

Traceback (most recent call last):
   File "<pyshell#24>", line 1, in <module>
     t.next()
   File "<pyshell#21>", line 5, in test
     raise IndexError
IndexError



From aahz at pythoncraft.com  Sat Jul 18 02:47:55 2009
From: aahz at pythoncraft.com (Aahz)
Date: Fri, 17 Jul 2009 17:47:55 -0700
Subject: [Python-ideas] Where-statement (Proposal for
	function	expressions)
In-Reply-To: <200907181039.57256.steve@pearwood.info>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<200907180024.31209.steve@pearwood.info>
	<5d1a32000907171328y6da57c7j30b5bfe154e90f32@mail.gmail.com>
	<200907181039.57256.steve@pearwood.info>
Message-ID: <20090718004755.GA15099@panix.com>

[quote-trimming ahead]

On Sat, Jul 18, 2009, Steven D'Aprano wrote:
>
> In a real sense, the proposed 'where' block breaks the flow of reading 
> code. Normally, reading a function proceeds in an orderly fashion from 
> the start of the function to the end in (mostly) sequential order, a 
> bottom-up process:
> 
> But the where clause introduces *look-ahead* to the process:
> 
> Look-ahead is simply harder on the reader than reading sequentially.
> 
> You can abuse any tool, write spaghetti code in any language, but some 
> idioms encourage misuse, and in my opinion this is one of them.

This is the clearest indictment yet -- I've mostly been ignoring this
thread figuring it'll die a natural death on its own, but this makes me
add my own -1 to the ``where`` clause.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"If you think it's expensive to hire a professional to do the job, wait
until you hire an amateur."  --Red Adair


From steve at pearwood.info  Sat Jul 18 02:59:57 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 18 Jul 2009 10:59:57 +1000
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
Message-ID: <200907181059.58523.steve@pearwood.info>

On Sat, 18 Jul 2009 09:41:10 am Michael wrote:
> Here are two examples of why allowing return inside a finally block
> is a bad idea:
>
> def f():
>    try:
>      return 3
>    finally:
>      return 4


Given that the finally block is guaranteed to run after exiting the try 
block, I don't see anything objectionable about that. Surprising, 
perhaps, possibly even an artifact of the CPython implementation, but 
why is it a "bad idea" that needs to be protected against?



> def f():
>    try:
>      raise Exception()
>    finally:
>      return 4

That's no weirder than:


def f():
    try:
        raise Exception()
    except Exception:
        return 4

or:

def f():
    try:
        raise Exception()
    except Exception:
        pass
    return 4


-- 
Steven D'Aprano


From zuo at chopin.edu.pl  Sat Jul 18 03:43:43 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Sat, 18 Jul 2009 03:43:43 +0200
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <200907181059.58523.steve@pearwood.info>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
	<200907181059.58523.steve@pearwood.info>
Message-ID: <op.uw8pq5svfvx12t@jerzozwiesz.home.aster.pl>

18-07-2009, 02:59 Steven D'Aprano <steve at pearwood.info> wrote:

> On Sat, 18 Jul 2009 09:41:10 am Michael wrote:
>> Here are two examples of why allowing return inside a finally block
>> is a bad idea:
>>
>> def f():
>>    try:
>>      return 3
>>    finally:
>>      return 4
>
>
> Given that the finally block is guaranteed to run after exiting the try
> block, I don't see anything objectionable about that. Surprising,
> perhaps, possibly even an artifact of the CPython implementation, but
> why is it a "bad idea" that needs to be protected against?
>
>
>
>> def f():
>>    try:
>>      raise Exception()
>>    finally:
>>      return 4
>
> That's no weirder than:
>
>
> def f():
>     try:
>         raise Exception()
>     except Exception:
>         return 4
>
> or:
>
> def f():
>     try:
>         raise Exception()
>     except Exception:
>         pass
>     return 4

The problem is that finally clause is not for catching exceptions
but for doing clean-up actions. And here finally-clause *stops*
propagation of any exception! (also from lower levels!)

def f()
     try:
         raise KeyError
     except KeyError:
         raise Exception
     finally:
         return 4

...returns 4 without obstacles!

It's weird and missleading. I thing it's definitely a bug.

-- 
Jan Kaliszewski <zuo at chopin.edu.pl>


From greg.ewing at canterbury.ac.nz  Sat Jul 18 03:51:33 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 18 Jul 2009 13:51:33 +1200
Subject: [Python-ideas] Where-statement (Proposal for
	function	expressions)
In-Reply-To: <87my73fqyv.fsf@benfinney.id.au>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<eae285400907160837v7f483df9t3d757cd46e2429af@mail.gmail.com>
	<4A5FDBBA.6@canterbury.ac.nz>
	<eae285400907170120u5932eef2obaa667ccd87c30b7@mail.gmail.com>
	<de32cc030907170450x2f40671bj35c6322142012cb2@mail.gmail.com>
	<87my73fqyv.fsf@benfinney.id.au>
Message-ID: <4A612AA5.9020108@canterbury.ac.nz>

Ben Finney wrote:

> I haven't found that any of the examples improve the clarity of the
> code over using a named function.

This isn't about unnamed functions. The functions
defined in a where block still have names.

-- 
Greg




From m.lenzen at gmail.com  Sat Jul 18 03:54:15 2009
From: m.lenzen at gmail.com (Michael Lenzen)
Date: Fri, 17 Jul 2009 20:54:15 -0500
Subject: [Python-ideas] Proposal for extending the collections module - bags
 / multisets, ordered sets / unique lists
Message-ID: <4A612B47.3050706@gmail.com>

In a nutshell, I want to add 2 new classes (and respective frozen 
varients) to the collections module, namely a bag (multiset) class and a 
setlist (ordered set/unique list) class.  I know this has been floated 
before, but I think it merits more consideration.

Everything I'm writing here is available with better formatting here: 
http://code.google.com/p/python-data-structures/wiki/collections
The latest version of my implementation is here: 
http://python-data-structures.googlecode.com/svn/trunk/collections_.py

I look at the basic data structures of Python like this (ignoring 
mutability which I will mention later) :

             | Unordered | Ordered |
-----------------------------------
Non-Unique  |           |   list  |
-----------------------------------
Unique      |    set    |         |
-----------------------------------

Python is missing 2 basic collections.  First, an unordered non-unique 
collection, a bag aka multiset, the simplest collection there is. You 
can't even really call them a data structure because there's no 
structure to them (which is their beauty). Secondly it's missing an 
ordered unique collection, an ordered set or unique list, what I'm 
calling a setlist.

That is the scope of when I started out, of course it grew and grew.  
The last addition but the first I will list, because I think it is the 
coolest and provides the most functionality, is the new function I 
propose adding to collections.  The function is simply called 
'collection' and has the following signature:

collection(iterable=None, mutable=False, ordered=False, unique=False)

What it does is return a collection (instantiated from iterable if 
passed) with the passed properties.  This will only be possible with 
then new classes I propose, because they don't exist yet.

New ABCs (abstract base classes)
--------------------------------
After starting to study the ABCs built in to Python I came up with 2 
more ABCs that I think should be included:

Collection - implements Sized, Iterable and Container.  Sequence, Set 
and Mapping would then all inherit Collection instead of the three 
aforementioned classes. Built-in classes that would be subclasses of 
Collection are set, frozenset, list and tuple.  Collection would provide 
2 methods:
   * _from_iterable(cls, it) - This is a classmethod that I took from 
Set, for convenience of returning new instantiations
   * __getitem__(key, value) - This is an abstract method. Every 
Collection would have to make its elements accessible. Of course set and 
frozenset do not currently implement this method so they will have to be 
changed, more later.

Mutable - I think there should be one metaclass for all mutable types 
just like there is currently Iterable, Sized and Container. We could 
then check instanceof(obj, Mutable). MutableSequence, MutableSet and 
MutableMapping would inherit Mutable, or we might be able to just do 
away with them altogether.  There are 3 abstractmethods defined:
   * pop(self) - This is the simplest to implement, all mutable classes 
already do.
   * __setitem__(self, key, value) - This is already implemented for 
list and dict but would have to be defined for set, more later
   * __delitem__(self, key) Same as above.

Extending set to fit the ABCs
-----------------------------
I mentioned before that sets don't currently fit into my model. Here are 
my propositions for the three methods that would need to be implemented.
   * set.__setitem__(self, elem, value) - Set whether or not elem is in 
the set based on the boolean evaluation of value. s[elem] = 1 would call 
s.add(elem) and s[elem] = False would call s.remove(elem), appropriately 
raising a KeyError if elem not in s. The reason I chose this way is that 
the only thing you can set about an element of a set is whether or not 
it is present.
   * __getitem__(self, item) - Simply returns item in self. Just like a 
bag returns the multiplicity of an item when you get it, this returns 
the number of times item appears in self. This also corresponds to the 
definition of setitem.
   * __delitem__(self, item) - This is the simplest case, just make it 
an alias for set.remove(item)

I think this is the most controversial part of my proposition, but the 
more I think about it, the more I like it. []s aren't reserved for one 
single purpose in Python, lists restrict the index to an integer while 
dicts allow any Hashable object. list allows slicing inside the []s 
while it is meaningless for a dict.

Finally, the New Classes
------------------------
These are the 2 new Collections (4 if you count the frozen variants) to 
fill in the holes I mentioned in the beginning.

bags - These would be true multisets, AKA unordered tuples.  See the 
wikipedia page on Multisets for more info. The benefits of a bag vs a 
list are that order doesn't matter for equality testing and that 
searching for an item is constant time. The latter is the real convincer 
for me.  I think this should replace the Counter class in the 
collections module which doesn't provide anything more than 
defaultdict(int) does.

setlist - AKA unique list, ordered set, bijection int->hashable, and 
ordered collection of unique elements.  I find myself often creating a 
list of unique elements, then an inverse dict to look up the location 
quickly or test inclusion.  I chose the name setlist because it's not a 
set xor a list, it's both. Then it occurred to me that bands' setlists 
are a perfect example of this data structure, an ordered collection of 
unique elements, real bands that is, 
http://en.wikipedia.org/wiki/Vertigo_Tour#The_encores

-Michael Lenzen


From greg.ewing at canterbury.ac.nz  Sat Jul 18 04:01:06 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 18 Jul 2009 14:01:06 +1200
Subject: [Python-ideas] Where-statement (Proposal
	for	function	expressions)
In-Reply-To: <4A607803.2030801@mrabarnett.plus.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<fb6fbf560907121731u2578ecd1o739fa0b78d1a69ab@mail.gmail.com>
	<184a9f5a0907130351p3edff72enacd28f34be37b268@mail.gmail.com>
	<de32cc030907130819l1a27a1d1r20c62d0e2ee0a46a@mail.gmail.com>
	<3bdda690907140441t4a74b27ci893eea0430bd182c@mail.gmail.com>
	<184a9f5a0907140721r7f9e673dmc4314cff2023e398@mail.gmail.com>
	<79990c6b0907140804n3a63d7bh1336d26c5abdc895@mail.gmail.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>
	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>
	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>
	<4A5D8B63.4040509@canterbury.ac.nz>
	<op.uw57mcv8fvx12t@jerzozwiesz.home.aster.pl>
	<4A5FE431.1070407@canterbury.ac.nz>
	<4A607803.2030801@mrabarnett.plus.com>
Message-ID: <4A612CE2.7070909@canterbury.ac.nz>

MRAB wrote:

> "do" might be a sufficiently generic word to cover all the cases.
>
>     if a == b where:
>         ...
>     do:
>         ...

It's probably okay for 'if', but to me it doesn't read
right with 'def' or 'class'.

But in any case, I'm -0.9 on allowing 'where' on
arbitrary statements in the first place. Seems like
feature creep on a massive scale to me. (Feature
stampede?)

-- 
Greg


From greg.ewing at canterbury.ac.nz  Sat Jul 18 04:05:22 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 18 Jul 2009 14:05:22 +1200
Subject: [Python-ideas] Where-statement (Proposal for
	function	expressions)
In-Reply-To: <79990c6b0907170623y2fec30e4h2a44a300f301adf7@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<184a9f5a0907160821q79780f12vd6b9a65e870bffd4@mail.gmail.com>
	<79990c6b0907161151x967e8f8ncf0d3d54344177f3@mail.gmail.com>
	<184a9f5a0907161726n27b4f55fn79aa1ff0974e0f30@mail.gmail.com>
	<79990c6b0907170623y2fec30e4h2a44a300f301adf7@mail.gmail.com>
Message-ID: <4A612DE2.7000905@canterbury.ac.nz>

Paul Moore wrote:

> That somewhat assumes that a where block is only used for assignments
> (or more accurately, that there's a special exception for the left
> side of an assignment not being included). That's at best a wart.

The more I think about it, the more I think it would be
best for the where block *not* to start a new scope.
That would completely eliminate all these scope issues,
along with break/return/yield problems, etc.

-- 
Greg


From greg.ewing at canterbury.ac.nz  Sat Jul 18 04:08:20 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 18 Jul 2009 14:08:20 +1200
Subject: [Python-ideas] Where-statement (Proposal for
	function	expressions)
In-Reply-To: <79990c6b0907170636p355dce83i328b9ca200d8f069@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<184a9f5a0907140956n7a9c8ab8ucd6c81d54c0c6ecb@mail.gmail.com>
	<79990c6b0907141216t223aefeaj620209e219b06d7c@mail.gmail.com>
	<91ad5bf80907142050m408c144do7d4c57c31e92a7ad@mail.gmail.com>
	<3bdda690907142251n67f1ddaek9e1f8693dd99f050@mail.gmail.com>
	<4A5D8B63.4040509@canterbury.ac.nz>
	<op.uw57mcv8fvx12t@jerzozwiesz.home.aster.pl>
	<4A5FE431.1070407@canterbury.ac.nz>
	<4A607803.2030801@mrabarnett.plus.com>
	<5d1a32000907170625v5ff9dd2dmddbf83e17f183e71@mail.gmail.com>
	<79990c6b0907170636p355dce83i328b9ca200d8f069@mail.gmail.com>
Message-ID: <4A612E94.40105@canterbury.ac.nz>

Paul Moore wrote:

> Given that the defined semantics are in terms of
> defining then calling a temporary function,

That's another thing that would be saved if we gave
up the idea of a separate scope.

-- 
Greg


From guido at python.org  Sat Jul 18 04:13:54 2009
From: guido at python.org (Guido van Rossum)
Date: Fri, 17 Jul 2009 19:13:54 -0700
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
Message-ID: <ca471dc20907171913v6015044eh8fe1d59fce16d46e@mail.gmail.com>

On Fri, Jul 17, 2009 at 4:41 PM, Michael<fuzzyman at gmail.com> wrote:
> Here are two examples of why allowing return inside a finally block is a bad
> idea:
>
> def f():
> ?try:
> ? ?return 3
> ?finally:
> ? ?return 4
>
> def f():
> ?try:
> ? ?raise Exception()
> ?finally:
> ? ?return 4

Well, too bad. As others have (gently) tried to point out, there will
always remain a gray area for what should be accepted and what
shouldn't. Feel free to put in your company's style guide that this is
a bad idea. In most cases it probably is. But I don't think the parser
should police this particular issue. (And were you surprised anyway?
Since you agree that an exception raised in a finally block has
well-defined semantics, why wouldn't a return statement?)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)


From bruce at leapyear.org  Sat Jul 18 04:25:03 2009
From: bruce at leapyear.org (Bruce Leban)
Date: Fri, 17 Jul 2009 19:25:03 -0700
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <ca471dc20907171913v6015044eh8fe1d59fce16d46e@mail.gmail.com>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com> 
	<ca471dc20907171913v6015044eh8fe1d59fce16d46e@mail.gmail.com>
Message-ID: <cf5b87740907171925k10693d89v8523ffb3aebec20f@mail.gmail.com>

On Fri, Jul 17, 2009 at 7:13 PM, Guido van Rossum <guido at python.org> wrote:

> Since you agree that an exception raised in a finally block has
> well-defined semantics, why wouldn't a return statement?)
>

I think that to some extent exception > return. That is in both of these
cases

   try:
     return
   finally:
     raise Exception()

   try:
     raise Exception()
   finally:
     return

it makes sense to me that the caller would see an exception. I disagree that
it should be a syntax error. But it could be a runtime error when an
exception would get silently eaten.

--- Bruce
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090717/dd5c8b31/attachment.html>

From cmjohnson.mailinglist at gmail.com  Sat Jul 18 04:36:37 2009
From: cmjohnson.mailinglist at gmail.com (Carl Johnson)
Date: Fri, 17 Jul 2009 16:36:37 -1000
Subject: [Python-ideas] Proposal for extending the collections module -
	bags / multisets, ordered sets / unique lists
In-Reply-To: <4A612B47.3050706@gmail.com>
References: <4A612B47.3050706@gmail.com>
Message-ID: <3bdda690907171936x7d9d571as7a840931543051d7@mail.gmail.com>

Excuse my ignorance, but it's been a while since I've taken an
algorithms class: Would a bag have any Big-O performance advantage
over just using a list but ignoring the fact that it's in an order?
Non-big O but otherwise practical speed advantages?

As for the ordered set, there I can see the uses much more clearly,
and the proposal to included it has come up on this list before. At
the same time though, I think some of those uses can also be realized
using the new collections.Counter class. Then again, if there's
interest, why not?

-- Carl Johnson


From ncoghlan at gmail.com  Sat Jul 18 04:48:16 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 18 Jul 2009 12:48:16 +1000
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
Message-ID: <4A6137F0.7040600@gmail.com>

Michael wrote:
> def f():
>   try:
>     raise Exception()
>   finally:
>     return 4

I actually did this on purpose recently. I felt somewhat evil while I
was doing it, but it was the most concise way of getting the behaviour I
wanted.

I'll probably go back and change it at some though, since exploiting
this trick will likely confuse someone in the future.

That said, I think this is in the realm of style guides and lint tools
rather than the compiler.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ben+python at benfinney.id.au  Sat Jul 18 04:54:10 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 18 Jul 2009 12:54:10 +1000
Subject: [Python-ideas] Where-statement (Proposal for
	function	expressions)
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<eae285400907160837v7f483df9t3d757cd46e2429af@mail.gmail.com>
	<4A5FDBBA.6@canterbury.ac.nz>
	<eae285400907170120u5932eef2obaa667ccd87c30b7@mail.gmail.com>
	<de32cc030907170450x2f40671bj35c6322142012cb2@mail.gmail.com>
	<87my73fqyv.fsf@benfinney.id.au>
	<4A612AA5.9020108@canterbury.ac.nz>
Message-ID: <87iqhqd7l9.fsf@benfinney.id.au>

Greg Ewing <greg.ewing at canterbury.ac.nz> writes:

> Ben Finney wrote:
> 
> > I haven't found that any of the examples improve the clarity of the
> > code over using a named function.
> 
> This isn't about unnamed functions. The functions defined in a where
> block still have names.

So this is about introducing a new syntax for named functions? That
drives it even further from clarity, in my estimation.

-- 
 \       ?An idea isn't responsible for the people who believe in it.? |
  `\                                      ?Donald Robert Perry Marquis |
_o__)                                                                  |
Ben Finney



From ncoghlan at gmail.com  Sat Jul 18 04:58:18 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 18 Jul 2009 12:58:18 +1000
Subject: [Python-ideas] Proposal for extending the collections module -
 bags / multisets, ordered sets / unique lists
In-Reply-To: <4A612B47.3050706@gmail.com>
References: <4A612B47.3050706@gmail.com>
Message-ID: <4A613A4A.3010607@gmail.com>

Michael Lenzen wrote:
> In a nutshell, I want to add 2 new classes (and respective frozen
> varients) to the collections module, namely a bag (multiset) class and a
> setlist (ordered set/unique list) class.  I know this has been floated
> before, but I think it merits more consideration.

A Counter class (aka bag/multiset) and an OrderedDict class were added
to collections for 3.1. These classes will also be available in 2.7.

Counter *is* a bag implementation, while OrderedDict can trivially be
used as the basis for an OrderedSet implementation.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From curt at hagenlocher.org  Sat Jul 18 05:04:42 2009
From: curt at hagenlocher.org (Curt Hagenlocher)
Date: Fri, 17 Jul 2009 20:04:42 -0700
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <ca471dc20907171913v6015044eh8fe1d59fce16d46e@mail.gmail.com>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
	<ca471dc20907171913v6015044eh8fe1d59fce16d46e@mail.gmail.com>
Message-ID: <d2155e360907172004o250f19b8h21fec2c0020581f6@mail.gmail.com>

On Fri, Jul 17, 2009 at 7:13 PM, Guido van Rossum <guido at python.org> wrote:
>
> Since you agree that an exception raised in a finally block has
> well-defined semantics, why wouldn't a return statement?

Because I'm not always fortunate enough to be programming in Python,
and I don't use any other language where "return" can swallow
exceptions.

I see how this behavior is internally consistent and even potentially
useful. But as a reader of code, I've got internal models of the
semantics of both "finally" and "return", and because I associate
"finally" with exceptional circumstances, my model for "finally"
activates more strongly in this case than my model for "return".
Ultimately, it has an adverse effect on my ability to reason about the
code.

On the other hand, even if I was certain this was a design error, I
wouldn't feel it was worth losing backward compatibility over.

--
Curt Hagenlocher
curt at hagenlocher.org


From greg.ewing at canterbury.ac.nz  Sat Jul 18 05:07:03 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 18 Jul 2009 15:07:03 +1200
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
Message-ID: <4A613C57.8070208@canterbury.ac.nz>

Michael wrote:
> Here are two examples of why allowing return inside a finally block is  
> a bad idea:

Don't do those things, then!

Nobody is *forcing* you to write returns inside
finally blocks.

-- 
Greg


From cmjohnson.mailinglist at gmail.com  Sat Jul 18 05:19:18 2009
From: cmjohnson.mailinglist at gmail.com (Carl Johnson)
Date: Fri, 17 Jul 2009 17:19:18 -1000
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <200907181039.57256.steve@pearwood.info>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<200907180024.31209.steve@pearwood.info>
	<5d1a32000907171328y6da57c7j30b5bfe154e90f32@mail.gmail.com>
	<200907181039.57256.steve@pearwood.info>
Message-ID: <3bdda690907172019hb70bda5h51d76ef31058423a@mail.gmail.com>

Steven D'Aprano wrote:


> It seems to me that the "where" construct makes writing code easier than
> reading code. It is a top-down construct: first you come up with some
> abstract expression:
>
> element = w + x.y - f(z) where:
>
> and then you need to think about how to implement the construct:
>
> ? ?w = 2
> ? ?x = thingy()
> ? ?z = 4
>
> That roughly models the process of writing code in the first place, so I
> can see the attraction. But it doesn't model the process of *reading*
> code:
>
> "Hmmm, element is equal to w + x.y - f(z), okay, but I haven't seen any
> of those names before, are they globals? Ah, there's a 'where'
> statement, better pop that expression into short-term memory while I
> read the block and find out what they are..."
>
> In a real sense, the proposed 'where' block breaks the flow of reading
> code. Normally, reading a function proceeds in an orderly fashion from
> the start of the function to the end in (mostly) sequential order, a
> bottom-up process:

This objection seems to be a very convincing one. At the very least, I
think we should give up on the idea of having the "where" block use a
different scope, since that seems to cause more problems than it
solves.

For me, the original reason why where clauses looked appealing is that
I think there are some very limited cases in which the code reads
better out of order. My canonical example for this is "new_list =
sorted(old_list, key=...)" It just seems like sometimes you don't want
to hear about the key func until after you know that you're going to
sort a list. A simple "where" (or even "do ? where" which might read a
little better) statement would solve this problem by basically just
letting you one line out of order.

do new_list = sorted(old_list, key=key) where:
    def key(item):
        do_data_normalizing_stuff?

I also think that most decorator definitions would look better with
the return at the top rather than the bottom:

def my_debug_decorator(f):
    do return inner where:
        def inner(*args, **kwargs):
            print(args, kwargs)
            return f(*args, **kwargs)

Yes, this nests a level deeper than usual (flat is better than
nested), but I think it's a lot less confusing than the usual form of
decorators with the return at the end where your eye doesn't know
where to rest.

On the other hand, it's not that hard to make a decorator decorator to
get rid of the weird return clause, and having any where statement at
all invites the abuse of nested wheres:

do x += y where:
    do y = z where:
        do z = x where:
            x = 2
print(x) # 4

Is the temptation for abuse here so strong that it will out weigh the
potential for readability gains? Does this construct violate TOOWTDI?

My answer is a definite? maybe.

So for now, I think "where" is not ready for prime time.

My 2 cents,

-- Carl Johnson


From zuo at chopin.edu.pl  Sat Jul 18 05:30:25 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Sat, 18 Jul 2009 05:30:25 +0200
Subject: [Python-ideas] Proposal for extending the collections module -
 bags / multisets, ordered sets / unique lists
In-Reply-To: <4A613A4A.3010607@gmail.com>
References: <4A612B47.3050706@gmail.com> <4A613A4A.3010607@gmail.com>
Message-ID: <op.uw8uozxnfvx12t@jerzozwiesz.home.aster.pl>

Nick Coghlan <ncoghlan at gmail.com> wrote:

> Counter *is* a bag implementation, while OrderedDict can trivially be
> used as the basis for an OrderedSet implementation.

Yeah, but it'd be nice to have OrderedSet in collections.

+1 from me :)

*j


From greg.ewing at canterbury.ac.nz  Sat Jul 18 05:33:59 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 18 Jul 2009 15:33:59 +1200
Subject: [Python-ideas] Where-statement (Proposal for
	function	expressions)
In-Reply-To: <200907181039.57256.steve@pearwood.info>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<200907180024.31209.steve@pearwood.info>
	<5d1a32000907171328y6da57c7j30b5bfe154e90f32@mail.gmail.com>
	<200907181039.57256.steve@pearwood.info>
Message-ID: <4A6142A7.2050900@canterbury.ac.nz>

Steven D'Aprano wrote:

> But the where clause introduces *look-ahead* to the process:
> 
> def parrot():
>     x = 1
>     result = x+y+z where:  # look-ahead
>         y = 2
>         z = 3
>     return result

I think this example is too rarified to give much idea of
how readable a where-statement might be. In real life
you wouldn't use arbitrary names like x, y, z. You'd
choose something to make the expression meaningful in
its own right, so you can get the gist of what's going
on just from reading the first line. Then you only
need to look inside the where block if you want to
know the fine details.

Also, I don't think that "look ahead" is necessarily
harmful unless the stuff ahead has side effects.

We seem to be happy with "look somewhere else" when
calling a function, whose definition could appear
textually before or after the point of call.

-- 
Greg


From mwm-keyword-python.b4bdba at mired.org  Sat Jul 18 05:48:04 2009
From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer)
Date: Fri, 17 Jul 2009 23:48:04 -0400
Subject: [Python-ideas] Proposal for extending the collections module
	-	bags / multisets, ordered sets / unique lists
In-Reply-To: <4A613A4A.3010607@gmail.com>
References: <4A612B47.3050706@gmail.com>
	<4A613A4A.3010607@gmail.com>
Message-ID: <20090717234804.036ce5f7@bhuda.mired.org>

On Sat, 18 Jul 2009 12:58:18 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:

> Michael Lenzen wrote:
> > In a nutshell, I want to add 2 new classes (and respective frozen
> > varients) to the collections module, namely a bag (multiset) class and a
> > setlist (ordered set/unique list) class.  I know this has been floated
> > before, but I think it merits more consideration.
> 
> A Counter class (aka bag/multiset) and an OrderedDict class were added
> to collections for 3.1. These classes will also be available in 2.7.
> 
> Counter *is* a bag implementation, while OrderedDict can trivially be
> used as the basis for an OrderedSet implementation.

If that's good enough, why do we have set?

More importantly, will the ones that you don't have to write yourself
all have the same API? It would be nice to be able to change between
the four cases as the requirements change without having to audit
every use of them to turn the spelling of add_element_to_type_x into
add_element_to_type_y, and so on.

       <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


From greg.ewing at canterbury.ac.nz  Sat Jul 18 06:00:35 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 18 Jul 2009 16:00:35 +1200
Subject: [Python-ideas] Where-statement (Proposal
	for	function	expressions)
In-Reply-To: <87iqhqd7l9.fsf@benfinney.id.au>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<eae285400907160837v7f483df9t3d757cd46e2429af@mail.gmail.com>
	<4A5FDBBA.6@canterbury.ac.nz>
	<eae285400907170120u5932eef2obaa667ccd87c30b7@mail.gmail.com>
	<de32cc030907170450x2f40671bj35c6322142012cb2@mail.gmail.com>
	<87my73fqyv.fsf@benfinney.id.au> <4A612AA5.9020108@canterbury.ac.nz>
	<87iqhqd7l9.fsf@benfinney.id.au>
Message-ID: <4A6148E3.9090200@canterbury.ac.nz>

Ben Finney wrote:

> So this is about introducing a new syntax for named functions?

No, the named functions are declared in exactly
the same way as they ever were.

It isn't even particularly about functions at all,
that just happens to be one of its potential uses.

It's about being able to put higher-level code
ahead of lower-level code, without having to
relegate the lower-level code to somewhere far
away.

-- 
Greg


From mwm-keyword-python.b4bdba at mired.org  Sat Jul 18 05:58:45 2009
From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer)
Date: Fri, 17 Jul 2009 23:58:45 -0400
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <4A6142A7.2050900@canterbury.ac.nz>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<200907180024.31209.steve@pearwood.info>
	<5d1a32000907171328y6da57c7j30b5bfe154e90f32@mail.gmail.com>
	<200907181039.57256.steve@pearwood.info>
	<4A6142A7.2050900@canterbury.ac.nz>
Message-ID: <20090717235845.17f87929@bhuda.mired.org>

On Sat, 18 Jul 2009 15:33:59 +1200
Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> Steven D'Aprano wrote:
> 
> > But the where clause introduces *look-ahead* to the process:
> > 
> > def parrot():
> >     x = 1
> >     result = x+y+z where:  # look-ahead
> >         y = 2
> >         z = 3
> >     return result
> 
> I think this example is too rarified to give much idea of
> how readable a where-statement might be. 

I, on the other hand, am convinced that this example is dead on. By
requiring the definition to be contextually close to the use, you'll
encourage people to use shorter, less meaningful names. Or can you
honestly say that you use names of the same length and carrying as
much meaning for local variables as you do for globals, attributes and
parameters?

> In real life you wouldn't use arbitrary names like x, y, z. You'd
> choose something to make the expression meaningful in its own right,
> so you can get the gist of what's going on just from reading the
> first line.

What where you're pointing your pronouns! I know you don't speak for
me, and I'm pretty sure you don't speak for Steven (but I don't
either, and he may agree with you).

I'd considered where to be promising, assuming that the problems it
has could be worked through. But Steven's observations are deadly. Not
that the original impulse wasn't good - trying to find things that
scratch the itches that cause repeated "function definition in an
expression" proposals is almost certainly worthwhile.

    <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


From greg.ewing at canterbury.ac.nz  Sat Jul 18 06:20:43 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 18 Jul 2009 16:20:43 +1200
Subject: [Python-ideas] Where-statement (Proposal for
	function	expressions)
In-Reply-To: <3bdda690907172019hb70bda5h51d76ef31058423a@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<200907180024.31209.steve@pearwood.info>
	<5d1a32000907171328y6da57c7j30b5bfe154e90f32@mail.gmail.com>
	<200907181039.57256.steve@pearwood.info>
	<3bdda690907172019hb70bda5h51d76ef31058423a@mail.gmail.com>
Message-ID: <4A614D9B.7060500@canterbury.ac.nz>

> Steven D'Aprano wrote:
> > In a real sense, the proposed 'where' block breaks the flow of reading
> > code. Normally, reading a function proceeds in an orderly fashion from
> > the start of the function to the end in (mostly) sequential order, a
> > bottom-up process:

Here you're assuming that the order of *execution* of the
code is necessarily the best order for *reading* it in
order to understand what it does.

In general I think that's mostly false. When encountering
a program for the first time, one tends to look for the
"main" function first, to get an overall idea of what's
being done, then recursively work one's way down into lower
level functions.

Python supports this on a large scale by allowing functions
and classes to be written pretty much in any order,
regardless of the order of execution.

There's not much support for it on a small scale, though.
There is a little. List comprehensions are one example,
where you get to see the high-level intent first -- make
a list of stuff -- and then you're shown the lower-level
details.

Another one that's easy to overlook is the assignment
statement. Nobody seems to be bothered by the fact that
the *right* hand side has to be evaluated before assigning
to the left hand side.

In a sense, the where-block is in part an attempt to extend
the assignment statement so that the "right hand side" can
span more than one statement.

-- 
Greg


From m.lenzen at gmail.com  Sat Jul 18 06:51:49 2009
From: m.lenzen at gmail.com (Michael Lenzen)
Date: Fri, 17 Jul 2009 23:51:49 -0500
Subject: [Python-ideas] Proposal for extending the collections module -
 bags / multisets, ordered sets / unique lists
In-Reply-To: <3bdda690907171936x7d9d571as7a840931543051d7@mail.gmail.com>
References: <4A612B47.3050706@gmail.com>
	<3bdda690907171936x7d9d571as7a840931543051d7@mail.gmail.com>
Message-ID: <4A6154E5.7090000@gmail.com>

A Counter class is NOT a multiset, it allows negative and zero 
elements.  This makes as much sense to me as having negative elements in 
a Set.  This is all besides the fact that Counter is nothing more than 
defaultdict(int) so it should be removed regardless of whether or not 
bags are added.

-Michael Lenzen

On 07/17/2009 09:36 PM, Carl Johnson wrote:
> Excuse my ignorance, but it's been a while since I've taken an
> algorithms class: Would a bag have any Big-O performance advantage
> over just using a list but ignoring the fact that it's in an order?
> Non-big O but otherwise practical speed advantages?
>
> As for the ordered set, there I can see the uses much more clearly,
> and the proposal to included it has come up on this list before. At
> the same time though, I think some of those uses can also be realized
> using the new collections.Counter class. Then again, if there's
> interest, why not?
>
> -- Carl Johnson
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>    


From m.lenzen at gmail.com  Sat Jul 18 06:57:06 2009
From: m.lenzen at gmail.com (Michael Lenzen)
Date: Fri, 17 Jul 2009 23:57:06 -0500
Subject: [Python-ideas] Proposal for extending the collections module -
 bags / multisets, ordered sets / unique lists
In-Reply-To: <4A613A4A.3010607@gmail.com>
References: <4A612B47.3050706@gmail.com> <4A613A4A.3010607@gmail.com>
Message-ID: <4A615622.9090308@gmail.com>

I posted this in the wrong thread before, sorry.

A Counter class is NOT a multiset/bag, it allows negative and zero 
elements.  This makes as much sense to me as having negative elements in 
a Set.  This is all besides the fact that Counter is nothing more than 
defaultdict(int) so it should be removed regardless of whether or not 
bags are added.

-Michael Lenzen

On 07/17/2009 09:58 PM, Nick Coghlan wrote:
> Michael Lenzen wrote:
>    
>> In a nutshell, I want to add 2 new classes (and respective frozen
>> varients) to the collections module, namely a bag (multiset) class and a
>> setlist (ordered set/unique list) class.  I know this has been floated
>> before, but I think it merits more consideration.
>>      
>
> A Counter class (aka bag/multiset) and an OrderedDict class were added
> to collections for 3.1. These classes will also be available in 2.7.
>
> Counter *is* a bag implementation, while OrderedDict can trivially be
> used as the basis for an OrderedSet implementation.
>
> Cheers,
> Nick.
>
>    
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090717/ebbcbf39/attachment.html>

From ben+python at benfinney.id.au  Sat Jul 18 07:07:09 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 18 Jul 2009 15:07:09 +1000
Subject: [Python-ideas] Where-statement (Proposal for
	function	expressions)
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<200907180024.31209.steve@pearwood.info>
	<5d1a32000907171328y6da57c7j30b5bfe154e90f32@mail.gmail.com>
	<200907181039.57256.steve@pearwood.info>
	<3bdda690907172019hb70bda5h51d76ef31058423a@mail.gmail.com>
	<4A614D9B.7060500@canterbury.ac.nz>
Message-ID: <8763dqd1fm.fsf@benfinney.id.au>

Greg Ewing <greg.ewing at canterbury.ac.nz> writes:

> In a sense, the where-block is in part an attempt to extend the
> assignment statement so that the "right hand side" can span more than
> one statement.

That pretty much spells its doom, then. The BDFL is quite firm (as
demonstrated in, e.g., the pronouncement against multi-line lambda
<URL:http://www.python.org/dev/peps/pep-3099/>) that multi-statement
expressions are verboten.

-- 
 \         ?Pinky, are you pondering what I'm pondering?? ?I think so, |
  `\   Brain, but why does a forklift have to be so big if all it does |
_o__)                           is lift forks?? ?_Pinky and The Brain_ |
Ben Finney



From m.lenzen at gmail.com  Sat Jul 18 07:17:44 2009
From: m.lenzen at gmail.com (Michael Lenzen)
Date: Sat, 18 Jul 2009 00:17:44 -0500
Subject: [Python-ideas] Proposal for extending the collections module -
 bags / multisets, ordered sets / unique lists
In-Reply-To: <3bdda690907171936x7d9d571as7a840931543051d7@mail.gmail.com>
References: <4A612B47.3050706@gmail.com>
	<3bdda690907171936x7d9d571as7a840931543051d7@mail.gmail.com>
Message-ID: <4A615AF8.8050406@gmail.com>

Sorry, the previous post was in the wrong thread.

Yes, checking for inclusion (elem in bag) and removing an elem would be 
O(1) time as opposed to O(n).  I think the former is the real case for 
bags over lists.  As for real world applications it makes a huge 
difference in a program I am running, I am repeatedly searching a 
collection of almost 1 million elements for an item and it takes a long 
time using a list.  A list just iterates through all of the elements 
until it finds what it's looking for whereas a bag just hashes the element.

-Michael Lenzen

On 07/17/2009 09:36 PM, Carl Johnson wrote:
> Excuse my ignorance, but it's been a while since I've taken an
> algorithms class: Would a bag have any Big-O performance advantage
> over just using a list but ignoring the fact that it's in an order?
> Non-big O but otherwise practical speed advantages?
>
> As for the ordered set, there I can see the uses much more clearly,
> and the proposal to included it has come up on this list before. At
> the same time though, I think some of those uses can also be realized
> using the new collections.Counter class. Then again, if there's
> interest, why not?
>
> -- Carl Johnson
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas


From stephen at xemacs.org  Sat Jul 18 07:17:05 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 18 Jul 2009 14:17:05 +0900
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <4A613C57.8070208@canterbury.ac.nz>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
	<4A613C57.8070208@canterbury.ac.nz>
Message-ID: <87skguh8oe.fsf@uwakimon.sk.tsukuba.ac.jp>

Greg Ewing writes:
 > Michael wrote:
 > > Here are two examples of why allowing return inside a finally block is  
 > > a bad idea:
 > 
 > Don't do those things, then!
 > 
 > Nobody is *forcing* you to write returns inside
 > finally blocks.

+1

Why not just emphasize in the documentation that return (etc) in a
finally: suite *will* get executed *after* return or an exception is
raised in the try: suite?  "These stunts were performed by
professionals.  Don't try this at home, kids."

More specifically, maybe there should be an explicit warning that in a
finally: suite

    if exit_condition:
        return
    do_work()
    # end of suite

has (perhaps surprisingly, YMMV) different semantics from

    if not exit_condition:
        do_work()
    # end of suite

WDOT?


From ben+python at benfinney.id.au  Sat Jul 18 07:27:37 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 18 Jul 2009 15:27:37 +1000
Subject: [Python-ideas] Make return inside a finally a SyntaxError
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
	<4A6137F0.7040600@gmail.com>
Message-ID: <87ws66blx2.fsf@benfinney.id.au>

Nick Coghlan <ncoghlan at gmail.com> writes:

> That said, I think this is in the realm of style guides and lint tools
> rather than the compiler.

That's a good idea. The pylint folks seem quite receptive to patches;
perhaps you (Michael) could present a new test to catch this class of
likely-error constructs.

-- 
 \     ?[W]e are still the first generation of users, and for all that |
  `\      we may have invented the net, we still don't really get it.? |
_o__)                                                   ?Douglas Adams |
Ben Finney



From stephen at xemacs.org  Sat Jul 18 07:35:22 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 18 Jul 2009 14:35:22 +0900
Subject: [Python-ideas] Where-statement
	(Proposal	for	function	expressions)
In-Reply-To: <4A6148E3.9090200@canterbury.ac.nz>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<eae285400907160837v7f483df9t3d757cd46e2429af@mail.gmail.com>
	<4A5FDBBA.6@canterbury.ac.nz>
	<eae285400907170120u5932eef2obaa667ccd87c30b7@mail.gmail.com>
	<de32cc030907170450x2f40671bj35c6322142012cb2@mail.gmail.com>
	<87my73fqyv.fsf@benfinney.id.au>
	<4A612AA5.9020108@canterbury.ac.nz>
	<87iqhqd7l9.fsf@benfinney.id.au>
	<4A6148E3.9090200@canterbury.ac.nz>
Message-ID: <87r5weh7tx.fsf@uwakimon.sk.tsukuba.ac.jp>

Greg Ewing writes:

 > It's about being able to put higher-level code
 > ahead of lower-level code, without having to
 > relegate the lower-level code to somewhere far
 > away.

In other words, we need better programmers' editors which can
temporarily bring the low-level code close to the high-level code, not
more syntax.



From ben+python at benfinney.id.au  Sat Jul 18 07:50:11 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 18 Jul 2009 15:50:11 +1000
Subject: [Python-ideas] Where-statement
	(Proposal	for	function	expressions)
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<eae285400907160837v7f483df9t3d757cd46e2429af@mail.gmail.com>
	<4A5FDBBA.6@canterbury.ac.nz>
	<eae285400907170120u5932eef2obaa667ccd87c30b7@mail.gmail.com>
	<de32cc030907170450x2f40671bj35c6322142012cb2@mail.gmail.com>
	<87my73fqyv.fsf@benfinney.id.au>
	<4A612AA5.9020108@canterbury.ac.nz>
	<87iqhqd7l9.fsf@benfinney.id.au>
	<4A6148E3.9090200@canterbury.ac.nz>
	<87r5weh7tx.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <87skgubkvg.fsf@benfinney.id.au>

"Stephen J. Turnbull" <stephen at xemacs.org>
writes:

> In other words, we need better programmers' editors which can
> temporarily bring the low-level code close to the high-level code, not
> more syntax.

That assumes that we only look at program code using such editors, and
ignores all the time spent looking at the code on paper, on web sites,
in messages to each other, etc.

If the source code is hard enough to read that we need special tools to
make it readable, there's not much point any more in pretending that
it's plain text. I hope we never approach that.

-- 
 \      ?I like my dental hygenist, I think she's very pretty; so when |
  `\    I go to have my teeth cleaned, while I'm in the waiting room I |
_o__)                    eat an entire box of cookies.? ?Steven Wright |
Ben Finney



From zuo at chopin.edu.pl  Sat Jul 18 07:52:18 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Sat, 18 Jul 2009 07:52:18 +0200
Subject: [Python-ideas] Where-statement (Proposal for function
 expressions)
In-Reply-To: <8763dqd1fm.fsf@benfinney.id.au>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<200907180024.31209.steve@pearwood.info>
	<5d1a32000907171328y6da57c7j30b5bfe154e90f32@mail.gmail.com>
	<200907181039.57256.steve@pearwood.info>
	<3bdda690907172019hb70bda5h51d76ef31058423a@mail.gmail.com>
	<4A614D9B.7060500@canterbury.ac.nz> <8763dqd1fm.fsf@benfinney.id.au>
Message-ID: <op.uw809gcjfvx12t@jerzozwiesz.home.aster.pl>

Ben Finney <ben+python at benfinney.id.au> wrote:

> Greg Ewing <greg.ewing at canterbury.ac.nz> writes:
>
>> In a sense, the where-block is in part an attempt to extend the
>> assignment statement so that the "right hand side" can span more
>> than one statement.
>
> That pretty much spells its doom, then. The BDFL is quite firm (as
> demonstrated in, e.g., the pronouncement against multi-line lambda
> <URL:http://www.python.org/dev/peps/pep-3099/>) that multi-statement
> expressions are verboten.

But where-statement is not an expression at all -- yet it's a sequence
of statements. And even in that PEP you can read that a new syntax for
anonymous functions would be welcone:

"At one point lambda was slated for removal in Python 3000.
Unfortunately no one was able to come up with a better way of providing
anonymous functions. And so lambda is here to stay."

IMHO 'where' would be better than lambda in many contexts, especially
if there is a need for function as argument, e.g. for filter(), map(),
sort(key=...) etc., and also for some list comprehensions and generator
expressions.

Regards,
-- 
Jan Kaliszewski <zuo at chopin.edu.pl>


From steve at pearwood.info  Sat Jul 18 08:10:37 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 18 Jul 2009 16:10:37 +1000
Subject: [Python-ideas] Where-statement (Proposal for
	=?iso-8859-1?q?function=09expressions?=)
In-Reply-To: <4A614D9B.7060500@canterbury.ac.nz>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907172019hb70bda5h51d76ef31058423a@mail.gmail.com>
	<4A614D9B.7060500@canterbury.ac.nz>
Message-ID: <200907181610.38520.steve@pearwood.info>

On Sat, 18 Jul 2009 02:20:43 pm Greg Ewing wrote:
> > Steven D'Aprano wrote:
> > > In a real sense, the proposed 'where' block breaks the flow of
> > > reading code. Normally, reading a function proceeds in an orderly
> > > fashion from the start of the function to the end in (mostly)
> > > sequential order, a bottom-up process:
>
> Here you're assuming that the order of *execution* of the
> code is necessarily the best order for *reading* it in
> order to understand what it does.
>
> In general I think that's mostly false. When encountering
> a program for the first time, one tends to look for the
> "main" function first, to get an overall idea of what's
> being done, then recursively work one's way down into lower
> level functions.

I was explicitly talking about reading a *function*, not the entire 
program. In general, one does not try to keep the entire program in 
one's brain at once (unless it's a trivial program) -- one swaps 
individual routines in and out as required:

"Hmm, let's see how the main function works... okay, first it grabs 
input from the user, now that I know that, I can pop the details from 
short-term memory and just treat it as a 'get input' black box. Next it 
creates a Pager object, what does that do... oh, it creates pages, duh, 
now I can forget the details and treat it as a black box as well... "

(At least I do this.)

But for understanding a single function, you do need to keep the whole 
thing in your mind at once: you can't treat it as a black box, because 
that defeats the purpose of trying to *understand* it. (If you want a 
black box, just read the doc string and function signature and you're 
done.) That's one of the reasons why giant monolithic functions are so 
difficult to understand, even if all they do is one thing. It's also 
why spaghetti code within a single function is so harmful: it defeats 
the reader's ability to compartmentalise functionality into easily 
understandable chunks.


> Python supports this on a large scale by allowing functions
> and classes to be written pretty much in any order,
> regardless of the order of execution.

Yes, and so it should -- a certain amount of "spaghetti" is unavoidable 
in any non-trivial program, but so long as it's kept under control, 
it's not too bad. But a single function is different.


> There's not much support for it on a small scale, though.
> There is a little. List comprehensions are one example,
> where you get to see the high-level intent first -- make
> a list of stuff -- and then you're shown the lower-level
> details.

That's hardly different from a for loop though.


> Another one that's easy to overlook is the assignment
> statement. Nobody seems to be bothered by the fact that
> the *right* hand side has to be evaluated before assigning
> to the left hand side.

I've often wondered whether coding would be simpler (at least for 
beginners) if we wrote assignment left-to-right like this:

2*x + 1 => y

instead of

y = 2*x + 1

That might be a nice experiment for a teaching language some day: does 
left-to-right assignment reduce or eliminate the confusion that 
beginners experience over assignment?

In fact, the more I think about it, the more I like the idea.

Oh yes... Hypertalk does that, only very verbosely:

put 2*x + 1 into y

And doesn't Cobol do something similar?



> In a sense, the where-block is in part an attempt to extend
> the assignment statement so that the "right hand side" can
> span more than one statement.

Yes, I get that. I understand the rationale for it, I just think it's a 
bad idea.




-- 
Steven D'Aprano


From pyideas at rebertia.com  Sat Jul 18 08:18:15 2009
From: pyideas at rebertia.com (Chris Rebert)
Date: Fri, 17 Jul 2009 23:18:15 -0700
Subject: [Python-ideas] Proposal for extending the collections module -
	bags / multisets, ordered sets / unique lists
In-Reply-To: <4A615622.9090308@gmail.com>
References: <4A612B47.3050706@gmail.com> <4A613A4A.3010607@gmail.com>
	<4A615622.9090308@gmail.com>
Message-ID: <50697b2c0907172318t4de4b425md996ff37a7170b01@mail.gmail.com>

> On 07/17/2009 09:58 PM, Nick Coghlan wrote:
>
> Michael Lenzen wrote:
>
>
> In a nutshell, I want to add 2 new classes (and respective frozen
> varients) to the collections module, namely a bag (multiset) class and a
> setlist (ordered set/unique list) class.  I know this has been floated
> before, but I think it merits more consideration.
>
>
> A Counter class (aka bag/multiset) and an OrderedDict class were added
> to collections for 3.1. These classes will also be available in 2.7.
>
> Counter *is* a bag implementation, while OrderedDict can trivially be
> used as the basis for an OrderedSet implementation.
On Fri, Jul 17, 2009 at 9:57 PM, Michael Lenzen<m.lenzen at gmail.com> wrote:
> I posted this in the wrong thread before, sorry.
>
> A Counter class is NOT a multiset/bag, it allows negative and zero
> elements.  This makes as much sense to me as having negative elements in a
> Set.  This is all besides the fact that Counter is nothing more than
> defaultdict(int) so it should be removed regardless of whether or not bags
> are added.

Truth be told, it's more than just defaultdict(int). It adds
.elements() and .most_common().

Seems bag-like to me.
- Unordered? Check.
- Allows duplicates? Check.
- O(1) containment test? Check.
- Counts multiplicity of elements? Check.
- Iterable? Check.

The only non-bag thing about it is allowing 0 and negative
multiplicities, which I agree is unintuitive; I don't like that
"feature" either.

+0.75 on adding a proper Bag. It could be subclassed from or borrow
code from Counter.

Cheers,
Chris
-- 
http://blog.rebertia.com


From janssen at parc.com  Sat Jul 18 08:24:36 2009
From: janssen at parc.com (Bill Janssen)
Date: Fri, 17 Jul 2009 23:24:36 PDT
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <87skgubkvg.fsf@benfinney.id.au>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<eae285400907160837v7f483df9t3d757cd46e2429af@mail.gmail.com>
	<4A5FDBBA.6@canterbury.ac.nz>
	<eae285400907170120u5932eef2obaa667ccd87c30b7@mail.gmail.com>
	<de32cc030907170450x2f40671bj35c6322142012cb2@mail.gmail.com>
	<87my73fqyv.fsf@benfinney.id.au>
	<4A612AA5.9020108@canterbury.ac.nz>
	<87iqhqd7l9.fsf@benfinney.id.au>
	<4A6148E3.9090200@canterbury.ac.nz>
	<87r5weh7tx.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87skgubkvg.fsf@benfinney.id.au>
Message-ID: <72753.1247898276@parc.com>

Ben Finney <ben+python at benfinney.id.au> wrote:

> That assumes that we only look at program code using such editors, and
> ignores all the time spent looking at the code on paper, on web sites,
> in messages to each other, etc.

Is there a MIME type for Python code?  Something a Web browser or email
client could use to catch Python snippets, and interpose a reasonable
edit/inspect mode for that code?  Still wouldn't help with paper, but
I can imagine other uses for such a registration.

"text/python" seems reasonable, but the registration procedure seems to
suggest "text/prs.python" ("part of products that are not distributed
commercially").

Bill


From pyideas at rebertia.com  Sat Jul 18 08:30:44 2009
From: pyideas at rebertia.com (Chris Rebert)
Date: Fri, 17 Jul 2009 23:30:44 -0700
Subject: [Python-ideas] Proposal for extending the collections module -
	bags / multisets, ordered sets / unique lists
In-Reply-To: <50697b2c0907172318t4de4b425md996ff37a7170b01@mail.gmail.com>
References: <4A612B47.3050706@gmail.com> <4A613A4A.3010607@gmail.com>
	<4A615622.9090308@gmail.com>
	<50697b2c0907172318t4de4b425md996ff37a7170b01@mail.gmail.com>
Message-ID: <50697b2c0907172330k5c7dc88aj3adc8d463993708@mail.gmail.com>

On Fri, Jul 17, 2009 at 11:18 PM, Chris Rebert<pyideas at rebertia.com> wrote:
>> On 07/17/2009 09:58 PM, Nick Coghlan wrote:
>>
>> Michael Lenzen wrote:
>>
>>
>> In a nutshell, I want to add 2 new classes (and respective frozen
>> varients) to the collections module, namely a bag (multiset) class and a
>> setlist (ordered set/unique list) class. ?I know this has been floated
>> before, but I think it merits more consideration.
>>
>>
>> A Counter class (aka bag/multiset) and an OrderedDict class were added
>> to collections for 3.1. These classes will also be available in 2.7.
>>
>> Counter *is* a bag implementation, while OrderedDict can trivially be
>> used as the basis for an OrderedSet implementation.
> On Fri, Jul 17, 2009 at 9:57 PM, Michael Lenzen<m.lenzen at gmail.com> wrote:
>> I posted this in the wrong thread before, sorry.
>>
>> A Counter class is NOT a multiset/bag, it allows negative and zero
>> elements. ?This makes as much sense to me as having negative elements in a
>> Set. ?This is all besides the fact that Counter is nothing more than
>> defaultdict(int) so it should be removed regardless of whether or not bags
>> are added.
>
> Truth be told, it's more than just defaultdict(int). It adds
> .elements() and .most_common().
>
> Seems bag-like to me.
> - Unordered? Check.
> - Allows duplicates? Check.
> - O(1) containment test? Check.
> - Counts multiplicity of elements? Check.
> - Iterable? Check.
>
> The only non-bag thing about it is allowing 0 and negative
> multiplicities, which I agree is unintuitive; I don't like that
> "feature" either.

Actually, from the docs, it also appears (I don't have 3.0 handy to
test) to get len() wrong, using the dict definition of "number of
unique items" rather than just "number of items" as would be more
appropriate for a bag.

In the event a Bag is not added, +1 for adding a method to Counter to
return `sum(count if count > 0 else 0 for count in
a_counter.values())`

Cheers,
Chris
-- 
http://blog.rebertia.com


From steve at pearwood.info  Sat Jul 18 08:41:49 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 18 Jul 2009 16:41:49 +1000
Subject: [Python-ideas] Where-statement
	(=?utf-8?q?Proposal=09for=09function=09expressions?=)
In-Reply-To: <87skgubkvg.fsf@benfinney.id.au>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<87r5weh7tx.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87skgubkvg.fsf@benfinney.id.au>
Message-ID: <200907181641.50119.steve@pearwood.info>

On Sat, 18 Jul 2009 03:50:11 pm Ben Finney wrote:

> If the source code is hard enough to read that we need special tools
> to make it readable, there's not much point any more in pretending
> that it's plain text. I hope we never approach that.

But source code isn't plain text. It has syntax, and grammar. Source 
code is a structured collection of tokens -- you can't put arbitrary 
plain text in source code and expect it to execute, or compile.

There's value in treating source code like (say) HTML or XML -- it's 
structured text, human readable, and in a pinch if all else fails human 
writable as well, but normally you're expected to use a smart editor 
that understands the tokens and structure of the file. (Except for 
those who think there's something virtuous about keeping track of 
nested tags yourself.)

To a certain extent we already do that: people say "you'll be far more 
productive if you use vi or emacs than if you use Microsoft Notepad". 
We accept that you simply can't be a productive, professional 
programmer using Notepad. (It is a bonus that, if you're stuck without 
your proper tools, you can edit code with Notepad, or ed. Great. It 
would be nice to be able to repair a car with nothing but a 
screwdriver, but nobody honestly thinks that's practical.)

And we do expect people to edit source code using a *text* editor, a 
program that understands about text elements like lines and characters: 
nobody edits source with a hex editor, or by direct manipulation of the 
bytes, so there's already precedence for increasing specialisation.

Perhaps it's time for programmers to start using token editors instead 
of text editors. Of course it will be a big paradigm shift, and we'll 
need to think about how token files will interact with utilities like 
grep that expect text files, but none of these things are problems, 
they're opportunities.


-- 
Steven D'Aprano


From ben+python at benfinney.id.au  Sat Jul 18 09:04:47 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 18 Jul 2009 17:04:47 +1000
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<eae285400907160837v7f483df9t3d757cd46e2429af@mail.gmail.com>
	<4A5FDBBA.6@canterbury.ac.nz>
	<eae285400907170120u5932eef2obaa667ccd87c30b7@mail.gmail.com>
	<de32cc030907170450x2f40671bj35c6322142012cb2@mail.gmail.com>
	<87my73fqyv.fsf@benfinney.id.au>
	<4A612AA5.9020108@canterbury.ac.nz>
	<87iqhqd7l9.fsf@benfinney.id.au>
	<4A6148E3.9090200@canterbury.ac.nz>
	<87r5weh7tx.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87skgubkvg.fsf@benfinney.id.au> <72753.1247898276@parc.com>
Message-ID: <87ocribhf4.fsf@benfinney.id.au>

Bill Janssen <janssen at parc.com> writes:

> Is there a MIME type for Python code? Something a Web browser or email
> client could use to catch Python snippets, and interpose a reasonable
> edit/inspect mode for that code? Still wouldn't help with paper, but I
> can imagine other uses for such a registration.

My system-installed ?/etc/mime.types? has an entry for each of
?application/x-python-code? (for compiled bytecode) and ?text/x-python?
(for Python source).

-- 
 \          ?When we call others dogmatic, what we really object to is |
  `\   their holding dogmas that are different from our own.? ?Charles |
_o__)                                                           Issawi |
Ben Finney



From stephen at xemacs.org  Sat Jul 18 11:25:04 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 18 Jul 2009 18:25:04 +0900
Subject: [Python-ideas]
	Where-statement	(Proposal	for	function	expressions)
In-Reply-To: <87skgubkvg.fsf@benfinney.id.au>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<eae285400907160837v7f483df9t3d757cd46e2429af@mail.gmail.com>
	<4A5FDBBA.6@canterbury.ac.nz>
	<eae285400907170120u5932eef2obaa667ccd87c30b7@mail.gmail.com>
	<de32cc030907170450x2f40671bj35c6322142012cb2@mail.gmail.com>
	<87my73fqyv.fsf@benfinney.id.au>
	<4A612AA5.9020108@canterbury.ac.nz>
	<87iqhqd7l9.fsf@benfinney.id.au>
	<4A6148E3.9090200@canterbury.ac.nz>
	<87r5weh7tx.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87skgubkvg.fsf@benfinney.id.au>
Message-ID: <87prbygx73.fsf@uwakimon.sk.tsukuba.ac.jp>

Ben Finney writes:
 > "Stephen J. Turnbull" <stephen at xemacs.org> writes:

 > > In other words, we need better programmers' editors which can
 > > temporarily bring the low-level code close to the high-level code, not
 > > more syntax.

 > That assumes that we only look at program code using such editors,
 > and ignores all the time spent looking at the code on paper, on web
 > sites, in messages to each other, etc.

No, actually it is based on a belief that the problem of serializing a
program in a readable way is in general intractible.

 > If the source code is hard enough to read that we need special tools to
 > make it readable,

Source code is *not* plain text.  In all modern languages it is
explicitly tree-structured.  For that reason, having special tools to
help read it (tags, programmers' editors, IDEs) is a long-established
practice.

The point of the where clause is to allow that tree structure to be
expressed in top-down fashion, keeping subordinate nodes "textually
close" to the root.  Problem is, *that only works for very flat
semantics*, in fact only for a root node and one generation of
descendents (and not very many of them, with none very complex, at
that).  In general, it's just not possible to keep nodes of the
"where" tree close to *both* their parents *and* their siblings at
high levels of the tree.  It seems to me that even for people who like
and use a device like the where clause will quickly run into its
limitations.

It is for *that* reason that I suggested that tools, not syntax, are
the way to go here.



From g.brandl at gmx.net  Sat Jul 18 11:52:50 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 18 Jul 2009 11:52:50 +0200
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
Message-ID: <h3s62s$vot$1@ger.gmane.org>

Michael schrieb:
> Here are two examples of why allowing return inside a finally block is  
> a bad idea:
> 
> def f():
>    try:
>      return 3
>    finally:
>      return 4
> 
> def f():
>    try:
>      raise Exception()
>    finally:
>      return 4

I've been bitten by this as well, putting all my cleanup code including the
return unthinkingly into the finally clause.  It took a while of debugging
to find this instance, and realize that the behavior is correct if unexpected.

On the other hand, there may be cases where it is actually the most concise
way to express what you want.  I guess it's like the parameter defaults "wart":
once you've fallen over it once, you'll not make the same mistake again,
and use it when it's correct to do so.

However, why break and continue behave inconsistently, I can't see right now.
Maybe continue needs some more trickery implementation-wise?

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.



From g.brandl at gmx.net  Sat Jul 18 12:01:12 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 18 Jul 2009 12:01:12 +0200
Subject: [Python-ideas] Augment dis.dis to autocompile strings
In-Reply-To: <4A6100A5.8000008@gmail.com>
References: <h3nrtu$oog$1@ger.gmane.org>	<200907171232.21702.steve@pearwood.info>
	<4A6100A5.8000008@gmail.com>
Message-ID: <h3s6in$1vv$1@ger.gmane.org>

Nick Coghlan schrieb:
> Steven D'Aprano wrote:
>> On Fri, 17 Jul 2009 04:35:40 am Terry Reedy wrote:
>> 
>>> Proposal: if the input is a string, do the above (do what I mean)
>>> instead of raising
>>> TypeError: don't know how to disassemble str objects
>>>
>>> Any negatives before I submit a feature request?
>> 
>> +1 on the idea, but how do you determine which of the following are 
>> required?
>> 
>> compile('x+1', '', 'eval')
>> compile('x = x+1', '', 'single')
>> compile('while x < 42: x += 1', '', 'exec')
>> 
>> 
>> Or do you just assume 'exec'?
> 
> We could define it as trying the three in order (first 'eval', then
> 'single', then 'exec') moving on to the next option if it raises syntax
> error:
>
> from dis import dis
> def dis_str(source):
>   modes = ('eval', 'single', 'exec')
>   for mode in modes:
>     try:
>       c = compile(source, '', mode)
>       break
>     except SyntaxError:
>       if mode is modes[-1]:
>         raise
>   return dis(c)

Allowing 'single' has five unfortunate side effects: first, newlines at
the end make a difference; second, there will be PRINT_EXPRs sprinkled
in the disassembly seemingly coming from nowhere, third, statements after
the first will disappear:

>>> dis_str("for x in range(10): x\n")
  1           0 SETUP_LOOP              24 (to 27)
              3 LOAD_NAME                0 (range)
              6 LOAD_CONST               0 (10)
              9 CALL_FUNCTION            1
             12 GET_ITER
        >>   13 FOR_ITER                10 (to 26)
             16 STORE_NAME               1 (x)
             19 LOAD_NAME                1 (x)
             22 PRINT_EXPR
             23 JUMP_ABSOLUTE           13
        >>   26 POP_BLOCK
        >>   27 LOAD_CONST               1 (None)
             30 RETURN_VALUE

>>> dis_str("for x in range(10): x")
  1           0 SETUP_LOOP              24 (to 27)
              3 LOAD_NAME                0 (range)
              6 LOAD_CONST               0 (10)
              9 CALL_FUNCTION            1
             12 GET_ITER
        >>   13 FOR_ITER                10 (to 26)
             16 STORE_NAME               1 (x)
             19 LOAD_NAME                1 (x)
             22 POP_TOP
             23 JUMP_ABSOLUTE           13
        >>   26 POP_BLOCK
        >>   27 LOAD_CONST               1 (None)
             30 RETURN_VALUE

>>> dis_str("x=1\nx=2\n")
  1           0 LOAD_CONST               0 (1)
              3 STORE_NAME               0 (x)
              6 LOAD_CONST               1 (None)
              9 RETURN_VALUE

I don't think using "single" makes sense here; trying "eval" can't hurt,
though.

Georg


-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.



From zuo at chopin.edu.pl  Sat Jul 18 13:00:26 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Sat, 18 Jul 2009 13:00:26 +0200
Subject: [Python-ideas] Where-statement (Proposal for function
 expressions)
In-Reply-To: <200907181610.38520.steve@pearwood.info>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907172019hb70bda5h51d76ef31058423a@mail.gmail.com>
	<4A614D9B.7060500@canterbury.ac.nz>
	<200907181610.38520.steve@pearwood.info>
Message-ID: <op.uw9fi0tefvx12t@jerzozwiesz.home.aster.pl>

[OT]

Steven D'Aprano <steve at pearwood.info> wrote:

> I've often wondered whether coding would be simpler (at least for
> beginners) if we wrote assignment left-to-right like this:
>
> 2*x + 1 => y
>
> instead of
>
> y = 2*x + 1
>
> That might be a nice experiment for a teaching language some day: does
> left-to-right assignment reduce or eliminate the confusion that
> beginners experience over assignment?
>
> In fact, the more I think about it, the more I like the idea.
>
> Oh yes... Hypertalk does that, only very verbosely:
>
> put 2*x + 1 into y
>
> And doesn't Cobol do something similar?

Chuck does it, and just with '=>' :)
(see: http://chuck.cs.princeton.edu/ )

*j


From fuzzyman at gmail.com  Sat Jul 18 13:27:46 2009
From: fuzzyman at gmail.com (Michael Foord)
Date: Sat, 18 Jul 2009 12:27:46 +0100
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <87skguh8oe.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
	<4A613C57.8070208@canterbury.ac.nz>
	<87skguh8oe.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <6f4025010907180427ucd949ffl9f3fef77c6a97d5b@mail.gmail.com>

2009/7/18 Stephen J. Turnbull <stephen at xemacs.org>

> Greg Ewing writes:
>  > Michael wrote:
>  > > Here are two examples of why allowing return inside a finally block is
>  > > a bad idea:
>  >
>  > Don't do those things, then!
>  >
>  > Nobody is *forcing* you to write returns inside
>  > finally blocks.
>
> +1
>
> Why not just emphasize in the documentation that return (etc) in a
> finally: suite *will* get executed *after* return or an exception is
> raised in the try: suite?


That exceptions can be silently swallowed in a finally block (which is
'expected' and usually intended to propagate exceptions) in the presence of
a return (or a break apparently) is worrying.

Another solution would be to have the exception raised instead of swallowed.
There is a harder migration regarding backwards compatibility though - you
can only warn when the exception is swallowed which may never be seen by the
programmer.

Michael





> "These stunts were performed by
> professionals.  Don't try this at home, kids."
>
> More specifically, maybe there should be an explicit warning that in a
> finally: suite
>
>    if exit_condition:
>        return
>    do_work()
>    # end of suite
>
> has (perhaps surprisingly, YMMV) different semantics from
>
>    if not exit_condition:
>        do_work()
>    # end of suite
>
> WDOT?
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
http://www.ironpythoninaction.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090718/4fd58bc3/attachment.html>

From fuzzyman at gmail.com  Sat Jul 18 13:38:53 2009
From: fuzzyman at gmail.com (Michael Foord)
Date: Sat, 18 Jul 2009 12:38:53 +0100
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <ca471dc20907171913v6015044eh8fe1d59fce16d46e@mail.gmail.com>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
	<ca471dc20907171913v6015044eh8fe1d59fce16d46e@mail.gmail.com>
Message-ID: <6f4025010907180438q5e46854bq7547bbb662056112@mail.gmail.com>

2009/7/18 Guido van Rossum <guido at python.org>

> On Fri, Jul 17, 2009 at 4:41 PM, Michael<fuzzyman at gmail.com> wrote:
> > Here are two examples of why allowing return inside a finally block is a
> bad
> > idea:
> >
> > def f():
> >  try:
> >    return 3
> >  finally:
> >    return 4
> >
> > def f():
> >  try:
> >    raise Exception()
> >  finally:
> >    return 4
>
> Well, too bad. As others have (gently) tried to point out, there will
> always remain a gray area for what should be accepted and what
> shouldn't. Feel free to put in your company's style guide that this is
> a bad idea. In most cases it probably is. But I don't think the parser
> should police this particular issue. (And were you surprised anyway?
> Since you agree that an exception raised in a finally block has
> well-defined semantics, why wouldn't a return statement?)



I was surprised - I *thought* the semantics of finally were well defined;
that exceptions inside the try block will be propagated unless a new
exception is raised inside the finally. That return (and break it
transpires) silently and implicitly swallow those exceptions is surprising.

It is also *odd*. If you *want* behavior then it is trivial to use an except
instead of a finally, so I don't see a use case for it.

All the best,

Michael


>
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/<http://www.python.org/%7Eguido/>
> )
>



-- 
http://www.ironpythoninaction.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090718/f67d740c/attachment.html>

From fuzzyman at gmail.com  Sat Jul 18 13:42:30 2009
From: fuzzyman at gmail.com (Michael Foord)
Date: Sat, 18 Jul 2009 12:42:30 +0100
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <200907181059.58523.steve@pearwood.info>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
	<200907181059.58523.steve@pearwood.info>
Message-ID: <6f4025010907180442n39cb344fxc7b0d725bb2894c@mail.gmail.com>

2009/7/18 Steven D'Aprano <steve at pearwood.info>

> On Sat, 18 Jul 2009 09:41:10 am Michael wrote:
> > Here are two examples of why allowing return inside a finally block
> > is a bad idea:
> >
> > def f():
> >    try:
> >      return 3
> >    finally:
> >      return 4
>
>
> Given that the finally block is guaranteed to run after exiting the try
> block, I don't see anything objectionable about that. Surprising,
> perhaps, possibly even an artifact of the CPython implementation, but
> why is it a "bad idea" that needs to be protected against?
>

It's arbitrary - and constructs with arbitrary interpretations are generally
not a *good* thing at the very least. It also makes it harder to read code,
where a return in a try:... finally block may not do anything.



>
>
>
> > def f():
> >    try:
> >      raise Exception()
> >    finally:
> >      return 4
>
> That's no weirder than:
>
>
> def f():
>    try:
>        raise Exception()
>     except Exception:
>        return 4
>
> or:
>
> def f():
>    try:
>        raise Exception()
>     except Exception:
>        pass
>    return 4
>

It is much weirder. In the case of explicitly using an except you are
catching the error and choosing not to propagate it. In a finally where the
normal semantics are that the exception is re-raised the return silently
circumvents that.

Michael


>
>
> --
> Steven D'Aprano
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
http://www.ironpythoninaction.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090718/f5b0e282/attachment.html>

From zuo at chopin.edu.pl  Sat Jul 18 14:01:46 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Sat, 18 Jul 2009 14:01:46 +0200
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <6f4025010907180438q5e46854bq7547bbb662056112@mail.gmail.com>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
	<ca471dc20907171913v6015044eh8fe1d59fce16d46e@mail.gmail.com>
	<6f4025010907180438q5e46854bq7547bbb662056112@mail.gmail.com>
Message-ID: <op.uw9ic8drfvx12t@jerzozwiesz.home.aster.pl>

Michael Foord <fuzzyman at gmail.com> wrote:

> I was surprised - I *thought* the semantics of finally were well defined;
> that exceptions inside the try block will be propagated unless a new
> exception is raised inside the finally. That return (and break it
> transpires) silently and implicitly swallow those exceptions is  
> surprising.
>
> It is also *odd*. If you *want* behavior then it is trivial to use an  
> except instead of a finally, so I don't see a use case for it.

+1 and even more.

-- 
Jan Kaliszewski <zuo at chopin.edu.pl>


From terry at jon.es  Sat Jul 18 14:07:06 2009
From: terry at jon.es (Terry Jones)
Date: Sat, 18 Jul 2009 14:07:06 +0200
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: Your message at 12:42:30 on Saturday, 18 July 2009
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
	<200907181059.58523.steve@pearwood.info>
	<6f4025010907180442n39cb344fxc7b0d725bb2894c@mail.gmail.com>
Message-ID: <19041.47850.152959.565539@jon.es>

I agree that having finally subsume returns and exceptions in the try block
will at first be surprising to people, and that it can lead to easy-to-create
but difficult-to-find bugs.

But I think the current behavior is definitely for the best:

- It's dead simple. The finally clause is always done last, and is always
  responsible for the return of the function *no matter what* may have
  happened in the try clause.

- If finally were to automatically propagate any exception from the try
  clause, how would you turn that off when you needed/wanted to? You'd soon
  wind up wanting another keyword, perhaps seriouslyfinally, that was
  guaranteed to be run at the very end.

  If finally propagated exceptions, what should it do if there's an
  exception raised in the finally block?  It begins to get complicated.
  The primitive blanket solution is better, IMO.

- Besides, if you do want a finally clause to propagate an exception from
  its try clause, that's simple as you can just store the exception
  somewhere in an except clause and then let the finally detect it and
  re-raise it, if appropriate.

By keeping finally's semantics simple and resisting the temptation to make
it more sophisticated (e.g., propagating exceptions or return values) you
wind up with a better tool for the programmer who does know what they're
doing. The more sophisticated behavior can easily be added as you need it
on a case-by-case basis. The added functionality shouldn't be provided
unconditionally by the language because you then can't disable it.  I.e., I
prefer the control I get from the current semantics of finally.

Terry


From zuo at chopin.edu.pl  Sat Jul 18 14:29:26 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Sat, 18 Jul 2009 14:29:26 +0200
Subject: [Python-ideas] Where-statement (Proposal for function
 expressions)
In-Reply-To: <87prbygx73.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907151612j78af8f11t1a308ecddb6568d0@mail.gmail.com>
	<5d1a32000907151851h36e3d8daj11e4b29b0c5c607c@mail.gmail.com>
	<m2vdltufis.fsf@cs.uu.nl>
	<5d1a32000907160639x27abcb24teb0e016744f4aaf5@mail.gmail.com>
	<184a9f5a0907160751w8613514l95fdfffafa1167d1@mail.gmail.com>
	<79990c6b0907160806t7c5e0c62q4f30864c9dbd0e5a@mail.gmail.com>
	<eae285400907160837v7f483df9t3d757cd46e2429af@mail.gmail.com>
	<4A5FDBBA.6@canterbury.ac.nz>
	<eae285400907170120u5932eef2obaa667ccd87c30b7@mail.gmail.com>
	<de32cc030907170450x2f40671bj35c6322142012cb2@mail.gmail.com>
	<87my73fqyv.fsf@benfinney.id.au> <4A612AA5.9020108@canterbury.ac.nz>
	<87iqhqd7l9.fsf@benfinney.id.au> <4A6148E3.9090200@canterbury.ac.nz>
	<87r5weh7tx.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87skgubkvg.fsf@benfinney.id.au>
	<87prbygx73.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <op.uw9jncyyfvx12t@jerzozwiesz.home.aster.pl>

Stephen J. Turnbull <stephen at xemacs.org> wrote:

> Problem is, *that only works for very flat
> semantics*, in fact only for a root node and one generation of
> descendents (and not very many of them, with none very complex,
> at that).

Agree that where clause is good for short and rather flat structures.
The same thing is with list-comprehensions/generator expressions --
probably it is not a good idea to use them like that:

(
      2*item for item in (seq4
          for seq4 in sorted(seq3
              for seq3 in set(seq2
                  for seq2 in filter(None, reversed(seq1
                      for seq1 in sorted(seq0
                          for seq0 in map(lambda x, y: y(x),
                                          adict0.keys(),
                                          adict0.values()
                          )
                      if set(seq0) > set(adict0.values()))
                  if len(seq1) > 3))
              )
          if seq3.issubset(adict3.items()))
      if seq4)
)

:)


Me wrote:

> IMHO 'where' would be better than lambda in many contexts, especially
> if there is a need for function as argument, e.g. for filter(), map(),
> sort(key=...) etc., and also for some list comprehensions and generator
> expressions.

Of course, 'where' is not a statement/multi-line lambda, but in practice
it'd be take role of it, being IMHO really useful.

Still I'd prefer:

a = sorted(b, key=how) where def how(item):  # or even 'using'
      DO SOMETHING                            # instead of 'where def'
      return key


...rather than:

def how(item):
      DO SOMETHING
      return key

a = sorted(b, key=how)


Mainly, not because I need it visually but because in such situations
(say 'lambda-situations') I'm not interested *how to do* until I think
about *what to do*.

Regards,
-- 
Jan Kaliszewski <zuo at chopin.edu.pl>


From gerald.britton at gmail.com  Sat Jul 18 14:33:07 2009
From: gerald.britton at gmail.com (Gerald Britton)
Date: Sat, 18 Jul 2009 08:33:07 -0400
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <op.uw9fi0tefvx12t@jerzozwiesz.home.aster.pl>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907172019hb70bda5h51d76ef31058423a@mail.gmail.com> 
	<4A614D9B.7060500@canterbury.ac.nz>
	<200907181610.38520.steve@pearwood.info> 
	<op.uw9fi0tefvx12t@jerzozwiesz.home.aster.pl>
Message-ID: <5d1a32000907180533n74ac5690x4c0ccb4646753ca6@mail.gmail.com>

About return, yield and their ilk in where clauses:

don't do it!

Seriously,  a where clause should be restricted to providing
definitions for the things in the statement for which the where clause
is specified.  Today, we can't do a return (with argument) in a
generator function, which makes sense.  A where clause should not
allow returns or yields or in general actions that explicitly transfer
control beyond the statement.


From fuzzyman at gmail.com  Sat Jul 18 14:37:00 2009
From: fuzzyman at gmail.com (Michael Foord)
Date: Sat, 18 Jul 2009 13:37:00 +0100
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <19041.47850.152959.565539@jon.es>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
	<200907181059.58523.steve@pearwood.info>
	<6f4025010907180442n39cb344fxc7b0d725bb2894c@mail.gmail.com>
	<19041.47850.152959.565539@jon.es>
Message-ID: <6f4025010907180537k313e1b2bva099281fa1413725@mail.gmail.com>

2009/7/18 Terry Jones <terry at jon.es>

> I agree that having finally subsume returns and exceptions in the try block
> will at first be surprising to people, and that it can lead to
> easy-to-create
> but difficult-to-find bugs.
>
> But I think the current behavior is definitely for the best:
>
> - It's dead simple. The finally clause is always done last, and is always
>  responsible for the return of the function *no matter what* may have
>  happened in the try clause.



Finally is normally dead simple but the explanation usually includes what it
does with exceptions:

Finally is *always* run last. Exceptions that occur in the try block are
re-raised once the finally has completed. Exceptions inside the finally
block will be re-raised

The bit that has to be *added* in Python (and usually isn't added because it
isn't at all obvious) is:

Exceptions are not reraised if the finally returns or breaks.


>
>
> - If finally were to automatically propagate any exception from the try
>  clause, how would you turn that off when you needed/wanted to? You'd soon
>  wind up wanting another keyword, perhaps seriouslyfinally, that was
>  guaranteed to be run at the very end.



No, you'd use the already present and obvious except where you have the
*choice* of re-raising or ignoring the exception.


>
>
>  If finally propagated exceptions,


It does.


> what should it do if there's an
>  exception raised in the finally block?


It re-raises it on exit (except for two special cases - return and break).
Try it.


>  It begins to get complicated.




No, it gets simpler - no special cases for the exception handling of finally
in the presence of a return.

Michael


>
>  The primitive blanket solution is better, IMO.
>
> - Besides, if you do want a finally clause to propagate an exception from
>  its try clause, that's simple as you can just store the exception
>  somewhere in an except clause and then let the finally detect it and
>  re-raise it, if appropriate.
>
> By keeping finally's semantics simple and resisting the temptation to make
> it more sophisticated (e.g., propagating exceptions or return values) you
> wind up with a better tool for the programmer who does know what they're
> doing. The more sophisticated behavior can easily be added as you need it
> on a case-by-case basis. The added functionality shouldn't be provided
> unconditionally by the language because you then can't disable it.  I.e., I
> prefer the control I get from the current semantics of finally.
>
> Terry
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
http://www.ironpythoninaction.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090718/a579aa05/attachment.html>

From p.f.moore at gmail.com  Sat Jul 18 14:41:27 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 18 Jul 2009 13:41:27 +0100
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <6f4025010907180537k313e1b2bva099281fa1413725@mail.gmail.com>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
	<200907181059.58523.steve@pearwood.info>
	<6f4025010907180442n39cb344fxc7b0d725bb2894c@mail.gmail.com>
	<19041.47850.152959.565539@jon.es>
	<6f4025010907180537k313e1b2bva099281fa1413725@mail.gmail.com>
Message-ID: <79990c6b0907180541h4c714e16g61805198921e51ad@mail.gmail.com>

2009/7/18 Michael Foord <fuzzyman at gmail.com>:
> Finally is normally dead simple but the explanation usually includes what it
> does with exceptions:
>
> Finally is *always* run last. Exceptions that occur in the try block are
> re-raised once the finally has completed. Exceptions inside the finally
> block will be re-raised
>
> The bit that has to be *added* in Python (and usually isn't added because it
> isn't at all obvious) is:
>
> Exceptions are not reraised if the finally returns or breaks.

While I'll concede that it's a bit subtle, the finally doesn't
"complete" if you return or break - those two statements cause
premature exit. As such, it's quite right that the exception isn't
re-raised in those cases, as the finally clause never "completed".

Subtle, yes. Inconsistent, no.

Paul.


From daniel at stutzbachenterprises.com  Sat Jul 18 14:45:44 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Sat, 18 Jul 2009 07:45:44 -0500
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <200907181039.57256.steve@pearwood.info>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<200907180024.31209.steve@pearwood.info>
	<5d1a32000907171328y6da57c7j30b5bfe154e90f32@mail.gmail.com>
	<200907181039.57256.steve@pearwood.info>
Message-ID: <eae285400907180545u4d12272fwfea29f738d34ca47@mail.gmail.com>

On Fri, Jul 17, 2009 at 7:39 PM, Steven D'Aprano <steve at pearwood.info>wrote:

> In a real sense, the proposed 'where' block breaks the flow of reading
> code. Normally, reading a function proceeds in an orderly fashion from
> the start of the function to the end in (mostly) sequential order, a
> bottom-up process:
>

Normally, that's true, but that's not true when defining a sub-function.
Consider the following:

def foo(items):
    def compute_sort_value(item):
        # compute value based on item's properties
        return value
   # a bunch of other code is here
    items.sort(compute_sort_value)

Above, the body of compute_sort_value appears long before it is executed.
Now consider:

def foo(items):
    # a bunch of other code is here
    items.sort(key=mean) where:
        def compute_sort_value(item):
            return value

Now, the body of compute_sort_value appears exactly where it is executed:
within the call to items.sort().

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090718/0eb29417/attachment.html>

From terry at jon.es  Sat Jul 18 14:49:58 2009
From: terry at jon.es (Terry Jones)
Date: Sat, 18 Jul 2009 14:49:58 +0200
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: Your message at 13:37:00 on Saturday, 18 July 2009
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
	<200907181059.58523.steve@pearwood.info>
	<6f4025010907180442n39cb344fxc7b0d725bb2894c@mail.gmail.com>
	<19041.47850.152959.565539@jon.es>
	<6f4025010907180537k313e1b2bva099281fa1413725@mail.gmail.com>
Message-ID: <19041.50422.747915.707091@jon.es>

>>>>> "Michael" == Michael Foord <fuzzyman at gmail.com> writes:
Michael> The bit that has to be *added* in Python (and usually isn't added
Michael> because it isn't at all obvious) is:

Michael> Exceptions are not reraised if the finally returns or breaks.

OK.

>> If finally propagated exceptions,
Michael> It does.

Sorry - I meant propagates the try exception even in the case that the
finally does a return, break, or raises itself.

>> It begins to get complicated.

Michael> No, it gets simpler - no special cases for the exception handling
Michael> of finally in the presence of a return.

I think it's more complex. Either, or both, of the try block and the
finally block may return/raise etc. What to do when those things conflict?
Which block's opinion has priority? It seems clear (to me) that the finally
block should have the final word, no matter what - including (perhaps even
especially) if its ultimate conclusion conflicts with what the try block did.

Terry


From ironfroggy at gmail.com  Sat Jul 18 14:52:47 2009
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Sat, 18 Jul 2009 08:52:47 -0400
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
Message-ID: <76fd5acf0907180552g73010d78ndfaf82c4b7a0f730@mail.gmail.com>

I honestly can't read those examples and be surprised. Do I have
finally-colored glasses? Maybe "What does f do? It returns 3, but then
in finally returns 4" reads understandably, if silly, to me.

I will admit that a finally with a non-conditional return or raise
will effectively be a blank except: but in that case, as well, I say
"well, don't do that"

On Fri, Jul 17, 2009 at 7:41 PM, Michael<fuzzyman at gmail.com> wrote:
> Here are two examples of why allowing return inside a finally block is a bad
> idea:
>
> def f():
> ?try:
> ? ?return 3
> ?finally:
> ? ?return 4
>
> def f():
> ?try:
> ? ?raise Exception()
> ?finally:
> ? ?return 4
>
>
> Michael Foord
>
> --
> http://www.ironpythoninaction.com
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://techblog.ironfroggy.com/
Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy


From stephen at xemacs.org  Sat Jul 18 14:54:38 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 18 Jul 2009 21:54:38 +0900
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <6f4025010907180427ucd949ffl9f3fef77c6a97d5b@mail.gmail.com>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
	<4A613C57.8070208@canterbury.ac.nz>
	<87skguh8oe.fsf@uwakimon.sk.tsukuba.ac.jp>
	<6f4025010907180427ucd949ffl9f3fef77c6a97d5b@mail.gmail.com>
Message-ID: <87ljmmgnht.fsf@uwakimon.sk.tsukuba.ac.jp>

Michael Foord writes:

 > That exceptions can be silently swallowed in a finally block (which
 > is 'expected' and usually intended to propagate exceptions)

I would say "intended to be executed regardless of control transfer
out of the try block".

 > in the presence of a return (or a break apparently) is worrying.

 > Another solution would be to have the exception raised instead of
 > swallowed.

That would really piss off anybody who deliberately put a return in a
finally block, I think.



From zuo at chopin.edu.pl  Sat Jul 18 14:58:21 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Sat, 18 Jul 2009 14:58:21 +0200
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <19041.47850.152959.565539@jon.es>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
	<200907181059.58523.steve@pearwood.info>
	<6f4025010907180442n39cb344fxc7b0d725bb2894c@mail.gmail.com>
	<19041.47850.152959.565539@jon.es>
Message-ID: <op.uw9kzjcffvx12t@jerzozwiesz.home.aster.pl>

Dnia 18-07-2009 o 14:07:06 Terry Jones <terry at jon.es> napisa?(a):

> I agree that having finally subsume returns and exceptions in the try  
> block will at first be surprising to people, and that it can lead to  
> easy-to-create but difficult-to-find bugs.

IMHO it is enough to treat it as a bug.

> But I think the current behavior is definitely for the best:
[snip]
> - If finally were to automatically propagate any exception from the try
>   clause, how would you turn that off when you needed/wanted to? You'd  
>   soon wind up wanting another keyword, perhaps seriouslyfinally, that
>   was guaranteed to be run at the very end.

No, simply:

try:
     try:
        ...
     finally:
        ...
except Foo:
     ...

>   If finally propagated exceptions, what should it do if there's an
>   exception raised in the finally block?  It begins to get complicated.
>   The primitive blanket solution is better, IMO.

But normally finally propagates exceptions. The problem is that 'return'
breakes that rule. A rule that is both logicall and intuitive.

> - Besides, if you do want a finally clause to propagate an exception from
>   its try clause, that's simple as you can just store the exception
>   somewhere in an except clause and then let the finally detect it and
>   re-raise it, if appropriate.

Do you think about:

try:
     exc = None
     ....
except BaseException as exc:
     pass
finally
     if exc:
        raise exc
     else:
        return foobar

...?

Easy, this example is a joke. The workaround is: not to use return in
finally.
Never. (After all if you need catch an exception, you should use except).

> By keeping finally's semantics simple and resisting the temptation to  
> make it more sophisticated (e.g., propagating exceptions or return  
> values) you wind up with a better tool for the programmer who does
> know what they're doing.

IMHO rather the current behaviour is sophisticated, or rather surprising
and weird.

Regards,

zuo


From zuo at chopin.edu.pl  Sat Jul 18 14:59:11 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Sat, 18 Jul 2009 14:59:11 +0200
Subject: [Python-ideas] Where-statement (Proposal for function
 expressions)
In-Reply-To: <5d1a32000907180533n74ac5690x4c0ccb4646753ca6@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<3bdda690907172019hb70bda5h51d76ef31058423a@mail.gmail.com>
	<4A614D9B.7060500@canterbury.ac.nz>
	<200907181610.38520.steve@pearwood.info>
	<op.uw9fi0tefvx12t@jerzozwiesz.home.aster.pl>
	<5d1a32000907180533n74ac5690x4c0ccb4646753ca6@mail.gmail.com>
Message-ID: <op.uw9k0x0sfvx12t@jerzozwiesz.home.aster.pl>

Gerald Britton <gerald.britton at gmail.com> wrote:

> About return, yield and their ilk in where clauses:
>
> don't do it!
>
> Seriously,  a where clause should be restricted to providing
> definitions for the things in the statement for which the where clause
> is specified.  Today, we can't do a return (with argument) in a
> generator function, which makes sense.  A where clause should not
> allow returns or yields or in general actions that explicitly transfer
> control beyond the statement.

+1

-- 
Jan Kaliszewski <zuo at chopin.edu.pl>


From fuzzyman at gmail.com  Sat Jul 18 15:06:54 2009
From: fuzzyman at gmail.com (Michael Foord)
Date: Sat, 18 Jul 2009 14:06:54 +0100
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <87ljmmgnht.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
	<4A613C57.8070208@canterbury.ac.nz>
	<87skguh8oe.fsf@uwakimon.sk.tsukuba.ac.jp>
	<6f4025010907180427ucd949ffl9f3fef77c6a97d5b@mail.gmail.com>
	<87ljmmgnht.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <6f4025010907180606ucb5df24t9bf144237e47e6c4@mail.gmail.com>

2009/7/18 Stephen J. Turnbull <stephen at xemacs.org>

> Michael Foord writes:
>
>  > That exceptions can be silently swallowed in a finally block (which
>  > is 'expected' and usually intended to propagate exceptions)
>
> I would say "intended to be executed regardless of control transfer
> out of the try block".
>
>  > in the presence of a return (or a break apparently) is worrying.
>
>  > Another solution would be to have the exception raised instead of
>  > swallowed.
>
> That would really piss off anybody who deliberately put a return in a
> finally block, I think.
>
>
I think the current behavior (swallowing an exception that most people would
expect to be re-raised) surprises and pisses people off. Guido has already
made a pronouncement so not much point continuing this discussion.

Michael

-- 
http://www.ironpythoninaction.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090718/e090b9f4/attachment.html>

From zuo at chopin.edu.pl  Sat Jul 18 15:14:11 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Sat, 18 Jul 2009 15:14:11 +0200
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <6f4025010907180606ucb5df24t9bf144237e47e6c4@mail.gmail.com>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
	<4A613C57.8070208@canterbury.ac.nz>
	<87skguh8oe.fsf@uwakimon.sk.tsukuba.ac.jp>
	<6f4025010907180427ucd949ffl9f3fef77c6a97d5b@mail.gmail.com>
	<87ljmmgnht.fsf@uwakimon.sk.tsukuba.ac.jp>
	<6f4025010907180606ucb5df24t9bf144237e47e6c4@mail.gmail.com>
Message-ID: <op.uw9lpxwefvx12t@jerzozwiesz.home.aster.pl>

Michael Foord <fuzzyman at gmail.com> wrote:

> 2009/7/18 Stephen J. Turnbull <stephen at xemacs.org>
>
>> That would really piss off anybody who deliberately put a return in a
>> finally block, I think.
>>
>>
> I think the current behavior (swallowing an exception that most people  
> would
> expect to be re-raised) surprises and pisses people off. Guido has  
> already
> made a pronouncement so not much point continuing this discussion.

But maybe we can convince BDFL that it should be -- not SyntaxError but
-- RuntimeError?

*j


From stephen at xemacs.org  Sat Jul 18 15:20:48 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 18 Jul 2009 22:20:48 +0900
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <6f4025010907180442n39cb344fxc7b0d725bb2894c@mail.gmail.com>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
	<200907181059.58523.steve@pearwood.info>
	<6f4025010907180442n39cb344fxc7b0d725bb2894c@mail.gmail.com>
Message-ID: <87k526gma7.fsf@uwakimon.sk.tsukuba.ac.jp>

Michael Foord writes:

 > It's arbitrary - and constructs with arbitrary interpretations are generally
 > not a *good* thing at the very least. It also makes it harder to read code,
 > where a return in a try:... finally block may not do anything.

But that's true of *all* code in a try block:

    def foo():
        try:
            x = 42
        finally:
            x = "gotcha!"
        return x

or

    def bar():
        try:
            x = some_function_normally_returning_42()
            return x
        except Exception:
            return "didn't see that coming!"

 > It is much weirder. In the case of explicitly using an except you are
 > catching the error and choosing not to propagate it. In a finally where the
 > normal semantics are that the exception is re-raised the return silently
 > circumvents that.

AFAICS "finally" is a restriction of call-with-current-continuation.
Ie, the block is "spliced into" the control flow at the exit, and if
control "flows off the end" of the finally block, then we go back to
doing what we were about to do (return, propagate an exception).  But
if the block exits otherwise, then we don't.  What's "weird" about
that?

Eg, the finally block could do something that itself raises an
exception.  If so, I guess you would say that instead of dealing with
the "new" exception, we should blow it off and reraise the original
exception?






From ncoghlan at gmail.com  Sat Jul 18 15:47:57 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 18 Jul 2009 23:47:57 +1000
Subject: [Python-ideas] Augment dis.dis to autocompile strings
In-Reply-To: <h3s6in$1vv$1@ger.gmane.org>
References: <h3nrtu$oog$1@ger.gmane.org>	<200907171232.21702.steve@pearwood.info>	<4A6100A5.8000008@gmail.com>
	<h3s6in$1vv$1@ger.gmane.org>
Message-ID: <4A61D28D.10007@gmail.com>

Georg Brandl wrote:
> I don't think using "single" makes sense here; trying "eval" can't hurt,
> though.

Agreed. Only having the two modes to worry about greatly simplifies the
code as well:

from dis import dis
def dis_str(source):
  try:
    c = compile(source, '', 'eval')
  except SyntaxError:
    c = compile(source, '', 'exec')
  return dis(c)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Sat Jul 18 15:54:07 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 18 Jul 2009 23:54:07 +1000
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <79990c6b0907180541h4c714e16g61805198921e51ad@mail.gmail.com>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>	<200907181059.58523.steve@pearwood.info>	<6f4025010907180442n39cb344fxc7b0d725bb2894c@mail.gmail.com>	<19041.47850.152959.565539@jon.es>	<6f4025010907180537k313e1b2bva099281fa1413725@mail.gmail.com>
	<79990c6b0907180541h4c714e16g61805198921e51ad@mail.gmail.com>
Message-ID: <4A61D3FF.7030806@gmail.com>

Paul Moore wrote:
> While I'll concede that it's a bit subtle, the finally doesn't
> "complete" if you return or break - those two statements cause
> premature exit. As such, it's quite right that the exception isn't
> re-raised in those cases, as the finally clause never "completed".
> 
> Subtle, yes. Inconsistent, no.

This is the reason the current semantics make sense to me as well:
return, break and raise all mean that the finally block doesn't actually
finish, so the implicit "re-raise" at the end of the finally block is
never executed.

If anything was going to be made more consistent, I would suggest it
would be to allow continue in a finally statement with similar semantics
to break. I suspect there may be implementation factors that would make
that difficult though.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Sat Jul 18 16:01:32 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 19 Jul 2009 00:01:32 +1000
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <6f4025010907180438q5e46854bq7547bbb662056112@mail.gmail.com>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>	<ca471dc20907171913v6015044eh8fe1d59fce16d46e@mail.gmail.com>
	<6f4025010907180438q5e46854bq7547bbb662056112@mail.gmail.com>
Message-ID: <4A61D5BC.2080706@gmail.com>

Michael Foord wrote:
> I was surprised - I *thought* the semantics of finally were well
> defined; that exceptions inside the try block will be propagated unless
> a new exception is raised inside the finally. That return (and break it
> transpires) silently and implicitly swallow those exceptions is surprising.
> 
> It is also *odd*. If you *want* behavior then it is trivial to use an
> except instead of a finally, so I don't see a use case for it.

I think the perception of whether this behaviour is odd or not depends
on your perception of how a finally block works. If you just see it as a
normal sequence of instructions with an implicit "re-raise current
exception" at the end of the normal flow of control, then the fact that
return, break and raise can all bypass that re-raise by exiting the
block differently isn't particularly surprising (the only surprising
thing is that this doesn't also hold for continue).

However, if your mental model is that finally blocks are special
sections of code that promise to reraise the exception *even if* the
finally block exits by a means other than control flowing off the end,
then I can see why the current behaviour would be surprising.

This sounds like a documentation problem to me - the explanation of the
finally block in the tutorial and language reference should be pushing
people towards the first interpretation, since it is the one actually
used by the language.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From stephen at xemacs.org  Sat Jul 18 16:14:59 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 18 Jul 2009 23:14:59 +0900
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <6f4025010907180606ucb5df24t9bf144237e47e6c4@mail.gmail.com>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
	<4A613C57.8070208@canterbury.ac.nz>
	<87skguh8oe.fsf@uwakimon.sk.tsukuba.ac.jp>
	<6f4025010907180427ucd949ffl9f3fef77c6a97d5b@mail.gmail.com>
	<87ljmmgnht.fsf@uwakimon.sk.tsukuba.ac.jp>
	<6f4025010907180606ucb5df24t9bf144237e47e6c4@mail.gmail.com>
Message-ID: <87iqhqgjrw.fsf@uwakimon.sk.tsukuba.ac.jp>

Michael Foord writes:

 > I think the current behavior (swallowing an exception that most people would
 > expect to be re-raised) surprises and pisses people off. Guido has already
 > made a pronouncement so not much point continuing this discussion.

If you say so.  I think that there's an education and documentation
issue here, though.  You write elsewhere:

    Finally is normally dead simple but the explanation usually
    includes what it does with exceptions:

    Finally is *always* run last. Exceptions that occur in the try
    block are re-raised once the finally has completed. Exceptions
    inside the finally block will be re-raised

[Isn't that wrong?  An exception in the finally block will terminate
the finally block, no?  There's no "re-"raise about it.]

    The bit that has to be *added* in Python (and usually isn't added
    because it isn't at all obvious) is:

    Exceptions are not reraised if the finally returns or breaks.

Well, yes, except that I don't think that your explanation accurately
reflects what finally is supposed to do.  I would write:

    On *all* exits from the try block, control is first transferred to
    the finally block.  If control flows off its end, the exit
    proceeds as if there had been no finally block: values are
    returned by return, exceptions are propagated to the innermost
    containing handler, and otherwise control flows to the containing
    suite.

    An exception raised (either by called code or an explicit raise)
    in a finally block will supersede a return or exception generated
    in the try block.  Control transfer constructs (break, return) in
    a finally block also supersede return statements or exceptions
    generated in the try block.

I suppose the difference from what you wrote is a reflection of
finally's heritage as a construct completely independent of
exceptions.


From ncoghlan at gmail.com  Sat Jul 18 16:18:11 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 19 Jul 2009 00:18:11 +1000
Subject: [Python-ideas] Experiment: Adding "re" to string objects.
In-Reply-To: <4A61B027.4050106@voidspace.org.uk>
References: <20090718002836.GA26311@tummy.com>	<4A611CAA.5020108@mrabarnett.plus.com>
	<4A61B027.4050106@voidspace.org.uk>
Message-ID: <4A61D9A3.8020701@gmail.com>

<switching lists from python-dev to python-ideas>

Michael Foord wrote:
> MRAB wrote:
>> [snip]
>> Why not drop the ".re" part? You would, however, then need a new name
>> for the re split, eg "re_split".
>>
>> Or you could make the string the pattern, eg r'whatever(.*)'.match(s).
> 
> +1 for re support built-in to strings.

The idea of adding a mutable attribute to strings (even if it plays no
part in string equality) sends shivers down my spine. It also seems like
it would be difficult to do this in a way that didn't increase the size
of all strings by at least one pointer slot.

However, the idea of adding more convenience classes to the re module
may still have some merit. In particular, when checking against multiple
regexes, being able to do something like the following might be helpful:

m = re.Matcher(s)
if m.match(r'whatever(.*)'):
      print m.group(1)
elif m.match(r'something (.*) else(.*)'):
      print m.group(2)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From lists at cheimes.de  Sat Jul 18 16:30:19 2009
From: lists at cheimes.de (Christian Heimes)
Date: Sat, 18 Jul 2009 16:30:19 +0200
Subject: [Python-ideas] Experiment: Adding "re" to string objects.
In-Reply-To: <4A61D9A3.8020701@gmail.com>
References: <20090718002836.GA26311@tummy.com>	<4A611CAA.5020108@mrabarnett.plus.com>	<4A61B027.4050106@voidspace.org.uk>
	<4A61D9A3.8020701@gmail.com>
Message-ID: <h3sm9r$nub$1@ger.gmane.org>

Nick Coghlan wrote:
> The idea of adding a mutable attribute to strings (even if it plays no
> part in string equality) sends shivers down my spine. It also seems like
> it would be difficult to do this in a way that didn't increase the size
> of all strings by at least one pointer slot.

I've more arguments against the re idea:

* regular expressions are rarely used in Python. I have just a couple of
scripts that use re

* we shouldn't encourage people in using re when there is a simpler
solution. Python isn't Perl.

* Several modules and a C extension must be loaded during *every*
interpreter startup. Everybody must pay the speed penalty and the memory
usage of the interpreter increases with every module, too. Lazy loading
may be a workaround, though

* Beautiful is better than ugly.

* Explicit is better than implicit.

* Simple is better than complex.

* Readability counts.


Christian



From ncoghlan at gmail.com  Sat Jul 18 16:37:33 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 19 Jul 2009 00:37:33 +1000
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
Message-ID: <4A61DE2D.7000304@gmail.com>

Michael wrote:
> Here are two examples of why allowing return inside a finally block is a
> bad idea:
> 
> def f():
>   try:
>     return 3
>   finally:
>     return 4
> 
> def f():
>   try:
>     raise Exception()
>   finally:
>     return 4
> 
> 
> Michael Foord

I just remembered the case where I found it convenient to exploit this
behaviour for a quick and dirty script where I wanted as much of a list
that worked as I could get before one of the operations threw an
exception or I ran out of items:

def f(iter):
  s = []
  try:
    for x in iter:
      s.append(op(x))
  finally:
    return s

The fact that that would be better rewritten using a narrower except
statement (see postscript below) doesn't change the fact that it got the
job done (which was what was important at the time).

Cheers,
Nick.

P.S. The "better" approach to the above situation:

def f(iter):
  s = []
  try:
    for x in iter:
      s.append(op(x))
  except RuntimeError:
    pass
  return s

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From m.lenzen at gmail.com  Sat Jul 18 17:11:47 2009
From: m.lenzen at gmail.com (Michael Lenzen)
Date: Sat, 18 Jul 2009 10:11:47 -0500
Subject: [Python-ideas] Proposal for extending the collections module -
 bags / multisets, ordered sets / unique lists
In-Reply-To: <50697b2c0907172330k5c7dc88aj3adc8d463993708@mail.gmail.com>
References: <4A612B47.3050706@gmail.com> <4A613A4A.3010607@gmail.com>	
	<4A615622.9090308@gmail.com>	
	<50697b2c0907172318t4de4b425md996ff37a7170b01@mail.gmail.com>
	<50697b2c0907172330k5c7dc88aj3adc8d463993708@mail.gmail.com>
Message-ID: <4A61E633.4050301@gmail.com>


On 07/18/2009 01:30 AM, Chris Rebert wrote:
> On Fri, Jul 17, 2009 at 11:18 PM, Chris Rebert<pyideas at rebertia.com>  wrote:
>> Truth be told, it's more than just defaultdict(int). It adds
>> .elements() and .most_common().
>>
>> Seems bag-like to me.
>> - Unordered? Check.
>> - Allows duplicates? Check.
>> - O(1) containment test? Check.
>> - Counts multiplicity of elements? Check.
>> - Iterable? Check.
>>
>> The only non-bag thing about it is allowing 0 and negative
>> multiplicities, which I agree is unintuitive; I don't like that
>> "feature" either.
>
> Actually, from the docs, it also appears (I don't have 3.0 handy to
> test) to get len() wrong, using the dict definition of "number of
> unique items" rather than just "number of items" as would be more
> appropriate for a bag.
>
> In the event a Bag is not added, +1 for adding a method to Counter to
> return `sum(count if count>  0 else 0 for count in
> a_counter.values())`
>
> Cheers,
> Chris


As well as getting len() wrong, it gets iteration wrong.  It iterates 
over elements with counts of 0 and -1 as well as only iterating once 
over elements that appear multiple times.  Yes you can iterate over 
.elements(), but this should be the default not a special case.

As for adding most_common, it just calls
heapq.nlargest(n, self.items(), key=_itemgetter(1))
which anyone can do, and my bag class does.

My bag class behaves like a collection and provides a .unique_elements() 
method that returns the underlying set.  You can .add(elem) and 
.delete(elem) just like you can with a set, or you can manually change 
their multiplicities like in Counter with bag[elem] = 5 or bag[elem] -= 2.

If Counter is supposed to be a collection of elements, this makes no sense:
 >>> c = Counter()
 >>> c['a'] += 1
 >>> c['a'] -= 1
 >>> 'a' in c
True

-Michael Lenzen


From steve at pearwood.info  Sat Jul 18 18:36:45 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 19 Jul 2009 02:36:45 +1000
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <eae285400907180545u4d12272fwfea29f738d34ca47@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<200907181039.57256.steve@pearwood.info>
	<eae285400907180545u4d12272fwfea29f738d34ca47@mail.gmail.com>
Message-ID: <200907190236.46560.steve@pearwood.info>

On Sat, 18 Jul 2009 10:45:44 pm Daniel Stutzbach wrote:
> On Fri, Jul 17, 2009 at 7:39 PM, Steven D'Aprano 
<steve at pearwood.info>wrote:
> > In a real sense, the proposed 'where' block breaks the flow of
> > reading code. Normally, reading a function proceeds in an orderly
> > fashion from the start of the function to the end in (mostly)
> > sequential order, a bottom-up process:
>
> Normally, that's true, but that's not true when defining a
> sub-function. Consider the following:
>
> def foo(items):
>     def compute_sort_value(item):
>         # compute value based on item's properties
>         return value
>    # a bunch of other code is here
>     items.sort(compute_sort_value)
>
> Above, the body of compute_sort_value appears long before it is
> executed. 

Right. That's no different from this case:


def bar(items):
    c_s_v = list(
        # comment goes here
        'csv')
    # a bunch of other code is here
    items.extend(c_s_v)


def is an executable statement which creates a function; list() is an 
executable function (technically a type) which creates an list.

In your example, the inner function compute_sort_value is created when 
foo is executed. Python functions are first class objects, and so you 
can create functions and pass them to functions without necessarily 
calling the __call__ method.

In my example, the inner list c_s_v is created when bar is executed. 
Lists are first class objects, and so you can create lists and pass 
them to functions without necessarily calling the __getitem__ method.

In both functions, the objects compute_sort_value and c_s_v are created 
before they are used.


> Now consider: 
>
> def foo(items):
>     # a bunch of other code is here
>     items.sort(key=mean) where:
>         def compute_sort_value(item):
>             return value

Presumably you meant to write:

    items.sort(key=compute_sort_value) where:


> Now, the body of compute_sort_value appears exactly where it is
> executed: within the call to items.sort().

No it doesn't. The function definition appears outside the call to 
sort(). For the function definition to appear inside the call to sort, 
you'd need to write something like:

    items.sort(key=lambda item: value)

In this case, the lambda only occurs inside the call to sort() -- it's 
inaccessible to anything else (excluding trickery inside of sort() 
itself). By comparison, the compute_sort_value() function exists inside 
the where-block, and therefore is accessible to everything else inside 
that block:

    items.sort(key=compute_sort_value) where:
        def compute_sort_value(item):
            return value
        print compute_sort_value('testing testing 1 2 3')





-- 
Steven D'Aprano


From mrts.pydev at gmail.com  Sat Jul 18 20:03:47 2009
From: mrts.pydev at gmail.com (=?ISO-8859-1?Q?Mart_S=F5mermaa?=)
Date: Sat, 18 Jul 2009 21:03:47 +0300
Subject: [Python-ideas] Proposal for extending the collections module -
	bags / multisets, ordered sets / unique lists
In-Reply-To: <4A61E633.4050301@gmail.com>
References: <4A612B47.3050706@gmail.com> <4A613A4A.3010607@gmail.com>
	<4A615622.9090308@gmail.com>
	<50697b2c0907172318t4de4b425md996ff37a7170b01@mail.gmail.com>
	<50697b2c0907172330k5c7dc88aj3adc8d463993708@mail.gmail.com>
	<4A61E633.4050301@gmail.com>
Message-ID: <ad1f81530907181103i3e9c610fx5ca5e17dbf9bd5b@mail.gmail.com>

I drove a similar discussion a few months ago (April or May), nobody
was opposing then, so I'd say go ahead and file a feature request.

Of course, I'm personally strongly +1, especially for ordered set/unique list.

Various implementations:

http://code.activestate.com/recipes/576694/
http://www.peterbe.com/plog/uniqifiers-benchmark/

A list with no duplicate values is a common problem judging on the following:

http://www.google.com/codesearch?q=lang%3Apython+list+unique
http://www.google.com/codesearch?q=lang%3Apython+list+duplicates
http://code.activestate.com/recipes/52560/
http://www.google.com/search?q=python+unique+list


From m.lenzen at gmail.com  Sat Jul 18 20:35:09 2009
From: m.lenzen at gmail.com (Michael Lenzen)
Date: Sat, 18 Jul 2009 13:35:09 -0500
Subject: [Python-ideas] Proposal for extending the collections module -
 bags / multisets, ordered sets / unique lists
In-Reply-To: <ad1f81530907181103i3e9c610fx5ca5e17dbf9bd5b@mail.gmail.com>
References: <4A612B47.3050706@gmail.com>
	<4A613A4A.3010607@gmail.com>	<4A615622.9090308@gmail.com>	<50697b2c0907172318t4de4b425md996ff37a7170b01@mail.gmail.com>	<50697b2c0907172330k5c7dc88aj3adc8d463993708@mail.gmail.com>	<4A61E633.4050301@gmail.com>
	<ad1f81530907181103i3e9c610fx5ca5e17dbf9bd5b@mail.gmail.com>
Message-ID: <4A6215DD.9060105@gmail.com>

I was thinking of submitting a PEP, but I wanted to get feedback first 
and finish up my reference implementation.

-Michael Lenzen
http://code.google.com/p/python-data-structures/wiki/collections

On 07/18/2009 01:03 PM, Mart S?mermaa wrote:
> I drove a similar discussion a few months ago (April or May), nobody
> was opposing then, so I'd say go ahead and file a feature request.
>
> Of course, I'm personally strongly +1, especially for ordered set/unique list.
>
> Various implementations:
>
> http://code.activestate.com/recipes/576694/
> http://www.peterbe.com/plog/uniqifiers-benchmark/
>
> A list with no duplicate values is a common problem judging on the following:
>
> http://www.google.com/codesearch?q=lang%3Apython+list+unique
> http://www.google.com/codesearch?q=lang%3Apython+list+duplicates
> http://code.activestate.com/recipes/52560/
> http://www.google.com/search?q=python+unique+list
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas


From pyideas at rebertia.com  Sat Jul 18 20:56:30 2009
From: pyideas at rebertia.com (Chris Rebert)
Date: Sat, 18 Jul 2009 11:56:30 -0700
Subject: [Python-ideas] Proposal for extending the collections module -
	bags / multisets, ordered sets / unique lists
In-Reply-To: <4A6215DD.9060105@gmail.com>
References: <4A612B47.3050706@gmail.com> <4A613A4A.3010607@gmail.com>
	<4A615622.9090308@gmail.com>
	<50697b2c0907172318t4de4b425md996ff37a7170b01@mail.gmail.com>
	<50697b2c0907172330k5c7dc88aj3adc8d463993708@mail.gmail.com>
	<4A61E633.4050301@gmail.com>
	<ad1f81530907181103i3e9c610fx5ca5e17dbf9bd5b@mail.gmail.com>
	<4A6215DD.9060105@gmail.com>
Message-ID: <50697b2c0907181156r762dbb36s180c788b29eb851a@mail.gmail.com>

> On 07/18/2009 01:03 PM, Mart S?mermaa wrote:
>>
>> I drove a similar discussion a few months ago (April or May), nobody
>> was opposing then, so I'd say go ahead and file a feature request.
>>
>> Of course, I'm personally strongly +1, especially for ordered set/unique
>> list.
>>
>> Various implementations:
>>
>> http://code.activestate.com/recipes/576694/
>> http://www.peterbe.com/plog/uniqifiers-benchmark/
>>
>> A list with no duplicate values is a common problem judging on the
>> following:
>>
>> http://www.google.com/codesearch?q=lang%3Apython+list+unique
>> http://www.google.com/codesearch?q=lang%3Apython+list+duplicates
>> http://code.activestate.com/recipes/52560/
>> http://www.google.com/search?q=python+unique+list

On Sat, Jul 18, 2009 at 11:35 AM, Michael Lenzen<m.lenzen at gmail.com> wrote:
> I was thinking of submitting a PEP, but I wanted to get feedback first and
> finish up my reference implementation.

A PEP is probably not be necessary just to add a class to the std lib.
Counter was added without a PEP after all. You would need to convince
the committers though.

Cheers,
Chris
-- 
http://blog.rebertia.com


From m.lenzen at gmail.com  Sat Jul 18 21:07:39 2009
From: m.lenzen at gmail.com (Michael Lenzen)
Date: Sat, 18 Jul 2009 14:07:39 -0500
Subject: [Python-ideas] Proposal for extending the collections module -
 bags / multisets, ordered sets / unique lists
In-Reply-To: <50697b2c0907181156r762dbb36s180c788b29eb851a@mail.gmail.com>
References: <4A612B47.3050706@gmail.com> <4A613A4A.3010607@gmail.com>	
	<4A615622.9090308@gmail.com>	
	<50697b2c0907172318t4de4b425md996ff37a7170b01@mail.gmail.com>	
	<50697b2c0907172330k5c7dc88aj3adc8d463993708@mail.gmail.com>	
	<4A61E633.4050301@gmail.com>	
	<ad1f81530907181103i3e9c610fx5ca5e17dbf9bd5b@mail.gmail.com>	
	<4A6215DD.9060105@gmail.com>
	<50697b2c0907181156r762dbb36s180c788b29eb851a@mail.gmail.com>
Message-ID: <4A621D7B.60900@gmail.com>

My full proposition includes 2 new classes, 2 new abstract base classes, 
modifying set and a new `collection` function to abstract away the new 
multitude of classes.

The collection function is my favorite part, you just pass it parameters 
about the collection you want (mutable, ordered and/or unique) and it 
returns the collection that fits without you ever having to figure it out.

-Michael Lenzen

>
> On Sat, Jul 18, 2009 at 11:35 AM, Michael Lenzen<m.lenzen at gmail.com>  wrote:
>> I was thinking of submitting a PEP, but I wanted to get feedback first and
>> finish up my reference implementation.
>
> A PEP is probably not be necessary just to add a class to the std lib.
> Counter was added without a PEP after all. You would need to convince
> the committers though.
>
> Cheers,
> Chris


From p.f.moore at gmail.com  Sat Jul 18 21:28:19 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 18 Jul 2009 20:28:19 +0100
Subject: [Python-ideas] Proposal for extending the collections module -
	bags / multisets, ordered sets / unique lists
In-Reply-To: <50697b2c0907181156r762dbb36s180c788b29eb851a@mail.gmail.com>
References: <4A612B47.3050706@gmail.com> <4A613A4A.3010607@gmail.com>
	<4A615622.9090308@gmail.com>
	<50697b2c0907172318t4de4b425md996ff37a7170b01@mail.gmail.com>
	<50697b2c0907172330k5c7dc88aj3adc8d463993708@mail.gmail.com>
	<4A61E633.4050301@gmail.com>
	<ad1f81530907181103i3e9c610fx5ca5e17dbf9bd5b@mail.gmail.com>
	<4A6215DD.9060105@gmail.com>
	<50697b2c0907181156r762dbb36s180c788b29eb851a@mail.gmail.com>
Message-ID: <79990c6b0907181228p4f641d94jdbf80ba9b2267d1@mail.gmail.com>

2009/7/18 Chris Rebert <pyideas at rebertia.com>:
> A PEP is probably not be necessary just to add a class to the std lib.
> Counter was added without a PEP after all. You would need to convince
> the committers though.

For something like this, you could release your code as a separate
module. Then let it get some popular support before asking for it to
be promoted to the standard library. (Of course the proposed set
changes might not be possible this way, but they are the most likely
aspect to get rejected, because of backward compatibility concerns, in
any case).

Paul.


From m.lenzen at gmail.com  Sat Jul 18 22:00:19 2009
From: m.lenzen at gmail.com (Michael Lenzen)
Date: Sat, 18 Jul 2009 15:00:19 -0500
Subject: [Python-ideas] Proposal for extending the collections module -
 bags / multisets, ordered sets / unique lists
In-Reply-To: <79990c6b0907181228p4f641d94jdbf80ba9b2267d1@mail.gmail.com>
References: <4A612B47.3050706@gmail.com> <4A613A4A.3010607@gmail.com>	
	<4A615622.9090308@gmail.com>	
	<50697b2c0907172318t4de4b425md996ff37a7170b01@mail.gmail.com>	
	<50697b2c0907172330k5c7dc88aj3adc8d463993708@mail.gmail.com>	
	<4A61E633.4050301@gmail.com>	
	<ad1f81530907181103i3e9c610fx5ca5e17dbf9bd5b@mail.gmail.com>	
	<4A6215DD.9060105@gmail.com>	
	<50697b2c0907181156r762dbb36s180c788b29eb851a@mail.gmail.com>
	<79990c6b0907181228p4f641d94jdbf80ba9b2267d1@mail.gmail.com>
Message-ID: <4A6229D3.3040008@gmail.com>

On 07/18/2009 02:28 PM, Paul Moore wrote:
> 2009/7/18 Chris Rebert<pyideas at rebertia.com>:
>> A PEP is probably not be necessary just to add a class to the std lib.
>> Counter was added without a PEP after all. You would need to convince
>> the committers though.
>
> For something like this, you could release your code as a separate
> module. Then let it get some popular support before asking for it to
> be promoted to the standard library. (Of course the proposed set
> changes might not be possible this way, but they are the most likely
> aspect to get rejected, because of backward compatibility concerns, in
> any case).
>
> Paul.

My module (collections_) is a drop in replacement for collections, it 
imports everything from there, then adds it to __all__ so that you can 
import it from collections_.  The only caveat is that you have to import 
collections_.set and collections_.frozenset to override the default 
implementations, but it works.

-Michael Lenzen


From gerald.britton at gmail.com  Sat Jul 18 22:11:57 2009
From: gerald.britton at gmail.com (Gerald Britton)
Date: Sat, 18 Jul 2009 16:11:57 -0400
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <200907190236.46560.steve@pearwood.info>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<200907181039.57256.steve@pearwood.info>
	<eae285400907180545u4d12272fwfea29f738d34ca47@mail.gmail.com> 
	<200907190236.46560.steve@pearwood.info>
Message-ID: <5d1a32000907181311ucbfabefk162d54f34cec32da@mail.gmail.com>

>
> ? ?items.sort(key=compute_sort_value) where:
>
>
>> Now, the body of compute_sort_value appears exactly where it is
>> executed: within the call to items.sort().
>
> No it doesn't. The function definition appears outside the call to
> sort(). For the function definition to appear inside the call to sort,
> you'd need to write something like

Yes, it does appear within the call.  The call is more than just the
parameters in parentheses.  It begins with "items.sort" and ends (in
this case) with then end of the "where" clause

>
> ? ?items.sort(key=lambda item: value)

The idea behind "where" clauses is not to replace lambdas.  In fact,
they are useful together:

     items.sort(key=mysort) where:
         mysort =  sort_key_function_factory(
                          with, many, arguments)

which to many is preferable over:

    items.sort(key=lambda item:sort_key_function_factory(with, many, arguments))

Even though you can split it:

    items.sort(key=lambda item:
                     sort_key_function_factory(with, many, arguments)  )


>
> In this case, the lambda only occurs inside the call to sort() -- it's
> inaccessible to anything else (excluding trickery inside of sort()
> itself). By comparison, the compute_sort_value() function exists inside
> the where-block, and therefore is accessible to everything else inside
> that block:
>
> ? ?items.sort(key=compute_sort_value) where:
> ? ? ? ?def compute_sort_value(item):
> ? ? ? ? ? ?return value
> ? ? ? ?print compute_sort_value('testing testing 1 2 3')

That would be useful for debugging.

Really though, the whole thing comes down to a matter of taste and
visual presentation.  Some will find "where" completely intuitive and
a natural way to work.  Others will loathe it (though as has been
pointed out, it really makes using Haskell functions easier to write
and to read.)

On the whole though, it's a +1 for me


From santagada at gmail.com  Sat Jul 18 22:19:16 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Sat, 18 Jul 2009 17:19:16 -0300
Subject: [Python-ideas] Proposal for extending the collections module -
	bags / multisets, ordered sets / unique lists
In-Reply-To: <4A621D7B.60900@gmail.com>
References: <4A612B47.3050706@gmail.com> <4A613A4A.3010607@gmail.com>	
	<4A615622.9090308@gmail.com>	
	<50697b2c0907172318t4de4b425md996ff37a7170b01@mail.gmail.com>	
	<50697b2c0907172330k5c7dc88aj3adc8d463993708@mail.gmail.com>	
	<4A61E633.4050301@gmail.com>	
	<ad1f81530907181103i3e9c610fx5ca5e17dbf9bd5b@mail.gmail.com>	
	<4A6215DD.9060105@gmail.com>
	<50697b2c0907181156r762dbb36s180c788b29eb851a@mail.gmail.com>
	<4A621D7B.60900@gmail.com>
Message-ID: <8B867CFB-8F7B-4410-9654-E523607AB7FD@gmail.com>


On Jul 18, 2009, at 4:07 PM, Michael Lenzen wrote:

> My full proposition includes 2 new classes, 2 new abstract base  
> classes, modifying set and a new `collection` function to abstract  
> away the new multitude of classes.
>
> The collection function is my favorite part, you just pass it  
> parameters about the collection you want (mutable, ordered and/or  
> unique) and it returns the collection that fits without you ever  
> having to figure it out.


I was silent on this because I wanted someone that has a say in this  
stuff to stop it but as you really think this is important I have to  
point out that this function is probably unaceptable, it makes code  
that much harder to read (instead of set or list I have to see each  
parameter and form on my mind what they map to) and also GvR already  
said before that he doesn't like functions that have flags that make  
them do completely different things based on those.

Multiset seems interesting, some of the ABCs you mentioned seems to be  
just over engineering (maybe they just need a better justification)  
but this function in my view is unacceptable.

--
Leonardo Santagada
santagada at gmail.com





From pyideas at rebertia.com  Sat Jul 18 22:32:44 2009
From: pyideas at rebertia.com (Chris Rebert)
Date: Sat, 18 Jul 2009 13:32:44 -0700
Subject: [Python-ideas] Proposal for extending the collections module -
	bags / multisets, ordered sets / unique lists
In-Reply-To: <4A612B47.3050706@gmail.com>
References: <4A612B47.3050706@gmail.com>
Message-ID: <50697b2c0907181332y1c10a95fl94dd394129a47bbc@mail.gmail.com>

On Fri, Jul 17, 2009 at 6:54 PM, Michael Lenzen<m.lenzen at gmail.com> wrote:
<snip>
> New ABCs (abstract base classes)
> --------------------------------
<snip>
> Mutable - I think there should be one metaclass for all mutable types just
> like there is currently Iterable, Sized and Container. We could then check
> instanceof(obj, Mutable). MutableSequence, MutableSet and MutableMapping
> would inherit Mutable, or we might be able to just do away with them
> altogether. ?There are 3 abstractmethods defined:
> ?* pop(self) - This is the simplest to implement, all mutable classes
> already do.
> ?* __setitem__(self, key, value) - This is already implemented for list and
> dict but would have to be defined for set, more later
> ?* __delitem__(self, key) Same as above.

I think this would be better named MutableCollection personally,
otherwise it's liable to be confused with the more general mutable vs.
immutable types distinction.

<snip>
> Extending set to fit the ABCs
> -----------------------------
> I mentioned before that sets don't currently fit into my model. Here are my
> propositions for the three methods that would need to be implemented.
> ?* set.__setitem__(self, elem, value) - Set whether or not elem is in the
> set based on the boolean evaluation of value. s[elem] = 1 would call
> s.add(elem) and s[elem] = False would call s.remove(elem), appropriately
> raising a KeyError if elem not in s. The reason I chose this way is that the
> only thing you can set about an element of a set is whether or not it is
> present.

The implicit bool(), which I assume is what you mean by "boolean
evaluation", here concerns me, as it could mask bugs where something
tries to treat a set like a mapping. I would recommend throwing
TypeError if given a non-int and ValueError if given any integer
besides 0 or 1, so as to more strongly enforce the set axioms.

Cheers,
Chris
-- 
http://blog.rebertia.com


From grosser.meister.morti at gmx.net  Sat Jul 18 22:45:46 2009
From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=)
Date: Sat, 18 Jul 2009 22:45:46 +0200
Subject: [Python-ideas] Proposal for extending the collections module
 -	bags / multisets, ordered sets / unique lists
In-Reply-To: <8B867CFB-8F7B-4410-9654-E523607AB7FD@gmail.com>
References: <4A612B47.3050706@gmail.com>
	<4A613A4A.3010607@gmail.com>		<4A615622.9090308@gmail.com>		<50697b2c0907172318t4de4b425md996ff37a7170b01@mail.gmail.com>		<50697b2c0907172330k5c7dc88aj3adc8d463993708@mail.gmail.com>		<4A61E633.4050301@gmail.com>		<ad1f81530907181103i3e9c610fx5ca5e17dbf9bd5b@mail.gmail.com>		<4A6215DD.9060105@gmail.com>	<50697b2c0907181156r762dbb36s180c788b29eb851a@mail.gmail.com>	<4A621D7B.60900@gmail.com>
	<8B867CFB-8F7B-4410-9654-E523607AB7FD@gmail.com>
Message-ID: <4A62347A.8080500@gmx.net>

On 07/18/2009 10:19 PM, Leonardo Santagada wrote:
> 
> On Jul 18, 2009, at 4:07 PM, Michael Lenzen wrote:
> 
>> My full proposition includes 2 new classes, 2 new abstract base
>> classes, modifying set and a new `collection` function to abstract
>> away the new multitude of classes.
>>
>> The collection function is my favorite part, you just pass it
>> parameters about the collection you want (mutable, ordered and/or
>> unique) and it returns the collection that fits without you ever
>> having to figure it out.
> 
> 
> I was silent on this because I wanted someone that has a say in this
> stuff to stop it but as you really think this is important I have to
> point out that this function is probably unaceptable, it makes code that
> much harder to read (instead of set or list I have to see each parameter
> and form on my mind what they map to) and also GvR already said before
> that he doesn't like functions that have flags that make them do
> completely different things based on those.
> 
> Multiset seems interesting, some of the ABCs you mentioned seems to be
> just over engineering (maybe they just need a better justification) but
> this function in my view is unacceptable.
> 

I don't understand what's so bad about such a collection factory. As a
programmer I don't care what class an object is. I just want an object that
behaves a certain way. E.g. I want to say: I need a random accessible
collection, or I need a collection with unique entries, or I need a collection
with a fast (near to O(1)) way to append elements, or all of the above. What
class this object will have I don't care.

Where is the problem?

	-panzi


From m.lenzen at gmail.com  Sat Jul 18 22:54:39 2009
From: m.lenzen at gmail.com (Michael Lenzen)
Date: Sat, 18 Jul 2009 15:54:39 -0500
Subject: [Python-ideas] Proposal for extending the collections module -
 bags / multisets, ordered sets / unique lists
In-Reply-To: <8B867CFB-8F7B-4410-9654-E523607AB7FD@gmail.com>
References: <4A612B47.3050706@gmail.com> <4A613A4A.3010607@gmail.com>	
	<4A615622.9090308@gmail.com>	
	<50697b2c0907172318t4de4b425md996ff37a7170b01@mail.gmail.com>	
	<50697b2c0907172330k5c7dc88aj3adc8d463993708@mail.gmail.com>	
	<4A61E633.4050301@gmail.com>	
	<ad1f81530907181103i3e9c610fx5ca5e17dbf9bd5b@mail.gmail.com>	
	<4A6215DD.9060105@gmail.com>
	<50697b2c0907181156r762dbb36s180c788b29eb851a@mail.gmail.com>
	<4A621D7B.60900@gmail.com>
	<8B867CFB-8F7B-4410-9654-E523607AB7FD@gmail.com>
Message-ID: <4A62368F.8050602@gmail.com>

On 07/18/2009 03:19 PM, Leonardo Santagada wrote:
>
> I was silent on this because I wanted someone that has a say in this
> stuff to stop it but as you really think this is important I have to
> point out that this function is probably unaceptable, it makes code that
> much harder to read (instead of set or list I have to see each parameter
> and form on my mind what they map to) and also GvR already said before
> that he doesn't like functions that have flags that make them do
> completely different things based on those.
>
> Multiset seems interesting, some of the ABCs you mentioned seems to be
> just over engineering (maybe they just need a better justification) but
> this function in my view is unacceptable.
>
> --
> Leonardo Santagada
> santagada at gmail.com
>
>
>

Thanks, this is why I posted here before I proposed anything formally, I 
wasn't aware of the precedence.  Maybe I didn't explain it clearly 
enough, or maybe I'm just wrong.

The Collection ABC was mostly to unify operations of the different 
collections for the collection function, so that a generic collection 
returned by the collection function meant something, you wouldn't have 
to look back to see how it will perform.  For example all collections 
would have to provide a count(elem) method, or all mutable collections 
could have to provide add and remove methods.

I don't understand how this code becomes unreadable.

  > hand = collection(ordered=False, unique=False)
  > for i in range(5):
  >     hand.add(deck.pop_card())
  > for card in cards:
  >     if hand.count(card) == 2:
  >         print('You have a pair.')

-Michael Lenzen


From m.lenzen at gmail.com  Sat Jul 18 23:23:33 2009
From: m.lenzen at gmail.com (Michael Lenzen)
Date: Sat, 18 Jul 2009 16:23:33 -0500
Subject: [Python-ideas] Proposal for extending the collections module -
 bags / multisets, ordered sets / unique lists
In-Reply-To: <50697b2c0907181332y1c10a95fl94dd394129a47bbc@mail.gmail.com>
References: <4A612B47.3050706@gmail.com>
	<50697b2c0907181332y1c10a95fl94dd394129a47bbc@mail.gmail.com>
Message-ID: <4A623D55.9030902@gmail.com>

On 07/18/2009 03:32 PM, Chris Rebert wrote:
> On Fri, Jul 17, 2009 at 6:54 PM, Michael Lenzen<m.lenzen at gmail.com>  wrote:
> <snip>
>> New ABCs (abstract base classes)
>> --------------------------------
> <snip>
>> Mutable - I think there should be one metaclass for all mutable types just
>> like there is currently Iterable, Sized and Container. We could then check
>> instanceof(obj, Mutable). MutableSequence, MutableSet and MutableMapping
>> would inherit Mutable, or we might be able to just do away with them
>> altogether.  There are 3 abstractmethods defined:
>>   * pop(self) - This is the simplest to implement, all mutable classes
>> already do.
>>   * __setitem__(self, key, value) - This is already implemented for list and
>> dict but would have to be defined for set, more later
>>   * __delitem__(self, key) Same as above.
>
> I think this would be better named MutableCollection personally,
> otherwise it's liable to be confused with the more general mutable vs.
> immutable types distinction.
>
> <snip>
>> Extending set to fit the ABCs
>> -----------------------------
>> I mentioned before that sets don't currently fit into my model. Here are my
>> propositions for the three methods that would need to be implemented.
>>   * set.__setitem__(self, elem, value) - Set whether or not elem is in the
>> set based on the boolean evaluation of value. s[elem] = 1 would call
>> s.add(elem) and s[elem] = False would call s.remove(elem), appropriately
>> raising a KeyError if elem not in s. The reason I chose this way is that the
>> only thing you can set about an element of a set is whether or not it is
>> present.
>
> The implicit bool(), which I assume is what you mean by "boolean
> evaluation", here concerns me, as it could mask bugs where something
> tries to treat a set like a mapping. I would recommend throwing
> TypeError if given a non-int and ValueError if given any integer
> besides 0 or 1, so as to more strongly enforce the set axioms.
>
> Cheers,
> Chris

Sounds good to me.  I changed my implementation and the description on 
the wiki.  I also changed bags to enforce the int restriction.

-Lenzen


From rylesny at gmail.com  Sun Jul 19 01:18:21 2009
From: rylesny at gmail.com (ryles)
Date: Sat, 18 Jul 2009 16:18:21 -0700 (PDT)
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
Message-ID: <1dc5053f-b23d-4e1d-9dd3-8943493fdd4c@p36g2000vbn.googlegroups.com>

On Jul 17, 8:18 pm, MRAB <pyt... at mrabarnett.plus.com> wrote:
> Isn't Python for consenting adults, or something like that? :-)

I agree. And the language reference does shed some light on this:

http://docs.python.org/reference/compound_stmts.html#the-try-statement

There are a lot of other things which are bad ideas as well. If you
personally don't want to permit return inside finally blocks, tools
like pylint or pychecker can be made to verify this.


From rylesny at gmail.com  Sun Jul 19 01:10:46 2009
From: rylesny at gmail.com (ryles)
Date: Sat, 18 Jul 2009 16:10:46 -0700 (PDT)
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <4A6114DC.9090207@mrabarnett.plus.com>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com> 
	<4A6114DC.9090207@mrabarnett.plus.com>
Message-ID: <1d163489-703f-4c39-a0a5-75319c74af37@g23g2000vbr.googlegroups.com>

On Jul 17, 8:18?pm, MRAB <pyt... at mrabarnett.plus.com> wrote:
>
> Isn't Python for consenting adults, or something like that? :-)

I agree. And the language reference does shed some light on this
behavior:

http://docs.python.org/reference/compound_stmts.html#the-try-statement

If you don't want return used inside finally blocks then tools like
pylint or pychecker can be made to detect this.


From greg.ewing at canterbury.ac.nz  Sun Jul 19 02:06:15 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 19 Jul 2009 12:06:15 +1200
Subject: [Python-ideas] Where-statement (Proposal
	for	function	expressions)
In-Reply-To: <8763dqd1fm.fsf@benfinney.id.au>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<200907180024.31209.steve@pearwood.info>
	<5d1a32000907171328y6da57c7j30b5bfe154e90f32@mail.gmail.com>
	<200907181039.57256.steve@pearwood.info>
	<3bdda690907172019hb70bda5h51d76ef31058423a@mail.gmail.com>
	<4A614D9B.7060500@canterbury.ac.nz> <8763dqd1fm.fsf@benfinney.id.au>
Message-ID: <4A626377.8050001@canterbury.ac.nz>

Ben Finney wrote:

> That pretty much spells its doom, then. The BDFL is quite firm (as
> demonstrated in, e.g., the pronouncement against multi-line lambda
> <URL:http://www.python.org/dev/peps/pep-3099/>) that multi-statement
> expressions are verboten.

But it's not a multi-statement expression. It's
a statement with a suite, and it works entirely
within the existing framework of the grammar.

-- 
Greg


From guido at python.org  Sun Jul 19 03:14:36 2009
From: guido at python.org (Guido van Rossum)
Date: Sat, 18 Jul 2009 18:14:36 -0700
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <4A626377.8050001@canterbury.ac.nz>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<200907180024.31209.steve@pearwood.info>
	<5d1a32000907171328y6da57c7j30b5bfe154e90f32@mail.gmail.com> 
	<200907181039.57256.steve@pearwood.info>
	<3bdda690907172019hb70bda5h51d76ef31058423a@mail.gmail.com> 
	<4A614D9B.7060500@canterbury.ac.nz> <8763dqd1fm.fsf@benfinney.id.au> 
	<4A626377.8050001@canterbury.ac.nz>
Message-ID: <ca471dc20907181814q4b76d527qc1cad3410f4df8d1@mail.gmail.com>

On Sat, Jul 18, 2009 at 5:06 PM, Greg Ewing<greg.ewing at canterbury.ac.nz> wrote:
> Ben Finney wrote:
>> That pretty much spells its doom, then. The BDFL is quite firm (as
>> demonstrated in, e.g., the pronouncement against multi-line lambda
>> <URL:http://www.python.org/dev/peps/pep-3099/>) that multi-statement
>> expressions are verboten.
>
> But it's not a multi-statement expression. It's
> a statement with a suite, and it works entirely
> within the existing framework of the grammar.

The 'where' proposal actually looks nice to me. (Maybe I've been
studying functional languages too much lately :-). My biggest problem
with this concrete proposal is that 'where' means something completely
different in SQL (which I've also studied too much lately :-).

However, I think we should focus on keeping the language stable rather
than keep tinkering with it. Let's help 3rd party developers port
their work to 3.1 rather than planning 3.1's obsolescence.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)


From steve at pearwood.info  Sun Jul 19 03:24:38 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 19 Jul 2009 11:24:38 +1000
Subject: [Python-ideas]
	=?iso-8859-1?q?Proposal_for_extending_the_collecti?=
	=?iso-8859-1?q?ons_module_-=09bags_/_multisets=2C_ordered_sets_/_u?=
	=?iso-8859-1?q?nique_lists?=
In-Reply-To: <4A62347A.8080500@gmx.net>
References: <4A612B47.3050706@gmail.com>
	<8B867CFB-8F7B-4410-9654-E523607AB7FD@gmail.com>
	<4A62347A.8080500@gmx.net>
Message-ID: <200907191124.39534.steve@pearwood.info>

On Sun, 19 Jul 2009 06:45:46 am Mathias Panzenb?ck wrote:
> I don't understand what's so bad about such a collection factory. As
> a programmer I don't care what class an object is. I just want an
> object that behaves a certain way. E.g. I want to say: I need a
> random accessible collection, or I need a collection with unique
> entries, or I need a collection with a fast (near to O(1)) way to
> append elements, or all of the above. What class this object will
> have I don't care.
>
> Where is the problem?


The problem is that you *do* need to care what the class is, because 
different classes have different interfaces:


>>> def selector(code):
...     return {0: list, 1: tuple, 2: set}[code]
...
>>> k = selector(0)
>>> instance1 = k([1, 2, 3, 4])
>>> k = selector(2)
>>> instance2 = k([1, 2, 3, 4])

So far so good. But now:

>>> instance1.append(5)
>>> instance2.append(5)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'set' object has no attribute 'append'


Unless you can guarantee identical interfaces, you can't use the 
different collection classes as drop-in replacements for each other. 
And you can't have identical interfaces once you have mutable and 
immutable types, so the whole idea is still-born.




-- 
Steven D'Aprano


From steve at pearwood.info  Sun Jul 19 03:28:37 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 19 Jul 2009 11:28:37 +1000
Subject: [Python-ideas] Proposal for extending the collections module -
	bags / multisets, ordered sets / unique lists
In-Reply-To: <4A62368F.8050602@gmail.com>
References: <4A612B47.3050706@gmail.com>
	<8B867CFB-8F7B-4410-9654-E523607AB7FD@gmail.com>
	<4A62368F.8050602@gmail.com>
Message-ID: <200907191128.37436.steve@pearwood.info>

On Sun, 19 Jul 2009 06:54:39 am Michael Lenzen wrote:

> I don't understand how this code becomes unreadable.
>
>   > hand = collection(ordered=False, unique=False)
>   > for i in range(5):
>   >     hand.add(deck.pop_card())

Do you expect that code to work if the caller had used "mutable=False" 
in the call to collection()? 


>   > for card in cards:
>   >     if hand.count(card) == 2:
>   >         print('You have a pair.')

Why would a class with unique=True have a count() method? 



-- 
Steven D'Aprano


From greg.ewing at canterbury.ac.nz  Sun Jul 19 03:34:53 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 19 Jul 2009 13:34:53 +1200
Subject: [Python-ideas] Make return inside a finally a SyntaxError
In-Reply-To: <6f4025010907180537k313e1b2bva099281fa1413725@mail.gmail.com>
References: <5A21A18A-BD6B-41C5-9181-F39A41EF672A@gmail.com>
	<200907181059.58523.steve@pearwood.info>
	<6f4025010907180442n39cb344fxc7b0d725bb2894c@mail.gmail.com>
	<19041.47850.152959.565539@jon.es>
	<6f4025010907180537k313e1b2bva099281fa1413725@mail.gmail.com>
Message-ID: <4A62783D.2080508@canterbury.ac.nz>

Michael Foord wrote:
> 
> Exceptions that occur in the try block are 
> re-raised once the finally has completed.

That's correct if you understand "completed" to mean
"by falling off the end".

A doc enhancement is perhaps in order to make that
clearer.

-- 
Greg


From grosser.meister.morti at gmx.net  Sun Jul 19 03:44:00 2009
From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=)
Date: Sun, 19 Jul 2009 03:44:00 +0200
Subject: [Python-ideas] Proposal for extending the collections module -
 bags / multisets, ordered sets / unique lists
In-Reply-To: <200907191124.39534.steve@pearwood.info>
References: <4A612B47.3050706@gmail.com>	<8B867CFB-8F7B-4410-9654-E523607AB7FD@gmail.com>	<4A62347A.8080500@gmx.net>
	<200907191124.39534.steve@pearwood.info>
Message-ID: <4A627A60.6040306@gmx.net>

On 07/19/2009 03:24 AM, Steven D'Aprano wrote:
> On Sun, 19 Jul 2009 06:45:46 am Mathias Panzenb?ck wrote:
>> I don't understand what's so bad about such a collection factory. As
>> a programmer I don't care what class an object is. I just want an
>> object that behaves a certain way. E.g. I want to say: I need a
>> random accessible collection, or I need a collection with unique
>> entries, or I need a collection with a fast (near to O(1)) way to
>> append elements, or all of the above. What class this object will
>> have I don't care.
>>
>> Where is the problem?
>
>
> The problem is that you *do* need to care what the class is, because
> different classes have different interfaces:
>
>
>>>> def selector(code):
> ...     return {0: list, 1: tuple, 2: set}[code]
> ...
>>>> k = selector(0)
>>>> instance1 = k([1, 2, 3, 4])
>>>> k = selector(2)
>>>> instance2 = k([1, 2, 3, 4])
>
> So far so good. But now:
>
>>>> instance1.append(5)
>>>> instance2.append(5)
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> AttributeError: 'set' object has no attribute 'append'
>
>
> Unless you can guarantee identical interfaces, you can't use the
> different collection classes as drop-in replacements for each other.
> And you can't have identical interfaces once you have mutable and
> immutable types, so the whole idea is still-born.

True. Ok, I didn't realize that the possible returned classes where that
different. However, I always thought it's odd that pythons list has append but
the set class has the method add. Whats the reason for that anyway?

What I thought of was a function that would return (in Java terms) instances of:
ArrayList, LinkedList, HashSet, TreeSet, LinkedHashSet, ConcurrentSkipListSet,
CopyOnWriteArraySet, CopyOnWriteArrayList, Vector etc.
(In Java terms: All cast to the super type Collection.)

So I don't have to study the class hierarchy and documentation on what class to
use best (and make apropriate changes when there is a new, better fitting class
introduced). The factory will choose the best fitting class for me.

However, I don't think there are enough classes one could choose from in the
python standard library for this to be appropriate.

	-panzi


From python at rcn.com  Sun Jul 19 03:45:01 2009
From: python at rcn.com (Raymond Hettinger)
Date: Sat, 18 Jul 2009 18:45:01 -0700
Subject: [Python-ideas] Proposal for extending the collections module -
	bags / multisets, ordered sets / unique lists
In-Reply-To: <4A612B47.3050706@gmail.com>
References: <4A612B47.3050706@gmail.com>
Message-ID: <003D2989-9032-4C81-A97F-B1DC2CCB9934@rcn.com>


On Jul 17, 2009, at 6:54 PM, Michael Lenzen wrote:

> In a nutshell, I want to add 2 new classes (and respective frozen  
> varients) to the collections module, namely a bag (multiset) class  
> and a setlist (ordered set/unique list) class.  I know this has been  
> floated before, but I think it merits more consideration.

FWIW, my goal for the collections module is to keep it relatively  
small and not to include every variant someone can think of.   We've  
already rejected red-black trees, pairing heaps, and blist (which is a  
list variant that allows fast insertions).

Starting with 2.7 and 3.1, we do have a Counter object that has some  
multiset capabilities -- I want to see how that fares in the real  
world before entertaining a proposal for another multiset/bag variant.

With respect to ordered sets, I started by posting an efficient recipe  
on ASPN so that we could see how it gets used.  Though I'm a fan of  
the idea in general, I've found few use cases and have noted nearly  
zero uptake on the recipe (so the OrderedSet idea may just be  
YAGNI).    The wording of your proposal indicates that it is motivated  
by a desire to complete the grid of possible variants (frozen/ 
unfrozen, itemized/multivalued, ordered/unordered, etc), but it would  
have been better to focus on use cases (existing needs that aren't  
being met by the current toolsets).

After posting the pure Python recipe for ordered sets, my plan had  
been to wait to see how it was received.  If there was an active  
interest (none so far), I've already worked out the steps needed  to  
extend existing C code to make the built-in sets remember their  
insertion order (no need for a separate type, just make the regular  
sets remember their order) -- this can be done with very little impact  
on performance.

So that leaves your proposed extensions to the abstract base classes.   
My issue with those is that collections ABCs suffer from the same  
problem as the itertools module -- the more of them you add, the  
harder it is to learn and remember which ones to use (it is possible  
to go wild with collections abstractions and then find that zero value  
has been added).  When developing the ABCs, Guido rejected the idea of  
adding intermediate layers of abstraction (IIRC, he documented his  
rationale in the PEP).

My own thought on the subject is that the collections ABC development  
needs to be somewhat restrained.  They are new and have not yet been  
widely adopted.  The community is just now gaining experience with  
them and those experiences are vital for making informed choices on  
how or whether to grow the collections ABCs.

Also, the collections ABCs that we already have have not been fully  
exercised or debugged.  The ones we have need to be shaken-out before  
additional efforts are undertaken to extend them (or to create  
intermediate layers of abstraction).

 >  The last addition but the first I will list, because I think it is  
the coolest
 > and provides the most functionality, is the new function I propose
 > adding to collections.  The function is simply called 'collection'  
and has
 > the following signature:
 >
 > collection(iterable=None, mutable=False, ordered=False, unique=False)

There are a couple of problems here.  Unifying many collections under  
a single API has the unfortunate side-effect of freezing development  
on those collections (it becomes hard to change one without changing  
them all).  Another issue is that the API doesn't seem comfortable to  
me -- when I want a set or list or dict, I just say so -- I don't want  
to go through a menu of all of the possible variants.  Also, it is not  
like those variants are readily substitutable for one another -- if  
your code relies on hashability, then you don't really have the option  
of switching the mutability flag.  To me, this factory function is  
like a comp sci exercise in hyper-generalization -- it doesn't make  
any real-world code any cleaner.

Sorry for the sour notes.  You're definitely showing insight into the  
relationship between various collection classes but I don't think the  
ideas represent good, use-case driven language development.   The  
proposals are all too theoretically based; instead, the evolution of  
the collections module needs to follow a slower path, one that lets  
the existing tools mature.  And tools like OrderedSet need to have  
some early life as a recipe to see how people use it and to learn from  
their experiences.


Raymond


From steve at pearwood.info  Sun Jul 19 04:08:57 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 19 Jul 2009 12:08:57 +1000
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <5d1a32000907181311ucbfabefk162d54f34cec32da@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<200907190236.46560.steve@pearwood.info>
	<5d1a32000907181311ucbfabefk162d54f34cec32da@mail.gmail.com>
Message-ID: <200907191208.57387.steve@pearwood.info>

On Sun, 19 Jul 2009 06:11:57 am Gerald Britton wrote:
> > ? ?items.sort(key=compute_sort_value) where:
> >> Now, the body of compute_sort_value appears exactly where it is
> >> executed: within the call to items.sort().
> >
> > No it doesn't. The function definition appears outside the call to
> > sort(). For the function definition to appear inside the call to
> > sort, you'd need to write something like
>
> Yes, it does appear within the call.  The call is more than just the
> parameters in parentheses.  It begins with "items.sort" and ends (in
> this case) with then end of the "where" clause

That's a definition of "within" that I'm not really happy with. And I 
think it's wrong. Consider the timing of calls:


import time
class MyList(list):
    def sort(self, *args, **kwargs):
        print "Call to sort() started at %s seconds" % time.time()
        super(MyList, self).sort(*args, **kwargs)
        print "Call to sort() ended at %s seconds" % time.time()

items = MyList([-9, 7, -4, 2, -1, 0, 3])
items.sort(key=func) where:
    print "Enter the where-block at %s seconds" % time.time()
    time.sleep(555)
    def func(item):
        return abs(item)


You need to create func before you can pass it as an argument to sort(), 
so I would expect the above to print something like:

Enter the where-block at 1234560000.23 seconds
Call to sort() started at 1234560555.24 seconds
Call to sort() ended at 1234560555.25 seconds

To my mind, to say that the contents of the where-block occur "within" 
the call to sort, it would imply that you get output like this:

Call to sort() started at 1234560000.23 seconds
Enter the where-block at 1234560000.24 seconds
Call to sort() ended at 1234560555.25 seconds


> > ? ?items.sort(key=lambda item: value)
>
> The idea behind "where" clauses is not to replace lambdas.  In fact,
> they are useful together:
>
>      items.sort(key=mysort) where:
>          mysort =  sort_key_function_factory(
>                           with, many, arguments)


If you're going to say that where and lambda are useful together, 
shouldn't your example actually show where and lambda together?



> which to many is preferable over:
>
>     items.sort(key=lambda item:sort_key_function_factory(with, many,
> arguments))

That's equivalent to:

def f(item):
    function = sort_key_function_factory(with, many, arguments)
    return function

items.sort(key=f)

which almost certainly is not what you what.

I think what you meant was:

items.sort(key=lambda item:
           sort_key_function_factory(with, many, arguments)(item)
          )  # note that you call the function returned by the factory

which at least works, but it adds a pointless layer of indirection. What 
you should have is:

items.sort(key=sort_key_function_factory(with, many, arguments))

which of course is simpler than your earlier example with the 
where-block.



-- 
Steven D'Aprano


From steve at pearwood.info  Sun Jul 19 04:14:14 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 19 Jul 2009 12:14:14 +1000
Subject: [Python-ideas] Proposal for extending the collections module -
	bags / multisets, ordered sets / unique lists
In-Reply-To: <4A627A60.6040306@gmx.net>
References: <4A612B47.3050706@gmail.com>
	<200907191124.39534.steve@pearwood.info> <4A627A60.6040306@gmx.net>
Message-ID: <200907191214.14702.steve@pearwood.info>

On Sun, 19 Jul 2009 11:44:00 am Mathias Panzenb?ck wrote:

> I always thought it's odd that pythons list has append but
> the set class has the method add. Whats the reason for that anyway?

You can't append an item to the end of the set, because sets don't have 
an end. They're unordered.


-- 
Steven D'Aprano


From greg.ewing at canterbury.ac.nz  Sun Jul 19 04:15:20 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 19 Jul 2009 14:15:20 +1200
Subject: [Python-ideas] Proposal for extending the collections module -
 bags / multisets, ordered sets / unique lists
In-Reply-To: <4A627A60.6040306@gmx.net>
References: <4A612B47.3050706@gmail.com>
	<8B867CFB-8F7B-4410-9654-E523607AB7FD@gmail.com>
	<4A62347A.8080500@gmx.net>
	<200907191124.39534.steve@pearwood.info> <4A627A60.6040306@gmx.net>
Message-ID: <4A6281B8.4010807@canterbury.ac.nz>

Mathias Panzenb?ck wrote:

> I always thought it's odd that pythons list has append but
> the set class has the method add. Whats the reason for that anyway?

In English, "append" means "add at the end", which doesn't
make sense for an unordered collection.

On the other hand, "add" doesn't imply anything about
ordering, so using it for lists would make the intent
less clear.

-- 
Greg



From cmjohnson.mailinglist at gmail.com  Sun Jul 19 04:21:24 2009
From: cmjohnson.mailinglist at gmail.com (Carl Johnson)
Date: Sat, 18 Jul 2009 16:21:24 -1000
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <200907191208.57387.steve@pearwood.info>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<200907190236.46560.steve@pearwood.info>
	<5d1a32000907181311ucbfabefk162d54f34cec32da@mail.gmail.com>
	<200907191208.57387.steve@pearwood.info>
Message-ID: <3bdda690907181921o23d8e5a4nfb428ae1d5d4ca51@mail.gmail.com>

Steven D'Aprano wrote:

> That's a definition of "within" that I'm not really happy with. And I
> think it's wrong. Consider the timing of calls:


> To my mind, to say that the contents of the where-block occur "within"
> the call to sort, it would imply that you get output like this:
>
> Call to sort() started at 1234560000.23 seconds
> Enter the where-block at 1234560000.24 seconds
> Call to sort() ended at 1234560555.25 seconds

By that definition though, a lambda isn't compiled "within" the sort
expression. First the lambda is compiled then it is passed as an
argument to sort. But you earlier wrote that,

> For the function definition to appear inside the call to sort,
> you'd need to write something like:
>
>   items.sort(key=lambda item: value)

But that doesn't follow at all. Items.sort isn't called until there
are arguments that can be passed to it. The timing of this would be
the exact same as a where block:

>>> import time
>>> class MyList(list):
...    def sort(self, *args, **kwargs):
...        print("Call to sort() started at %s seconds" % time.time())
...        super(MyList, self).sort(*args, **kwargs)
...        print("Call to sort() ended at %s seconds" % time.time())
...
>>> items = MyList([-9, 7, -4, 2, -1, 0, 3])
>>>
>>>
>>> def do_stuff():
...    print("Enter the fake where-block at %s seconds" % time.time())
...    time.sleep(10)
...
>>> items.sort(key=lambda item, default=do_stuff(): abs(item))
Enter the where-block at 1247970018.89 seconds
Call to sort() started at 1247970028.89 seconds
Call to sort() ended at 1247970028.89 seconds


-- Carl


From mwm-keyword-python.b4bdba at mired.org  Sun Jul 19 06:31:48 2009
From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer)
Date: Sun, 19 Jul 2009 00:31:48 -0400
Subject: [Python-ideas] Proposal for extending the collections module
	-	bags / multisets, ordered sets / unique lists
In-Reply-To: <200907191128.37436.steve@pearwood.info>
References: <4A612B47.3050706@gmail.com>
	<8B867CFB-8F7B-4410-9654-E523607AB7FD@gmail.com>
	<4A62368F.8050602@gmail.com>
	<200907191128.37436.steve@pearwood.info>
Message-ID: <20090719003148.223db2e5@bhuda.mired.org>

On Sun, 19 Jul 2009 11:28:37 +1000
Steven D'Aprano <steve at pearwood.info> wrote:
> >   > for card in cards:
> >   >     if hand.count(card) == 2:
> >   >         print('You have a pair.')
> Why would a class with unique=True have a count() method? 

So that you can write code that doesn't care which of the various
flavors of collection it gets passed, but will work on them
anyway.

It supports a property of classes in OO programming called
"polymorphism". It's a good thing. That python's collections don't
have it has always been a minor wart, and it gets bigger as we get
more types of collections.

     <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


From mwm-keyword-python.b4bdba at mired.org  Sun Jul 19 06:56:37 2009
From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer)
Date: Sun, 19 Jul 2009 00:56:37 -0400
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <200907190236.46560.steve@pearwood.info>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<200907181039.57256.steve@pearwood.info>
	<eae285400907180545u4d12272fwfea29f738d34ca47@mail.gmail.com>
	<200907190236.46560.steve@pearwood.info>
Message-ID: <20090719005637.5610e8bb@bhuda.mired.org>

I wrote this, then pretty much rejected this due to the "upside-down"
nature of where, but the BDFL said he sort of liked it, except for the
name - and that we should be shaking out 3.X instead of extending the
language. So, just to get it on record, how about a B&D version of the
where statement? Let's call it the "with" clause. The grammar
extension (based on the 2.6 grammer) would be:

stmt_with_with: small_stmt 'with' name_list':' bind_suite
bind_suite = bind_stmt | NEWLINE INDENT bind_stmt+ DEDENT
bind_stmt = expr_stmt | funcdef | classdef
name_list = NAME (',' NAME)*

The semantics look like this (modulo the non-hygenic nature of the
macro conversion):

old_locals = locals()
old_names = set(old_locals)
bind_suite
new_names = old_names.union(name_list)
for key in locals:
    if key not in new_names:
        del key
small_stmt
for key in name_list:
    if key in old_locals:
        eval '%s = %s' % (key, old_locals[key])
    else:
	del key

The bind_suite is effectively run in a new scope, as the current scope
won't be affected by any bindings in it. In particular, global and
friends can't be used in bind_suite. The statement that the with
clause is attached to is then executed in the enclosing scope - as
that's where any bindings will happen, with the variables in
name_list, and no others, added to it. After that, we restore the
existence/values of the variables in name_list to their old
value. Yes, this means if you do something like:

x = a + b with x, a, b:
    a = 2
    b = 3

then the value of x is going to get lost afterwards. That's a bug in
your code.

     <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


From santagada at gmail.com  Sun Jul 19 09:00:48 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Sun, 19 Jul 2009 04:00:48 -0300
Subject: [Python-ideas] Proposal for extending the collections module
	-	bags / multisets, ordered sets / unique lists
In-Reply-To: <20090719003148.223db2e5@bhuda.mired.org>
References: <4A612B47.3050706@gmail.com>
	<8B867CFB-8F7B-4410-9654-E523607AB7FD@gmail.com>
	<4A62368F.8050602@gmail.com>
	<200907191128.37436.steve@pearwood.info>
	<20090719003148.223db2e5@bhuda.mired.org>
Message-ID: <A36FE314-D362-4BF4-AC95-C01C6BBDCD28@gmail.com>


On Jul 19, 2009, at 1:31 AM, Mike Meyer wrote:

> On Sun, 19 Jul 2009 11:28:37 +1000
> Steven D'Aprano <steve at pearwood.info> wrote:
>>>> for card in cards:
>>>>    if hand.count(card) == 2:
>>>>        print('You have a pair.')
>> Why would a class with unique=True have a count() method?
>
> So that you can write code that doesn't care which of the various
> flavors of collection it gets passed, but will work on them
> anyway.
>
> It supports a property of classes in OO programming called
> "polymorphism". It's a good thing. That python's collections don't
> have it has always been a minor wart, and it gets bigger as we get
> more types of collections.


For example ints don't support slicing and that is not an omission of  
polymorphism, there is no sense in having that on them. A dict or set  
having a count method would make no sense either. This has nothing to  
do with polymorphism.

add on lists or append on sets could be discussed (maybe append on  
sets makes more sense as they already have pop), but adding count in  
dict or set would not be pragmatic.

--
Leonardo Santagada
santagada at gmail.com





From g.brandl at gmx.net  Sun Jul 19 09:07:34 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 19 Jul 2009 09:07:34 +0200
Subject: [Python-ideas] Experiment: Adding "re" to string objects.
In-Reply-To: <4A61D9A3.8020701@gmail.com>
References: <20090718002836.GA26311@tummy.com>	<4A611CAA.5020108@mrabarnett.plus.com>	<4A61B027.4050106@voidspace.org.uk>
	<4A61D9A3.8020701@gmail.com>
Message-ID: <h3ugp0$jgr$1@ger.gmane.org>

Nick Coghlan schrieb:

> However, the idea of adding more convenience classes to the re module
> may still have some merit. In particular, when checking against multiple
> regexes, being able to do something like the following might be helpful:
> 
> m = re.Matcher(s)
> if m.match(r'whatever(.*)'):
>       print m.group(1)
> elif m.match(r'something (.*) else(.*)'):
>       print m.group(2)

That one looks very useful, and will prevent more proposals of new syntax
along the lines of "if rex.match(...) as m" :)

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.



From mwm-keyword-python.b4bdba at mired.org  Sun Jul 19 09:47:28 2009
From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer)
Date: Sun, 19 Jul 2009 03:47:28 -0400
Subject: [Python-ideas] Proposal for extending the collections module
	-	bags / multisets, ordered sets / unique lists
In-Reply-To: <A36FE314-D362-4BF4-AC95-C01C6BBDCD28@gmail.com>
References: <4A612B47.3050706@gmail.com>
	<8B867CFB-8F7B-4410-9654-E523607AB7FD@gmail.com>
	<4A62368F.8050602@gmail.com>
	<200907191128.37436.steve@pearwood.info>
	<20090719003148.223db2e5@bhuda.mired.org>
	<A36FE314-D362-4BF4-AC95-C01C6BBDCD28@gmail.com>
Message-ID: <20090719034728.1e9f5ee7@bhuda.mired.org>

On Sun, 19 Jul 2009 04:00:48 -0300
Leonardo Santagada <santagada at gmail.com> wrote:
> On Jul 19, 2009, at 1:31 AM, Mike Meyer wrote:
> > On Sun, 19 Jul 2009 11:28:37 +1000
> > Steven D'Aprano <steve at pearwood.info> wrote:
> >>>> for card in cards:
> >>>>    if hand.count(card) == 2:
> >>>>        print('You have a pair.')
> >> Why would a class with unique=True have a count() method?
> >
> > So that you can write code that doesn't care which of the various
> > flavors of collection it gets passed, but will work on them
> > anyway.
> >
> > It supports a property of classes in OO programming called
> > "polymorphism". It's a good thing. That python's collections don't
> > have it has always been a minor wart, and it gets bigger as we get
> > more types of collections.
> For example ints don't support slicing and that is not an omission of  
> polymorphism, there is no sense in having that on them.

Given that there aren't any types related to int for which slicing
makes sense, this is true.

> A dict or set having a count method would make no sense either. This
> has nothing to do with polymorphism.

Actually, count on a dict makes *lots* of sense. I can think of a lot
of uses for counting the number of occurrences of values in a
dictionary, mostly revolving around using dictionaries to implement
sparse arrays of various kinds.

As for sets, the "count" method is obviously useful on multisets. And
it takes on reasonable values for the subset of multisets with unique
values. Yes, it's either 1 or 0, and yes, it's isomorphic to testing
membership, but it makes sense. An algorithm that uses count on
multisets should work just fine on sets - that's what polymorphism
buys for you. Saying that "count" doesn't make sense on sets is a lot
like saying "length" and "width" don't make sense on squares, since
those are equal - which would mean code written to handle rectangles
wouldn't work on squares, unless they were crammed into rectangle
objects.

Part of the problem is that Python's collections types are a
collection (no pun intended) of things that have shown themselves to
be useful in practice.  This has resulted in types that have nearly
identical sets of operations on them, but the operations have
different names, so the types - which would be polymorphic if they'd
been designed to work together - aren't polymorphic. I get bit by this
fairly often. Most recently spending last friday afternoon auditing
code to make sure I'd turned all the list methods into the appropriate
set methods in a graph class's node walking method.

> add on lists or append on sets could be discussed (maybe append on  
> sets makes more sense as they already have pop), but adding count in  
> dict or set would not be pragmatic.

Nah, it'd just be useful.

     <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


From steve at pearwood.info  Sun Jul 19 10:49:09 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 19 Jul 2009 18:49:09 +1000
Subject: [Python-ideas]
	=?iso-8859-1?q?Proposal_for_extending_the_collecti?=
	=?iso-8859-1?q?ons_module_-=09bags_/_multisets=2C_ordered_sets_/_u?=
	=?iso-8859-1?q?nique_lists?=
In-Reply-To: <20090719034728.1e9f5ee7@bhuda.mired.org>
References: <4A612B47.3050706@gmail.com>
	<A36FE314-D362-4BF4-AC95-C01C6BBDCD28@gmail.com>
	<20090719034728.1e9f5ee7@bhuda.mired.org>
Message-ID: <200907191849.10616.steve@pearwood.info>

On Sun, 19 Jul 2009 05:47:28 pm Mike Meyer wrote:

> > A dict or set having a count method would make no sense either.
> > This has nothing to do with polymorphism.
>
> Actually, count on a dict makes *lots* of sense. I can think of a lot
> of uses for counting the number of occurrences of values in a
> dictionary, mostly revolving around using dictionaries to implement
> sparse arrays of various kinds.

But dicts don't allow you to look up on values, you look up on keys.

dict.count(value) might be useful -- I doubt that there are "a lot of 
uses", but there are probably a few -- but it's hardly a fundamental 
operation on a dict. I'd argue it would be better as a function 
count_value(mapping, value) rather than a method. That way, you can 
make it polymorphic on the first argument, without requiring that all 
mappings inherit from each other.


Arguably you might have dict.count(key) as a dict method, but since that 
can only return 0 or 1, it's just another way of spelling

key in dict

which is faster, shorter and easier to read.


> As for sets, the "count" method is obviously useful on multisets. 

But not for sets, where it is just another way of spelling "element in 
set".


> And 
> it takes on reasonable values for the subset of multisets with unique
> values. Yes, it's either 1 or 0, and yes, it's isomorphic to testing
> membership, but it makes sense. An algorithm that uses count on
> multisets should work just fine on sets - that's what polymorphism
> buys for you. 

That can't be true, because their behaviour is different and they 
therefore aren't interchangeable. If obj is either a set or a multiset, 
the state of obj after:

obj.add(element)
obj.add(element)
obj.remove(element)

will be different.

An algorithm that expects obj to be empty will fail if obj is a 
multiset, and an algorithm that expects obj will not be empty will fail 
if it is a set. Either way, they aren't interchangeable.


> Saying that "count" doesn't make sense on sets is a lot 
> like saying "length" and "width" don't make sense on squares, since
> those are equal - which would mean code written to handle rectangles
> wouldn't work on squares, unless they were crammed into rectangle
> objects.

Funny you should mention that. Squares and rectangles is just another 
example of the Circle And Ellipse Problem:

http://www.c2.com/cgi/wiki?CircleAndEllipseProblem

You can't expect to use squares everywhere you use rectangles. Assuming 
mutable objects:

rectangle.width = 2*rectangle.height
assert rectangle.width == 2*rectangle.height

will work, but:

square.width = 2*square.height

if problematic. There are two "reasonable" approaches to it:

(1) keep the invariant width==height by setting both the width and the 
height to twice the existing height, which will then cause:

assert square.width == 2*square.height

to fail;  or

(2) treat the square as if it were a rectangle, and stretch the object 
as requested, but that will break the invariant that squares are, well, 
square.

Either way, you break something.

Note that the problem is slightly different if you use immutable 
objects. Now you can't stretch a square into a rectangular shape, but 
how do you create your objects? Both of:

square(width=5, height=10)
rectangle(width=5)

will fail. (Although rectangle could make height optional, and default 
to the same value as width. However, that's also problematic: "In the 
face of ambiguity, refuse the temptation to guess".)



> Part of the problem is that Python's collections types are a
> collection (no pun intended) of things that have shown themselves to
> be useful in practice.  This has resulted in types that have nearly
> identical sets of operations on them, but the operations have
> different names, so the types - which would be polymorphic if they'd
> been designed to work together - aren't polymorphic. 

They wouldn't be polymorphic, because they aren't polymorphic. The 
following is a good example of why they're not polymorphic, but some 
algorithms are agnostic to the choice of data structure:

> I get bit by 
> this fairly often. Most recently spending last friday afternoon
> auditing code to make sure I'd turned all the list methods into the
> appropriate set methods in a graph class's node walking method.

Lists are ordered, sets are not. You can *insert* or *append* to a list, 
but not to a set, because sets aren't ordered.

True, one could artificially pretend that sets are polymorphic to lists 
by using misleading method names:

set.add -> set.append

and do-nothing methods like set.sort() and set.reverse(). But apart from 
satisfying people who have read too many OO books, what would be the 
point?



-- 
Steven D'Aprano


From solipsis at pitrou.net  Sun Jul 19 12:40:07 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 19 Jul 2009 10:40:07 +0000 (UTC)
Subject: [Python-ideas] Experiment: Adding "re" to string objects.
References: <20090718002836.GA26311@tummy.com>	<4A611CAA.5020108@mrabarnett.plus.com>	<4A61B027.4050106@voidspace.org.uk>
	<4A61D9A3.8020701@gmail.com> <h3ugp0$jgr$1@ger.gmane.org>
Message-ID: <loom.20090719T103450-684@post.gmane.org>

Georg Brandl <g.brandl at ...> writes:
> 
> Nick Coghlan schrieb:
> 
> > However, the idea of adding more convenience classes to the re module
> > may still have some merit. In particular, when checking against multiple
> > regexes, being able to do something like the following might be helpful:
> > 
> > m = re.Matcher(s)
> > if m.match(r'whatever(.*)'):
> >       print m.group(1)
> > elif m.match(r'something (.*) else(.*)'):
> >       print m.group(2)

-0.5. Right now, objects in the re module are constructed from a regular
expression pattern -- one of the reasons being that these patterns are compiled
to bytecode form, and the objects help retain the bytecode. Having another
object type constructed from the string-to-match is confusing. Besides, keeping
some kind of internal state about the last matched pattern, only for
"convenience" purposes, isn't pretty either.




From steve at pearwood.info  Sun Jul 19 13:39:04 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 19 Jul 2009 21:39:04 +1000
Subject: [Python-ideas] Experiment: Adding "re" to string objects.
In-Reply-To: <h3sm9r$nub$1@ger.gmane.org>
References: <20090718002836.GA26311@tummy.com> <4A61D9A3.8020701@gmail.com>
	<h3sm9r$nub$1@ger.gmane.org>
Message-ID: <200907192139.05316.steve@pearwood.info>

On Sun, 19 Jul 2009 12:30:19 am Christian Heimes wrote:
> Nick Coghlan wrote:
> > The idea of adding a mutable attribute to strings (even if it plays
> > no part in string equality) sends shivers down my spine. It also
> > seems like it would be difficult to do this in a way that didn't
> > increase the size of all strings by at least one pointer slot.
>
> I've more arguments against the re idea:
[snip]

Apologies for the Metoo, but I'm with Nick and Christian on this. It 
sounds like a terrible idea to me, just to avoid a temporary name in 
the standard idiom:

m = re.match(r'whatever(.*)', s)
if m:
    m.group(1)


-1 on the suggestion.



-- 
Steven D'Aprano


From python at mrabarnett.plus.com  Sun Jul 19 16:16:15 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 19 Jul 2009 15:16:15 +0100
Subject: [Python-ideas] Experiment: Adding "re" to string objects.
In-Reply-To: <h3ugp0$jgr$1@ger.gmane.org>
References: <20090718002836.GA26311@tummy.com>	<4A611CAA.5020108@mrabarnett.plus.com>	<4A61B027.4050106@voidspace.org.uk>	<4A61D9A3.8020701@gmail.com>
	<h3ugp0$jgr$1@ger.gmane.org>
Message-ID: <4A632AAF.3090009@mrabarnett.plus.com>

Georg Brandl wrote:
> Nick Coghlan schrieb:
> 
>> However, the idea of adding more convenience classes to the re module
>> may still have some merit. In particular, when checking against multiple
>> regexes, being able to do something like the following might be helpful:
>>
>> m = re.Matcher(s)
>> if m.match(r'whatever(.*)'):
>>       print m.group(1)
>> elif m.match(r'something (.*) else(.*)'):
>>       print m.group(2)
> 
> That one looks very useful, and will prevent more proposals of new syntax
> along the lines of "if rex.match(...) as m" :)
> 
Or re.match could accept a tuple of patterns and the MatchObject could
have an 'index' property to say which one matched.


From daniel at stutzbachenterprises.com  Sun Jul 19 16:22:59 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Sun, 19 Jul 2009 09:22:59 -0500
Subject: [Python-ideas] Proposal for extending the collections module -
	bags / multisets, ordered sets / unique lists
In-Reply-To: <003D2989-9032-4C81-A97F-B1DC2CCB9934@rcn.com>
References: <4A612B47.3050706@gmail.com>
	<003D2989-9032-4C81-A97F-B1DC2CCB9934@rcn.com>
Message-ID: <eae285400907190722k6e2c685bx3e1e8912c2ba774a@mail.gmail.com>

On Sat, Jul 18, 2009 at 8:45 PM, Raymond Hettinger <python at rcn.com> wrote:

> On Jul 17, 2009, at 6:54 PM, Michael Lenzen wrote:
>
>> In a nutshell, I want to add 2 new classes (and respective frozen
>> varients) to the collections module, namely a bag (multiset) class and a
>> setlist (ordered set/unique list) class.  I know this has been floated
>> before, but I think it merits more consideration.
>>
>
> FWIW, my goal for the collections module is to keep it relatively small and
> not to include every variant someone can think of.   We've already rejected
> red-black trees, pairing heaps, and blist (which is a list variant that
> allows fast insertions).
>

Michael Lenzen:

If you're interested in maintaining a more extensive collections module on
PyPi, I'd be happy to collaborate with you as the author of 2 of the 3
aforementioned rejected data structures. :-)

In addition to pairing heaps (deprecated) and blist (on PyPi), I've also got
a few other data structures that may or may not interest you:
- HeapDict (on PyPi): a dictionary where .popitem() returns the item with
the lowest value
- LRU: a dictionary that maintains hard references to the most recently used
n items and weak references to the rest
- MultiValueDict: x[5] = 1, x[5] = 2, print x[5] => set([1,2])
- WeakRefSet: the set analog to a WeakRefDict

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090719/d21b838a/attachment.html>

From daniel at stutzbachenterprises.com  Sun Jul 19 16:28:32 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Sun, 19 Jul 2009 09:28:32 -0500
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <200907191208.57387.steve@pearwood.info>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<200907190236.46560.steve@pearwood.info>
	<5d1a32000907181311ucbfabefk162d54f34cec32da@mail.gmail.com>
	<200907191208.57387.steve@pearwood.info>
Message-ID: <eae285400907190728r773f7601w88b58013a7b9f00e@mail.gmail.com>

On Sat, Jul 18, 2009 at 9:08 PM, Steven D'Aprano <steve at pearwood.info>wrote:

> > Yes, it does appear within the call.  The call is more than just the
> > parameters in parentheses.  It begins with "items.sort" and ends (in
> > this case) with then end of the "where" clause
>
> That's a definition of "within" that I'm not really happy with. And I
> think it's wrong. Consider the timing of calls:
>

I think we're arguing semantics there.  There are some reasonable
definitions of "within" where the "where" statement isn't within the call,
and other reasonable definitions of "within" where it after the call.
You've already given examples of the later.  Here's an example of the
former:

for i in range(5):
  x.append(some_function(i))

I think we can agree that the body of the "for" loop is within the "for"
statement?

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090719/821d4f59/attachment.html>

From mwm-keyword-python.b4bdba at mired.org  Sun Jul 19 17:53:43 2009
From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer)
Date: Sun, 19 Jul 2009 11:53:43 -0400
Subject: [Python-ideas] Proposal for extending the collections module
	-	bags / multisets, ordered sets / unique lists
In-Reply-To: <200907191849.10616.steve@pearwood.info>
References: <4A612B47.3050706@gmail.com>
	<A36FE314-D362-4BF4-AC95-C01C6BBDCD28@gmail.com>
	<20090719034728.1e9f5ee7@bhuda.mired.org>
	<200907191849.10616.steve@pearwood.info>
Message-ID: <20090719115343.171f1e30@bhuda.mired.org>

On Sun, 19 Jul 2009 18:49:09 +1000
Steven D'Aprano <steve at pearwood.info> wrote:

> On Sun, 19 Jul 2009 05:47:28 pm Mike Meyer wrote:
> 
> > > A dict or set having a count method would make no sense either.
> > > This has nothing to do with polymorphism.
> >
> > Actually, count on a dict makes *lots* of sense. I can think of a lot
> > of uses for counting the number of occurrences of values in a
> > dictionary, mostly revolving around using dictionaries to implement
> > sparse arrays of various kinds.
> 
> But dicts don't allow you to look up on values, you look up on keys.

So? I wasn't talking about looking up values, I was talking about
counting them. Of course, lists don't do that very well either - you
can only look up two values, or maybe it's one. And that's just
another thing that makes changing an implementations from lists to dicts

> dict.count(value) might be useful -- I doubt that there are "a lot of 
> uses", but there are probably a few -- but it's hardly a fundamental 
> operation on a dict. I'd argue it would be better as a function 
> count_value(mapping, value) rather than a method. That way, you can 
> make it polymorphic on the first argument, without requiring that all 
> mappings inherit from each other.

That would be fine - except that count_values(list, value) is another
way of spelling list.count(value).

> Arguably you might have dict.count(key) as a dict method,

You might argue that, but I wasn't.

> > As for sets, the "count" method is obviously useful on multisets. 
> But not for sets, where it is just another way of spelling "element in 
> set".

True. And if you know you're working with sets, that's the preferred
spelling.

> 
> > And 
> > it takes on reasonable values for the subset of multisets with unique
> > values. Yes, it's either 1 or 0, and yes, it's isomorphic to testing
> > membership, but it makes sense. An algorithm that uses count on
> > multisets should work just fine on sets - that's what polymorphism
> > buys for you. 
> 
> That can't be true, because their behaviour is different and they 
> therefore aren't interchangeable. If obj is either a set or a multiset, 
> the state of obj after:
>
> obj.add(element)
> obj.add(element)
> obj.remove(element)
> 
> will be different.

That just means the two classes don't always behave the same, which is
why we give them different names.

> An algorithm that expects obj to be empty will fail if obj is a 
> multiset, and an algorithm that expects obj will not be empty will fail 
> if it is a set. Either way, they aren't interchangeable.

So? An algorithm that doesn't depend on that behavior will still
work. Yes, multisets and sets are different types, so not all
algorithms that work on one will work properly on the other. Making
them polymorphic will make the set of algorithms that work on both
larger, which helps with not repeating yourself.

> > Saying that "count" doesn't make sense on sets is a lot 
> > like saying "length" and "width" don't make sense on squares, since
> > those are equal - which would mean code written to handle rectangles
> > wouldn't work on squares, unless they were crammed into rectangle
> > objects.
> 
> Funny you should mention that. Squares and rectangles is just another 
> example of the Circle And Ellipse Problem:
> 
> http://www.c2.com/cgi/wiki?CircleAndEllipseProblem
> 
> You can't expect to use squares everywhere you use rectangles.

I would never argue that you can.  If there were identical, we
wouldn't have two names for them.

> Assuming mutable objects:

That's an *excellent* place to start if you're going to show they're
different kinds of objects! Ditto for sets vs. multisets. It's their
*values* that determine which class they belong to, so changing their
values is the easiest way to break them.

> Note that the problem is slightly different if you use immutable 
> objects. Now you can't stretch a square into a rectangular shape, but 
> how do you create your objects? Both of:

But only slightly - you have to know what type of object you're
creating if you're going to create them. 

This is a well-know - and fairly well-analyzed - problem. Created
types need to be contravariant instead of covariant.
> > Part of the problem is that Python's collections types are a
> > collection (no pun intended) of things that have shown themselves to
> > be useful in practice.  This has resulted in types that have nearly
> > identical sets of operations on them, but the operations have
> > different names, so the types - which would be polymorphic if they'd
> > been designed to work together - aren't polymorphic. 
> They wouldn't be polymorphic, because they aren't polymorphic. The 
> following is a good example of why they're not polymorphic, but some 
> algorithms are agnostic to the choice of data structure:

They aren't polymorphic because python has chosen names that make them
non-polymorphic.

> > I get bit by 
> > this fairly often. Most recently spending last friday afternoon
> > auditing code to make sure I'd turned all the list methods into the
> > appropriate set methods in a graph class's node walking method.
> Lists are ordered, sets are not. You can *insert* or *append* to a list, 
> but not to a set, because sets aren't ordered.

Of course I can insert and append to a set. Just because the set loses
the ordering information doesn't mean the element wasn't added to the
set (or not, depending).

> True, one could artificially pretend that sets are polymorphic to lists 
> by using misleading method names:
> 
> set.add -> set.append
> 
> and do-nothing methods like set.sort() and set.reverse(). But apart from 
> satisfying people who have read too many OO books, what would be the 
> point?

Saving time for people who are used to having a well-designed
hierarchy of collections by letting them not waste time dealing with
name differences over implementation details. Things like repeating
myself by writing foo_for_sets and foo_for_dicts and foo_for_lists, or
changing method names, or mapping between two representations so I can
use code that chose one set of names, or dealing with the headaches of
maintaining two copies of a structure.

Python has been moving this direction for a long time - int/long
unification, lists being acceptable most places that tuples are,
adding iterator interfaces to the collection types - all are things
that make using those types in a polymorphic manner easier. This is
just another step in that direction.

	<mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


From zuo at chopin.edu.pl  Sun Jul 19 20:29:18 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Sun, 19 Jul 2009 20:29:18 +0200
Subject: [Python-ideas] Where-statement (Proposal for function
 expressions)
In-Reply-To: <ca471dc20907181814q4b76d527qc1cad3410f4df8d1@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<200907180024.31209.steve@pearwood.info>
	<5d1a32000907171328y6da57c7j30b5bfe154e90f32@mail.gmail.com>
	<200907181039.57256.steve@pearwood.info>
	<3bdda690907172019hb70bda5h51d76ef31058423a@mail.gmail.com>
	<4A614D9B.7060500@canterbury.ac.nz> <8763dqd1fm.fsf@benfinney.id.au>
	<4A626377.8050001@canterbury.ac.nz>
	<ca471dc20907181814q4b76d527qc1cad3410f4df8d1@mail.gmail.com>
Message-ID: <op.uxbuy4o9fvx12t@jerzozwiesz.home.aster.pl>

Guido van Rossum <guido at python.org> wrote:

> The 'where' proposal actually looks nice to me. (Maybe I've been
> studying functional languages too much lately :-). My biggest problem
> with this concrete proposal is that 'where' means something completely
> different in SQL (which I've also studied too much lately :-).
>
> However, I think we should focus on keeping the language stable rather
> than keep tinkering with it. Let's help 3rd party developers port
> their work to 3.1 rather than planning 3.1's obsolescence.

I believe it's at the other 'end for stick' of development process and
doesn't hurt 3.1 at all.

It could appear at the earliest in 3.2 or 3.3 in __future__... but I
think it doesn't mean we should freeze working on new ideas, even if
they have chance to be released not earlier that in three or five years.

Mike Meyer <mwm-keyword-python.b4bdba at mired.org> wrote:

> but the BDFL said he sort of liked it, except for the
> name - and that we should be shaking out 3.X instead of extending the
> language. So, just to get it on record, how about a B&D version of the
> where statement? Let's call it the "with" clause.

But we already have 'with' in the language, doing completely different
things.

I propose 'using'.


Regards,

zuo

-- 
Jan Kaliszewski <zuo at chopin.edu.pl>


From zuo at chopin.edu.pl  Sun Jul 19 20:45:32 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Sun, 19 Jul 2009 20:45:32 +0200
Subject: [Python-ideas] Experiment: Adding "re" to string objects.
In-Reply-To: <loom.20090719T103450-684@post.gmane.org>
References: <20090718002836.GA26311@tummy.com>
	<4A611CAA.5020108@mrabarnett.plus.com>
	<4A61B027.4050106@voidspace.org.uk>
	<4A61D9A3.8020701@gmail.com> <h3ugp0$jgr$1@ger.gmane.org>
	<loom.20090719T103450-684@post.gmane.org>
Message-ID: <op.uxbvp6ghfvx12t@jerzozwiesz.home.aster.pl>

Antoine Pitrou <solipsis at pitrou.net> wrote:

> Georg Brandl <g.brandl at ...> writes:
>>
>> Nick Coghlan schrieb:
>>
>> > However, the idea of adding more convenience classes to the re module
>> > may still have some merit. In particular, when checking against  
>> multiple
>> > regexes, being able to do something like the following might be  
>> helpful:
>> >
>> > m = re.Matcher(s)
>> > if m.match(r'whatever(.*)'):
>> >       print m.group(1)
>> > elif m.match(r'something (.*) else(.*)'):
>> >       print m.group(2)
>
> -0.5. Right now, objects in the re module are constructed from a regular
> expression pattern -- one of the reasons being that these patterns are  
> compiled
> to bytecode form, and the objects help retain the bytecode. Having  
> another
> object type constructed from the string-to-match is confusing.

Maybe it should be limited to using compiled regexps, not strings?

> Besides, keeping
> some kind of internal state about the last matched pattern, only for
> "convenience" purposes, isn't pretty either.

But 'practicality beats purity' :)

-- 
Jan Kaliszewski <zuo at chopin.edu.pl>


From zuo at chopin.edu.pl  Sun Jul 19 20:51:56 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Sun, 19 Jul 2009 20:51:56 +0200
Subject: [Python-ideas] Proposal for extending the collections module -
 bags / multisets, ordered sets / unique lists
In-Reply-To: <20090719115343.171f1e30@bhuda.mired.org>
References: <4A612B47.3050706@gmail.com>
	<A36FE314-D362-4BF4-AC95-C01C6BBDCD28@gmail.com>
	<20090719034728.1e9f5ee7@bhuda.mired.org>
	<200907191849.10616.steve@pearwood.info>
	<20090719115343.171f1e30@bhuda.mired.org>
Message-ID: <op.uxbv0uvhfvx12t@jerzozwiesz.home.aster.pl>

Mike Meyer <mwm-keyword-python.b4bdba at mired.org> wrote:

> On Sun, 19 Jul 2009 18:49:09 +1000
> Steven D'Aprano <steve at pearwood.info> wrote:
>
>> On Sun, 19 Jul 2009 05:47:28 pm Mike Meyer wrote:
>>
>> > > A dict or set having a count method would make no sense either.
>> > > This has nothing to do with polymorphism.
>> >
>> > Actually, count on a dict makes *lots* of sense. I can think of a lot
>> > of uses for counting the number of occurrences of values in a
>> > dictionary, mostly revolving around using dictionaries to implement
>> > sparse arrays of various kinds.
>>
>> But dicts don't allow you to look up on values, you look up on keys.
>
> So? I wasn't talking about looking up values, I was talking about
> counting them. Of course, lists don't do that very well either - you
> can only look up two values, or maybe it's one. And that's just
> another thing that makes changing an implementations from lists to dicts
>
>> dict.count(value) might be useful -- I doubt that there are "a lot of
>> uses", but there are probably a few -- but it's hardly a fundamental
>> operation on a dict. I'd argue it would be better as a function
>> count_value(mapping, value) rather than a method.

You seem to have forgot that now we have dict views.

I think it should be a method -- but of dict values-view rather than of
dict itself.

-- 
Jan Kaliszewski <zuo at chopin.edu.pl>


From zuo at chopin.edu.pl  Sun Jul 19 21:00:01 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Sun, 19 Jul 2009 21:00:01 +0200
Subject: [Python-ideas] Proposal for extending the collections module -
 bags / multisets, ordered sets / unique lists
In-Reply-To: <200907191124.39534.steve@pearwood.info>
References: <4A612B47.3050706@gmail.com>
	<8B867CFB-8F7B-4410-9654-E523607AB7FD@gmail.com>
	<4A62347A.8080500@gmx.net> <200907191124.39534.steve@pearwood.info>
Message-ID: <op.uxbwebpmfvx12t@jerzozwiesz.home.aster.pl>

Dnia 19-07-2009 o 03:24:38 Steven D'Aprano <steve at pearwood.info>  
napisa?(a):

> On Sun, 19 Jul 2009 06:45:46 am Mathias Panzenb?ck wrote:
>> I don't understand what's so bad about such a collection factory. As
>> a programmer I don't care what class an object is. I just want an
>> object that behaves a certain way. E.g. I want to say: I need a
>> random accessible collection, or I need a collection with unique
>> entries, or I need a collection with a fast (near to O(1)) way to
>> append elements, or all of the above. What class this object will
>> have I don't care.
>>
>> Where is the problem?
>
>
> The problem is that you *do* need to care what the class is, because
> different classes have different interfaces:

It could be a factory similar to namedtuple: creating not objects of
container type, but container types:

OrdSet = collection(ordered=True, unique=True)

numbers = OrdSet([1, 2, 3 ,4 ,5])
letters = OrdSet(['x', 'y', 'z'])
names = OrdSet(['Ann', 'Berta', 'Claude', 'Danuta'])

-- 
Jan Kaliszewski <zuo at chopin.edu.pl>


From m.lenzen at gmail.com  Sun Jul 19 22:01:47 2009
From: m.lenzen at gmail.com (Michael Lenzen)
Date: Sun, 19 Jul 2009 15:01:47 -0500
Subject: [Python-ideas] Proposal for extending the collections module -
 bags / multisets, ordered sets / unique lists
In-Reply-To: <003D2989-9032-4C81-A97F-B1DC2CCB9934@rcn.com>
References: <4A612B47.3050706@gmail.com>
	<003D2989-9032-4C81-A97F-B1DC2CCB9934@rcn.com>
Message-ID: <4A637BAB.9000106@gmail.com>

On 07/18/2009 08:45 PM, Raymond Hettinger wrote:

First of all, thanks for the input.  I was hoping you would read and 
comment on this.

>
> Starting with 2.7 and 3.1, we do have a Counter object that has some
> multiset capabilities -- I want to see how that fares in the real world
> before entertaining a proposal for another multiset/bag variant.
>

I'm not really a fan of the Counter class, I think that it has some 
fundamental flaws.  The biggest problem that I see is that it doesn't 
behave like a collection with multiple elements, when you iterate over 
it you only get the unique elements and len(Counter) returns the number 
of unique elements instead of the number of items in the multiset.  Then 
there's the fact that non-positive counts are allowed:

 >>> c = Counter()
 >>> c['a'] += 1
 >>> c['a'] -= 1
 >>> 'a' in c
True

This isn't a collection, it's something else - a counter.  I'm not 
saying it's not useful, but it's not a multiset.  Why I think it should 
be removed is that `defaultdict(int)` now provides exactly the same 
functionality.

> With respect to ordered sets, I started by posting an efficient recipe
> on ASPN so that we could see how it gets used. Though I'm a fan of the
> idea in general, I've found few use cases and have noted nearly zero
> uptake on the recipe (so the OrderedSet idea may just be YAGNI). The
> wording of your proposal indicates that it is motivated by a desire to
> complete the grid of possible variants (frozen/unfrozen,
> itemized/multivalued, ordered/unordered, etc), but it would have been
> better to focus on use cases (existing needs that aren't being met by
> the current toolsets).
>
> After posting the pure Python recipe for ordered sets, my plan had been
> to wait to see how it was received. If there was an active interest
> (none so far), I've already worked out the steps needed to extend
> existing C code to make the built-in sets remember their insertion order
> (no need for a separate type, just make the regular sets remember their
> order) -- this can be done with very little impact on performance.

Your implementation of ordered sets also doesn't work for me.  I want to 
be able to access elements by index, take slices of the list, just be 
able to use it as a list in general.  I think it might be a good 
addition to the current set class, it's basically an improved set.

>
> There are a couple of problems here. Unifying many collections under a
> single API has the unfortunate side-effect of freezing development on
> those collections (it becomes hard to change one without changing them
> all). Another issue is that the API doesn't seem comfortable to me --
> when I want a set or list or dict, I just say so -- I don't want to go
> through a menu of all of the possible variants. Also, it is not like
> those variants are readily substitutable for one another -- if your code
> relies on hashability, then you don't really have the option of
> switching the mutability flag. To me, this factory function is like a
> comp sci exercise in hyper-generalization -- it doesn't make any
> real-world code any cleaner.

I'm not as gung ho about the ABCs as I was before.  But, the way I see 
it is that the collection factory/ABC would simplify some situations, 
especially for new programmers.  They wouldn't have to learn the 
difference between bags, sets, lists and setlists right away.  They 
would just have an easy way to create a collection that provided a basic 
API, maybe just add, remove, contains and count methods.  All the 
classes are still available by name to those that want them, and that is 
how I would probably use them.

> Sorry for the sour notes. You're definitely showing insight into the
> relationship between various collection classes but I don't think the
> ideas represent good, use-case driven language development. The
> proposals are all too theoretically based; instead, the evolution of the
> collections module needs to follow a slower path, one that lets the
> existing tools mature. And tools like OrderedSet need to have some early
> life as a recipe to see how people use it and to learn from their
> experiences.
>

I agree that I need use-cases and will go through my old code and post 
some in the near future.  I also think that use cases will follow if the 
option is provided, that it is the responsibility of the language 
developers to push innovation and a certain paradigm.

I am firmly convinced that there need to be bag and setlist classes 
included at least in the collections module, if not the standard 
library.  Python is supposed to be *batteries included* and I see these 
as glaring omissions.  If my arguments were too theoretical it's because 
I'm a mathematician at heart and am more apt to be convinced by a 
theoretical argument as opposed to a utilitarian one.

-Michael Lenzen


From ncoghlan at gmail.com  Sun Jul 19 22:02:33 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 20 Jul 2009 06:02:33 +1000
Subject: [Python-ideas] Experiment: Adding "re" to string objects.
In-Reply-To: <loom.20090719T103450-684@post.gmane.org>
References: <20090718002836.GA26311@tummy.com>	<4A611CAA.5020108@mrabarnett.plus.com>	<4A61B027.4050106@voidspace.org.uk>	<4A61D9A3.8020701@gmail.com>
	<h3ugp0$jgr$1@ger.gmane.org>
	<loom.20090719T103450-684@post.gmane.org>
Message-ID: <4A637BD9.4060600@gmail.com>

Antoine Pitrou wrote:
> Georg Brandl <g.brandl at ...> writes:
>> Nick Coghlan schrieb:
>>
>>> However, the idea of adding more convenience classes to the re module
>>> may still have some merit. In particular, when checking against multiple
>>> regexes, being able to do something like the following might be helpful:
>>>
>>> m = re.Matcher(s)
>>> if m.match(r'whatever(.*)'):
>>>       print m.group(1)
>>> elif m.match(r'something (.*) else(.*)'):
>>>       print m.group(2)
> 
> -0.5. Right now, objects in the re module are constructed from a regular
> expression pattern -- one of the reasons being that these patterns are compiled
> to bytecode form, and the objects help retain the bytecode. Having another
> object type constructed from the string-to-match is confusing. Besides, keeping
> some kind of internal state about the last matched pattern, only for
> "convenience" purposes, isn't pretty either.

As Georg pointed out, re.match() is still the number one use case that
gets brought up whenever people start talking about embedded assignment
 statements, and it is when matching against multiple patterns that the
problems hurts the most. The idea of a helper class in the re module to
simplify that idiom isn't a silly one, even if I would object
strenuously to the idea of building regex support directly into the
basic string types.

That said, I agree that my suggestion above is potentially confusing,
which is why I kicked this thread over here rather than continuing it on
python-dev. At the moment it is just a half-baked idea that occurred to
me when reading the python-dev thread - it would need somebody to pick
it up and iterate on the design a bit to come up with something
potentially worthy of inclusion in the re module.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Sun Jul 19 22:07:40 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 20 Jul 2009 06:07:40 +1000
Subject: [Python-ideas] Proposal for extending the collections module
 -	bags / multisets, ordered sets / unique lists
In-Reply-To: <20090719115343.171f1e30@bhuda.mired.org>
References: <4A612B47.3050706@gmail.com>	<A36FE314-D362-4BF4-AC95-C01C6BBDCD28@gmail.com>	<20090719034728.1e9f5ee7@bhuda.mired.org>	<200907191849.10616.steve@pearwood.info>
	<20090719115343.171f1e30@bhuda.mired.org>
Message-ID: <4A637D0C.6070907@gmail.com>

Mike Meyer wrote:
> Saving time for people who are used to having a well-designed
> hierarchy of collections by letting them not waste time dealing with
> name differences over implementation details. Things like repeating
> myself by writing foo_for_sets and foo_for_dicts and foo_for_lists, or
> changing method names, or mapping between two representations so I can
> use code that chose one set of names, or dealing with the headaches of
> maintaining two copies of a structure.

Blurring semantic distinctions (add != append) to make
overgeneralisation easier doesn't sound like a good idea to me.

Choose the data structure that makes the most sense for the current
algorithm. If someone gives you a different data structure, convert it
at the beginning and convert it back at the end (think of it as a
variant of the decorate-sort-undecorate approach to sorting).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From m.lenzen at gmail.com  Sun Jul 19 22:34:16 2009
From: m.lenzen at gmail.com (Michael Lenzen)
Date: Sun, 19 Jul 2009 15:34:16 -0500
Subject: [Python-ideas] Proposal for extending the collections module -
 bags / multisets, ordered sets / unique lists
In-Reply-To: <eae285400907190722k6e2c685bx3e1e8912c2ba774a@mail.gmail.com>
References: <4A612B47.3050706@gmail.com>	
	<003D2989-9032-4C81-A97F-B1DC2CCB9934@rcn.com>
	<eae285400907190722k6e2c685bx3e1e8912c2ba774a@mail.gmail.com>
Message-ID: <4A638348.2020100@gmail.com>

On 07/19/2009 09:22 AM, Daniel Stutzbach wrote:
> Michael Lenzen:
>
> If you're interested in maintaining a more extensive collections module
> on PyPi, I'd be happy to collaborate with you as the author of 2 of the
> 3 aforementioned rejected data structures. :-)
>
> In addition to pairing heaps (deprecated) and blist (on PyPi), I've also
> got a few other data structures that may or may not interest you:
> - HeapDict (on PyPi): a dictionary where .popitem() returns the item
> with the lowest value
> - LRU: a dictionary that maintains hard references to the most recently
> used n items and weak references to the rest
> - MultiValueDict: x[5] = 1, x[5] = 2, print x[5] => set([1,2])
> - WeakRefSet: the set analog to a WeakRefDict
>
>     --
>     Daniel Stutzbach, Ph.D.
>     President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
>

I am definitely interested in maintaining a more extensive collections 
module regardless of whether or not my suggestions make it into the 
Python library.  I must admit though, that I am not a proponent of 
pushing all of the data structures into the standard library.

I don't have any experience creating a package, although it doesn't seem 
too tough, so as much help as you'd like to give would be great.  We'd 
also have to come up with a better name that 'collections_'.  If you 
want to join the project, just shoot me an email.

-Michael Lenzen
http://code.google.com/p/python-data-structures/


From python at mrabarnett.plus.com  Sun Jul 19 23:20:08 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 19 Jul 2009 22:20:08 +0100
Subject: [Python-ideas] Experiment: Adding "re" to string objects.
In-Reply-To: <op.uxbvp6ghfvx12t@jerzozwiesz.home.aster.pl>
References: <20090718002836.GA26311@tummy.com>	<4A611CAA.5020108@mrabarnett.plus.com>	<4A61B027.4050106@voidspace.org.uk>	<4A61D9A3.8020701@gmail.com>
	<h3ugp0$jgr$1@ger.gmane.org>	<loom.20090719T103450-684@post.gmane.org>
	<op.uxbvp6ghfvx12t@jerzozwiesz.home.aster.pl>
Message-ID: <4A638E08.5070402@mrabarnett.plus.com>

Jan Kaliszewski wrote:
> Antoine Pitrou <solipsis at pitrou.net> wrote:
> 
>> Georg Brandl <g.brandl at ...> writes:
>>>
>>> Nick Coghlan schrieb:
>>>
>>> > However, the idea of adding more convenience classes to the re module
>>> > may still have some merit. In particular, when checking against 
>>> multiple
>>> > regexes, being able to do something like the following might be 
>>> helpful:
>>> >
>>> > m = re.Matcher(s)
>>> > if m.match(r'whatever(.*)'):
>>> >       print m.group(1)
>>> > elif m.match(r'something (.*) else(.*)'):
>>> >       print m.group(2)
>>
>> -0.5. Right now, objects in the re module are constructed from a regular
>> expression pattern -- one of the reasons being that these patterns are 
>> compiled
>> to bytecode form, and the objects help retain the bytecode. Having 
>> another
>> object type constructed from the string-to-match is confusing.
> 
> Maybe it should be limited to using compiled regexps, not strings?
> 
The patterns are currently cached in a dict in the re module in case
they're wanted again.

>> Besides, keeping
>> some kind of internal state about the last matched pattern, only for
>> "convenience" purposes, isn't pretty either.
> 
> But 'practicality beats purity' :)
> 
Only if it feels Pythonic. :-)


From python at rcn.com  Mon Jul 20 00:33:08 2009
From: python at rcn.com (Raymond Hettinger)
Date: Sun, 19 Jul 2009 15:33:08 -0700
Subject: [Python-ideas] Proposal for extending the collections module
	-	bags / multisets, ordered sets / unique lists
In-Reply-To: <4A637D0C.6070907@gmail.com>
References: <4A612B47.3050706@gmail.com>	<A36FE314-D362-4BF4-AC95-C01C6BBDCD28@gmail.com>	<20090719034728.1e9f5ee7@bhuda.mired.org>	<200907191849.10616.steve@pearwood.info>
	<20090719115343.171f1e30@bhuda.mired.org>
	<4A637D0C.6070907@gmail.com>
Message-ID: <A0FCFFC3-E006-4690-AF11-ABA65B1627EC@rcn.com>


On Jul 19, 2009, at 1:07 PM, Nick Coghlan wrote:
> Blurring semantic distinctions (add != append) to make
> overgeneralisation easier doesn't sound like a good idea to me.


Well said.


Raymond


From solipsis at pitrou.net  Mon Jul 20 00:36:25 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 19 Jul 2009 22:36:25 +0000 (UTC)
Subject: [Python-ideas] Experiment: Adding "re" to string objects.
References: <20090718002836.GA26311@tummy.com>	<4A611CAA.5020108@mrabarnett.plus.com>	<4A61B027.4050106@voidspace.org.uk>	<4A61D9A3.8020701@gmail.com>
	<h3ugp0$jgr$1@ger.gmane.org>
	<loom.20090719T103450-684@post.gmane.org>
	<4A637BD9.4060600@gmail.com>
Message-ID: <loom.20090719T223239-603@post.gmane.org>

Nick Coghlan <ncoghlan at ...> writes:
> 
> The idea of a helper class in the re module to
> simplify that idiom isn't a silly one, even if I would object
> strenuously to the idea of building regex support directly into the
> basic string types.

My point is that simplifying that idiom (only by a very slight amount, by the
way) is not worth the cognitive overhead of making the re API more complicated
and less regular. There should be one obvious way to do it, after all.





From grosser.meister.morti at gmx.net  Mon Jul 20 02:32:31 2009
From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=)
Date: Mon, 20 Jul 2009 02:32:31 +0200
Subject: [Python-ideas] Proposal for extending the collections module
 -	bags / multisets, ordered sets / unique lists
In-Reply-To: <200907191214.14702.steve@pearwood.info>
References: <4A612B47.3050706@gmail.com>	<200907191124.39534.steve@pearwood.info>
	<4A627A60.6040306@gmx.net> <200907191214.14702.steve@pearwood.info>
Message-ID: <4A63BB1F.7030203@gmx.net>

On 07/19/2009 04:14 AM, Steven D'Aprano wrote:
> On Sun, 19 Jul 2009 11:44:00 am Mathias Panzenb?ck wrote:
>
>> I always thought it's odd that pythons list has append but
>> the set class has the method add. Whats the reason for that anyway?
>
> You can't append an item to the end of the set, because sets don't have
> an end. They're unordered.
>
>

Yes, but you can add an element to a list. That the position where it is added
is the end is under the interface of the super class just coincidence.

	-panzi


From guido at python.org  Mon Jul 20 02:58:15 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 19 Jul 2009 17:58:15 -0700
Subject: [Python-ideas] Where-statement (Proposal for function
	expressions)
In-Reply-To: <op.uxbuy4o9fvx12t@jerzozwiesz.home.aster.pl>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>
	<200907180024.31209.steve@pearwood.info>
	<5d1a32000907171328y6da57c7j30b5bfe154e90f32@mail.gmail.com> 
	<200907181039.57256.steve@pearwood.info>
	<3bdda690907172019hb70bda5h51d76ef31058423a@mail.gmail.com> 
	<4A614D9B.7060500@canterbury.ac.nz> <8763dqd1fm.fsf@benfinney.id.au> 
	<4A626377.8050001@canterbury.ac.nz>
	<ca471dc20907181814q4b76d527qc1cad3410f4df8d1@mail.gmail.com> 
	<op.uxbuy4o9fvx12t@jerzozwiesz.home.aster.pl>
Message-ID: <ca471dc20907191758y2703a1f2mfe2afad347fbc70@mail.gmail.com>

On Sun, Jul 19, 2009 at 11:29 AM, Jan Kaliszewski<zuo at chopin.edu.pl> wrote:
> Guido van Rossum <guido at python.org> wrote:
>
>> The 'where' proposal actually looks nice to me. (Maybe I've been
>> studying functional languages too much lately :-). My biggest problem
>> with this concrete proposal is that 'where' means something completely
>> different in SQL (which I've also studied too much lately :-).
>>
>> However, I think we should focus on keeping the language stable rather
>> than keep tinkering with it. Let's help 3rd party developers port
>> their work to 3.1 rather than planning 3.1's obsolescence.
>
> I believe it's at the other 'end for stick' of development process and
> doesn't hurt 3.1 at all.

Actually, adding too many new features post 3.1 *could* hurt 3.1 -- it
could slow adoption because people might decide to wait for 3.2 which
they expect to be even better. Plus the implementation of 3.2 features
uses up energy that would be better spent elsewhere (the Python
community has a chronic shortage of hands to help with menial task
like porting 3rd party code to 3.1).

Another argument against new features is that the people who have the
hardest time moving to 3.1 are those who are most bothered by change
in general. If they view the language as "moving too fast" they might
decide to move to a language that moves more slowly, or just stick
with Python 2.4, which is nearly as bad.

A more cynical view (the like of which I've heard expressed about Perl
6) would be that python-ideas is where we keep those folks occupied
who love to argue about new features but are unlikely to contribute
anything... :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)


From python at mrabarnett.plus.com  Mon Jul 20 03:23:08 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 20 Jul 2009 02:23:08 +0100
Subject: [Python-ideas] Where-statement (Proposal for
	function	expressions)
In-Reply-To: <ca471dc20907191758y2703a1f2mfe2afad347fbc70@mail.gmail.com>
References: <8ef5e1a3-9bf7-4269-a562-d76b42ac1165@d4g2000vbm.googlegroups.com>	<200907180024.31209.steve@pearwood.info>	<5d1a32000907171328y6da57c7j30b5bfe154e90f32@mail.gmail.com>
	<200907181039.57256.steve@pearwood.info>	<3bdda690907172019hb70bda5h51d76ef31058423a@mail.gmail.com>
	<4A614D9B.7060500@canterbury.ac.nz>
	<8763dqd1fm.fsf@benfinney.id.au>
	<4A626377.8050001@canterbury.ac.nz>	<ca471dc20907181814q4b76d527qc1cad3410f4df8d1@mail.gmail.com>
	<op.uxbuy4o9fvx12t@jerzozwiesz.home.aster.pl>
	<ca471dc20907191758y2703a1f2mfe2afad347fbc70@mail.gmail.com>
Message-ID: <4A63C6FC.7040408@mrabarnett.plus.com>

Guido van Rossum wrote:
> On Sun, Jul 19, 2009 at 11:29 AM, Jan Kaliszewski<zuo at chopin.edu.pl> wrote:
>> Guido van Rossum <guido at python.org> wrote:
>>
>>> The 'where' proposal actually looks nice to me. (Maybe I've been
>>> studying functional languages too much lately :-). My biggest problem
>>> with this concrete proposal is that 'where' means something completely
>>> different in SQL (which I've also studied too much lately :-).
>>>
>>> However, I think we should focus on keeping the language stable rather
>>> than keep tinkering with it. Let's help 3rd party developers port
>>> their work to 3.1 rather than planning 3.1's obsolescence.
>> I believe it's at the other 'end for stick' of development process and
>> doesn't hurt 3.1 at all.
> 
> Actually, adding too many new features post 3.1 *could* hurt 3.1 -- it
> could slow adoption because people might decide to wait for 3.2 which
> they expect to be even better. Plus the implementation of 3.2 features
> uses up energy that would be better spent elsewhere (the Python
> community has a chronic shortage of hands to help with menial task
> like porting 3rd party code to 3.1).
> 
> Another argument against new features is that the people who have the
> hardest time moving to 3.1 are those who are most bothered by change
> in general. If they view the language as "moving too fast" they might
> decide to move to a language that moves more slowly, or just stick
> with Python 2.4, which is nearly as bad.
> 
> A more cynical view (the like of which I've heard expressed about Perl
> 6) would be that python-ideas is where we keep those folks occupied
> who love to argue about new features but are unlikely to contribute
> anything... :-)
> 
Or it's where the bikeshedding gets done; if an idea makes out of
python-ideas then any critic of the new feature can be pointed to the
relevant thread and told "already discussed, see!". :-)


From jnoller at gmail.com  Mon Jul 20 03:57:24 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Sun, 19 Jul 2009 21:57:24 -0400
Subject: [Python-ideas] Changing the default install location,
	script versioning (Packages 	and PEP 370)
Message-ID: <4222a8490907191857m2705b298gd329357ef6912169@mail.gmail.com>

So, I've finally been able to kick my python 2.5 boat anchor, and I'm
now relishing in the delightful future of 2.6. Two things struck me
though, digging into PEP 370's (per user site-packages) behavior.

The first is this - when you install a package (say, pip) into the
.local directory via the --user command, the package ends up in the
correct lib directory; but the binary get dropped into ./local/bin -
this seems wrong. The reason being, is that what if I also have python
2.7 (which i do) installed, as well as python 3.1 and the
release-which-will-not-be-named (3.0) - if I install that same package
into one of the other versions, a new binary would be written with the
same name - a script-pythonversion might also be installed, but the
fact that the original script was overwritten seems broken to me.

I think the "best" fix for this is to make the bin/ directory mirror
the lib layout - each version would get it's own bin directory:

.local/
    bin/
        python2.6/
        python3.1/
    lib/
        python2.6/
        python3.1/

Of course, doing this begs the question - why have /bin and /lib be
top-level directories, instead favoring a layout which looks like:

.local/
    python2.6/
        bin/
        lib/

The data and doc directories which some packages might include should
follow the same convention, to avoid conflicts in those directories as
well.

The second behavior is more of a "man I wish for a unicorn" type of
thing - I think that --user should be removed, and by removed, I mean
made the default action when "python setup.py install" is invoked.
Users should need to instead pass in a --global flag to alter the
behavior to install into the system's site-packages and bin
directories. The reason is simple - installation to the global site is
typically a super-user task, and can side-effect all users, it's
growing into more of a no-no as more OS vendors grow to rely on the
packaged/included versions of python.

.002 cents

jesse


From solipsis at pitrou.net  Mon Jul 20 10:32:35 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 20 Jul 2009 08:32:35 +0000 (UTC)
Subject: [Python-ideas]
	=?utf-8?q?Changing_the_default_install_location=2C?=
	=?utf-8?q?=09script_versioning_=28Packages_=09and_PEP_370=29?=
References: <4222a8490907191857m2705b298gd329357ef6912169@mail.gmail.com>
Message-ID: <loom.20090720T082847-338@post.gmane.org>

Jesse Noller <jnoller at ...> writes:
> 
> The reason being, is that what if I also have python
> 2.7 (which i do) installed, as well as python 3.1 and the
> release-which-will-not-be-named (3.0) - if I install that same package
> into one of the other versions, a new binary would be written with the
> same name - a script-pythonversion might also be installed, but the
> fact that the original script was overwritten seems broken to me.

A script is a script. What version of Python it runs with shouldn't be your (the
user's) concern. Do you have two Apache binaries, one compiled with gcc 3.x and
one compiled with gcc 4.x?

> I think the "best" fix for this is to make the bin/ directory mirror
> the lib layout - each version would get it's own bin directory:
> 
> .local/
>     bin/
>         python2.6/
>         python3.1/

-1. The point of .local/bin is that it's (supposedly) standard, so that you have
only one path to add to $PATH. Putting scripts in versioned subdirectories
totally defeats its purpose.





From doug.hellmann at gmail.com  Mon Jul 20 16:19:41 2009
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Mon, 20 Jul 2009 10:19:41 -0400
Subject: [Python-ideas] Changing the default install location,
	script versioning (Packages 	and PEP 370)
In-Reply-To: <loom.20090720T082847-338@post.gmane.org>
References: <4222a8490907191857m2705b298gd329357ef6912169@mail.gmail.com>
	<loom.20090720T082847-338@post.gmane.org>
Message-ID: <75F0120D-6B2B-4C38-9943-49CF8A1765E1@gmail.com>


On Jul 20, 2009, at 4:32 AM, Antoine Pitrou wrote:

> Jesse Noller <jnoller at ...> writes:
>>
>> The reason being, is that what if I also have python
>> 2.7 (which i do) installed, as well as python 3.1 and the
>> release-which-will-not-be-named (3.0) - if I install that same  
>> package
>> into one of the other versions, a new binary would be written with  
>> the
>> same name - a script-pythonversion might also be installed, but the
>> fact that the original script was overwritten seems broken to me.
>
> A script is a script. What version of Python it runs with shouldn't  
> be your (the
> user's) concern. Do you have two Apache binaries, one compiled with  
> gcc 3.x and
> one compiled with gcc 4.x?

That analogy doesn't hold because the compiler isn't selected at  
runtime.  The interpreter *is*, and at least with setuptools the  
script is hard-wired to run the version of the compiler used to  
install it (not just 2.6 vs. 2.7 but the actual path to the  
interpreter is embedded in the script).  So if you install one package  
with 2.6 and another with 2.7, you end up with mingled scripts but  
they (rightly) don't see the other libraries.  That's inconsistent.

>> I think the "best" fix for this is to make the bin/ directory mirror
>> the lib layout - each version would get it's own bin directory:
>>
>> .local/
>>    bin/
>>        python2.6/
>>        python3.1/
>
> -1. The point of .local/bin is that it's (supposedly) standard, so  
> that you have
> only one path to add to $PATH. Putting scripts in versioned  
> subdirectories
> totally defeats its purpose.

+1 because using versioned subdirectories lets me install apps that  
may only work under one of the versions of Python I have installed on  
my system.

Of course, a possibly better solution would be to have each app in its  
own directory with its own copy of its dependencies, but virtualenv  
makes that easy enough.

Doug



From solipsis at pitrou.net  Mon Jul 20 18:18:46 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 20 Jul 2009 16:18:46 +0000 (UTC)
Subject: [Python-ideas]
	=?utf-8?q?Changing_the_default_install_location=2C?=
	=?utf-8?q?=09script_versioning_=28Packages_=09and_PEP_370=29?=
References: <4222a8490907191857m2705b298gd329357ef6912169@mail.gmail.com>
	<loom.20090720T082847-338@post.gmane.org>
	<75F0120D-6B2B-4C38-9943-49CF8A1765E1@gmail.com>
Message-ID: <loom.20090720T160738-261@post.gmane.org>

Doug Hellmann <doug.hellmann at ...> writes:
> 
> +1 because using versioned subdirectories lets me install apps that  
> may only work under one of the versions of Python I have installed on  
> my system.

I don't follow the reasoning here. If app A only works with Python version 2.5,
install it using Python 2.5, and if app B only works with Python version 2.6,
install it using Python 2.6. The shebang line will point to the exact Python
version (using setuptools, that is), and the executable's location is obviously
orthogonal to selecting the proper Python interpreter.

If, on the other hand, for whatever reason you want to install the same
application A using two different Python interpreters, I think it's fair to say
that it's a very unusual need and that Python shouldn't go out of its way to
satisfy it (especially if it worsens the experience for everyone else).

(I understand that the few applications which are themselves dedicated to
package management, such as easy_install or pip, need some kind of
multi-versioning of the executable. But, at least for easy_install, it /already/
comes multi-versioned (easy_install-2.6, etc.) and, besides, this situation is
the exception, not the rule)

> Of course, a possibly better solution would be to have each app in its  
> own directory with its own copy of its dependencies, but virtualenv  
> makes that easy enough.

Yes, so bottom line: if you have very precise and exclusive requirements
regarding what ends up where, you might just as well create a virtualenv, or
manually override --install rather than require everyone else to switch to a
more complicated setup. ~/.local/bin is simple and standard, please don't change
that in order to better accomodate a couple of fringe use cases.




From doug.hellmann at gmail.com  Mon Jul 20 19:44:38 2009
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Mon, 20 Jul 2009 13:44:38 -0400
Subject: [Python-ideas] Changing the default install location,
	script versioning (Packages 	and PEP 370)
In-Reply-To: <loom.20090720T160738-261@post.gmane.org>
References: <4222a8490907191857m2705b298gd329357ef6912169@mail.gmail.com>
	<loom.20090720T082847-338@post.gmane.org>
	<75F0120D-6B2B-4C38-9943-49CF8A1765E1@gmail.com>
	<loom.20090720T160738-261@post.gmane.org>
Message-ID: <E7E1CAC0-A802-41A5-B47D-1A6B633F6C75@gmail.com>


On Jul 20, 2009, at 12:18 PM, Antoine Pitrou wrote:

> Doug Hellmann <doug.hellmann at ...> writes:
>> Of course, a possibly better solution would be to have each app in  
>> its
>> own directory with its own copy of its dependencies, but virtualenv
>> makes that easy enough.
>
> Yes, so bottom line: if you have very precise and exclusive  
> requirements
> regarding what ends up where, you might just as well create a  
> virtualenv, or
> manually override --install rather than require everyone else to  
> switch to a
> more complicated setup. ~/.local/bin is simple and standard, please  
> don't change
> that in order to better accomodate a couple of fringe use cases.

This comes up often enough, with people other than me, that calling it  
a "fringe case" seems like a mischaracterization.

Why is the lib directory ~/.local/lib/pythonVERSION if ~/.local isn't  
intended to support more than one version of the interpreter?

Doug



From ncoghlan at gmail.com  Mon Jul 20 22:29:15 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 21 Jul 2009 06:29:15 +1000
Subject: [Python-ideas] Changing the default install location,
 script versioning (Packages  and PEP 370)
In-Reply-To: <loom.20090720T082847-338@post.gmane.org>
References: <4222a8490907191857m2705b298gd329357ef6912169@mail.gmail.com>
	<loom.20090720T082847-338@post.gmane.org>
Message-ID: <4A64D39B.6040608@gmail.com>

Antoine Pitrou wrote:
> Jesse Noller <jnoller at ...> writes:
>> I think the "best" fix for this is to make the bin/ directory mirror
>> the lib layout - each version would get it's own bin directory:
>>
>> .local/
>>     bin/
>>         python2.6/
>>         python3.1/
> 
> -1. The point of .local/bin is that it's (supposedly) standard, so that you have
> only one path to add to $PATH. Putting scripts in versioned subdirectories
> totally defeats its purpose.

Also -1 because the -m switch was added to address exactly this problem
of interpreter version specific copies of scripts without needing a
proliferation of script files in the system bin directories.

That said, does distutils have the equivalent of Python's "make
altinstall" command to tell the distribution to install versioned
scripts (e.g. easy-install-2.6) without altering non-versioned symlinks
(e.g. easy-install)?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From jnoller at gmail.com  Mon Jul 20 23:18:33 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 20 Jul 2009 17:18:33 -0400
Subject: [Python-ideas] Changing the default install location,
	script 	versioning (Packages and PEP 370)
In-Reply-To: <4A64D39B.6040608@gmail.com>
References: <4222a8490907191857m2705b298gd329357ef6912169@mail.gmail.com>
	<loom.20090720T082847-338@post.gmane.org> <4A64D39B.6040608@gmail.com>
Message-ID: <4222a8490907201418m7a064d88ia6bc9617d2af129@mail.gmail.com>

On Mon, Jul 20, 2009 at 4:29 PM, Nick Coghlan<ncoghlan at gmail.com> wrote:
> Antoine Pitrou wrote:
>> Jesse Noller <jnoller at ...> writes:
>>> I think the "best" fix for this is to make the bin/ directory mirror
>>> the lib layout - each version would get it's own bin directory:
>>>
>>> .local/
>>> ? ? bin/
>>> ? ? ? ? python2.6/
>>> ? ? ? ? python3.1/
>>
>> -1. The point of .local/bin is that it's (supposedly) standard, so that you have
>> only one path to add to $PATH. Putting scripts in versioned subdirectories
>> totally defeats its purpose.
>
> Also -1 because the -m switch was added to address exactly this problem
> of interpreter version specific copies of scripts without needing a
> proliferation of script files in the system bin directories.
>
> That said, does distutils have the equivalent of Python's "make
> altinstall" command to tell the distribution to install versioned
> scripts (e.g. easy-install-2.6) without altering non-versioned symlinks
> (e.g. easy-install)?
>
> Cheers,
> Nick.

So python -m "setuptools.commands.easy_install" <args> or python
`which easy_install` is a-ok for people? I find it much easier to tell
someone "run easy_install" or "run pylint" rather than either one of
the previous examples.

I mean, if I have two versions of python (say I'm developing for 2.6
and 3.1) and I want my tool chain (pip, virtualenv, pylint, fabric,
and so on) to be installed into both, I have two choices:

1: as soon as I install into --user remove the unversioned binary name
(for example, pylint) and stick to pylint-2.6. This breaks down if the
upstream package doesn't add the versioned name of the script, which
mean I need to move it after install.

2. Skip scripts entirely, and execute the module by hand.

python -m may have been added to stop the proliferation of script
files - but from a pure usability standpoint, as someone who writes
libraries which contain executables standpoint, running "myfoo" is
much simpler to explain to noobs and easier to remember overall.

jesse


From p.f.moore at gmail.com  Mon Jul 20 23:38:48 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 20 Jul 2009 22:38:48 +0100
Subject: [Python-ideas] Changing the default install location,
	script 	versioning (Packages and PEP 370)
In-Reply-To: <4222a8490907201418m7a064d88ia6bc9617d2af129@mail.gmail.com>
References: <4222a8490907191857m2705b298gd329357ef6912169@mail.gmail.com>
	<loom.20090720T082847-338@post.gmane.org> <4A64D39B.6040608@gmail.com>
	<4222a8490907201418m7a064d88ia6bc9617d2af129@mail.gmail.com>
Message-ID: <79990c6b0907201438w3b759f6cob32c9007efaaaa1f@mail.gmail.com>

2009/7/20 Jesse Noller <jnoller at gmail.com>:
> So python -m "setuptools.commands.easy_install" <args> or python
> `which easy_install` is a-ok for people? I find it much easier to tell
> someone "run easy_install" or "run pylint" rather than either one of
> the previous examples.

No, but "python -m easy_install" is fine. The fact that setuptools
doesn't provide a form designed for use with python -m doesn't mean
that there's a problem with the feature, just that setuptools wasn't
designed with it in mind.

Paul.


From jafo at tummy.com  Mon Jul 20 23:47:05 2009
From: jafo at tummy.com (Sean Reifschneider)
Date: Mon, 20 Jul 2009 15:47:05 -0600
Subject: [Python-ideas]  Experiment: Adding "re" to string objects.
In-Reply-To: <4A61D9A3.8020701@gmail.com>
Message-ID: <20090720214705.GA29607@tummy.com>

I've combined the various discussions into a single "digest" of the
discussions and responded to them below.  Thanks so much everyone for
the discussion.

MRAB wrote:
>Why not drop the ".re" part? You would, however, then need a new name
>for the re split, eg "re_split".

Because of the "str.re.match()" and other attributes that depend on having
had str.re.match or search called, I thought it best to consolidate them
into a single area, to make it more obvious that they were related.

But, I'm not tied to that idea, it just seemed to make sense.

MRAB wrote:
>Or you could make the string the pattern, eg r'whatever(.*)'.match(s).

The problem with that is that it you already have the "s", but you may not
have a regex object.  So the above may lead to:

   regex = r'whatever(.*)'
   if regex.match(s):
      regex.group(1)

As compared with:

   if s.re.match(r'whatever(.*)'):
      s.re.group(1)

In which case we're probably talking about going the direction that Nick
mentioned below, with a helper class.

Nick Coghlan wrote:
>The idea of adding a mutable attribute to strings (even if it plays no
>part in string equality) sends shivers down my spine.

I'm having a hard time seeing this as a mutable attribute.  Now, this would
be unprecedented in that the value returned by the re.group() type calls
would vary depending on the re.match() type calls, nothing else has similar
sorts of side-effects on string objects.  But it just doesn't "feel
mutable" to me because of this.

>It also seems like
>it would be difficult to do this in a way that didn't increase the size
>of all strings by at least one pointer slot.

I gather that's a big deal?  I don't honestly know, to me it probably
isn't, but I'll admit there are many cases I don't really care about.  :-)

Nick Coghlan wrote:
>However, the idea of adding more convenience classes to the re module
>may still have some merit. In particular, when checking against multiple
>regexes, being able to do something like the following might be helpful:
>
>m = re.Matcher(s)
>if m.match(r'whatever(.*)'):
>      print m.group(1)
>elif m.match(r'something (.*) else(.*)'):
>      print m.group(2)

I think that's just another way of implementing the "restr()" class I made
in the "filtertools" module.  Possibly the biggest difference being that
the "restr()" is able to be used as a string, instead of having to do
something like:

   m = re.Matcher(s)
   if not m.match(r'whatever(.*)'):
      print "Didn't match line: '%s'" % m.origstr
(or:)
      print "Didn't match line: '%s'" % s

Christian Heimes wrote:
>* regular expressions are rarely used in Python. I have just a couple of
>scripts that use re

Oh really?

   guin:~$ cd ~/cvs/python-trunk/Lib
   guin:Lib$ find . -type f -name \*.py -exec egrep 'import re\>' '{}' + | wc -l
   137
   guin:Lib$ find . -type f -name \*.py -exec egrep 'import os\>' '{}' + | wc -l
   447
   guin:Lib$ find . -type f -name \*.py -exec egrep 'import sys\>' '{}' + | wc -l
   643
   guin:Lib$ find . -type f -name \*.py -exec egrep 'import time\>' '{}' + | wc -l
   142
   guin:Lib$ find . -type f -name \*.py -exec egrep 'import socket\>' '{}' + | wc -l
   54
   guin:Lib$ find . -type f -name \*.py -exec egrep 'import datetime\>' '{}' + | wc -l
   13
   guin:Lib$ find . -type f -name \*.py -exec egrep 'import math\>' '{}' + | wc -l
   23
   guin:Lib$ find . -type f -name \*.py -exec egrep 'import pickle\>' '{}' + | wc -l
   31
   guin:Lib$ 

Admittedly, this is a very crude metric, but I'm not sure it's fair to say
that regular expressions are rarely used.  YOU may rarely use them, but I
probably use them in one out of 3 Python programs I write.  I do a lot of
stuff where I process the output of other commands or log or text files
though...

Christian Heimes wrote:
>* we shouldn't encourage people in using re when there is a simpler
>solution. Python isn't Perl.

I know people like to say "Python isn't Perl", but one of the reasons Perl
and other languages make dealing with regexes so easy is that it's
something that is useful in a lot of cases.  Python isn't LISP or other
languages either, but we still take the good stuff from them.

With all due respect, "Python isn't Perl" seems more designed to invoke an
emotional response rather than be based on hard reasoning.

Christian Heimes wrote:
>* Several modules and a C extension must be loaded during *every*
>interpreter startup. Everybody must pay the speed penalty and the memory
>usage of the interpreter increases with every module, too. Lazy loading
>may be a workaround, though

I had expected this would be loaded lazily.  Possibly even to the extent
that it can address Nick's concern about using another pointer slot in
strings.  I think it can be resolved, but I don't have the solution at
hand.

Note that "str.re" is *NOT* just an import of "re" into the string module.
The implementation of restr() in filtertools uses an inner class that the
re attribute is an instance of, and could load the re module only when
used.

I would agree that if it had to cause re to be loaded for strings to be
used at interpreter startup, that it would make this idea unusable.

Christian Heimes wrote:
>* Beautiful is better than ugly.
>* Explicit is better than implicit.
>* Simple is better than complex.
>* Readability counts.

I'll presume you mean this tongue-in-cheek, because most of these arguments
could apply to this proposal, making some handling of regexes more
beautiful, simple, and readable.  Explicit?  Provide more details of what
you are thinking there...

Steven D'Aprano wrote:
>Apologies for the Metoo, but I'm with Nick and Christian on this. It 
>sounds like a terrible idea to me, just to avoid a temporary name in 
>the standard idiom:
>
>m = re.match(r'whatever(.*)', s)
>if m:
>    m.group(1)

It's not so much about adding a temporary name, it's about the above being
an ugly construct.  Particularly in more complex cases:

   m = re.match(r'whatever(.*)', s)
   if m:
       m.group(1)
   m = re.match(r'something else(.*)', s)
   if m:
       m.group(1)

instead of:

   if s.re.match(r'whatever(.*)') or s.re.match(r'something else(.*)'):
       s.re.group(1)

Georg Brandl wrote:
>That one looks very useful, and will prevent more proposals of new syntax
>along the lines of "if rex.match(...) as m" :)

Well, "if rex.match() as m" is more general, but if it's really the primary
reason for it, that's a good reason for it.  :-)

Antoine Pitrou wrote:
>-0.5. Right now, objects in the re module are constructed from a regular
>expression pattern -- one of the reasons being that these patterns are compiled
>to bytecode form, and the objects help retain the bytecode.

Sure, but the compiled regex cache in my testing pretty much eradicated any
performance reasons for compiling regexes.

Antoine Pitrou wrote:
>Having another
>object type constructed from the string-to-match is confusing.

I think you mean "the string-to-match against"?  I'm not sure that it's
really more confusing that way though.  It makes sense considering that you
can't assign and compare in Python, so there's some argument for it being
"more pythonic".

Antoine Pitrou wrote:
>Besides, keeping
>some kind of internal state about the last matched pattern, only for
>"convenience" purposes, isn't pretty either.

I guess that depends on whether you use convenience in the pejorative or
not.  :-)  This isn't the Most Manly Programming Contest (tm), making
programming more convenient is a GOOD thing.  It's kind of like the
caching of compiled regexes -- that's a convenience thing.

Jan Kaliszewski wrote:
>Maybe it should be limited to using compiled regexps, not strings?

I don't think so, though I could see it being able to take compiled
regexes as well as strings.  If you HAVE to compile the regexes then you're
already making a temporary object and I'm not sure how you would gain
anything from this pattern.

Thanks,
Sean
-- 
 Do bad programmers wake up on Christmas morning to find coal in
 their sockets?  -- Sean Reifschneider
Sean Reifschneider, Member of Technical Staff <jafo at tummy.com>
tummy.com, ltd. - Linux Consulting since 1995: Ask me about High Availability



From jnoller at gmail.com  Mon Jul 20 23:50:47 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 20 Jul 2009 17:50:47 -0400
Subject: [Python-ideas] Changing the default install location,
	script 	versioning (Packages and PEP 370)
In-Reply-To: <79990c6b0907201438w3b759f6cob32c9007efaaaa1f@mail.gmail.com>
References: <4222a8490907191857m2705b298gd329357ef6912169@mail.gmail.com>
	<loom.20090720T082847-338@post.gmane.org> <4A64D39B.6040608@gmail.com>
	<4222a8490907201418m7a064d88ia6bc9617d2af129@mail.gmail.com>
	<79990c6b0907201438w3b759f6cob32c9007efaaaa1f@mail.gmail.com>
Message-ID: <4222a8490907201450t478721aekbbb0a3d8e185f2f7@mail.gmail.com>

On Mon, Jul 20, 2009 at 5:38 PM, Paul Moore<p.f.moore at gmail.com> wrote:
> 2009/7/20 Jesse Noller <jnoller at gmail.com>:
>> So python -m "setuptools.commands.easy_install" <args> or python
>> `which easy_install` is a-ok for people? I find it much easier to tell
>> someone "run easy_install" or "run pylint" rather than either one of
>> the previous examples.
>
> No, but "python -m easy_install" is fine. The fact that setuptools
> doesn't provide a form designed for use with python -m doesn't mean
> that there's a problem with the feature, just that setuptools wasn't
> designed with it in mind.
>
> Paul.
>

I don't know if easy_install does or doesn't - I simply used it as an
example. What I don't parse is that python -m <module> is somehow a
replacement for ./script - the logic within a script can do a lot more
than just firing off the __main__ of a module. Are we saying that
"scripts are considered harmful" and recommend people only support -m
for this?

There's over 7000 packages, applications and libraries in the
cheeseshop right now. A fair number of them would run face first into
the non-versioned binary problem. I guess distutils (in a future
version) should just deprecate the scripts/entry points options
entirely?

I really don't think this is an edge case, or should be unsupported.
Sure, the same problem exists outside of the .local directory - you
run into this installing things into the default system-level
site-packages and /usr/bin /usr/local/bin directories, but there's no
real reason we can't make this work better in the context of .local

jesse


From p.f.moore at gmail.com  Tue Jul 21 00:11:15 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 20 Jul 2009 23:11:15 +0100
Subject: [Python-ideas] Changing the default install location,
	script 	versioning (Packages and PEP 370)
In-Reply-To: <4222a8490907201450t478721aekbbb0a3d8e185f2f7@mail.gmail.com>
References: <4222a8490907191857m2705b298gd329357ef6912169@mail.gmail.com>
	<loom.20090720T082847-338@post.gmane.org> <4A64D39B.6040608@gmail.com>
	<4222a8490907201418m7a064d88ia6bc9617d2af129@mail.gmail.com>
	<79990c6b0907201438w3b759f6cob32c9007efaaaa1f@mail.gmail.com>
	<4222a8490907201450t478721aekbbb0a3d8e185f2f7@mail.gmail.com>
Message-ID: <79990c6b0907201511r6b23b949vc62e08f53491bbef@mail.gmail.com>

2009/7/20 Jesse Noller <jnoller at gmail.com>:
> On Mon, Jul 20, 2009 at 5:38 PM, Paul Moore<p.f.moore at gmail.com> wrote:
>> 2009/7/20 Jesse Noller <jnoller at gmail.com>:
>>> So python -m "setuptools.commands.easy_install" <args> or python
>>> `which easy_install` is a-ok for people? I find it much easier to tell
>>> someone "run easy_install" or "run pylint" rather than either one of
>>> the previous examples.
>>
>> No, but "python -m easy_install" is fine. The fact that setuptools
>> doesn't provide a form designed for use with python -m doesn't mean
>> that there's a problem with the feature, just that setuptools wasn't
>> designed with it in mind.
>>
>> Paul.
>>
>
> I don't know if easy_install does or doesn't - I simply used it as an
> example. What I don't parse is that python -m <module> is somehow a
> replacement for ./script - the logic within a script can do a lot more
> than just firing off the __main__ of a module. Are we saying that
> "scripts are considered harmful" and recommend people only support -m
> for this?
>
> There's over 7000 packages, applications and libraries in the
> cheeseshop right now. A fair number of them would run face first into
> the non-versioned binary problem. I guess distutils (in a future
> version) should just deprecate the scripts/entry points options
> entirely?
>
> I really don't think this is an edge case, or should be unsupported.
> Sure, the same problem exists outside of the .local directory - you
> run into this installing things into the default system-level
> site-packages and /usr/bin /usr/local/bin directories, but there's no
> real reason we can't make this work better in the context of .local

My perspective is somewhat different, as I'm a Windows user, and many,
many cases where scripts are used don't work cross-platform.

Note: If you have a pure-python script foo.py, distributed with
package bar, why not just put foo.py on sys.path as part of the
install, and run it as python -m foo? How is that appreciably harder
than putting it in a bin directory?

Some cross-platform notes, which may not be obvious to people (if they
are obvious to you, my apologies - I don't mean to be patronising):

- Many Unix users don't like their scripts to end in .py
- Scripts that don't end in .py don't work on Windows
- .bat file wrappers on Windows have "odd" properties
- You made the point that the #! line is an issue on Unix

python -m module is cross-platform, and properly linked to the Python
version. It's less transparent than an executable script, but you have
to pick a compromise somewhere (and I appreciate that cross-platform
isn't always foremost in people's minds).

The biggest problem with -m is that not enough packages are designed
to support it. I hope I can add a "yet" to that, but it needs more
people seeing the benefits (you Unix people, remember us poor Windows
users! :-) :-)) if that's to happen.

Paul.

PS If you really want an executable script, you can always write an
alias or one-line shell script to do "python -m foo"... I don't know
if that suits your requirement.

PPS This is a defence of -m only, I don't have a view on the issue of
whether .local/bin should be versioned, as I don't use the feature
myself.


From dstanek at dstanek.com  Tue Jul 21 00:33:14 2009
From: dstanek at dstanek.com (David Stanek)
Date: Mon, 20 Jul 2009 18:33:14 -0400
Subject: [Python-ideas] Changing the default install location,
	script 	versioning (Packages and PEP 370)
In-Reply-To: <4222a8490907201450t478721aekbbb0a3d8e185f2f7@mail.gmail.com>
References: <4222a8490907191857m2705b298gd329357ef6912169@mail.gmail.com>
	<loom.20090720T082847-338@post.gmane.org> <4A64D39B.6040608@gmail.com>
	<4222a8490907201418m7a064d88ia6bc9617d2af129@mail.gmail.com>
	<79990c6b0907201438w3b759f6cob32c9007efaaaa1f@mail.gmail.com>
	<4222a8490907201450t478721aekbbb0a3d8e185f2f7@mail.gmail.com>
Message-ID: <de32cc030907201533p5a41b785td40de0b6de7140e7@mail.gmail.com>

On Mon, Jul 20, 2009 at 5:50 PM, Jesse Noller<jnoller at gmail.com> wrote:
> On Mon, Jul 20, 2009 at 5:38 PM, Paul Moore<p.f.moore at gmail.com> wrote:
>> 2009/7/20 Jesse Noller <jnoller at gmail.com>:
>>> So python -m "setuptools.commands.easy_install" <args> or python
>>> `which easy_install` is a-ok for people? I find it much easier to tell
>>> someone "run easy_install" or "run pylint" rather than either one of
>>> the previous examples.
>>
>> No, but "python -m easy_install" is fine. The fact that setuptools
>> doesn't provide a form designed for use with python -m doesn't mean
>> that there's a problem with the feature, just that setuptools wasn't
>> designed with it in mind.
>>
>> Paul.
>>
>
> I don't know if easy_install does or doesn't - I simply used it as an
> example. What I don't parse is that python -m <module> is somehow a
> replacement for ./script - the logic within a script can do a lot more
> than just firing off the __main__ of a module. Are we saying that
> "scripts are considered harmful" and recommend people only support -m
> for this?

I'm personally OK with saying that scripts are no longer directly
supported. That means that users will be forced to run scripts like:
  python -m mypackage.script -- --my-args

Is this accessible to new users and Windows users?

>
> There's over 7000 packages, applications and libraries in the
> cheeseshop right now. A fair number of them would run face first into
> the non-versioned binary problem. I guess distutils (in a future
> version) should just deprecate the scripts/entry points options
> entirely?

This is what scares me about using 'python -m'. I think you would have
to do this.

>
> I really don't think this is an edge case, or should be unsupported.
> Sure, the same problem exists outside of the .local directory - you
> run into this installing things into the default system-level
> site-packages and /usr/bin /usr/local/bin directories, but there's no
> real reason we can't make this work better in the context of .local

I currently have this issue all over the place. Every time I install
software on my Buildbot I have to remember to move the script  xyz to
xyz$PYVER. I'm running 4 different versions of Python and as you can
imagine this issues frustrates me.

-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek


From nad at acm.org  Tue Jul 21 00:59:55 2009
From: nad at acm.org (Ned Deily)
Date: Mon, 20 Jul 2009 15:59:55 -0700
Subject: [Python-ideas] Changing the default install location,
	script 	versioning (Packages and PEP 370)
References: <4222a8490907191857m2705b298gd329357ef6912169@mail.gmail.com>
	<loom.20090720T082847-338@post.gmane.org>
	<4A64D39B.6040608@gmail.com>
	<4222a8490907201418m7a064d88ia6bc9617d2af129@mail.gmail.com>
	<79990c6b0907201438w3b759f6cob32c9007efaaaa1f@mail.gmail.com>
	<4222a8490907201450t478721aekbbb0a3d8e185f2f7@mail.gmail.com>
Message-ID: <nad-1D35D0.15595420072009@news.gmane.org>

Jesse Noller wrote:
> Of course, doing this begs the question - why have /bin and /lib be
> top-level directories, instead favoring a layout which looks like:
> 
> .local/
>     python2.6/
>         bin/
>         lib/
> 
> The data and doc directories which some packages might include should
> follow the same convention, to avoid conflicts in those directories as
> well.

As you know, Jesse, there is a precedent for this kind of layout:  
Python framework installs on Mac OS X.  For those not familiar with it, 
the directory layout is something like this:

{/System/Library/ | /Library | ~/Library }/Frameworks/Python.framework/
   2.6/
       bin/
       include/
       lib/
       share/
   3.1/
       bin/
       include/
              ...

where
  /System/Library/... contains the Apple-supplied Python(s),
  /Library/... contains the sysadmin-installed set of Pythons
                   (for example, the default for python.org installers)
  ~/Library/... contains user-installed Pythons.

So, not only do framework installs support multiple versions at the user 
level, they do so at the system and vendor level as well, and support 
multiple versions of essentially all Python-installed objects (i.e. 
shared libraries, executables, scripts, etc).  (A side note: with OS X 
Python frameworks builds, ~/.local really isn't needed since 
~/Library/Frameworks/... already provides its functionality and more.)

As with any scheme supporting multiple versions, a downside to this is 
potential user confusion over how to manage access to these multiple 
versions.  For those who don't regularly follow Python on OS X matters, 
this continues to be the source of some of the most frequent support 
questions.  Those who understand the concept of search $PATH can figure 
out how to deal with it a lot easier than those who don't.

As a convenience, the python.org 2.x and 3.1 installers by default also 
install versioned symbolic links in /usr/local/bin to the standard 
python command line objects in their respective framework bin 
directories., i.e. pythonx.x, pydocx.x, idlex.x.  Something similar 
could be done automatically for distutil-installed scripts, of course.

-- 
 Ned Deily,
 nad at acm.org



From brett at python.org  Tue Jul 21 01:23:59 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 20 Jul 2009 16:23:59 -0700
Subject: [Python-ideas] Changing the default install location,
	script 	versioning (Packages and PEP 370)
In-Reply-To: <4222a8490907201450t478721aekbbb0a3d8e185f2f7@mail.gmail.com>
References: <4222a8490907191857m2705b298gd329357ef6912169@mail.gmail.com> 
	<loom.20090720T082847-338@post.gmane.org> <4A64D39B.6040608@gmail.com> 
	<4222a8490907201418m7a064d88ia6bc9617d2af129@mail.gmail.com> 
	<79990c6b0907201438w3b759f6cob32c9007efaaaa1f@mail.gmail.com> 
	<4222a8490907201450t478721aekbbb0a3d8e185f2f7@mail.gmail.com>
Message-ID: <bbaeab100907201623x7cf63d33ked1f6363429e6937@mail.gmail.com>

On Mon, Jul 20, 2009 at 14:50, Jesse Noller <jnoller at gmail.com> wrote:

> On Mon, Jul 20, 2009 at 5:38 PM, Paul Moore<p.f.moore at gmail.com> wrote:
> > 2009/7/20 Jesse Noller <jnoller at gmail.com>:
> >> So python -m "setuptools.commands.easy_install" <args> or python
> >> `which easy_install` is a-ok for people? I find it much easier to tell
> >> someone "run easy_install" or "run pylint" rather than either one of
> >> the previous examples.
> >
> > No, but "python -m easy_install" is fine. The fact that setuptools
> > doesn't provide a form designed for use with python -m doesn't mean
> > that there's a problem with the feature, just that setuptools wasn't
> > designed with it in mind.
> >
> > Paul.
> >
>
> I don't know if easy_install does or doesn't - I simply used it as an
> example. What I don't parse is that python -m <module> is somehow a
> replacement for ./script - the logic within a script can do a lot more
> than just firing off the __main__ of a module. Are we saying that
> "scripts are considered harmful" and recommend people only support -m
> for this?
>
> There's over 7000 packages, applications and libraries in the
> cheeseshop right now. A fair number of them would run face first into
> the non-versioned binary problem. I guess distutils (in a future
> version) should just deprecate the scripts/entry points options
> entirely?
>
> I really don't think this is an edge case, or should be unsupported.
> Sure, the same problem exists outside of the .local directory - you
> run into this installing things into the default system-level
> site-packages and /usr/bin /usr/local/bin directories, but there's no
> real reason we can't make this work better in the context of .local


Taking Paul's follow-up email to this into account, I think we should
definitely encourage people support runpy, but that doesn't do away with the
usefulness of scripts. Ignoring the convenience factor of having short
script names (``hg tip`` is much shorter than ``python2.6 -m mercurial --
tip``), there is also having to remember which interpreter you installed a
tool under. Did I install hg under my Python 2.3, 2.4, 2.5 or 2.6
interpreter? I actually had to check the hg script to find that out.

And there is also the situation where people are not targeting multiple
operating systems and thus have a use for the features a script gives. If I
am writing some Linux app in Python I might have some need to run something
under bash.

So while I totally support pushing people to start using runpy now that
__main__.py support exists for packages, I don't think we can just fluff off
script support.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090720/32625593/attachment.html>

From steve at pearwood.info  Tue Jul 21 01:52:58 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 21 Jul 2009 09:52:58 +1000
Subject: [Python-ideas] Experiment: Adding "re" to string objects.
In-Reply-To: <20090720214705.GA29607@tummy.com>
References: <20090720214705.GA29607@tummy.com>
Message-ID: <200907210952.59570.steve@pearwood.info>

On Tue, 21 Jul 2009 07:47:05 am Sean Reifschneider wrote:

>    if s.re.match(r'whatever(.*)'):
>       s.re.group(1)

To me, the above is far less attractive than the standard idiom. That's 
an aesthetic judgement which you might disagree with, but I believe 
there's a far more critical flaw in the above idiom: it operates by 
side-effect in an unsafe way.

Consider:

s = "yabba dabba doo"
if s.re.match(r'y.bba'):
    function(s)
    print s.re.group(0)

You might expect the above to print 'yabba', but consider:

def function(s):
    if s.re.match(r'.*(dabba)'):
        log(s)

And now the snippet above will mysteriously print "yabba dabba" instead.

It's (presumably) easy enough to work around this:

s = "yabba dabba doo"
if s.re.match(r'y.bba'):
    # Save a copy of the re.group in case it gets mutated
    group = s.re.group(0)
    print function(s)
    print group


but that:

(1) spoils what little convenience your proposal did have; 
(2) will lead to confusion when people discover that "Python's regexes 
are broken"; and
(3) probably means that you simply cannot use s.re.* in threads at all.


> I'm having a hard time seeing this as a mutable attribute.  Now, this
> would be unprecedented in that the value returned by the re.group()
> type calls would vary depending on the re.match() type calls, nothing
> else has similar sorts of side-effects on string objects.  But it
> just doesn't "feel mutable" to me because of this.

s.re.group changes it's value according to whether or not you have 
called s.re.match() or s.re.search(). Every time you call s.re.match() 
with a different argument, s.re.group() potentially changes its value. 
Why is this not mutable?


> Christian Heimes wrote:
> >* regular expressions are rarely used in Python. I have just a
> > couple of scripts that use re
>
> Oh really?

I won't speak for Christian, but in my opinion, regexes are overused in 
Python, by the sort of people who prefer to write:

import re
if re.match(r'.*\.py$', s):
    ...

instead of:

if s.endswith('.py'):
    ...


Hang around comp.lang.python for long enough, and you too will get a 
very jaundiced view of regexes being misused, usually by people who 
have come from a Perl background.


> Steven D'Aprano wrote:
> >Apologies for the Metoo, but I'm with Nick and Christian on this. It
> >sounds like a terrible idea to me, just to avoid a temporary name in
> >the standard idiom:
> >
> >m = re.match(r'whatever(.*)', s)
> >if m:
> >    m.group(1)
>
> It's not so much about adding a temporary name, it's about the above
> being an ugly construct.  Particularly in more complex cases:
>
>    m = re.match(r'whatever(.*)', s)
>    if m:
>        m.group(1)
>    m = re.match(r'something else(.*)', s)
>    if m:
>        m.group(1)
>
> instead of:
>
>    if s.re.match(r'whatever(.*)') or s.re.match(r'something
> else(.*)'): s.re.group(1)

Time for a convenience function:

# Untested
def multimatch(s, patterns):
    """Do a re.match on s using each pattern in patterns, 
    returning the first one to succeed, or None if they all fail."""
    for pattern in patterns:
        m = re.match(pattern, s)
        if m: return m


Perhaps that, and the obvious multisearch() function, should be added to 
the re module.


-- 
Steven D'Aprano


From m.lenzen at gmail.com  Tue Jul 21 05:40:40 2009
From: m.lenzen at gmail.com (Michael Lenzen)
Date: Mon, 20 Jul 2009 22:40:40 -0500
Subject: [Python-ideas] Proposal for extending the collections module -
 bags / multisets, ordered sets / unique lists
In-Reply-To: <4A612B47.3050706@gmail.com>
References: <4A612B47.3050706@gmail.com>
Message-ID: <4A6538B8.6090609@gmail.com>

On 07/17/2009 08:54 PM, Michael Lenzen wrote:
> In a nutshell, I want to add 2 new classes (and respective frozen
> varients) to the collections module, namely a bag (multiset) class and a
> setlist (ordered set/unique list) class. I know this has been floated
> before, but I think it merits more consideration.

So I have pared down my proposition to just adding bag and setlist 
classes, without the additional ABCs, modifications to set and 
collection factory.  I know I still haven't posted use cases, but I'm 
hoping supporters of this will post some too.  I haven't given up on the 
idea of a unified Collections interface and a collection factory, but 
baby steps are probably better.

My implementation is here:
http://python-data-structures.googlecode.com/svn/trunk/collections_.py

It is a drop in replacement for collections, so
 >>> import collections_ as collections
or
 >>> from collections_ import bag, setlist

If I should post this to another more appropriate list, just let me know 
where.

I plan on creating a package and submitting it to pypi , but I would 
like other people to test it first.

-Michael Lenzen


From solipsis at pitrou.net  Tue Jul 21 10:49:19 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 21 Jul 2009 08:49:19 +0000 (UTC)
Subject: [Python-ideas]
	=?utf-8?q?Changing_the_default_install_location=2C?=
	=?utf-8?q?=09script_versioning_=28Packages_=09and_PEP_370=29?=
References: <4222a8490907191857m2705b298gd329357ef6912169@mail.gmail.com>
	<loom.20090720T082847-338@post.gmane.org>
	<75F0120D-6B2B-4C38-9943-49CF8A1765E1@gmail.com>
	<loom.20090720T160738-261@post.gmane.org>
	<E7E1CAC0-A802-41A5-B47D-1A6B633F6C75@gmail.com>
Message-ID: <loom.20090721T084624-784@post.gmane.org>

Doug Hellmann <doug.hellmann at ...> writes:
> 
> Why is the lib directory ~/.local/lib/pythonVERSION if ~/.local isn't  
> intended to support more than one version of the interpreter?

Because scripts specify which version of the interpreter they use in their
shebang line, so they don't need a versioned location.
Conversely, libraries have no way to specify for which version of the
interpreter they were installed, so their location needs to be
versioned.
(besides, it is common for a library to be installed for several interpreters
because it may be required by several different apps)

This is the same difference as between /usr/bin and
/usr/lib/python-X.Y/site-packages.

Regards

Antoine.




From solipsis at pitrou.net  Tue Jul 21 11:01:01 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 21 Jul 2009 09:01:01 +0000 (UTC)
Subject: [Python-ideas]
	=?utf-8?q?Changing_the_default_install_location=2C?=
	=?utf-8?q?=09script=09versioning_=28Packages_and_PEP_370=29?=
References: <4222a8490907191857m2705b298gd329357ef6912169@mail.gmail.com>
	<loom.20090720T082847-338@post.gmane.org>
	<4A64D39B.6040608@gmail.com>
	<4222a8490907201418m7a064d88ia6bc9617d2af129@mail.gmail.com>
	<79990c6b0907201438w3b759f6cob32c9007efaaaa1f@mail.gmail.com>
	<4222a8490907201450t478721aekbbb0a3d8e185f2f7@mail.gmail.com>
Message-ID: <loom.20090721T085356-10@post.gmane.org>

Jesse Noller <jnoller at ...> writes:
> 
> I don't know if easy_install does or doesn't - I simply used it as an
> example. What I don't parse is that python -m <module> is somehow a
> replacement for ./script - the logic within a script can do a lot more
> than just firing off the __main__ of a module. Are we saying that
> "scripts are considered harmful" and recommend people only support -m
> for this?

I don't think so. I interpret Paul's suggestion as "if you need a
multi-versioned script, then you can use the 'python -m' feature instead". But
you can continue running scripts in the other cases.

> There's over 7000 packages, applications and libraries in the
> cheeseshop right now. A fair number of them would run face first into
> the non-versioned binary problem.

Well, that's your contention actually :)
My contention is that a /very small number of them/ would run into the
non-versioned binary problem, because only applications which are themselves
used for deployment (easy_install, pip, etc.) may need to be invoked with an
explicit version of the interpreter.

Once again, I don't understand why you care which version of the interpreter
random application X works with (for example Mercurial, as in Brett's message).
/Apart/ from easy_install, pip and friends, that is.

> I guess distutils (in a future
> version) should just deprecate the scripts/entry points options
> entirely?

Certainly not. Perhaps it could grow an option to generate versioned scripts
easily, however. By the way, easy_install already comes versioned on the systems
I work on (you can run `easy_install-2.6` explicitly, for example).

> I really don't think this is an edge case, or should be unsupported.
> Sure, the same problem exists outside of the .local directory - you
> run into this installing things into the default system-level
> site-packages and /usr/bin /usr/local/bin directories, but there's no
> real reason we can't make this work better in the context of .local

The real reason, IMO and as I've said, is that it makes life harder for people
who don't care.




From lie.1296 at gmail.com  Tue Jul 21 13:55:57 2009
From: lie.1296 at gmail.com (Lie Ryan)
Date: Tue, 21 Jul 2009 21:55:57 +1000
Subject: [Python-ideas] Experiment: Adding "re" to string objects.
In-Reply-To: <200907210952.59570.steve@pearwood.info>
References: <20090720214705.GA29607@tummy.com>
	<200907210952.59570.steve@pearwood.info>
Message-ID: <h44acg$8jq$1@ger.gmane.org>

Steven D'Aprano wrote:
> On Tue, 21 Jul 2009 07:47:05 am Sean Reifschneider wrote:
> 
>>    if s.re.match(r'whatever(.*)'):
>>       s.re.group(1)
> 
> To me, the above is far less attractive than the standard idiom. That's 
> an aesthetic judgement which you might disagree with, but I believe 
> there's a far more critical flaw in the above idiom: it operates by 
> side-effect in an unsafe way.

I agree with your concerns. Then how about restricting so
str.match/str.re.match would only return True or False? Basically,
str.match/str.re.match is only used to determine whether a string
matches a regular expression and not to extract information out of the
string.



From ncoghlan at gmail.com  Tue Jul 21 15:12:40 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 21 Jul 2009 23:12:40 +1000
Subject: [Python-ideas] Proposal for extending the collections module -
 bags / multisets, ordered sets / unique lists
In-Reply-To: <4A61E633.4050301@gmail.com>
References: <4A612B47.3050706@gmail.com>
	<4A613A4A.3010607@gmail.com>		<4A615622.9090308@gmail.com>		<50697b2c0907172318t4de4b425md996ff37a7170b01@mail.gmail.com>	<50697b2c0907172330k5c7dc88aj3adc8d463993708@mail.gmail.com>
	<4A61E633.4050301@gmail.com>
Message-ID: <4A65BEC8.4020205@gmail.com>

Michael Lenzen wrote:
> 
> On 07/18/2009 01:30 AM, Chris Rebert wrote:
>> On Fri, Jul 17, 2009 at 11:18 PM, Chris Rebert<pyideas at rebertia.com> 
>> wrote:
>>> Truth be told, it's more than just defaultdict(int). It adds
>>> .elements() and .most_common().
>>>
>>> Seems bag-like to me.
>>> - Unordered? Check.
>>> - Allows duplicates? Check.
>>> - O(1) containment test? Check.
>>> - Counts multiplicity of elements? Check.
>>> - Iterable? Check.
>>>
>>> The only non-bag thing about it is allowing 0 and negative
>>> multiplicities, which I agree is unintuitive; I don't like that
>>> "feature" either.
>>
>> Actually, from the docs, it also appears (I don't have 3.0 handy to
>> test) to get len() wrong, using the dict definition of "number of
>> unique items" rather than just "number of items" as would be more
>> appropriate for a bag.
>>
>> In the event a Bag is not added, +1 for adding a method to Counter to
>> return `sum(count if count>  0 else 0 for count in
>> a_counter.values())`
>>
>> Cheers,
>> Chris
> 
> 
> As well as getting len() wrong, it gets iteration wrong.  It iterates
> over elements with counts of 0 and -1 as well as only iterating once
> over elements that appear multiple times.  Yes you can iterate over
> .elements(), but this should be the default not a special case.
> 
> As for adding most_common, it just calls
> heapq.nlargest(n, self.items(), key=_itemgetter(1))
> which anyone can do, and my bag class does.
> 
> My bag class behaves like a collection and provides a .unique_elements()
> method that returns the underlying set.  You can .add(elem) and
> .delete(elem) just like you can with a set, or you can manually change
> their multiplicities like in Counter with bag[elem] = 5 or bag[elem] -= 2.
> 
> If Counter is supposed to be a collection of elements, this makes no sense:
>>>> c = Counter()
>>>> c['a'] += 1
>>>> c['a'] -= 1
>>>> 'a' in c
> True

I encourage you to put your questions/concerns regarding the new
collections.Counter class into a separate email and send them to
python-dev. It seems to me that it is possible some revisions could be
made to the API. Whether or not that happens will depend on the precise
use cases Raymond had in mind when he added it, but even if nothing
changes such an email thread should provide some more insight into the
rationale driving the API design choices.

Although rather than calling the current API wrong from the outset, I'd
suggest phrasing it as asking "why is the interface this way?". We don't
know whether or not the API is actually wrong without knowing the
objectives Raymond was setting out to achieve.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From p.f.moore at gmail.com  Tue Jul 21 19:36:50 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 21 Jul 2009 18:36:50 +0100
Subject: [Python-ideas] Changing the default install location,
	script 	versioning (Packages and PEP 370)
In-Reply-To: <bbaeab100907201623x7cf63d33ked1f6363429e6937@mail.gmail.com>
References: <4222a8490907191857m2705b298gd329357ef6912169@mail.gmail.com>
	<loom.20090720T082847-338@post.gmane.org> <4A64D39B.6040608@gmail.com>
	<4222a8490907201418m7a064d88ia6bc9617d2af129@mail.gmail.com>
	<79990c6b0907201438w3b759f6cob32c9007efaaaa1f@mail.gmail.com>
	<4222a8490907201450t478721aekbbb0a3d8e185f2f7@mail.gmail.com>
	<bbaeab100907201623x7cf63d33ked1f6363429e6937@mail.gmail.com>
Message-ID: <79990c6b0907211036g40886edfif05b0277274daeb7@mail.gmail.com>

2009/7/21 Brett Cannon <brett at python.org>:
> Taking Paul's follow-up email to this into account, I think we should
> definitely encourage people support runpy, but that doesn't do away with the
> usefulness of scripts. Ignoring the convenience factor of having short
> script names (``hg tip`` is much shorter than ``python2.6 -m mercurial --
> tip``), there is also having to remember which interpreter you installed a
> tool under. Did I install hg under my Python 2.3, 2.4, 2.5 or 2.6
> interpreter? I actually had to check the hg script to find that out.
> And there is also the situation where people are not targeting multiple
> operating systems and thus have a use for the features a script gives. If I
> am writing some Linux app in Python I might have some need to run something
> under bash.
> So while I totally support pushing people to start using runpy now that
> __main__.py support exists for packages, I don't think we can just fluff off
> script support.

Just to clarify - my personal view ("tainted" by Windows experience,
certainly) is that *applications* (such as Mercurial) should be built
as isolated, standalone executables, bundled with something like
py2exe. I fully appreciate that this is the precise opposite of common
practice on Linux (see below for some uninformed speculation on that).

I see runpy as being appropriate for the types of small "supporting"
scripts that come with general-use packages - things like Twisted's
twistd, Nose's nosetests, etc. Essentially, where the package is the
key thing, and the command is a related utility. When the command is
the main focus, I see that as an "application".

I don't know the right answer for "applications" under Linux, but I
don't see any reason why a wrapper shell script wouldn't work - it's
no less portable than a py2exe built executable. Installing a wrapper
is then the responsibility of the packaging system, rather than
Python. But as I say, my expertise is basically zero here.

Paul.


From jafo at tummy.com  Tue Jul 21 22:32:17 2009
From: jafo at tummy.com (Sean Reifschneider)
Date: Tue, 21 Jul 2009 14:32:17 -0600
Subject: [Python-ideas] Experiment: Adding "re" to string objects.
In-Reply-To: <200907210952.59570.steve@pearwood.info>
References: <20090720214705.GA29607@tummy.com>
	<200907210952.59570.steve@pearwood.info>
Message-ID: <4A6625D1.5090007@tummy.com>

On 07/20/2009 05:52 PM, Steven D'Aprano wrote:
> there's a far more critical flaw in the above idiom: it operates by 
> side-effect in an unsafe way.

But this is hardly unprecedented.

   def f(arg): arg.append('world')
   l = []
   f(l)
   l.append('hello')
   l[0] == 'hello'

The difference being that strings are immutable and lists are not, but I
don't see a particular concern with string.re being mutable, while still
keeping strings immutable.  If you call a function with side-effects, it's
going to result in side-effects.  That's the nature of the beast, and just
a part of programming.

But, I agree that it would be ideal if we could prevent this in most cases.
Do you have an idea about how to prevent this?

> Why is this not mutable?

I understand that the string.re is mutable, but saying that this "makes
strings mutable" is kind of misleading...

> Hang around comp.lang.python for long enough, and you too will get a 
> very jaundiced view of regexes being misused, usually by people who 
> have come from a Perl background.

Don't take this the wrong way, but...  Why do we care?  There will always
be people who misuse things.  If they are more familiar with using an re
than "endswith", we can certainly educate them when they are looking for
it, but...

Why should we penalize people who are not abusing regular expressions just
because some people might?

> Perhaps that, and the obvious multisearch() function, should be added to 
> the re module.

Agreed.

Sean
-- 
Sean Reifschneider, Member of Technical Staff <jafo at tummy.com>
tummy.com, ltd. - Linux Consulting since 1995: Ask me about High Availability

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 252 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090721/709b26e2/attachment.pgp>

From ncoghlan at gmail.com  Tue Jul 21 23:16:56 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 22 Jul 2009 07:16:56 +1000
Subject: [Python-ideas] Experiment: Adding "re" to string objects.
In-Reply-To: <4A6625D1.5090007@tummy.com>
References: <20090720214705.GA29607@tummy.com>	<200907210952.59570.steve@pearwood.info>
	<4A6625D1.5090007@tummy.com>
Message-ID: <4A663048.4080106@gmail.com>

Sean Reifschneider wrote:
>> Why is this not mutable?
> 
> I understand that the string.re is mutable, but saying that this "makes
> strings mutable" is kind of misleading...

Because it is blurring the distinction: while having a mutable attribute
that plays no part in equality checks technically preserves the
immutability of the strings themselves, it does mean that there are now
some operations on a string that alter the internal state of that
specific string. Better to have a separate type that is known to have
mutable internal state and hence may not be safe to share between
different operations.

It also just occurred to me that string interning completely destroys
any concept of storing mutable attributes on strings:

.>>> x = "aString"
.>>> y = "aString"
.>>> x is y
True


.>>> x = intern("I am a string")
.>>> y = intern("I am a string")
.>>> x is y
True

You can't make this safe without removing the intern() operation along
with the implicit interning of "identifier-like" strings. And that is
too critical to the normal operation of the language to ever happen.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Tue Jul 21 23:19:26 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 22 Jul 2009 07:19:26 +1000
Subject: [Python-ideas] Experiment: Adding "re" to string objects.
In-Reply-To: <200907210952.59570.steve@pearwood.info>
References: <20090720214705.GA29607@tummy.com>
	<200907210952.59570.steve@pearwood.info>
Message-ID: <4A6630DE.8050403@gmail.com>

Steven D'Aprano wrote:
> On Tue, 21 Jul 2009 07:47:05 am Sean Reifschneider wrote:
> 
>>    if s.re.match(r'whatever(.*)'):
>>       s.re.group(1)
> 
> To me, the above is far less attractive than the standard idiom. That's 
> an aesthetic judgement which you might disagree with, but I believe 
> there's a far more critical flaw in the above idiom: it operates by 
> side-effect in an unsafe way.
> 
> Consider:
> 
> s = "yabba dabba doo"
> if s.re.match(r'y.bba'):
>     function(s)
>     print s.re.group(0)
> 
> You might expect the above to print 'yabba', but consider:
> 
> def function(s):
>     if s.re.match(r'.*(dabba)'):
>         log(s)
> 
> And now the snippet above will mysteriously print "yabba dabba" instead.
> 
> It's (presumably) easy enough to work around this:
> 
> s = "yabba dabba doo"
> if s.re.match(r'y.bba'):
>     # Save a copy of the re.group in case it gets mutated
>     group = s.re.group(0)
>     print function(s)
>     print group

While I agree with this objection in the case of strings (which are long
embedded in developers minds as immutable, hence unable to be altered by
other functions or threads), it doesn't bother me for a new convenience
type which is known to be mutable (and hence subject to side effects as
in the example above).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From greg.ewing at canterbury.ac.nz  Wed Jul 22 01:27:57 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 22 Jul 2009 11:27:57 +1200
Subject: [Python-ideas] Experiment: Adding "re" to string objects.
In-Reply-To: <h44acg$8jq$1@ger.gmane.org>
References: <20090720214705.GA29607@tummy.com>
	<200907210952.59570.steve@pearwood.info> <h44acg$8jq$1@ger.gmane.org>
Message-ID: <4A664EFD.8030502@canterbury.ac.nz>

I'm -1 on adding any re-related functionality to the
string type. The re engine is far too much baggage to
pull into the Python core.

Despite what Perl-heads may want to believe, the
universe does not revolve around regular expressions.

-- 
Greg


From lists at cheimes.de  Wed Jul 22 01:54:11 2009
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 22 Jul 2009 01:54:11 +0200
Subject: [Python-ideas] Experiment: Adding "re" to string objects.
In-Reply-To: <4A6625D1.5090007@tummy.com>
References: <20090720214705.GA29607@tummy.com>	<200907210952.59570.steve@pearwood.info>
	<4A6625D1.5090007@tummy.com>
Message-ID: <h45kf3$qjh$1@ger.gmane.org>

Sean Reifschneider wrote:
> The difference being that strings are immutable and lists are not, but I
> don't see a particular concern with string.re being mutable, while still
> keeping strings immutable.  If you call a function with side-effects, it's
> going to result in side-effects.  That's the nature of the beast, and just
> a part of programming.

String objects may be used by and shared between multiple
subinterpreters. In order to keep the subinterpreters apart no shared
object must be mutable.
Strings and dicts are fundamental building blocks of the CPython
interpreter. If you chance any of the bricks you are changing the
foundation of the interpreter. You may open a can of worms much faster
than you might think.

I'm definitely -10 on any proposal that chances the PyStringObject
struct for regular expressions.

Christian



From steve at pearwood.info  Wed Jul 22 02:00:52 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 22 Jul 2009 10:00:52 +1000
Subject: [Python-ideas] Proposed convenience functions for re module
Message-ID: <200907221000.52871.steve@pearwood.info>

Following the thread "Experiment: Adding "re" to string objects.", I 
would like to propose the addition of two convenience functions to the 
re module:


def multimatch(s, *patterns):
? ? """Do a re.match on s using each pattern in patterns, 
? ? returning the first one to succeed, or None if they all fail."""
? ? for pattern in patterns:
? ? ? ? m = re.match(pattern, s)
? ? ? ? if m: return m

def multisearch(s, *patterns):
    """Do a re.search on s using each pattern in patterns, 
    returning the first one to succeed, or None if they all fail."""
    for pattern in patterns:
        m = re.search(pattern, s)
        if m: return m


The rationale is to make the following idiom easier:


m = re.match(s, pattern1)
if not m:
    m = re.match(s, pattern2)
    if not m:
        m = re.match(s, pattern3)
        if not m:
            m = re.match(s, pattern4)
if m:
    m.group()


which will become:

m = re.multimatch(s, pattern1, pattern2, pattern3, pattern4)
if m:
    m.group()


Is there any support or objections to this proposal? Any comments?



-- 
Steven D'Aprano


From python at mrabarnett.plus.com  Wed Jul 22 02:50:34 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 22 Jul 2009 01:50:34 +0100
Subject: [Python-ideas] Proposed convenience functions for re module
In-Reply-To: <200907221000.52871.steve@pearwood.info>
References: <200907221000.52871.steve@pearwood.info>
Message-ID: <4A66625A.1030404@mrabarnett.plus.com>

Steven D'Aprano wrote:
> Following the thread "Experiment: Adding "re" to string objects.", I 
> would like to propose the addition of two convenience functions to the 
> re module:
> 
> 
> def multimatch(s, *patterns):
>     """Do a re.match on s using each pattern in patterns, 
>     returning the first one to succeed, or None if they all fail."""
>     for pattern in patterns:
>         m = re.match(pattern, s)
>         if m: return m
> 
> def multisearch(s, *patterns):
>     """Do a re.search on s using each pattern in patterns, 
>     returning the first one to succeed, or None if they all fail."""
>     for pattern in patterns:
>         m = re.search(pattern, s)
>         if m: return m
> 
> 
> The rationale is to make the following idiom easier:
> 
> 
> m = re.match(s, pattern1)
> if not m:
>     m = re.match(s, pattern2)
>     if not m:
>         m = re.match(s, pattern3)
>         if not m:
>             m = re.match(s, pattern4)
> if m:
>     m.group()
> 
> 
> which will become:
> 
> m = re.multimatch(s, pattern1, pattern2, pattern3, pattern4)
> if m:
>     m.group()
> 
> 
> Is there any support or objections to this proposal? Any comments?
> 
Extend the current re.match and re.search to accept a tuple of patterns:

     m = re.match((pattern1, pattern2, pattern3, pattern4), s)
     if m:
         print m.group()

This format is already used by some string methods, eg str.startswith().


From gerald.britton at gmail.com  Wed Jul 22 03:21:03 2009
From: gerald.britton at gmail.com (Gerald Britton)
Date: Tue, 21 Jul 2009 21:21:03 -0400
Subject: [Python-ideas] Proposed convenience functions for re module
In-Reply-To: <4A66625A.1030404@mrabarnett.plus.com>
References: <200907221000.52871.steve@pearwood.info>
	<4A66625A.1030404@mrabarnett.plus.com>
Message-ID: <5d1a32000907211821p56bf07bava52951dfe0f9dbcd@mail.gmail.com>

Why not:

from itertools import starmap
for x in starmap(re.match, *patterns):
  if x: break

On Tue, Jul 21, 2009 at 8:50 PM, MRAB<python at mrabarnett.plus.com> wrote:
> Steven D'Aprano wrote:
>>
>> Following the thread "Experiment: Adding "re" to string objects.", I would
>> like to propose the addition of two convenience functions to the re module:
>>
>>
>> def multimatch(s, *patterns):
>> ? ?"""Do a re.match on s using each pattern in patterns, ? ?returning the
>> first one to succeed, or None if they all fail."""
>> ? ?for pattern in patterns:
>> ? ? ? ?m = re.match(pattern, s)
>> ? ? ? ?if m: return m
>>
>> def multisearch(s, *patterns):
>> ? ?"""Do a re.search on s using each pattern in patterns, ? ?returning the
>> first one to succeed, or None if they all fail."""
>> ? ?for pattern in patterns:
>> ? ? ? ?m = re.search(pattern, s)
>> ? ? ? ?if m: return m
>>
>>
>> The rationale is to make the following idiom easier:
>>
>>
>> m = re.match(s, pattern1)
>> if not m:
>> ? ?m = re.match(s, pattern2)
>> ? ?if not m:
>> ? ? ? ?m = re.match(s, pattern3)
>> ? ? ? ?if not m:
>> ? ? ? ? ? ?m = re.match(s, pattern4)
>> if m:
>> ? ?m.group()
>>
>>
>> which will become:
>>
>> m = re.multimatch(s, pattern1, pattern2, pattern3, pattern4)
>> if m:
>> ? ?m.group()
>>
>>
>> Is there any support or objections to this proposal? Any comments?
>>
> Extend the current re.match and re.search to accept a tuple of patterns:
>
> ? ?m = re.match((pattern1, pattern2, pattern3, pattern4), s)
> ? ?if m:
> ? ? ? ?print m.group()
>
> This format is already used by some string methods, eg str.startswith().
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
Gerald Britton


From tleeuwenburg at gmail.com  Wed Jul 22 04:01:03 2009
From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg)
Date: Wed, 22 Jul 2009 12:01:03 +1000
Subject: [Python-ideas] 2c about package directories
Message-ID: <43c8685c0907211901s2c771c73xf3359f419cc0c104@mail.gmail.com>

Hi all,

Apologies if any of this has been said before. I've been loosely following
the ongoing discussions about packaging utilities, plus a few people's
experiences with attempting to manage their own Python environments. I was
thinking about a potential directory layout approach which might be
attractive to some people and which I think could be useful for automatic
packaging.

/usr/local/bin (or whatever the system prefix is)
  python --> 'standard python version'
  python2.6 --> 'standard python 2.6'
  python2.7
  python3.1
  etc

/usr/local/lib/python (or whatever the system prefix is)
   /python2.6
      /core
      /site-packages
      /applications
          /application-name-packages
   /python2.7... etc

/home/username/.python/
   /python2.6
      /user-packages
      /applications
          /application-name-packages

With that layout, it should be possible for:
  -- Anyone to always be able to run a system python installation with
'core' libraries only
  -- Run a system python installation with 'sitewide' libraries
  -- Deploy application-specific libraries to a system python installation
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090722/d6a2019a/attachment.html>

From greg.ewing at canterbury.ac.nz  Wed Jul 22 08:17:46 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 22 Jul 2009 18:17:46 +1200
Subject: [Python-ideas] Proposed convenience functions for re module
In-Reply-To: <200907221000.52871.steve@pearwood.info>
References: <200907221000.52871.steve@pearwood.info>
Message-ID: <4A66AF0A.1070705@canterbury.ac.nz>

Steven D'Aprano wrote:

> def multimatch(s, *patterns):
>     """Do a re.match on s using each pattern in patterns, 
>     returning the first one to succeed, or None if they all fail."""
>     for pattern in patterns:
>         m = re.match(pattern, s)
>         if m: return m

How are you supposed to tell which one matched?

-- 
Greg


From greg.ewing at canterbury.ac.nz  Wed Jul 22 08:22:17 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 22 Jul 2009 18:22:17 +1200
Subject: [Python-ideas] Proposed convenience functions for re module
In-Reply-To: <4A66625A.1030404@mrabarnett.plus.com>
References: <200907221000.52871.steve@pearwood.info>
	<4A66625A.1030404@mrabarnett.plus.com>
Message-ID: <4A66B019.2090501@canterbury.ac.nz>

MRAB wrote:

> Extend the current re.match and re.search to accept a tuple of patterns:
> 
>     m = re.match((pattern0, pattern1, pattern2, pattern3), s)
>     if m:
>         print m.group()

Also give the match object an 'index' attribute indicating
which pattern matched, so you can do

   m = re.match((pattern1, pattern2, pattern3, pattern4), s)
   if m:
     if m.index == 0:
       # pattern0 matched
     elif m.index == 1:
       # pattern1 matched
     # etc.

-- 
Greg



From gagsl-py2 at yahoo.com.ar  Wed Jul 22 09:14:16 2009
From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina)
Date: Wed, 22 Jul 2009 04:14:16 -0300
Subject: [Python-ideas] Proposed convenience functions for re module
References: <200907221000.52871.steve@pearwood.info>
	<4A66AF0A.1070705@canterbury.ac.nz>
Message-ID: <op.uxgjp2yvx6zn5v@lepton.softlabbsas.com.ar>

En Wed, 22 Jul 2009 03:17:46 -0300, Greg Ewing
<greg.ewing-F+z8Qja7x9Xokq/tPzqvJg at public.gmane.org> escribi?:
> Steven D'Aprano wrote:
>
>> def multimatch(s, *patterns):
>>     """Do a re.match on s using each pattern in patterns,     returning  
>> the first one to succeed, or None if they all fail."""
>>     for pattern in patterns:
>>         m = re.match(pattern, s)
>>         if m: return m
>
> How are you supposed to tell which one matched?

m.re contains the matching expression

-- 
Gabriel Genellina


From rrr at ronadam.com  Wed Jul 22 10:02:54 2009
From: rrr at ronadam.com (Ron Adam)
Date: Wed, 22 Jul 2009 03:02:54 -0500
Subject: [Python-ideas] Proposed convenience functions for re module
In-Reply-To: <200907221000.52871.steve@pearwood.info>
References: <200907221000.52871.steve@pearwood.info>
Message-ID: <4A66C7AE.3050100@ronadam.com>



Steven D'Aprano wrote:
> Following the thread "Experiment: Adding "re" to string objects.", I 
> would like to propose the addition of two convenience functions to the 
> re module:
> 
> 
> def multimatch(s, *patterns):
>     """Do a re.match on s using each pattern in patterns, 
>     returning the first one to succeed, or None if they all fail."""
>     for pattern in patterns:
>         m = re.match(pattern, s)
>         if m: return m
> 
> def multisearch(s, *patterns):
>     """Do a re.search on s using each pattern in patterns, 
>     returning the first one to succeed, or None if they all fail."""
>     for pattern in patterns:
>         m = re.search(pattern, s)
>         if m: return m
> 
> 
> The rationale is to make the following idiom easier:
> 
> 
> m = re.match(s, pattern1)
> if not m:
>     m = re.match(s, pattern2)
>     if not m:
>         m = re.match(s, pattern3)
>         if not m:
>             m = re.match(s, pattern4)
> if m:
>     m.group()
> 
> 
> which will become:
> 
> m = re.multimatch(s, pattern1, pattern2, pattern3, pattern4)
> if m:
>     m.group()
> 
> 
> Is there any support or objections to this proposal? Any comments?


One of the needs I've run across is to enable the program user (possibly a 
non-programmer) to do logical searches on data.

It would be nice if the search patterns specified by the program user could 
be used directly by the functions.  Search functions of this type would 
take patterns that are more like what you would use for google or yahoo 
searches instead of the more complex language re requires.

Ron








From jafo at tummy.com  Wed Jul 22 11:13:28 2009
From: jafo at tummy.com (Sean Reifschneider)
Date: Wed, 22 Jul 2009 03:13:28 -0600
Subject: [Python-ideas] Experiment: Adding "re" to string objects.
In-Reply-To: <4A663048.4080106@gmail.com>
References: <20090720214705.GA29607@tummy.com>	<200907210952.59570.steve@pearwood.info>
	<4A6625D1.5090007@tummy.com> <4A663048.4080106@gmail.com>
Message-ID: <4A66D838.80803@tummy.com>

On 07/21/2009 03:16 PM, Nick Coghlan wrote:
> It also just occurred to me that string interning completely destroys
> any concept of storing mutable attributes on strings:

So that would prevent "re" from being added to strings, right?  Is there
some way around that?  I can't think of one, beyond making a sub-class like
I've done in the filtertools example...

Which kind of makes the whole idea fall down, because the benefit of having
it directly in strings is that you don't then have to use a bunch of
different APIs like "file.rereadlines()" or "re.readlines(fp)" or
"re.open('filename')" or "re.file(fp)"...  At that point, I wonder if it
makes more sense to put the match() and group() methods on the refile
object instead of on the string it returns.

Opinions?

Sean
-- 
Sean Reifschneider, Member of Technical Staff <jafo at tummy.com>
tummy.com, ltd. - Linux Consulting since 1995: Ask me about High Availability

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 252 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090722/1c55dec5/attachment.pgp>

From steve at pearwood.info  Wed Jul 22 13:58:42 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 22 Jul 2009 21:58:42 +1000
Subject: [Python-ideas] Proposed convenience functions for re module
In-Reply-To: <4A66C7AE.3050100@ronadam.com>
References: <200907221000.52871.steve@pearwood.info>
	<4A66C7AE.3050100@ronadam.com>
Message-ID: <200907222158.43260.steve@pearwood.info>

On Wed, 22 Jul 2009 06:02:54 pm Ron Adam wrote:
> One of the needs I've run across is to enable the program user
> (possibly a non-programmer) to do logical searches on data.
>
> It would be nice if the search patterns specified by the program user
> could be used directly by the functions. ?Search functions of this
> type would take patterns that are more like what you would use for
> google or yahoo searches instead of the more complex language re
> requires.

I'm not sure if I understand this correctly. Perhaps you could give an 
example or two?

Also, please don't overload my simple little proposal with a multitude 
of new functionality. My proposal is only meant to be a lightweight 
convenience function. Additional functionality probably belongs as a 
different function, maybe even a different module.


-- 
Steven D'Aprano


From jnoller at gmail.com  Wed Jul 22 15:41:33 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Wed, 22 Jul 2009 09:41:33 -0400
Subject: [Python-ideas] Changing the default install location,
	script 	versioning (Packages and PEP 370)
In-Reply-To: <loom.20090721T084624-784@post.gmane.org>
References: <4222a8490907191857m2705b298gd329357ef6912169@mail.gmail.com>
	<loom.20090720T082847-338@post.gmane.org>
	<75F0120D-6B2B-4C38-9943-49CF8A1765E1@gmail.com>
	<loom.20090720T160738-261@post.gmane.org>
	<E7E1CAC0-A802-41A5-B47D-1A6B633F6C75@gmail.com>
	<loom.20090721T084624-784@post.gmane.org>
Message-ID: <4222a8490907220641v7055c892l2591a301c2fc9cbc@mail.gmail.com>

On Tue, Jul 21, 2009 at 4:49 AM, Antoine Pitrou<solipsis at pitrou.net> wrote:
> Doug Hellmann <doug.hellmann at ...> writes:
>>
>> Why is the lib directory ~/.local/lib/pythonVERSION if ~/.local isn't
>> intended to support more than one version of the interpreter?
>
> Because scripts specify which version of the interpreter they use in their
> shebang line, so they don't need a versioned location.
> Conversely, libraries have no way to specify for which version of the
> interpreter they were installed, so their location needs to be
> versioned.
> (besides, it is common for a library to be installed for several interpreters
> because it may be required by several different apps)
>
> This is the same difference as between /usr/bin and
> /usr/lib/python-X.Y/site-packages.
>
> Regards
>
> Antoine.
>

I'm not dead yet! /monty python

I want to point out that the ./local/bin directory isn't the only
thing unversioned - any docs/ which get laid down by a package are
also being plopped in .local/docs - not .local/version/docs.


From solipsis at pitrou.net  Wed Jul 22 16:10:35 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 22 Jul 2009 14:10:35 +0000 (UTC)
Subject: [Python-ideas]
	=?utf-8?q?Changing_the_default_install_location=2C?=
	=?utf-8?q?=09script=09versioning_=28Packages_and_PEP_370=29?=
References: <4222a8490907191857m2705b298gd329357ef6912169@mail.gmail.com>
	<loom.20090720T082847-338@post.gmane.org>
	<75F0120D-6B2B-4C38-9943-49CF8A1765E1@gmail.com>
	<loom.20090720T160738-261@post.gmane.org>
	<E7E1CAC0-A802-41A5-B47D-1A6B633F6C75@gmail.com>
	<loom.20090721T084624-784@post.gmane.org>
	<4222a8490907220641v7055c892l2591a301c2fc9cbc@mail.gmail.com>
Message-ID: <loom.20090722T140926-830@post.gmane.org>

Jesse Noller <jnoller at ...> writes:
> 
> I'm not dead yet! /monty python
> 
> I want to point out that the ./local/bin directory isn't the only
> thing unversioned - any docs/ which get laid down by a package are
> also being plopped in .local/docs - not .local/version/docs.

Is it a problem?
I understand the docs could be versioned by package version, but why should they
be versioned by interpreter version?





From jnoller at gmail.com  Wed Jul 22 16:39:42 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Wed, 22 Jul 2009 10:39:42 -0400
Subject: [Python-ideas] Changing the default install location,
	script 	versioning (Packages and PEP 370)
In-Reply-To: <loom.20090722T140926-830@post.gmane.org>
References: <4222a8490907191857m2705b298gd329357ef6912169@mail.gmail.com>
	<loom.20090720T082847-338@post.gmane.org>
	<75F0120D-6B2B-4C38-9943-49CF8A1765E1@gmail.com>
	<loom.20090720T160738-261@post.gmane.org>
	<E7E1CAC0-A802-41A5-B47D-1A6B633F6C75@gmail.com>
	<loom.20090721T084624-784@post.gmane.org>
	<4222a8490907220641v7055c892l2591a301c2fc9cbc@mail.gmail.com>
	<loom.20090722T140926-830@post.gmane.org>
Message-ID: <4222a8490907220739q14068b98j81e4b869d4cd7cd9@mail.gmail.com>

On Wed, Jul 22, 2009 at 10:10 AM, Antoine Pitrou<solipsis at pitrou.net> wrote:
> Jesse Noller <jnoller at ...> writes:
>>
>> I'm not dead yet! /monty python
>>
>> I want to point out that the ./local/bin directory isn't the only
>> thing unversioned - any docs/ which get laid down by a package are
>> also being plopped in .local/docs - not .local/version/docs.
>
> Is it a problem?
> I understand the docs could be versioned by package version, but why should they
> be versioned by interpreter version?
>

Because they're artifacts of a package installed into a version of the
interpreter. Sure, "proper docs" will include a Python2.x section
*and* the 3.x section; then again, why not only package the 3.x docs
with the 3.x package?


From talin at acm.org  Wed Jul 22 18:23:25 2009
From: talin at acm.org (Talin)
Date: Wed, 22 Jul 2009 09:23:25 -0700
Subject: [Python-ideas] Proposed convenience functions for re module
In-Reply-To: <200907221000.52871.steve@pearwood.info>
References: <200907221000.52871.steve@pearwood.info>
Message-ID: <4A673CFD.4030005@acm.org>

Steven D'Aprano wrote:
> Following the thread "Experiment: Adding "re" to string objects.", I 
> would like to propose the addition of two convenience functions to the 
> re module:
> 
> 
> def multimatch(s, *patterns):
>     """Do a re.match on s using each pattern in patterns, 
>     returning the first one to succeed, or None if they all fail."""
>     for pattern in patterns:
>         m = re.match(pattern, s)
>         if m: return m
>

There's a cute trick that you can use to do this that is much more 
efficient than testing each regex expression individually:

   combined_pattern = "|".join("(%s)" % p for p in patterns)
   combined_re = re.compile(combined_pattern)

   m = combined_re.match(string)
   return m.lastindex

Basically, it combines all of the patterns into a single large regex, 
where each pattern is converted into a capturing group. It then returns 
match.lastindex, which is the index of the capturing group that matched. 
This is very efficient because now all of the patterns are combined into 
a single NFA which can prune possibilities very quickly.

This works for up to 99 patterns, which is the limit on the number of 
capturing groups that a regex can have.

I use this technique in my Python-based lexer, Reflex:

    http://pypi.python.org/pypi/reflex/0.1

Now, if we are talking about convenience functions, what I would really 
like to see is a class that wraps a string that allows matches to be 
done incrementally, where each successful match consumes the head of the 
string, leaving the remainder of the string for the next match.

This can be done very efficiently since the regex functions all take a 
start-index parameter. Essentially, the wrapper class would update the 
start index each time a successful match was performed.

So something like:

    stream = MatchStream(string)
    while 1:
       m = stream.match(combined_re)
       # m is a match object
       # Do something with m

Or even an iterator over matches (this assumes that you want to use the 
same regex each time, which may not be the case for a parser):

    stream = MatchStream(string)
    for m in stream.match(combined_re):
       # m is a match object
       # Do something with m

-- Talin


From python at mrabarnett.plus.com  Wed Jul 22 18:42:51 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 22 Jul 2009 17:42:51 +0100
Subject: [Python-ideas] Proposed convenience functions for re module
In-Reply-To: <4A673CFD.4030005@acm.org>
References: <200907221000.52871.steve@pearwood.info> <4A673CFD.4030005@acm.org>
Message-ID: <4A67418B.10507@mrabarnett.plus.com>

Talin wrote:
> Steven D'Aprano wrote:
>> Following the thread "Experiment: Adding "re" to string objects.", I 
>> would like to propose the addition of two convenience functions to the 
>> re module:
>>
>>
>> def multimatch(s, *patterns):
>>     """Do a re.match on s using each pattern in patterns,     
>> returning the first one to succeed, or None if they all fail."""
>>     for pattern in patterns:
>>         m = re.match(pattern, s)
>>         if m: return m
>>
> 
> There's a cute trick that you can use to do this that is much more 
> efficient than testing each regex expression individually:
> 
>   combined_pattern = "|".join("(%s)" % p for p in patterns)
>   combined_re = re.compile(combined_pattern)
> 
>   m = combined_re.match(string)
>   return m.lastindex
> 
> Basically, it combines all of the patterns into a single large regex, 
> where each pattern is converted into a capturing group. It then returns 
> match.lastindex, which is the index of the capturing group that matched. 
> This is very efficient because now all of the patterns are combined into 
> a single NFA which can prune possibilities very quickly.
> 
> This works for up to 99 patterns, which is the limit on the number of 
> capturing groups that a regex can have.
> 
[snip]
It won't work properly if the patterns themselves contain capture
groups.



From bjourne at gmail.com  Wed Jul 22 20:29:31 2009
From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=)
Date: Wed, 22 Jul 2009 20:29:31 +0200
Subject: [Python-ideas] Proposed convenience functions for re module
In-Reply-To: <200907221000.52871.steve@pearwood.info>
References: <200907221000.52871.steve@pearwood.info>
Message-ID: <740c3aec0907221129x638285a1n22075390c589ade1@mail.gmail.com>

2009/7/22 Steven D'Aprano <steve at pearwood.info>:
> The rationale is to make the following idiom easier:
>
>
> m = re.match(s, pattern1)
> if not m:
> ? ?m = re.match(s, pattern2)
> ? ?if not m:
> ? ? ? ?m = re.match(s, pattern3)
> ? ? ? ?if not m:
> ? ? ? ? ? ?m = re.match(s, pattern4)
> if m:
> ? ?m.group()
>
>
> which will become:
>
> m = re.multimatch(s, pattern1, pattern2, pattern3, pattern4)
> if m:
> ? ?m.group()
>
>
> Is there any support or objections to this proposal? Any comments?

I don't like it very much because it would only work for uncompiled
patterns. All functions in re has a RegexObject counterpart, but
multisearch() and multimatch() obviously would not. For the quoted
example I'd usually try to create one regex that matches all four
patterns, or use a loop:

for pat in (pattern1, pattern2, pattern3, pattern4):
    m = re.match(s, pat)
    if m:
        m.group()
        break


-- 
mvh Bj?rn


From python at mrabarnett.plus.com  Wed Jul 22 20:44:02 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 22 Jul 2009 19:44:02 +0100
Subject: [Python-ideas] Proposed convenience functions for re module
In-Reply-To: <740c3aec0907221129x638285a1n22075390c589ade1@mail.gmail.com>
References: <200907221000.52871.steve@pearwood.info>
	<740c3aec0907221129x638285a1n22075390c589ade1@mail.gmail.com>
Message-ID: <4A675DF2.6050603@mrabarnett.plus.com>

BJ?rn Lindqvist wrote:
> 2009/7/22 Steven D'Aprano <steve at pearwood.info>:
>> The rationale is to make the following idiom easier:
>>
>>
>> m = re.match(s, pattern1)
>> if not m:
>>    m = re.match(s, pattern2)
>>    if not m:
>>        m = re.match(s, pattern3)
>>        if not m:
>>            m = re.match(s, pattern4)
>> if m:
>>    m.group()
>>
>>
>> which will become:
>>
>> m = re.multimatch(s, pattern1, pattern2, pattern3, pattern4)
>> if m:
>>    m.group()
>>
>>
>> Is there any support or objections to this proposal? Any comments?
> 
> I don't like it very much because it would only work for uncompiled
> patterns. All functions in re has a RegexObject counterpart, but
> multisearch() and multimatch() obviously would not. For the quoted
> example I'd usually try to create one regex that matches all four
> patterns, or use a loop:
> 
> for pat in (pattern1, pattern2, pattern3, pattern4):
>     m = re.match(s, pat)
>     if m:
>         m.group()
>         break
> 
re.match and re.search will accept either a string or a compiled pattern
as the first argument. Never used it myself, but...


From g.brandl at gmx.net  Wed Jul 22 21:51:40 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 22 Jul 2009 21:51:40 +0200
Subject: [Python-ideas] Proposed convenience functions for re module
In-Reply-To: <4A673CFD.4030005@acm.org>
References: <200907221000.52871.steve@pearwood.info> <4A673CFD.4030005@acm.org>
Message-ID: <h47qlt$23k$1@ger.gmane.org>

Talin schrieb:

> Now, if we are talking about convenience functions, what I would really 
> like to see is a class that wraps a string that allows matches to be 
> done incrementally, where each successful match consumes the head of the 
> string, leaving the remainder of the string for the next match.
> 
> This can be done very efficiently since the regex functions all take a 
> start-index parameter. Essentially, the wrapper class would update the 
> start index each time a successful match was performed.
> 
> So something like:
> 
>     stream = MatchStream(string)
>     while 1:
>        m = stream.match(combined_re)
>        # m is a match object
>        # Do something with m
> 
> Or even an iterator over matches (this assumes that you want to use the 
> same regex each time, which may not be the case for a parser):
> 
>     stream = MatchStream(string)
>     for m in stream.match(combined_re):
>        # m is a match object
>        # Do something with m

You might be interested in the undocumented re.Scanner class :)

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.



From zuo at chopin.edu.pl  Wed Jul 22 23:25:30 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Wed, 22 Jul 2009 23:25:30 +0200
Subject: [Python-ideas] Proposed convenience functions for re module
In-Reply-To: <4A66C7AE.3050100@ronadam.com>
References: <200907221000.52871.steve@pearwood.info>
	<4A66C7AE.3050100@ronadam.com>
Message-ID: <op.uxhm4qw5fvx12t@jerzozwiesz.home.aster.pl>

Ron Adam <rrr at ronadam.com>"

> One of the needs I've run across is to enable the program user (possibly  
> a non-programmer) to do logical searches on data.
>
> It would be nice if the search patterns specified by the program user  
> could be used directly by the functions.  Search functions of this type  
> would take patterns that are more like what you would use for google or  
> yahoo searches instead of the more complex language re requires.

It's a proposition of another string matching module ("browsermatch"? :-))

+0.6

*j


From dickinsm at gmail.com  Wed Jul 22 23:38:11 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Wed, 22 Jul 2009 22:38:11 +0100
Subject: [Python-ideas] Proposed convenience functions for re module
In-Reply-To: <200907221000.52871.steve@pearwood.info>
References: <200907221000.52871.steve@pearwood.info>
Message-ID: <5c6f2a5d0907221438o542c06f2y9b33f6c50a6f9403@mail.gmail.com>

On Wed, Jul 22, 2009 at 1:00 AM, Steven D'Aprano<steve at pearwood.info> wrote:
> Following the thread "Experiment: Adding "re" to string objects.", I
> would like to propose the addition of two convenience functions to the
> re module:
>
>
> def multimatch(s, *patterns):
> ? ? """Do a re.match on s using each pattern in patterns,
> ? ? returning the first one to succeed, or None if they all fail."""
> ? ? for pattern in patterns:
> ? ? ? ? m = re.match(pattern, s)
> ? ? ? ? if m: return m
>
> def multisearch(s, *patterns):
> ? ?"""Do a re.search on s using each pattern in patterns,
> ? ?returning the first one to succeed, or None if they all fail."""
> ? ?for pattern in patterns:
> ? ? ? ?m = re.search(pattern, s)
> ? ? ? ?if m: return m
>
> [...]

Steven, could you show some examples of real(ish)-world use-cases
for one or both of these functions?  Preferably including the code that
might directly follow a multimatch or multisearch call.

It's probably because I haven't used regexes widely enough, but
in all the potential examples I can come up with, either

(1) the regexes are similar enough that they can be refactored into
a single regex (e.g., just concatenated with '|'), or

(2) they're distinct enough that each regex needs its own handing,
so that the multimatch/multisearch would need to be followed by
a multiway 'if/elif/else' anyway;  in this case, it seems that little
is gained.

--
Mark


From zuo at chopin.edu.pl  Thu Jul 23 00:03:35 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Thu, 23 Jul 2009 00:03:35 +0200
Subject: [Python-ideas] Proposed convenience functions for re module
In-Reply-To: <200907221000.52871.steve@pearwood.info>
References: <200907221000.52871.steve@pearwood.info>
Message-ID: <op.uxhov9rrfvx12t@jerzozwiesz.home.aster.pl>

22-07-2009, 02:00 Steven D'Aprano <steve at pearwood.info>:

> Following the thread "Experiment: Adding "re" to string objects.", I
> would like to propose the addition of two convenience functions to the
> re module:
>
>
> def multimatch(s, *patterns):
> ? ? """Do a re.match on s using each pattern in patterns,
> ? ? returning the first one to succeed, or None if they all fail."""
> ? ? for pattern in patterns:
> ? ? ? ? m = re.match(pattern, s)
> ? ? ? ? if m: return m
>
> def multisearch(s, *patterns):
>     """Do a re.search on s using each pattern in patterns,
>     returning the first one to succeed, or None if they all fail."""
>     for pattern in patterns:
>         m = re.search(pattern, s)
>         if m: return m
>
>
> The rationale is to make the following idiom easier:
>
>
> m = re.match(s, pattern1)
> if not m:
>     m = re.match(s, pattern2)
>     if not m:
>         m = re.match(s, pattern3)
>         if not m:
>             m = re.match(s, pattern4)
> if m:
>     m.group()
>
>
> which will become:
>
> m = re.multimatch(s, pattern1, pattern2, pattern3, pattern4)
> if m:
>     m.group()
>
>
> Is there any support or objections to this proposal? Any comments?

It sounds nice. But why not to use simply:

m = re.match(s, '|'.join(pattern1, pattern2, pattern3, pattern4))

And if we want the feature anyway, I'd prefer MRAB's:

>     m = re.match((pattern1, pattern2, pattern3, pattern4), s)
>     if m:
>         print m.group()
>
>  This format is already used by some string methods, eg str.startswith().

***

But if we are talking about convenience functions in re module, it'd
be IMHO very nice to have such functions:

def matchgrouping(pattern, string, flags=0, default=None):
? ? """Do a re.match on string using pattern,
? ? returning dict containing groups which could be
     got by index or by name."""

     match = re.match(pattern, string, flags)
     groups = collections.DefaultDict()
     groups.update(enumerate(match.groups()))
     groups.update(match.groupdict())
     return result

Plus the analogous function for searching).
Plus 2 analogous methods of RegexObject instances).


* Then e.g. -- instead of:

m = re.search(pattern, s)
if m:
     first_group = m.group(1)
     surname = m.group('surname')
else:
     first_group = None
     surname = None

-- we could write simply:

m = re.matchgrouping(pattern, s)
first_group = m[1]
surname = m['surname']


* And e.g. -- instead of:

withip = log_re.match(logline)
if withip and withip.group('ip_addr'):
     iplog.append(logline)

-- we could write simply:

if log_re.matchgrouping(logline)['ip_addr']:
     iplog.append(logline)


What do you think about it?

*j

-- 
Jan Kaliszewski (zuo) <zuo at chopin.edu.pl>


From solipsis at pitrou.net  Thu Jul 23 00:10:47 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 22 Jul 2009 22:10:47 +0000 (UTC)
Subject: [Python-ideas]
	=?utf-8?q?Changing_the_default_install_location=2C?=
	=?utf-8?q?=09script=09versioning_=28Packages_and_PEP_370=29?=
References: <4222a8490907191857m2705b298gd329357ef6912169@mail.gmail.com>
	<loom.20090720T082847-338@post.gmane.org>
	<75F0120D-6B2B-4C38-9943-49CF8A1765E1@gmail.com>
	<loom.20090720T160738-261@post.gmane.org>
	<E7E1CAC0-A802-41A5-B47D-1A6B633F6C75@gmail.com>
	<loom.20090721T084624-784@post.gmane.org>
	<4222a8490907220641v7055c892l2591a301c2fc9cbc@mail.gmail.com>
	<loom.20090722T140926-830@post.gmane.org>
	<4222a8490907220739q14068b98j81e4b869d4cd7cd9@mail.gmail.com>
Message-ID: <loom.20090722T220918-715@post.gmane.org>

Jesse Noller <jnoller at ...> writes:
> 
> Because they're artifacts of a package installed into a version of the
> interpreter. Sure, "proper docs" will include a Python2.x section
> *and* the 3.x section; then again, why not only package the 3.x docs
> with the 3.x package?

That's a good point. 
Incidentally, data files can be installed in the package's directory if
`package_data` is used in the setup script. Which obviously versions them.

Regards

Antoine.




From zuo at chopin.edu.pl  Thu Jul 23 00:14:20 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Thu, 23 Jul 2009 00:14:20 +0200
Subject: [Python-ideas] Proposed convenience functions for re module
In-Reply-To: <op.uxhov9rrfvx12t@jerzozwiesz.home.aster.pl>
References: <200907221000.52871.steve@pearwood.info>
	<op.uxhov9rrfvx12t@jerzozwiesz.home.aster.pl>
Message-ID: <op.uxhpd6t7fvx12t@jerzozwiesz.home.aster.pl>

Jan Kaliszewski <zuo at chopin.edu.pl> wrote:

> It sounds nice. But why not to use simply:
>
> m = re.match(s, '|'.join(pattern1, pattern2, pattern3, pattern4))

Sorry, I ment of course:

m = re.match('|'.join(pattern1, pattern2, pattern3, pattern4), s)

***

> ? ? """Do a re.match on string using pattern,
> ? ? returning dict containing groups which could be
>      got by index or by name."""

I ment: "...returning collections.DefaultDict..." (as you can see
in the code following).

Regards,

*j

-- 
Jan Kaliszewski <zuo at chopin.edu.pl>


From steve at pearwood.info  Thu Jul 23 01:00:01 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 23 Jul 2009 09:00:01 +1000
Subject: [Python-ideas] Proposed convenience functions for re module
In-Reply-To: <740c3aec0907221129x638285a1n22075390c589ade1@mail.gmail.com>
References: <200907221000.52871.steve@pearwood.info>
	<740c3aec0907221129x638285a1n22075390c589ade1@mail.gmail.com>
Message-ID: <200907230900.02025.steve@pearwood.info>

On Thu, 23 Jul 2009 04:29:31 am BJ?rn Lindqvist wrote:

> I don't like it very much because it would only work for uncompiled
> patterns. All functions in re has a RegexObject counterpart, but
> multisearch() and multimatch() obviously would not. 

That's incorrect -- they accept pre-compiled regexes as well as strings.

>>> pat = re.compile('a.c')
>>> multimatch("abcd", ['z.*z', pat])
<_sre.SRE_Match object at 0xb7cd8090>

> For the quoted 
> example I'd usually try to create one regex that matches all four
> patterns, or use a loop:
>
> for pat in (pattern1, pattern2, pattern3, pattern4):
>     m = re.match(s, pat)
>     if m:
>         m.group()
>         break

Apart from being in a function, my proposal (which you claim to dislike) 
is virtually identical to that code (which you say you use).




-- 
Steven D'Aprano


From steve at pearwood.info  Thu Jul 23 01:31:47 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 23 Jul 2009 09:31:47 +1000
Subject: [Python-ideas] Proposed convenience functions for re module
In-Reply-To: <5c6f2a5d0907221438o542c06f2y9b33f6c50a6f9403@mail.gmail.com>
References: <200907221000.52871.steve@pearwood.info>
	<5c6f2a5d0907221438o542c06f2y9b33f6c50a6f9403@mail.gmail.com>
Message-ID: <200907230931.48064.steve@pearwood.info>

On Thu, 23 Jul 2009 07:38:11 am Mark Dickinson wrote:

> Steven, could you show some examples of real(ish)-world use-cases
> for one or both of these functions?  Preferably including the code
> that might directly follow a multimatch or multisearch call.

I'm afraid that I don't use regexes anywhere near enough to champion 
this proposal in the face of serious opposition, or even skepticism. If 
this isn't a simple enough "no-brainer", then I'm going to have to pass 
the baton onto somebody else (assuming anyone actually likes this 
idea).

This idea came about from the thread started by Sean Reifschneider, 
proposing adding regexes to strings. I thought (and Sean seemed to 
agree) that these convenience functions would solve his primary 
use-case. So this proposal isn't scratching an itch I have.


> It's probably because I haven't used regexes widely enough, but
> in all the potential examples I can come up with, either
>
> (1) the regexes are similar enough that they can be refactored into
> a single regex (e.g., just concatenated with '|'), or
>
> (2) they're distinct enough that each regex needs its own handing,
> so that the multimatch/multisearch would need to be followed by
> a multiway 'if/elif/else' anyway;  in this case, it seems that little
> is gained.

These are both reasonable approaches. This proposal isn't supposed to 
solve every multiple-regex-handling problem.

So far support for this has been luke-warm. If anyone really likes this 
idea, please speak up, otherwise I'll let it drop.



-- 
Steven D'Aprano


From python at mrabarnett.plus.com  Thu Jul 23 02:10:42 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 23 Jul 2009 01:10:42 +0100
Subject: [Python-ideas] Proposed convenience functions for re module
In-Reply-To: <200907230931.48064.steve@pearwood.info>
References: <200907221000.52871.steve@pearwood.info>	<5c6f2a5d0907221438o542c06f2y9b33f6c50a6f9403@mail.gmail.com>
	<200907230931.48064.steve@pearwood.info>
Message-ID: <4A67AA82.6030103@mrabarnett.plus.com>

Steven D'Aprano wrote:
> On Thu, 23 Jul 2009 07:38:11 am Mark Dickinson wrote:
> 
>> Steven, could you show some examples of real(ish)-world use-cases
>> for one or both of these functions?  Preferably including the code
>> that might directly follow a multimatch or multisearch call.
> 
> I'm afraid that I don't use regexes anywhere near enough to champion 
> this proposal in the face of serious opposition, or even skepticism. If 
> this isn't a simple enough "no-brainer", then I'm going to have to pass 
> the baton onto somebody else (assuming anyone actually likes this 
> idea).
> 
> This idea came about from the thread started by Sean Reifschneider, 
> proposing adding regexes to strings. I thought (and Sean seemed to 
> agree) that these convenience functions would solve his primary 
> use-case. So this proposal isn't scratching an itch I have.
> 
> 
>> It's probably because I haven't used regexes widely enough, but
>> in all the potential examples I can come up with, either
>>
>> (1) the regexes are similar enough that they can be refactored into
>> a single regex (e.g., just concatenated with '|'), or
>>
>> (2) they're distinct enough that each regex needs its own handing,
>> so that the multimatch/multisearch would need to be followed by
>> a multiway 'if/elif/else' anyway;  in this case, it seems that little
>> is gained.
> 
> These are both reasonable approaches. This proposal isn't supposed to 
> solve every multiple-regex-handling problem.
> 
> So far support for this has been luke-warm. If anyone really likes this 
> idea, please speak up, otherwise I'll let it drop.
> 
If you want to try multiple regexes until one matches then the approach
with re.match accepting a tuple of patterns would, it seems to me, to be
the one that requires the smallest change and has the greatest
precedence (like str.startwith).


From greg.ewing at canterbury.ac.nz  Thu Jul 23 02:49:10 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 23 Jul 2009 12:49:10 +1200
Subject: [Python-ideas] Proposed convenience functions for re module
In-Reply-To: <h47qlt$23k$1@ger.gmane.org>
References: <200907221000.52871.steve@pearwood.info> <4A673CFD.4030005@acm.org>
	<h47qlt$23k$1@ger.gmane.org>
Message-ID: <4A67B386.6010308@canterbury.ac.nz>

Georg Brandl wrote:

> You might be interested in the undocumented re.Scanner class :)

It looks like it could be interesting, but with no
documentation, how are we supposed to tell?

Is there *any* information about it available
anywhere?

-- 
Greg


From robert.kern at gmail.com  Thu Jul 23 03:04:24 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 22 Jul 2009 20:04:24 -0500
Subject: [Python-ideas] Proposed convenience functions for re module
In-Reply-To: <4A67B386.6010308@canterbury.ac.nz>
References: <200907221000.52871.steve@pearwood.info>
	<4A673CFD.4030005@acm.org>	<h47qlt$23k$1@ger.gmane.org>
	<4A67B386.6010308@canterbury.ac.nz>
Message-ID: <h48cuq$j7d$1@ger.gmane.org>

On 2009-07-22 19:49, Greg Ewing wrote:
> Georg Brandl wrote:
>
>> You might be interested in the undocumented re.Scanner class :)
>
> It looks like it could be interesting, but with no
> documentation, how are we supposed to tell?
>
> Is there *any* information about it available
> anywhere?

http://mail.python.org/pipermail/python-dev/2003-April/035075.html

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From agkish at gmail.com  Thu Jul 23 08:28:33 2009
From: agkish at gmail.com (Andy Kish)
Date: Thu, 23 Jul 2009 02:28:33 -0400
Subject: [Python-ideas] universal set object for use in set manipulation
Message-ID: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com>

Hi all,

I think the addition of a universal set object would be a nice touch
for python sets. Manipulation of collections of sets becomes much
easier with it. Right now, certain operations are easy because empty
sets are easy to create while the dual operation requires special
cases:

    set_union = set()
    for s in sets:
        set_union |= s

    # this doesn't work
    set_intersection = set(???)
    for s in sets:
        set_intersection &= s

Instead you have to do something contorted like:

    if len(sets) == 0:
        set_intersection = set()
    else:
        sets_i = iter(sets)
        set_intersection = sets_i.next()
        for s in sets:
            set_intersection &= s

Implementation of a universal set would be pretty trivial. Trails of
EasyExtend [1] has an implementation (albeit used a bit differently)
that's basically the entirety of what's needed.

The above intersection case would end up looking something like:

    set_intersection = set.universal()
    for s in sets:
        set_intersection &= s

Thoughts? I'm happy to write the patch myself if people like this
idea.

Thanks,
Andy Kish.

[1] http://fiber-space.de/wordpress/?p=322


From g.brandl at gmx.net  Thu Jul 23 09:04:22 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 23 Jul 2009 09:04:22 +0200
Subject: [Python-ideas] universal set object for use in set manipulation
In-Reply-To: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com>
References: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com>
Message-ID: <h49237$r89$1@ger.gmane.org>

Andy Kish schrieb:

> Instead you have to do something contorted like:
> 
>     if len(sets) == 0:
>         set_intersection = set()
>     else:
>         sets_i = iter(sets)
>         set_intersection = sets_i.next()
>         for s in sets:
>             set_intersection &= s

Note that a simple ``reduce(operator.and_, sets)`` would suffice for > 1 set.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.



From andrew at bemusement.org  Thu Jul 23 09:12:07 2009
From: andrew at bemusement.org (Andrew Bennetts)
Date: Thu, 23 Jul 2009 17:12:07 +1000
Subject: [Python-ideas] universal set object for use in set manipulation
In-Reply-To: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com>
References: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com>
Message-ID: <20090723071207.GG28628@steerpike.home.puzzling.org>

Andy Kish wrote:
[...]
> 
> The above intersection case would end up looking something like:
> 
>     set_intersection = set.universal()
>     for s in sets:
>         set_intersection &= s

Or even:

   set_intersection = reduce(operator.and_, sets, set.universal())

Although, you can already pass multiple (or zero) sets to set.intersection().
So your special case version can be a little simpler...

   sets = list(sets)
   if len(sets) == 0:
       return set()
   return sets[0].intersection(sets[1:])

Which isn't as elegant, but it's also not so bad.  

-Andrew.



From agkish at gmail.com  Thu Jul 23 09:55:41 2009
From: agkish at gmail.com (Andy Kish)
Date: Thu, 23 Jul 2009 00:55:41 -0700 (PDT)
Subject: [Python-ideas] universal set object for use in set manipulation
In-Reply-To: <20090723071207.GG28628@steerpike.home.puzzling.org>
References: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com> 
	<20090723071207.GG28628@steerpike.home.puzzling.org>
Message-ID: <6f4e8779-ce7a-411d-8b19-24c9fd54a51a@h11g2000yqb.googlegroups.com>

If we want to go golfing :), my favorite solution is with the python
2.6 versions of intersection and union:

    set_union = set().union(*sets)
    set_intersection = set.universal().intersection(*sets)

Folds are nice. That's actually why I sent my initial email to the
list. It's really annoying to having appropriate identity element for
union built in while missing the *correct* identity element for
intersection.

Andy.

On Jul 23, 3:12?am, Andrew Bennetts <and... at bemusement.org> wrote:
> Andy Kish wrote:
>
> [...]
>
>
>
> > The above intersection case would end up looking something like:
>
> > ? ? set_intersection = set.universal()
> > ? ? for s in sets:
> > ? ? ? ? set_intersection &= s
>
> Or even:
>
> ? ?set_intersection = reduce(operator.and_, sets, set.universal())
>
> Although, you can already pass multiple (or zero) sets to set.intersection().
> So your special case version can be a little simpler...
>
> ? ?sets = list(sets)
> ? ?if len(sets) == 0:
> ? ? ? ?return set()
> ? ?return sets[0].intersection(sets[1:])
>
> Which isn't as elegant, but it's also not so bad. ?
>
> -Andrew.
>
> _______________________________________________
> Python-ideas mailing list
> Python-id... at python.orghttp://mail.python.org/mailman/listinfo/python-ideas


From dickinsm at gmail.com  Thu Jul 23 10:39:02 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Thu, 23 Jul 2009 09:39:02 +0100
Subject: [Python-ideas] universal set object for use in set manipulation
In-Reply-To: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com>
References: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com>
Message-ID: <5c6f2a5d0907230139t147e62a7h35a864739c144523@mail.gmail.com>

On Thu, Jul 23, 2009 at 7:28 AM, Andy Kish<agkish at gmail.com> wrote:
> Hi all,
>
> I think the addition of a universal set object would be a nice touch
> for python sets. Manipulation of collections of sets becomes much
> easier with it. [...]

What would set.universal() - {1, 2, 3}  return?  Would this be a
ValueError, or would you want to also implement all cofinite sets?

How would  'for x in set.universal()' behave?

What's len(set.universal())?

It seems to me that set.universal() can't be a full-fledged Python
set, so it would have to be something else.  And then

  x = set.universal().intersection(*sets)

will sometimes be returning a true set (i.e., when sets is nonempty),
and sometimes the reduced-functionality set.universal().  In many
cases you're still going to need to distinguish before doing anything
with x.

Not-all-binary-operations-have-to-have-an-identity-ly yours,

--
Mark


From rrr at ronadam.com  Thu Jul 23 11:14:20 2009
From: rrr at ronadam.com (Ron Adam)
Date: Thu, 23 Jul 2009 04:14:20 -0500
Subject: [Python-ideas] Proposed convenience functions for re module
In-Reply-To: <200907222158.43260.steve@pearwood.info>
References: <200907221000.52871.steve@pearwood.info>
	<4A66C7AE.3050100@ronadam.com>
	<200907222158.43260.steve@pearwood.info>
Message-ID: <4A6829EC.5020702@ronadam.com>



Steven D'Aprano wrote:
> On Wed, 22 Jul 2009 06:02:54 pm Ron Adam wrote:
>> One of the needs I've run across is to enable the program user
>> (possibly a non-programmer) to do logical searches on data.
>>
>> It would be nice if the search patterns specified by the program user
>> could be used directly by the functions.  Search functions of this
>> type would take patterns that are more like what you would use for
>> google or yahoo searches instead of the more complex language re
>> requires.
> 
> I'm not sure if I understand this correctly. Perhaps you could give an 
> example or two?
> 
> Also, please don't overload my simple little proposal with a multitude 
> of new functionality. My proposal is only meant to be a lightweight 
> convenience function. Additional functionality probably belongs as a 
> different function, maybe even a different module.




Yes, it would be a different module and not added directly to the re 
module.  While you are thinking of simplifying re for programmers, I'm 
thinking of simplified searches for users.  A different target and purpose.

I think your functions would make this idea easier to do.


It would be nice if we could do simple logical searches where.

      [word1 word2]            ;get results with either word1 or word2
      [+word1 +word2]          ;get results with both word1 and word2
      [word1 -word2]           ;get results with word1 and not with word2
      ["word one" "word two"]  ;use quotes to search for phrases

And possibly use '*' and '?' as simple wild cards but keep it easy to use 
and simple.  More complex searches should use the re module directly.

This would act as a filter for lists and would be suitable for adding a 
*simple* user search capability to many scrips and applications.

An example would be to enhance pydocs search of the summery lines. 
Currently if you type "modules key", if the key is multiple words, it only 
searches on the first word.  You can not do searches on multiple words or 
exclude results with certain words.

While we could allow regular expression input to work, for many 
applications it is overkill and it is too complex for many users.  For 
example I would not like to try and teach my parents all the subtleties of 
regular expressions when they are struggling to understand a lot more basic 
things.  They don't want to learn how to program computers, they just want 
to get a recipe that has [+chicken +"tomato sauce" -onions].

Ron

















From agkish at gmail.com  Thu Jul 23 06:59:12 2009
From: agkish at gmail.com (Andy Kish)
Date: Wed, 22 Jul 2009 21:59:12 -0700 (PDT)
Subject: [Python-ideas] universal set object for use in set manipulation
Message-ID: <91fdcb21-e7fa-4330-a385-be7b7a361457@24g2000yqm.googlegroups.com>

Hi all,

I think the addition of a universal set object would be a nice touch
for python sets. Manipulation of collections of sets becomes much
easier with it. Right now, certain operations are easy because empty
sets are easy to create while the dual operation requires special
cases:

    set_union = set()
    for s in sets:
        set_union |= s

    # this doesn't work
    set_intersection = set(???)
    for s in sets:
        set_intersection &= s

Instead you have to do something contorted like:

    if len(sets) == 0:
        set_intersection = set()
    else:
        sets_i = iter(sets)
        set_intersection = sets_i.next()
        for s in sets:
            set_intersection &= s

Implementation of a universal set would be pretty trivial. Trails of
EasyExtend [1] has an implementation (albeit used a bit differently)
that's basically the entirety of what's needed.

The above intersection case would end up looking something like:

    set_intersection = set.universal()
    for s in sets:
        set_intersection &= s

Thoughts? I'm happy to write the patch myself if people like this
idea.

Thanks,
Andy Kish.

[1] http://fiber-space.de/wordpress/?p=322


From andrew at bemusement.org  Thu Jul 23 12:19:03 2009
From: andrew at bemusement.org (Andrew Bennetts)
Date: Thu, 23 Jul 2009 20:19:03 +1000
Subject: [Python-ideas] universal set object for use in set manipulation
In-Reply-To: <6f4e8779-ce7a-411d-8b19-24c9fd54a51a@h11g2000yqb.googlegroups.com>
References: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com>
	<20090723071207.GG28628@steerpike.home.puzzling.org>
	<6f4e8779-ce7a-411d-8b19-24c9fd54a51a@h11g2000yqb.googlegroups.com>
Message-ID: <20090723101903.GH28628@steerpike.home.puzzling.org>

Andy Kish wrote:
> If we want to go golfing :), my favorite solution is with the python
> 2.6 versions of intersection and union:
> 
>     set_union = set().union(*sets)
>     set_intersection = set.universal().intersection(*sets)
> 
> Folds are nice. That's actually why I sent my initial email to the
> list. It's really annoying to having appropriate identity element for
> union built in while missing the *correct* identity element for
> intersection.

Well, not that my opinion counts for much, I'm +0 on it.  It's conceptually nice
as you say, but I doubt I (or many others) would use it much, and hanging it off
the set builtin doesn't feel totally satisfactory to me (but I don't have a
better suggestion).

I hope you find some more firmly opinionated responses to your idea.  :)

-Andrew.



From terry at jon.es  Thu Jul 23 13:16:24 2009
From: terry at jon.es (Terry Jones)
Date: Thu, 23 Jul 2009 13:16:24 +0200
Subject: [Python-ideas] universal set object for use in set manipulation
In-Reply-To: Your message at 02:28:33 on Thursday, 23 July 2009
References: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com>
Message-ID: <19048.18056.796635.368294@jon.es>

See also this thread:

  http://mail.python.org/pipermail/python-dev/2006-May/064977.html

I implemented some of what I was proposing there (in Python though, not C),
and am happy to send people the code if there's any interest.

These things can't be 100% clean (e.g., what happens when you call len on
an infinite set), but you can still get a lot of useful functionality if
what's provided matches your use case. Having efficient support for very
large sets via complements, having a Universal set, etc., would all be nice.

Anyway, see the above thread for more discussion / opinions.

Terry


From aahz at pythoncraft.com  Thu Jul 23 14:44:36 2009
From: aahz at pythoncraft.com (Aahz)
Date: Thu, 23 Jul 2009 05:44:36 -0700
Subject: [Python-ideas] Proposed convenience functions for re module
In-Reply-To: <4A6829EC.5020702@ronadam.com>
References: <200907221000.52871.steve@pearwood.info>
	<4A66C7AE.3050100@ronadam.com>
	<200907222158.43260.steve@pearwood.info>
	<4A6829EC.5020702@ronadam.com>
Message-ID: <20090723124436.GD7907@panix.com>

On Thu, Jul 23, 2009, Ron Adam wrote:
>
> While we could allow regular expression input to work, for many  
> applications it is overkill and it is too complex for many users.  For  
> example I would not like to try and teach my parents all the subtleties 
> of regular expressions when they are struggling to understand a lot more 
> basic things.  They don't want to learn how to program computers, they 
> just want to get a recipe that has [+chicken +"tomato sauce" -onions].

This sounds like a *great* addition to PyPI....  ;-)

(That is, something like this is unlikely to make it into Python unless
there's code that has seen uptake in the community.)
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"The volume of a pizza of thickness 'a' and radius 'z' is
given by pi*z*z*a"


From jnoller at gmail.com  Thu Jul 23 14:55:52 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 23 Jul 2009 08:55:52 -0400
Subject: [Python-ideas] Changing the default install location,
	script 	versioning (Packages and PEP 370)
In-Reply-To: <loom.20090722T220918-715@post.gmane.org>
References: <4222a8490907191857m2705b298gd329357ef6912169@mail.gmail.com>
	<loom.20090720T082847-338@post.gmane.org>
	<75F0120D-6B2B-4C38-9943-49CF8A1765E1@gmail.com>
	<loom.20090720T160738-261@post.gmane.org>
	<E7E1CAC0-A802-41A5-B47D-1A6B633F6C75@gmail.com>
	<loom.20090721T084624-784@post.gmane.org>
	<4222a8490907220641v7055c892l2591a301c2fc9cbc@mail.gmail.com>
	<loom.20090722T140926-830@post.gmane.org>
	<4222a8490907220739q14068b98j81e4b869d4cd7cd9@mail.gmail.com>
	<loom.20090722T220918-715@post.gmane.org>
Message-ID: <4222a8490907230555l31127dafn4ea979cb469d3a57@mail.gmail.com>

On Wed, Jul 22, 2009 at 6:10 PM, Antoine Pitrou<solipsis at pitrou.net> wrote:
> Jesse Noller <jnoller at ...> writes:
>>
>> Because they're artifacts of a package installed into a version of the
>> interpreter. Sure, "proper docs" will include a Python2.x section
>> *and* the 3.x section; then again, why not only package the 3.x docs
>> with the 3.x package?
>
> That's a good point.
> Incidentally, data files can be installed in the package's directory if
> `package_data` is used in the setup script. Which obviously versions them.
>

Yup; and there's a sticking point - I'm more than willing with some of
this to chalk this up to "bad package maintainers" - but given the
sorry state of python-packaging in general (not disparaging Tarek's
efforts) and the general confusion about the "One True Way to Do It",
I think we need to do *something* to at least mitigate the issues.

That's why I suggested versioning everything into versioned
directories to begin with - sure, this adds a slight problem in that
you have to do a export
"PATH=$PATH:.local/python2.6/bin:local/python3.1/bin" and so on, but I
feel that's a small problem compared to the "what version does this
script apply to?" confusion.

I'm trying to figure out the best way to handle this for people who
are just trying to get something done, and we both know 2.x and 3.x
are going to be sitting alongside one another for some time. People
like those in this thread can easily work around both the $PATH and
the ./local/bin issues - I just disable .local and stick with
virtualenv, but that doesn't help Jim Bob sitting next to me who is a
total newb.

jesse


From steve at pearwood.info  Thu Jul 23 17:26:08 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 24 Jul 2009 01:26:08 +1000
Subject: [Python-ideas] universal set object for use in set manipulation
In-Reply-To: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com>
References: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com>
Message-ID: <200907240126.08261.steve@pearwood.info>

On Thu, 23 Jul 2009 04:28:33 pm Andy Kish wrote:
> Hi all,
>
> I think the addition of a universal set object would be a nice touch
> for python sets. 

But you can't have a universal set until you know what the problem 
domain is. If the problem domain is citrus fruits, then the universal 
set is: {lemons, limes, grapefruits, oranges, mandarins, tangelos, 
kumquats, ... }

If the problem domain is Presidents of the USA, then the universal set 
is: {Washington, Lincoln, Bush, Clinton, Obama, Wilson, ... }

If the problem domain is integers, then it is {0, 1, -1, 2, -2, 
3, -3, ...}

The first two are finite, the third is infinite.


> Manipulation of collections of sets becomes much 
> easier with it. Right now, certain operations are easy because empty
> sets are easy to create while the dual operation requires special
> cases:
>
>     set_union = set()
>     for s in sets:
>         set_union |= s

Better written as:

>>> reduce(operator.or_, [set([1, 2, 3]), set([2, 4, 5])])
set([1, 2, 3, 4, 5])


>     # this doesn't work
>     set_intersection = set(???)
>     for s in sets:
>         set_intersection &= s

But this does:

>>> reduce(operator.and_, [set([1, 2, 3]), set([2, 4, 5])])
set([2])


> Implementation of a universal set would be pretty trivial.

You think so? I can think of a number of problems.

(1) What elements should go into the universal set? Strings, ints, 
floats, presidents... ?

(2) What is len(set.universal())?

(3) What does set.universal() print?

(4) What does set.universal().clear() do?

(5) For that matter, union(), copy(), difference(), issubset(), etc?

(The universal set for strings of length one is a subset of the 
universal set for all strings.)

I don't think there is a suitable solution for all of these issues. That 
would mean that set.universal() can't be a real set object, it has to 
be some sort of not-quite-a-set object that doesn't provide the entire 
set interface.



-- 
Steven D'Aprano


From ddborowitz at gmail.com  Thu Jul 23 18:19:13 2009
From: ddborowitz at gmail.com (David Borowitz)
Date: Thu, 23 Jul 2009 09:19:13 -0700
Subject: [Python-ideas] universal set object for use in set manipulation
In-Reply-To: <200907240126.08261.steve@pearwood.info>
References: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com> 
	<200907240126.08261.steve@pearwood.info>
Message-ID: <70e6cf130907230919ie6c2140rfec518e6310ad4ed@mail.gmail.com>

Terry Jones briefly mentioned support for very large sets via complements,
which I think would solve many of the issues you bring up.

On Thu, Jul 23, 2009 at 08:26, Steven D'Aprano <steve at pearwood.info> wrote:

> On Thu, 23 Jul 2009 04:28:33 pm Andy Kish wrote:
> > Hi all,
> >
> > I think the addition of a universal set object would be a nice touch
> > for python sets.
>
> But you can't have a universal set until you know what the problem
> domain is. If the problem domain is citrus fruits, then the universal
> set is: {lemons, limes, grapefruits, oranges, mandarins, tangelos,
> kumquats, ... }
>
> If the problem domain is Presidents of the USA, then the universal set
> is: {Washington, Lincoln, Bush, Clinton, Obama, Wilson, ... }
>
> If the problem domain is integers, then it is {0, 1, -1, 2, -2,
> 3, -3, ...}
>
> The first two are finite, the third is infinite.
>

The domain is inherently all objects that can be put into a Python set, so
the universal set is all Python objects. (I think for the purposes of Python
we could gloss over the set-theoretical paradoxiness of "set.universal() in
set.universal()" returning True :)

> Manipulation of collections of sets becomes much
> > easier with it. Right now, certain operations are easy because empty
> > sets are easy to create while the dual operation requires special
> > cases:
> >
> >     set_union = set()
> >     for s in sets:
> >         set_union |= s
>
> Better written as:
>
> >>> reduce(operator.or_, [set([1, 2, 3]), set([2, 4, 5])])
> set([1, 2, 3, 4, 5])
>
>
> >     # this doesn't work
> >     set_intersection = set(???)
> >     for s in sets:
> >         set_intersection &= s
>
> But this does:
>
> >>> reduce(operator.and_, [set([1, 2, 3]), set([2, 4, 5])])
> set([2])
>
>
> > Implementation of a universal set would be pretty trivial.
>
> You think so? I can think of a number of problems.
>
> (1) What elements should go into the universal set? Strings, ints,
> floats, presidents... ?
>

There are two ways of defining what is "in" a set. One is what
iter(set.universal()) iterates through, which there is no good answer for.
But the other (arguably just as common) way to define what is "in" a set s
is "the set of Python objects p such that "p in s" returns True." This
latter definition is trivial to implement for the universal set. If we keep
track of the complement of a nearly-universal set, that also gives us an
easy way to implement in.

(2) What is len(set.universal())?
>

inf. Perhaps there is a complication since inf is technically a float type?
I don't see it being a huge deal. ("Real" universal sets have uncountably
many elements, whereas AFAICT floating-point inf is countable; since we have
finite-memory systems I can't imagine it makes a difference.)

(3) What does set.universal() print?
>

Some special-case string; maybe "set(U)" for the universal set, or
"set(U-[elements,in,complement])" for general nearly-infinite sets.

(4) What does set.universal().clear() do?
>

set.universal() doesn't have to be a singleton; it could return a brand new,
mutable universal set. So set.universal() returns a new set, and .clear()
clears a set regardless of what it contains.

(5) For that matter, union(), copy(), difference(), issubset(), etc?
>

I think these all fall pretty easily out of the definition of a universal
set and a complement representation of nearly infinite sets.

(The universal set for strings of length one is a subset of the
> universal set for all strings.)
>

The universal set is a superset of all of these: it is the set of all Python
objects. (Yes, that's paradoxical, but again, I don't think that matters for
most use cases in Python.)

I don't think there is a suitable solution for all of these issues. That
> would mean that set.universal() can't be a real set object, it has to
> be some sort of not-quite-a-set object that doesn't provide the entire
> set interface.
>
>
Storing the complement of an infinite set is pretty straightforward way of
implementing sets that are universal except for a few (maybe zero) elements,
and can satisfy the entire set interface except for iteration. As you
suggest, it would be far trickier to provide general infinite sets, like the
set of all integers, but that's not what's being asked for here.

I'm not saying it won't be tricky to implement; it requires changing pretty
much all set methods, I've never touched CPython code. I'm also roughly +0
on the issue (not that my vote counts since I lurk far more than I
contribute). I'm just pointing out it's possible.


>
>
> --
> Steven D'Aprano
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
It is better to be quotable than to be honest.
   -Tom Stoppard

Borowitz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090723/4078d427/attachment.html>

From g.brandl at gmx.net  Thu Jul 23 18:22:47 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 23 Jul 2009 18:22:47 +0200
Subject: [Python-ideas] universal set object for use in set manipulation
In-Reply-To: <200907240126.08261.steve@pearwood.info>
References: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com>
	<200907240126.08261.steve@pearwood.info>
Message-ID: <h4a2qa$63j$1@ger.gmane.org>

Steven D'Aprano schrieb:

>> Implementation of a universal set would be pretty trivial.
> 
> You think so? I can think of a number of problems.

First, I don't think this is important enough to become a standard
Python type.  But regardless, these problems are quite easy to answer.

> (1) What elements should go into the universal set? Strings, ints, 
> floats, presidents... ?

Everything. Every possible (i.e. hashable) object is a (virtual) member
of the universal set.  You're thinking too mathematically here.  Python has
no notion of a category of elements for ordinary sets as well -- you can
put a president in a set together with integers without problems.

> (2) What is len(set.universal())?

Since it was apparently decided that len() can "lie", sys.maxsize would be
a nice value.  Otherwise, an exception.

> (3) What does set.universal() print?

'set.universal()'.

> (4) What does set.universal().clear() do?

It clears the set. Afterwards, the set is empty.

> (5) For that matter, union(), copy(), difference(), issubset(), etc?

union() is a no-op, as well as any operation that adds elements.
intersection() just returns the "other" set.
copy() just returns another instance of the universal set.
issubset() - the universal set is the subset of no other set except
the universal set.

To make difference() or remove() and pop() possible, one would have to
expand the notion to a "universal-except" set which again has a finite
number of exceptions.

> (The universal set for strings of length one is a subset of the 
> universal set for all strings.)

Again, see the comment for (1).

> I don't think there is a suitable solution for all of these issues. That 
> would mean that set.universal() can't be a real set object, it has to 
> be some sort of not-quite-a-set object that doesn't provide the entire 
> set interface.

Of course, it could not be an ordinary set object (i.e. one actually containing
references to its members) -- that would require an infinite amount of memory.
However, it *can* implement much of the set interface without a problem.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.



From p.f.moore at gmail.com  Thu Jul 23 18:50:11 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 23 Jul 2009 17:50:11 +0100
Subject: [Python-ideas] Proposed convenience functions for re module
In-Reply-To: <h48cuq$j7d$1@ger.gmane.org>
References: <200907221000.52871.steve@pearwood.info> <4A673CFD.4030005@acm.org>
	<h47qlt$23k$1@ger.gmane.org> <4A67B386.6010308@canterbury.ac.nz>
	<h48cuq$j7d$1@ger.gmane.org>
Message-ID: <79990c6b0907230950q20db9b91l7f91fed126ee4640@mail.gmail.com>

2009/7/23 Robert Kern <robert.kern at gmail.com>:
> On 2009-07-22 19:49, Greg Ewing wrote:
>>
>> Georg Brandl wrote:
>>
>>> You might be interested in the undocumented re.Scanner class :)
>>
>> It looks like it could be interesting, but with no
>> documentation, how are we supposed to tell?
>>
>> Is there *any* information about it available
>> anywhere?
>
> http://mail.python.org/pipermail/python-dev/2003-April/035075.html

Question: Is there any reason (other than lack of time) why it's undocumented?

I'd be willing to write some documentation, but only if it would stand
a chance of being accepted - this isn't an itch of mine, so I don't
want to spend ages arguing over whether the class should be
documented.

The source code says "experimental stuff (see python-dev discussions
for details)". I've not searched the python-dev archives yet, but it
seems to me that it'll never be anything other than experimental if
people don't know it's there and try it out...

Paul.


From robert.kern at gmail.com  Thu Jul 23 19:06:47 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 23 Jul 2009 12:06:47 -0500
Subject: [Python-ideas] Proposed convenience functions for re module
In-Reply-To: <79990c6b0907230950q20db9b91l7f91fed126ee4640@mail.gmail.com>
References: <200907221000.52871.steve@pearwood.info>
	<4A673CFD.4030005@acm.org>	<h47qlt$23k$1@ger.gmane.org>
	<4A67B386.6010308@canterbury.ac.nz>	<h48cuq$j7d$1@ger.gmane.org>
	<79990c6b0907230950q20db9b91l7f91fed126ee4640@mail.gmail.com>
Message-ID: <h4a5b8$eb6$2@ger.gmane.org>

On 2009-07-23 11:50, Paul Moore wrote:
> 2009/7/23 Robert Kern<robert.kern at gmail.com>:
>> On 2009-07-22 19:49, Greg Ewing wrote:
>>> Georg Brandl wrote:
>>>
>>>> You might be interested in the undocumented re.Scanner class :)
>>> It looks like it could be interesting, but with no
>>> documentation, how are we supposed to tell?
>>>
>>> Is there *any* information about it available
>>> anywhere?
>> http://mail.python.org/pipermail/python-dev/2003-April/035075.html
>
> Question: Is there any reason (other than lack of time) why it's undocumented?

http://mail.python.org/pipermail/python-dev/2003-April/035070.html

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From steve at pearwood.info  Thu Jul 23 20:00:11 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 24 Jul 2009 04:00:11 +1000
Subject: [Python-ideas] universal set object for use in set manipulation
In-Reply-To: <h4a2qa$63j$1@ger.gmane.org>
References: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com>
	<200907240126.08261.steve@pearwood.info>
	<h4a2qa$63j$1@ger.gmane.org>
Message-ID: <200907240400.11744.steve@pearwood.info>

On Fri, 24 Jul 2009 02:22:47 am Georg Brandl wrote:

> > (1) What elements should go into the universal set? Strings, ints,
> > floats, presidents... ?
>
> Everything. Every possible (i.e. hashable) object is a (virtual)
> member of the universal set.  You're thinking too mathematically
> here.  Python has no notion of a category of elements for ordinary
> sets as well -- you can put a president in a set together with
> integers without problems.

No, I'm not thinking mathematically, I'm thinking concretely. A set that 
contains "everything" is an abstract concept. For concrete problem 
solving, the universe is (nearly always) smaller than "everything". 
It's everything within the problem domain, not everything imaginable.


> > (2) What is len(set.universal())?
>
> Since it was apparently decided that len() can "lie", sys.maxsize
> would be a nice value.  Otherwise, an exception.

Both are problematic.

Firstly, if it raises an exception, then it means all set handling code 
needs to be written (e.g.):

try:
    len(s)
except Exception:
    # handle universal set
    print "What do I do here?"


instead of the simple:

len(s)


The same for anything which iterates over a set, or calls pop() or 
remove(). That's bad -- we're complicating all set-handling code, just 
to make *one* special case a tiny bit not-really-simpler (actually more 
complicated -- the solution with reduce is simpler than the suggested 
idiom).


Secondly, if it returns sys.maxint, that's not as bad, but it's still 
wrong -- sys.maxint is being used as a sentinel, a "magic value", 
rather than actually being the length. If you count the elements that 
are in the set, you get more than the length:


n = 0
U = set.universal()
for i in xrange(-sys.maxint, sys.maxint):
    # this may take a while...
    if i in U:
        n += 1
assert n > len(U)


To me, having len() be accurate is *far* more important than being able 
to supposedly simplify (complexify?) a one-liner using reduce into a 
three-liner using set.universal(). Having a built-in type lie about its 
length gives me the willies.


[...]
> Of course, it could not be an ordinary set object (i.e. one actually
> containing references to its members) -- that would require an
> infinite amount of memory. However, it *can* implement much of the
> set interface without a problem.

Fair enough, but it's the parts that it can't implement which are 
critical. It's not a set, it's something which is nearly a set but is 
supposed to be used in combination with real sets. Any time you write 
code that expects sets, you have to allow for the risk that you might 
be given a not-really-a-set universal set instead.



-- 
Steven D'Aprano


From ncoghlan at gmail.com  Thu Jul 23 23:01:40 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 24 Jul 2009 07:01:40 +1000
Subject: [Python-ideas] universal set object for use in set manipulation
In-Reply-To: <200907240126.08261.steve@pearwood.info>
References: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com>
	<200907240126.08261.steve@pearwood.info>
Message-ID: <4A68CFB4.2020801@gmail.com>

Steven D'Aprano wrote:
> I don't think there is a suitable solution for all of these issues. That 
> would mean that set.universal() can't be a real set object, it has to 
> be some sort of not-quite-a-set object that doesn't provide the entire 
> set interface.

That doesn't strike me as all that different from handling NaN and Inf
values in floating point mathematics though. Would set.universal() need
to be a special object that required special handling in some set
algorithms? Yes, it would, but that doesn't make it an invalid idea.

In fact, treating it like a NaN from the decimal module is probably the
answer to most of those questions: if a valid answer isn't defined,
raise an exception.

The semantics would need to be spelled out clearly however, as would a
position on whether or not to add support for complementary sets (i.e.
sets that are defined as "set.universal() ^ items_not_in_set").

Also the "universal" or "complementary" character of the set would need
to be embodied in mutable state on the set object so that in-place
operations such as "s |= set.universal()" can work correctly.

Cheers,
Nick.

P.S. I'm not necessarily +1, or even +0, on the idea until it is fleshed
out further. I just wanted to point out that there is precedent for
including the numerical equivalents of the universal set.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From tjreedy at udel.edu  Thu Jul 23 23:06:56 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 23 Jul 2009 17:06:56 -0400
Subject: [Python-ideas] universal set object for use in set manipulation
In-Reply-To: <19048.18056.796635.368294@jon.es>
References: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com>
	<19048.18056.796635.368294@jon.es>
Message-ID: <h4ajdg$367$1@ger.gmane.org>

Terry Jones wrote:
> See also this thread:
> 
>   http://mail.python.org/pipermail/python-dev/2006-May/064977.html
> 
> I implemented some of what I was proposing there (in Python though, not C),
> and am happy to send people the code if there's any interest.
> 
> These things can't be 100% clean (e.g., what happens when you call len on
> an infinite set), but you can still get a lot of useful functionality if
> what's provided matches your use case. Having efficient support for very
> large sets via complements, having a Universal set, etc., would all be nice.
> 
> Anyway, see the above thread for more discussion / opinions.

Did you register a module with PyPI?

In any case, it seems to me that adding a 'universal set' to the set 
class does not work.  On the otherhand, a separate set-complement class 
(which I am guessing is what you did) should work fine as far as it 
goes. Each instance would have normal set as its data member, and 
operations on complements and mixed operations would be defined in terms 
of operations on sets. A universal set would then be an instance with an 
empty set for a complement. I think the OP should look in this 
direction, possibly building on your existing code.

Terry Jan Reedy




From ncoghlan at gmail.com  Thu Jul 23 23:08:34 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 24 Jul 2009 07:08:34 +1000
Subject: [Python-ideas] universal set object for use in set manipulation
In-Reply-To: <200907240400.11744.steve@pearwood.info>
References: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com>	<200907240126.08261.steve@pearwood.info>	<h4a2qa$63j$1@ger.gmane.org>
	<200907240400.11744.steve@pearwood.info>
Message-ID: <4A68D152.60105@gmail.com>

Steven D'Aprano wrote:
> Firstly, if it raises an exception, then it means all set handling code 
> needs to be written (e.g.):
> 
> try:
>     len(s)
> except Exception:
>     # handle universal set
>     print "What do I do here?"
> 
> 
> instead of the simple:
> 
> len(s)

There is a lot of floating point code in the world that will blow up if
fed a Nan or an Inf value as input. Given that it is even harder for a
set.universal() instance to show up by accident, I don't see that it
would be a major problem if most set handling code remained "naive"
about the universal set.

That said, I'm still with Georg in not being convinced yet that the
mathematical elegance is worth the additional complexity.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From gerald.britton at gmail.com  Thu Jul 23 23:48:04 2009
From: gerald.britton at gmail.com (Gerald Britton)
Date: Thu, 23 Jul 2009 17:48:04 -0400
Subject: [Python-ideas] Proposed convenience functions for re module
In-Reply-To: <h4a5b8$eb6$2@ger.gmane.org>
References: <200907221000.52871.steve@pearwood.info>
	<4A673CFD.4030005@acm.org> 
	<h47qlt$23k$1@ger.gmane.org> <4A67B386.6010308@canterbury.ac.nz> 
	<h48cuq$j7d$1@ger.gmane.org>
	<79990c6b0907230950q20db9b91l7f91fed126ee4640@mail.gmail.com> 
	<h4a5b8$eb6$2@ger.gmane.org>
Message-ID: <5d1a32000907231448s14270c93l548da69a8ae8554@mail.gmail.com>

So, in 2003 it was discussed but considered not ready for prime time.
If six years is not enough to get it ready, I can't imagine what would
be.

On Thu, Jul 23, 2009 at 1:06 PM, Robert Kern<robert.kern at gmail.com> wrote:
> On 2009-07-23 11:50, Paul Moore wrote:
>>
>> 2009/7/23 Robert Kern<robert.kern at gmail.com>:
>>>
>>> On 2009-07-22 19:49, Greg Ewing wrote:
>>>>
>>>> Georg Brandl wrote:
>>>>
>>>>> You might be interested in the undocumented re.Scanner class :)
>>>>
>>>> It looks like it could be interesting, but with no
>>>> documentation, how are we supposed to tell?
>>>>
>>>> Is there *any* information about it available
>>>> anywhere?
>>>
>>> http://mail.python.org/pipermail/python-dev/2003-April/035075.html
>>
>> Question: Is there any reason (other than lack of time) why it's
>> undocumented?
>
> http://mail.python.org/pipermail/python-dev/2003-April/035070.html
>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless enigma
> ?that is made terrible by our own mad attempt to interpret it as though it
> had
> ?an underlying truth."
> ?-- Umberto Eco
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
Gerald Britton


From greg.ewing at canterbury.ac.nz  Fri Jul 24 00:13:03 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 24 Jul 2009 10:13:03 +1200
Subject: [Python-ideas] universal set object for use in set manipulation
In-Reply-To: <6f4e8779-ce7a-411d-8b19-24c9fd54a51a@h11g2000yqb.googlegroups.com>
References: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com>
	<20090723071207.GG28628@steerpike.home.puzzling.org>
	<6f4e8779-ce7a-411d-8b19-24c9fd54a51a@h11g2000yqb.googlegroups.com>
Message-ID: <4A68E06F.20900@canterbury.ac.nz>

Andy Kish wrote:

> It's really annoying to having appropriate identity element for
> union built in while missing the *correct* identity element for
> intersection.

What would

    for x in set.universal():
      ...

do?

-- 
Greg


From george.sakkis at gmail.com  Fri Jul 24 00:29:14 2009
From: george.sakkis at gmail.com (George Sakkis)
Date: Thu, 23 Jul 2009 18:29:14 -0400
Subject: [Python-ideas] universal set object for use in set manipulation
In-Reply-To: <4A68E06F.20900@canterbury.ac.nz>
References: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com>
	<20090723071207.GG28628@steerpike.home.puzzling.org>
	<6f4e8779-ce7a-411d-8b19-24c9fd54a51a@h11g2000yqb.googlegroups.com>
	<4A68E06F.20900@canterbury.ac.nz>
Message-ID: <91ad5bf80907231529h6a21c9d5ye29d7c96ba0b578d@mail.gmail.com>

On Thu, Jul 23, 2009 at 6:13 PM, Greg Ewing<greg.ewing at canterbury.ac.nz> wrote:
> Andy Kish wrote:
>
>> It's really annoying to having appropriate identity element for
>> union built in while missing the *correct* identity element for
>> intersection.
>
> What would
>
> ? for x in set.universal():
> ? ? ...
>
> do?

Overall it seems like a non-solution to a non-problem. Why don't we
add a dict.universal(), list.universal(), etc. while we're at it.
Easily a -1.

George


From python at rcn.com  Fri Jul 24 00:40:02 2009
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 23 Jul 2009 18:40:02 -0400 (EDT)
Subject: [Python-ideas] universal set object for use in set manipulation
Message-ID: <20090723184002.BKW34389@ms19.lnh.mail.rcn.net>

[Andy Kish]
>    set_intersection = set.universal()
>    for s in sets:
>        set_intersection &= s

-1 This complicates the API and the implementation for very little benefit (saving you from writing a tiny, clear helper function for an uncommon use case).

Currently, sets enjoy a near zero learning curve.  That would be lost by adding set.universal() whose semantics are not immediately obvious -- for example, with s=set.universal() what is the meaning of list(s) or frozenset(s) or s.pop(); what is its repr value; and what is s^set('abc') or s-set('abc') or other operations?  You may be able to come-up with definitions that work, but those won't be intuitive to most users.


Raymond


From benjamin at python.org  Fri Jul 24 00:45:05 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 23 Jul 2009 22:45:05 +0000 (UTC)
Subject: [Python-ideas] universal set object for use in set manipulation
References: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com>
	<200907240126.08261.steve@pearwood.info>
	<h4a2qa$63j$1@ger.gmane.org>
Message-ID: <loom.20090723T224419-343@post.gmane.org>

Georg Brandl <g.brandl at ...> writes:
> Steven D'Aprano schrieb:
> > (2) What is len(set.universal())?
> 
> Since it was apparently decided that len() can "lie", sys.maxsize would be
> a nice value.  Otherwise, an exception.

I don't that was ever decided. Guido just suggested it.






From robert.kern at gmail.com  Fri Jul 24 00:47:32 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 23 Jul 2009 17:47:32 -0500
Subject: [Python-ideas] Proposed convenience functions for re module
In-Reply-To: <5d1a32000907231448s14270c93l548da69a8ae8554@mail.gmail.com>
References: <200907221000.52871.steve@pearwood.info>	<4A673CFD.4030005@acm.org>
	<h47qlt$23k$1@ger.gmane.org> <4A67B386.6010308@canterbury.ac.nz>
	<h48cuq$j7d$1@ger.gmane.org>	<79990c6b0907230950q20db9b91l7f91fed126ee4640@mail.gmail.com>
	<h4a5b8$eb6$2@ger.gmane.org>
	<5d1a32000907231448s14270c93l548da69a8ae8554@mail.gmail.com>
Message-ID: <h4apa5$jak$1@ger.gmane.org>

On 2009-07-23 16:48, Gerald Britton wrote:
> So, in 2003 it was discussed but considered not ready for prime time.
> If six years is not enough to get it ready, I can't imagine what would
> be.

I'm pretty sure that no one has worked on it.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From mwm-keyword-python.b4bdba at mired.org  Fri Jul 24 01:04:54 2009
From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer)
Date: Thu, 23 Jul 2009 19:04:54 -0400
Subject: [Python-ideas] universal set object for use in set manipulation
In-Reply-To: <91ad5bf80907231529h6a21c9d5ye29d7c96ba0b578d@mail.gmail.com>
References: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com>
	<20090723071207.GG28628@steerpike.home.puzzling.org>
	<6f4e8779-ce7a-411d-8b19-24c9fd54a51a@h11g2000yqb.googlegroups.com>
	<4A68E06F.20900@canterbury.ac.nz>
	<91ad5bf80907231529h6a21c9d5ye29d7c96ba0b578d@mail.gmail.com>
Message-ID: <20090723190454.082285b2@bhuda.mired.org>

On Thu, 23 Jul 2009 18:29:14 -0400
George Sakkis <george.sakkis at gmail.com> wrote:
> Overall it seems like a non-solution to a non-problem.

It's definitely a real solution. The problem may not be very
important, though.

> Why don't we add a dict.universal(), list.universal(), etc. while
> we're at it.

Because a set of "all the things in the universe" makes sense as an
entity, whereas dict's or lists of "all the things in the universe"
don't. For lists, this implies some sort of ordering on everything in
the universe, which we've already abandoned as a bad idea. Dicts, on
the other hand, aren't simple collections, but maps, so you don't have
_a_ dict of everything in the universe, you have len(universe)**2 of
them. The def statement gives us the ability to express such maps.

Which may be how set.universal() stacks up: We can already create a
class that has the appropriate behaviors. It's not clear that that is
sufficiently useful to justify adding it to the language.

One thing that bugs me is the universal set is a *constant. It's value
doesn't change, so it shouldn't be a function or method, but an
attribute.

On the other hand, adding negative sets to the language provides an
easy way to express that constant, and certainly have more uses in
general. If we had to add one of the two, it'd be these. But I'm not
convinced that these have enough uses to justify adding them, either.

	  <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


From m.lenzen at gmail.com  Fri Jul 24 02:33:26 2009
From: m.lenzen at gmail.com (Michael Lenzen)
Date: Thu, 23 Jul 2009 19:33:26 -0500
Subject: [Python-ideas] universal set object for use in set manipulation
In-Reply-To: <20090723184002.BKW34389@ms19.lnh.mail.rcn.net>
References: <20090723184002.BKW34389@ms19.lnh.mail.rcn.net>
Message-ID: <4A690156.5@gmail.com>

-.5

You'd have to come up with an entire system for infinite sets for this 
to work, for the reasons Raymond pointed out.  You'd also have to be 
able to define what the universe is, integers, strings, strings of 
length x, etc.

-Michael Lenzen

On 07/23/2009 05:40 PM, Raymond Hettinger wrote:
> [Andy Kish]
>>     set_intersection = set.universal()
>>     for s in sets:
>>         set_intersection&= s
>
> -1 This complicates the API and the implementation for very little benefit (saving you from writing a tiny, clear helper function for an uncommon use case).
>
> Currently, sets enjoy a near zero learning curve.  That would be lost by adding set.universal() whose semantics are not immediately obvious -- for example, with s=set.universal() what is the meaning of list(s) or frozenset(s) or s.pop(); what is its repr value; and what is s^set('abc') or s-set('abc') or other operations?  You may be able to come-up with definitions that work, but those won't be intuitive to most users.
>
>
> Raymond
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas


From zuo at chopin.edu.pl  Fri Jul 24 02:48:38 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Fri, 24 Jul 2009 02:48:38 +0200
Subject: [Python-ideas] universal set object for use in set manipulation
In-Reply-To: <20090723190454.082285b2@bhuda.mired.org>
References: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com>
	<20090723071207.GG28628@steerpike.home.puzzling.org>
	<6f4e8779-ce7a-411d-8b19-24c9fd54a51a@h11g2000yqb.googlegroups.com>
	<4A68E06F.20900@canterbury.ac.nz>
	<91ad5bf80907231529h6a21c9d5ye29d7c96ba0b578d@mail.gmail.com>
	<20090723190454.082285b2@bhuda.mired.org>
Message-ID: <op.uxjq7c0rfvx12t@jerzozwiesz.home.aster.pl>

Mike Meyer <mwm-keyword-python.b4bdba at mired.org>:

> On the other hand, adding negative sets to the language provides an
> easy way to express that constant, and certainly have more uses in
> general. If we had to add one of the two, it'd be these. But I'm not
> convinced that these have enough uses to justify adding them, either.

Yeah, I feel, that if a group of people need them, they should have
a separate type ("negative" or "infitity-minus..." or "universe
without..."-) sets, in a separate module (not necessarily in std
library).

It would be a type which can interact in some ways with sets (like
e.g. dict views do...).

As it was noted -- some sets' features don't make sense here...
(len, iter).

But many do:

a in universe_minus()
-> True

a in universe_minus([a, b, c])
-> False

set([a, b]) < universe_minus()
-> True

set([a, b]) < universe_minus([b, c])
-> False

universe_minus().remove(a)
-> universe_minus([a])

universe_minus([a, b, c]).remove(a)
-> KeyError

universe_minus([a]).add(a)
-> universe_minus([])

universe_minus().add(a)
-> universe_minus([])

universe_minus() - set([a, b, c])
-> universe_minus([a, b, c])

universe_minus() & set([a, b, c])
-> set([a, b, c])

universe_minus() | set([a, b, c])
-> universe_minus()

universe_minus() - universe_minus([a, b])
-> set([a, b])

universe_minus() & universe_minus([a, b])
-> universe_minus([a, b])

universe_minus() | universe_minus([a, b])
-> universe_minus()

universe_minus([a, b, c]) - set([a, b])
-> universe_minus([a, b, c])

universe_minus([a, b, c]) | set([a, b])
-> universe_minus([c])

universe_minus([a, b]) ^ set([a, b, c])
-> universe_minus([c])

etc.


From python at mrabarnett.plus.com  Fri Jul 24 03:22:46 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 24 Jul 2009 02:22:46 +0100
Subject: [Python-ideas] Proposed convenience functions for re module
In-Reply-To: <h4apa5$jak$1@ger.gmane.org>
References: <200907221000.52871.steve@pearwood.info>	<4A673CFD.4030005@acm.org>	<h47qlt$23k$1@ger.gmane.org>
	<4A67B386.6010308@canterbury.ac.nz>	<h48cuq$j7d$1@ger.gmane.org>	<79990c6b0907230950q20db9b91l7f91fed126ee4640@mail.gmail.com>	<h4a5b8$eb6$2@ger.gmane.org>	<5d1a32000907231448s14270c93l548da69a8ae8554@mail.gmail.com>
	<h4apa5$jak$1@ger.gmane.org>
Message-ID: <4A690CE6.7000605@mrabarnett.plus.com>

Robert Kern wrote:
> On 2009-07-23 16:48, Gerald Britton wrote:
>> So, in 2003 it was discussed but considered not ready for prime time.
>> If six years is not enough to get it ready, I can't imagine what would
>> be.
> 
> I'm pretty sure that no one has worked on it.
> 
I worked on it in issue #2636 (turning unnamed capture groups into
non-capture groups and forbidding group references and named groups) and
also added a generator method.


From tjreedy at udel.edu  Fri Jul 24 07:53:11 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 24 Jul 2009 01:53:11 -0400
Subject: [Python-ideas] universal set object for use in set manipulation
In-Reply-To: <op.uxjq7c0rfvx12t@jerzozwiesz.home.aster.pl>
References: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com>	<20090723071207.GG28628@steerpike.home.puzzling.org>	<6f4e8779-ce7a-411d-8b19-24c9fd54a51a@h11g2000yqb.googlegroups.com>	<4A68E06F.20900@canterbury.ac.nz>	<91ad5bf80907231529h6a21c9d5ye29d7c96ba0b578d@mail.gmail.com>	<20090723190454.082285b2@bhuda.mired.org>
	<op.uxjq7c0rfvx12t@jerzozwiesz.home.aster.pl>
Message-ID: <h4bi86$2oo$2@ger.gmane.org>

Jan Kaliszewski wrote:
> Mike Meyer 
> <mwm-keyword-python.b4bdba at mired.org>:
> 
>> On the other hand, adding negative sets to the language provides an
>> easy way to express that constant, and certainly have more uses in
>> general. If we had to add one of the two, it'd be these. But I'm not
>> convinced that these have enough uses to justify adding them, either.
> 
> Yeah, I feel, that if a group of people need them, they should have
> a separate type ("negative" or "infitity-minus..." or "universe
> without..."-) sets, in a separate module (not necessarily in std
> library).

It should definitely start on PyPI. Since most of the work of methods 
would be done by the underlying system set (or frozenset) type, a Python 
implementation should be fast enough.
> 
> It would be a type which can interact in some ways with sets (like
> e.g. dict views do...).
> 
> As it was noted -- some sets' features don't make sense here...
> (len, iter).
> 
> But many do:
> 
> a in universe_minus()
> -> True
> 
> a in universe_minus([a, b, c])
> -> False
> 
> set([a, b]) < universe_minus()
> -> True
> 
> set([a, b]) < universe_minus([b, c])
> -> False
> 
> universe_minus().remove(a)
> -> universe_minus([a])
> 
> universe_minus([a, b, c]).remove(a)
> -> KeyError
> 
> universe_minus([a]).add(a)
> -> universe_minus([])
> 
> universe_minus().add(a)
> -> universe_minus([])
> 
> universe_minus() - set([a, b, c])
> -> universe_minus([a, b, c])
> 
> universe_minus() & set([a, b, c])
> -> set([a, b, c])
> 
> universe_minus() | set([a, b, c])
> -> universe_minus()
> 
> universe_minus() - universe_minus([a, b])
> -> set([a, b])
> 
> universe_minus() & universe_minus([a, b])
> -> universe_minus([a, b])
> 
> universe_minus() | universe_minus([a, b])
> -> universe_minus()
> 
> universe_minus([a, b, c]) - set([a, b])
> -> universe_minus([a, b, c])
> 
> universe_minus([a, b, c]) | set([a, b])
> -> universe_minus([c])
> 
> universe_minus([a, b]) ^ set([a, b, c])
> -> universe_minus([c])
> 
> etc.



From g.brandl at gmx.net  Fri Jul 24 11:46:24 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 24 Jul 2009 11:46:24 +0200
Subject: [Python-ideas] universal set object for use in set manipulation
In-Reply-To: <4A68E06F.20900@canterbury.ac.nz>
References: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com>	<20090723071207.GG28628@steerpike.home.puzzling.org>	<6f4e8779-ce7a-411d-8b19-24c9fd54a51a@h11g2000yqb.googlegroups.com>
	<4A68E06F.20900@canterbury.ac.nz>
Message-ID: <h4bvv3$8be$1@ger.gmane.org>

Greg Ewing schrieb:
> Andy Kish wrote:
> 
>> It's really annoying to having appropriate identity element for
>> union built in while missing the *correct* identity element for
>> intersection.
> 
> What would
> 
>     for x in set.universal():
>       ...

Since it has to yield all possible Python objects, it could start with
the integers.  It's not likely to run out of them any time soon

<duck>
Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.



From terry at jon.es  Fri Jul 24 12:07:20 2009
From: terry at jon.es (Terry Jones)
Date: Fri, 24 Jul 2009 12:07:20 +0200
Subject: [Python-ideas] universal set object for use in set manipulation
In-Reply-To: Your message at 17:06:56 on Thursday, 23 July 2009
References: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com>
	<19048.18056.796635.368294@jon.es> <h4ajdg$367$1@ger.gmane.org>
Message-ID: <19049.34776.390781.583216@jon.es>

>>>>> "Terry" == Terry Reedy <tjreedy at udel.edu> writes:
Terry> Terry Jones wrote:
>> See also this thread:
>> 
>> http://mail.python.org/pipermail/python-dev/2006-May/064977.html
>> 
>> I implemented some of what I was proposing there (in Python though, not C),
>> and am happy to send people the code if there's any interest.
[snip]
Terry> Did you register a module with PyPI?

No.

Terry> In any case, it seems to me that adding a 'universal set' to the set
Terry> class does not work.  On the otherhand, a separate set-complement
Terry> class (which I am guessing is what you did) should work fine as far
Terry> as it goes. Each instance would have normal set as its data member,
Terry> and operations on complements and mixed operations would be defined
Terry> in terms of operations on sets. A universal set would then be an
Terry> instance with an empty set for a complement.

Yes, that's all exactly what I did.  I'll clean it up a little and post it
somewhere, PyPI I guess.

BTW, in my original posting I was wondering about this being added to
Python as a generalization of the set type. After the discussion in the
above thread and thinking through the various oddities it would introduce,
I also concluded that would be a mistake - there's just too much that's a
bit weird.  Raymond is right that a strong virtue of the current set module
is its simplicity. It's not worth blowing that so as to add a more general
behavior that very few people will ever use.  But it would be a nice 3rd
party module.

Terry


From terry at jon.es  Fri Jul 24 14:38:31 2009
From: terry at jon.es (Terry Jones)
Date: Fri, 24 Jul 2009 14:38:31 +0200
Subject: [Python-ideas] universal set object for use in set manipulation
In-Reply-To: Your message at 17:06:56 on Thursday, 23 July 2009
References: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com>
	<19048.18056.796635.368294@jon.es> <h4ajdg$367$1@ger.gmane.org>
Message-ID: <19049.43847.52697.307391@jon.es>

>>>>> "Terry" == Terry Reedy <tjreedy at udel.edu> writes:
Terry> Did you register a module with PyPI?

OK, I put my original set code into http://pypi.python.org/pypi/compset/0.1

It's just a quick hack, but it works. I didn't implement all the ideas I
suggested in http://mail.python.org/pipermail/python-dev/2006-May/064977.html
(e.g., universeExclusionFunc) but they'd be easy to add.

Hopefully someone can take that code as a starting point for a more fully
fleshed out package. The test suite could do with some love too.

Terry


From ncoghlan at gmail.com  Sat Jul 25 01:55:56 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 25 Jul 2009 09:55:56 +1000
Subject: [Python-ideas] universal set object for use in set manipulation
In-Reply-To: <4A68E06F.20900@canterbury.ac.nz>
References: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com>	<20090723071207.GG28628@steerpike.home.puzzling.org>	<6f4e8779-ce7a-411d-8b19-24c9fd54a51a@h11g2000yqb.googlegroups.com>
	<4A68E06F.20900@canterbury.ac.nz>
Message-ID: <4A6A4A0C.4050605@gmail.com>

Greg Ewing wrote:
> Andy Kish wrote:
> 
>> It's really annoying to having appropriate identity element for
>> union built in while missing the *correct* identity element for
>> intersection.
> 
> What would
> 
>    for x in set.universal():
>      ...
> 
> do?

Raise an exception, just like a signalling NaN in decimal. An expanded
set concept with support for complementary set definitions is definitely
something that should cook on PyPI for a while though.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From terry at jon.es  Sat Jul 25 02:09:15 2009
From: terry at jon.es (Terry Jones)
Date: Sat, 25 Jul 2009 02:09:15 +0200
Subject: [Python-ideas] universal set object for use in set manipulation
In-Reply-To: Your message at 09:55:56 on Saturday, 25 July 2009
References: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com>
	<20090723071207.GG28628@steerpike.home.puzzling.org>
	<6f4e8779-ce7a-411d-8b19-24c9fd54a51a@h11g2000yqb.googlegroups.com>
	<4A68E06F.20900@canterbury.ac.nz> <4A6A4A0C.4050605@gmail.com>
Message-ID: <19050.19755.891763.900896@jon.es>

>>>>> "Nick" == Nick Coghlan <ncoghlan at gmail.com> writes:
Nick> Greg Ewing wrote:
>> Andy Kish wrote:
>> 
>>> It's really annoying to having appropriate identity element for
>>> union built in while missing the *correct* identity element for
>>> intersection.
>> 
>> What would
>> 
>> for x in set.universal():
>> ...
>> 
>> do?

Nick> Raise an exception, just like a signalling NaN in decimal.

Agreed. The code I stuck on PyPI today raises InfiniteSetError on __len__
and __iter__.

Nick> An expanded set concept with support for complementary set
Nick> definitions is definitely something that should cook on PyPI for a
Nick> while though.

Just to repeat the URL: http://pypi.python.org/pypi/compset/0.1

It's had exactly zero downloads :-)

Terry


From fuzzyman at gmail.com  Sat Jul 25 02:28:54 2009
From: fuzzyman at gmail.com (Michael Foord)
Date: Sat, 25 Jul 2009 01:28:54 +0100
Subject: [Python-ideas] universal set object for use in set manipulation
In-Reply-To: <19050.19755.891763.900896@jon.es>
References: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com>
	<20090723071207.GG28628@steerpike.home.puzzling.org>
	<6f4e8779-ce7a-411d-8b19-24c9fd54a51a@h11g2000yqb.googlegroups.com>
	<4A68E06F.20900@canterbury.ac.nz> <4A6A4A0C.4050605@gmail.com>
	<19050.19755.891763.900896@jon.es>
Message-ID: <6f4025010907241728j4ee2a6dar504c7b7354a599c3@mail.gmail.com>

2009/7/25 Terry Jones <terry at jon.es>

> >>>>> "Nick" == Nick Coghlan <ncoghlan at gmail.com> writes:
> Nick> Greg Ewing wrote:
> >> Andy Kish wrote:
> >>
> >>> It's really annoying to having appropriate identity element for
> >>> union built in while missing the *correct* identity element for
> >>> intersection.
> >>
> >> What would
> >>
> >> for x in set.universal():
> >> ...
> >>
> >> do?
>
> Nick> Raise an exception, just like a signalling NaN in decimal.
>
> Agreed. The code I stuck on PyPI today raises InfiniteSetError on __len__
> and __iter__.
>
> Nick> An expanded set concept with support for complementary set
> Nick> definitions is definitely something that should cook on PyPI for a
> Nick> while though.
>
> Just to repeat the URL: http://pypi.python.org/pypi/compset/0.1
>
> It's had exactly zero downloads :-)



Those numbers only update once a day anyway.

Michael


>
>
> Terry
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
http://www.ironpythoninaction.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090725/ea246157/attachment.html>

From jan.kanis at phil.uu.nl  Sat Jul 25 18:21:47 2009
From: jan.kanis at phil.uu.nl (Jan Kanis)
Date: Sat, 25 Jul 2009 18:21:47 +0200
Subject: [Python-ideas] universal set object for use in set manipulation
In-Reply-To: <20090723071207.GG28628@steerpike.home.puzzling.org>
References: <85ea580a0907222328g6bba007ekc32650efb368f266@mail.gmail.com> 
	<20090723071207.GG28628@steerpike.home.puzzling.org>
Message-ID: <59a221a0907250921p12918bffqc2cd53bc08fface7@mail.gmail.com>

Going back to the original problem...

On Thu, Jul 23, 2009 at 09:12, Andrew Bennetts<andrew at bemusement.org> wrote:
> Andy Kish wrote:
> [...]
>>
>> The above intersection case would end up looking something like:
>>
>> ? ? set_intersection = set.universal()
>> ? ? for s in sets:
>> ? ? ? ? set_intersection &= s
>
> Or even:
>
> ? set_intersection = reduce(operator.and_, sets, set.universal())
>
> Although, you can already pass multiple (or zero) sets to set.intersection().
> So your special case version can be a little simpler...
>
> ? sets = list(sets)
> ? if len(sets) == 0:
> ? ? ? return set()
> ? return sets[0].intersection(sets[1:])
>
> Which isn't as elegant, but it's also not so bad.

But set.intersection doesn't need to be called using dot notation, the
class attribute call works just as well:


  try:
      return set.intersection(*sets)
  except TypeError:
      return set()


I'd say this is as elegant as the OPs solution with the universal set.

Universal sets and other such cofinite sets could be nice, but I don't
think they should be in the python standard library.


From greg at krypto.org  Sat Jul 25 22:17:01 2009
From: greg at krypto.org (Gregory P. Smith)
Date: Sat, 25 Jul 2009 13:17:01 -0700
Subject: [Python-ideas] Changing the default install location,
	script 	versioning (Packages and PEP 370)
In-Reply-To: <4222a8490907191857m2705b298gd329357ef6912169@mail.gmail.com>
References: <4222a8490907191857m2705b298gd329357ef6912169@mail.gmail.com>
Message-ID: <52dc1c820907251317k1f3a0fd0w78b6a612f54471cc@mail.gmail.com>

On Sun, Jul 19, 2009 at 6:57 PM, Jesse Noller<jnoller at gmail.com> wrote:
> So, I've finally been able to kick my python 2.5 boat anchor, and I'm
> now relishing in the delightful future of 2.6. Two things struck me
> though, digging into PEP 370's (per user site-packages) behavior.
>
> The first is this - when you install a package (say, pip) into the
> .local directory via the --user command, the package ends up in the
> correct lib directory; but the binary get dropped into ./local/bin -
> this seems wrong. The reason being, is that what if I also have python
> 2.7 (which i do) installed, as well as python 3.1 and the
> release-which-will-not-be-named (3.0) - if I install that same package
> into one of the other versions, a new binary would be written with the
> same name - a script-pythonversion might also be installed, but the
> fact that the original script was overwritten seems broken to me.
>
> I think the "best" fix for this is to make the bin/ directory mirror
> the lib layout - each version would get it's own bin directory:
>
> .local/
> ? ?bin/
> ? ? ? ?python2.6/
> ? ? ? ?python3.1/
> ? ?lib/
> ? ? ? ?python2.6/
> ? ? ? ?python3.1/
>
> Of course, doing this begs the question - why have /bin and /lib be
> top-level directories, instead favoring a layout which looks like:
>
> .local/
> ? ?python2.6/
> ? ? ? ?bin/
> ? ? ? ?lib/
>
> The data and doc directories which some packages might include should
> follow the same convention, to avoid conflicts in those directories as
> well.

In addition to what you've mentioned here and in your blog post, the
~/.local directory is not well thought out when it comes to binary
extension modules and

It ignores the needs for multi-architecture systems that do not use
fat binaries such as all x86_64 Linux/BSD/Solaris systems.

It also ignores the fact that ~ may be shared across machines which
are not the same architecture.

Compiled modules .so, .dll, .whateveryouroses need to live in a
subdirectory below the pythonX.Y directory unique to the OS +
architecture they were compiled for.  Even that still won't be perfect
for shared home directories as it doesn't take into account which OS
distro version (read: external shared library dependencies) an
extension was compiled for but its a good start:

.local/
  python2.6/
    linux-x86_64/
    linux-i386/
    linux-mipsel/
    darwin-fatmultiarchbinaryorwhateverapplecallsthem/
    darwin-x86_64/
    netbsd-ppc/

> The second behavior is more of a "man I wish for a unicorn" type of
> thing - I think that --user should be removed, and by removed, I mean
> made the default action when "python setup.py install" is invoked.
> Users should need to instead pass in a --global flag to alter the
> behavior to install into the system's site-packages and bin
> directories.  The reason is simple - installation to the global site is
> typically a super-user task, and can side-effect all users, it's
> growing into more of a no-no as more OS vendors grow to rely on the
> packaged/included versions of python.

+10   -  Unicorns for everyone!

The system python belongs to the OS and should -never- be touched by
anything that isn't the OSs own package management system.  People
violate this all the time.  That is the path to the dark side.  Doing
that on any system I'm near leads to anger.  Anger leads to hate.
Hate leads to suffering and revocation of root privileges.

-gps