[Python-ideas] Reduce/fold and scan with generator expressions and comprehensions

[Stephen J. Turnbull]

Danilo J. S. Bellini writes:

 > I'm not talking about "simple cases", but about quite general
 > cases, general in the sense that it allows modeling time varying
 > models (and the proposal is about recursion, whose computational
 > power should be obvious).

I think I admitted that in the text you snipped.  If not, consider it
stipulated here.  My point is that for me functions like accumulate
are sufficient without new syntax, and if you are going to do time-
varying stuff in the kind of economics I studied and research, you are
going to need to carry around so much state that the relative amount
of simplicity you can gain with syntax is very small.

As for recursion, the syntax you proposed doesn't say "recursion" to
me, it says "let's add more complexity to a syntax that already is
hard to fit on a line so we can turn a sequence into a series."

 > Anyway, that makes this talking about computational accuracy sound
 > like an argument for my proposal here, not against it.

Not as I see it.  My point is that functions like accumulate already
get me as much of your proposal as I can see being useful in my own
applications, and so I'd rather spend effort on the inherent
complexity of accurate computation than on learning new syntax which
as far as I can see buys me no extra simplicity.

Consider the existing comprehension syntax.  I use it all the time
because it's very expressive, it "looks like" what I would write in a
declarative mathematical definition, and the constructive alternative
is not a function call, it's a loop, in many cases with a nested if
test as well.  However, when you try to express more than one loop
with a comprehension, you end up with a counterintuitive ordering
based on the current "this expands naturally to for loops and if
conditionals according to this simple rule" syntax, along with a
separation of the result expression from the loop where it's actually
produced.  It's not obvious to me that use of comprehension syntax
beyond [ f(x) for x in y if g(x) ] is all that useful, but that very
basic form is extremely useful by itself.

If your proposal will help make more complex comprehensions easy to
understand, then you've got something I'd like to have.  But I don't
yet see how it fixes that, and if not, I ain't gonna need it and I
don't want to have to explain it to my students when they ain't gonna
need it either.

 > Is there any previous discussion on this topic or something related
 > to it?  Where's the link? (I asked it before)

I don't have one off-hand (I really am interested only in the claimed
uses in economics).  As I wrote earlier, maybe David Mertz does.

 > Actually, that's an argument for this proposal, not against. I'm
 > proposing a way to avoid itertools.accumulate and functools.reduce
 > using an explicit syntax instead of the functions. Or does GvR
 > prefer map/filter instead of list comprehensions?

itertools functions are widely used and generally well-thought-of.
Avoiding accumulate is a non-goal AFAIK.  AIUI, Guido does think that
avoiding reduce is a goal, but I believe that's because he thinks the
semantics are just plain hard to understand.  If your syntax is in
large part a substitute for reduce, I doubt he will like it more than
reduce just because it's syntax.  (I do not speak for Guido; I offer
my observation in the hope it will help you prepare to present your
idea to him in the most favorable light.)

 > If you're right, then we should be consistent and propose the elimination
 > of list comprehension and every syntax alike. The arguments you're giving
 > against my proposal are the same.

"A foolish consistency is the hobgoblin of small minds."  If Emerson
hadn't lived a century before Tim, I'm sure it would be in the Zen.

That is, comprehensions are clearly useful, and I use them in almost
every program I write.  I still don't see when I'd ever strongly
prefer the syntax you propose to the itertools we already have, so I
don't care if it's a consistent extension.  That doesn't mean it's not
useful to you or others, it just means I'm not convinced -- and so I
will not join the proponents.  That's all.