a pyrex-inspired for-i-from-1-to-n construct backported to Python. Good idea? Bad Idea? PEP?

[Jp Calderone]

On Wed, Apr 02, 2003 at 09:14:24AM -0500, WP wrote:
> Greg Ewing wrote the following Pyrex snippet:
> >    for i from 0 <= i < n:
> >      result.append(a[i])
> 
> This pyrex snippet reminds me (because of its orthogonality to C) of one
> thing, more than tab indents, that I have always thought was bogus in
> Python, which is the handling of integer loops.

  Why?  Just because the current implementation of it happens to be slower
than native-compiled C for-loops?  You're entitled to your opinion, of
course, but Python is a high level language, so usually design decisions
made about it are not driven primarily by runtime efficiency concerns. ;)

> range(0,n) sucks, xrange(0,n) 

  range(n), xrange(n), btw.  0 is the default starting point.

> is an interesting hack which obscures the
> readability of code, and post 2.x versions of python, with generators, and
> other 'inverted' ways of accomplishing the obvious procedural effect of an
> integer for loop, are all just ways of obfuscating otherwise perfectly
> readable python code.  Okay, so I'm being opinionated. I know that. But
> does anyone else thing that for the extremely common (and therefore
> bytecode optimizeable, syntax customizeable) case where you need to do a
> for loop over a range (possibly large range) of integers, that a language
> level keyword construct (such as the above) would be preferable to
> Python's choices of range, xrange, generators, custom class-based
> iterators, and whatever else you can come up with, that the lack of the
> most common idiom, being expressed in a bytecode optimized fashion would
> be good?

  Yes, in fact.  The current behavior of range() is actually one of
Guido's regrets about Python, so you can be pretty sure that it will change
around the time of Python 3.0.  How exactly, who can say?  I'm sure he'll
come up with something good >:)

  Hopefully something better than that Pyrex-style loop you've got above. 
(How do you do steps with that, anyway?)

> 
> Of course, it all comes I suppose from integers being "first class objects" 
> and thus, not actually integers, they are actually immutable objects which 
> contain an integer, and so we get to the ugly condition where "i++" 
> actually decref's one object, and either looks up or creates another 
> object, and increfs it.  Why does it come from this?

  Furthermore, what language are you talking about, when you talk about
"i++"?  Surely not Python.

> Hardly a fast operation to implement, but certainly one of the most common
> idioms in all of programming.

  Ahh, worrying about efficiency again.

> 
> Going back (dizzyingly) from the bytecode/c implementation, to the syntax, 
> for a moment. Imagine if Python had a pyrex-like first-class iteration 
> case, where i (in the above example) is not an integer, but rather a 
> reference to the for-integer-bytecode emitted by the compiler, an object 
> which knows (in C) how to increment i's value quickly, and jump out of a 
> loop when the limit n is reached.

  I'm not sure you're using "first-class" correctly here, but I think I know
what you mean.

  This kind of optimization could be implemented without any syntax change. 
As long as the Python interpreter knows that range(10) is going to return
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9], it can skip actually creating the list of
integers, and just do a highly-optimized loop over those integers.

  Now, in its current state, it is impossible for the Python interpreter to
know this.  What if range() is rebound?  So this can't be done now.  But, if
some of the talk about forbidding the shadowing of builtins bears fruit, it
might someday be possible for CPython to make this optimization, and all
without any ugly syntax hacks.

> 
> Ever since Python 1.5.2, I have been hacking around on my own to see if I 
> could get huge speed improvements out of Python.  I have settled down to 
> the reality that Python+Pyrex, with syntactical speedbumps in between them, 
> are probably the way things are going to stay. But I can dream, can't I?

  That sounds like a good route.  For myself, it is a rare case where I need
any more speed out of the execution of my software than Python gives me, and
I'm much happier with the quality of software that Python's mostly-clean,
mostly-consistent, mostly-readible syntax gives me.  I wouldn't trade a
speedup, even a significant one, for this kind of syntax-creep.

  Jp

-- 
        "I quite agree with you," said the Duchess; "and the moral of
that is -- Be what you would seem to be' -- or, if you'd like it put
more simply -- Never imagine yourself not to be otherwise than what it
might appear to others that what you were or might have been was not 
otherwise than what you had been would have appeared to them to be
otherwise.'"       -- Lewis Carrol, "Alice in Wonderland"
-- 
 up 13 days, 12:00, 3 users, load average: 0.01, 0.02, 0.00