beginning index for generators

[Alex Martelli]

kosh <kosh at aesaeion.com> wrote:

> I was wondering if there is or there could be some way to pass a generator an
> optional starting index so that if it supported that slicing could be made
> more efficient. Right now if you do use a generator and do a [100:110] or any
> other kind of slice it reads all the values up to 100 also. 

You can pass any starting arguments you want when you call a generator,
just like you can when you call any other function.  You can, if you
wish, make that argument optional, and by implied agreement between the
call site and the generator you can give that argument any meaning that
the generator is willing and able to supply.  "This is the starting
index" is in no way different from any other meaning whatsoever.

This is all so obvious that I'm afraid you mean something very different
from what you're actually saying, but, if so, then you'll just have to
try to express yourself differently, because I can't read minds!-)


> I know a lot of generators have to do all previous parts before they can do

Right, of course, so presumably such generators would never let you pass
you that optional "this is the starting index" argument -- with them you
have to start from the beginning, period.

> the next part but it would be a nice optimization that can start at any index
> to take an optional argument to tell them what index they are currently
> working on. This could make strides more efficient also since it would not
> have to run the generator to skip the parts it was not using.

I suspect you don't mean by "stride" the same thing I do.  To me, being
an old Fortran/Linpack/etc programmer, a "stride" in a loop is the
amount by which the loop-control variable (an integer) is incremented
after each leg of the loop.  In Python, for example, the third argument
to range, xrange or slice is what I think of as "a stride", though
Python tends to call it "a step" instead (but then, in English, a stride
IS nothing more than a long step!-).  I don't see what passing an
optional starting index has to do with strides, or what it means to
"make strides more efficient".  You could of course design your
generators, if what they're computing is suitable for that, to take an
optional stride argument (presumably defaulting to 1) instead of, or in
addition to, an optional start argument(presumably defaulting to 0).
Again, this is so obvious that I suspec we're just talking at
cross-purposes...


> The reason I would like this to be part of the generator rather then making a
> special function to do this is that it would make it easier to refactor a
> piece of code to be a much more efficient generator from some other sequence
> type without having to change how it is called. 

You can always design and code a generator instead of "some other"
_function_ returning an iterator, or vice versa.  The generator is often
more convenient, but whether it's "much more efficient" is of course
dubious.  For example, all the stuff in itertools _could_ have been
coded as generators, but in fact they're coded as callables returning
iterators of other kinds (instances of C-coded iterator types), as
_that_ turns out to be the "much more efficient" implementation choice
in this case.


> I have no idea how feasible this is, what it would take to implement it etc
> but I did not get any matches searching this on google and wanted to throw it
> out here to see if others have run into situations in which this would be
> very useful also and would further simplify code and make it more readable.
> 
> I would also love to know of other ways that this problem could be solved but
> I would like it to be as transparent as possible to existing calling code.

Perhaps if you can give some strawman examples of how you would like to
be able to code something, with exactly what inputs and what results,
somebody can understand exactly what you're wishing for, because, as of
now, I'm totally incapable of such understanding, beyond the obvieties
above expounded (and I suspect they ARE obvious enough that, if that was
all you were after, you wouldn't have posted -- but this doesn't allow
me to divine what it IS you're after, because the obvieties I expounded
DO match 100% of what you have literally been writing in this post).


Alex