iterators and views of lists

[Steven D'Aprano]

On Tue, 15 Dec 2009 23:48:04 -0800, Brendan Miller wrote:

> On Tue, Dec 15, 2009 at 9:09 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>> On 12/15/2009 10:39 PM, Brendan Miller wrote:
>>> I'm wondering if anyone has done work towards creating more powerful
>>> iterators for python, or creating some more pythonic equivalent.
>>
>> For sequences, integer indexes let you do anything you want that the
>> container supports.
> 
> No, that's what I'm getting at... Most of the existing mutating
> algorithms in python (sort, reverse) operate over entire collections,
> not partial collections delimited by indexes... which would be really
> awkward anyway.

I'm sympathetic to your request for list views. I've often wanted some 
way to cleanly and neatly do this:

for item in seq[1:]:
    process(item)

without making an unnecessary copy of almost all of seq.

None of the alternatives are very nice. They all obscure the simplicity 
of the algorithm and add unnecessary effort:


for i, item in seq:
    if i != 0:
        process(item)


it = iter(seq)
try:
    it.next()  # throw the first value away
except StopIteration:
    pass
for item in it:
    process(item)


if not isinstance(seq, basestring):
    raise TypeError("this only works on strings")
for item in buffer(seq, 1):
    process(item)


Although that last one is the least horrible, particularly if you are 
prepared to drop the explicit type-check.



> Currently people slice and dice with well... slices, but those are
> copying, so if you want to operate over part of a range you make a copy,
> perform the operation, then copy the results back in.
> 
> I was thinking you'd want something like random access iterators in c++,
> or pointers in c, to write typical in place algorithmic code. To me,
> something like non-copying slices (maybe you'd call it a list view?)
> would seem functionally similar and maybe more pythonic.


Unless you have a really huge amount of data in your list, chances are 
that the cost of making a temporary copy will be relatively small. After 
all, under the hood you're just copying pointers (at least for CPython), 
not the entire data object. So most of the time the extra complexity of 
in-place algorithms is just premature optimization.

But not always.



-- 
Steven