[Python-ideas] should there be a difference between generators anditerators?

[Bruce Frederiksen]

Raymond Hettinger wrote:
> From: "Bruce Frederiksen" <dangyogi at gmail.com>
>> 1.  All of the itertools and map (and I've no doubt left some others 
>> out here) be extended to propagate the extra generators methods: 
>> close, throw and send.
>
> Try wrapping each input iterator with an autocloser:
>
> def closer(it):
>    for elem in it:
>        yield elem
>    if hasattr(it, 'close'):
>        it.close()
>
> map(somefunc, closer(someiterable))
I don't think that this adds anything.  If the 'it' iterable is run to 
completion (and, presumably, does what 'close' will do anyway), this 
adds nothing.  And if the 'it' iterable is not run to completion, then 
neither is closer, and so it still adds nothing.  (And placing the 'if 
hasattr' clause under a 'finally' doesn't help, because the 'finally' is 
only run if the generator runs to completion or 'close' is called, and 
nobody is ever going to call close here on closer).  What I'm looking 
for is making the following work:

with contextlib.closing(map(somefunc, someiterable)) as it:
    for i in it:
        if somecondition: break
# at this point someiterable.close should have been run

Intuitively, I would think that the above should work, but it doesn't.  
Intuitively, I would think that if  Python defines 'close' and 'throw' 
methods to allow generators to properly clean up after themselves, that 
itertools and the 'for' statement would both honor these so that the 
'with' statement isn't even required here.  I can see arguments for not 
calling 'close' automatically in 'for' statements (though none for not 
calling 'throw' automatically); but I don't see any arguments against 
itertools honoring these methods.  If any of the itertools is passing 
values back from a generator (rather than a simple iterator), it would 
be very nice to retain the full semantics of the generator.

Now, for map, you could suggest that Python programmers understand these 
subtleties and write instead:

with contextlib.closing(someiterable) as it:
    for i in map(somefunc, it):

and this works for map.  But what about chain?

with contextlib.closing(chain.from_iterable(map(somegenerator, 
someiterable))) as it:
    for i in it:

This is my specific situation, and I need 'close' to be called on the 
currently active somegenerator within chain before the line following 
'with' is executed.  In my particular case, I don't think I need close 
on someiterable, but it seems to make sense to do this too.

Currently, I'm not using the 'with' clause and am relying on CPython's 
garbage collector to immediately call __del__ (which is defined to call 
close) when the 'for' statement abandons chain.  But this doesn't work 
in jython or ironpython...

-bruce

And, BTW, thank you for adding from_iterable to chain!