Possible File iteration bug

[Billy Mays]

On 07/15/2011 10:28 AM, Thomas Rachel wrote:
> Am 15.07.2011 14:52 schrieb Billy Mays:
>
>> Also, in the python docs, file.next() mentions there
>> being a performance gain for using the file generator (iterator?) over
>> the readline function.
>
> Here, the question is if this performance gain is really relevant AKA
> "feelable". The file object seems to have another internal buffer
> distinct from the one used for iterating used for the readline()
> function. Why this is not the same buffer is unclear to me.
>
>
>> Really what would be useful is some sort of PauseIteration Exception
>> which doesn't close the generator when raised, but indicates to the
>> looping header that there is no more data for now.
>
> a None or other sentinel value would do this as well (as ChrisA already
> said).
>
>
> Thomas

A sentinel does provide a work around, but it also passes the problem 
onto the caller rather than the callee:

def getLines(f):
     lines = []

     while True:
         yield f.readline()

def bar(f):
     for line in getLines(f):
         if not line: # I now have to check here instead of in getLines
             break
         foo(line)


def baz(f):
     for line in getLines(f) if line: # this would be nice for generators
         foo(line)


bar() is the correct way to do things, but I think baz looks cleaner.  I 
found my self writing baz() first, finding it wasn't syntactically 
correct, and then converting it to bar().  The if portion of the loop 
would be nice for generators, since it seems like the proper place for 
the sentinel to be matched.  Also, with potentially infinite (but 
pauseable) data, there needs to be a nice way to catch stuff like this.

--
Bill