[Python-ideas] "continue with" for dynamic iterable injection

[Nathaniel Smith]

On Thu, Sep 25, 2014 at 2:50 AM, Nathaniel Smith <njs at pobox.com> wrote:
> The most elegant solution I know is:
>
> class PushbackAdaptor:
>     def __init__(self, iterable):
>         self.base = iter(iterable)
>         self.stack = []
>
>     def next(self):
>         if self.stack:
>             return self.stack.pop()
>         else:
>             return self.base.next()
>
>     def pushback(self, obj):
>         self.stack.append(obj)
>
> it = iter(character_source)
> for char in it:
>     ...
>     if state is IDENTIFIER and char not in IDENT_CHARS:
>         state = NEW_TOKEN
>         it.push_back(char)
>         continue
>     ...
>
> In modern python, I think the natural meaning for 'continue with' wouldn't
> be to special-case something like this. Instead, where 'continue' triggers a
> call to 'it.next()', I'd expect 'continue with x' to trigger a call to
> 'it.send(x)'. I suspect this might enable some nice idioms in coroutiney
> code, though I'm not very familiar with such.

In fact, given the 'send' definition of 'continue with x', the above
tokenization code would become simply:

def redoable(iterable):
    for obj in iterable:
        while yield obj == "redo":
            pass

for char in redoable(character_source):
    ...
    if state is IDENTIFIER and char not in IDENT_CHARS:
        state = NEW_TOKEN
        continue with "redo"
    ...

which I have to admit is fairly sexy.

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org