Best practice for operations on streams of text

[Beni Cherniavsky]

On May 8, 12:07 am, MRAB <goo... at mrabarnett.plus.com> wrote:
> def compound_filter(token_stream):
>      stream = lowercase_token(token_stream)
>      stream = remove_boring(stream)
>      stream = remove_dupes(stream)
>      for t in stream(t):
>          yield t

The last loop is superfluous.  You can just do::

def compound_filter(token_stream):
     stream = lowercase_token(token_stream)
     stream = remove_boring(stream)
     stream = remove_dupes(stream)
     return stream

which is simpler and slightly more efficient.  This works because from
the caller's perspective, a generator is just a function that returns
an iterator.  It doesn't matter whether it implements the iterator
itself by containing ``yield`` statements, or shamelessly passes on an
iterator implemented elsewhere.