A gnarly little python loop

[Steve Howell]

On Nov 11, 4:44 pm, Cameron Simpson <c... at zip.com.au> wrote:
> On 11Nov2012 11:16, Steve Howell <showel... at yahoo.com> wrote:
> | On Nov 11, 10:34 am, Peter Otten <__pete... at web.de> wrote:
> | > Steve Howell wrote:
> | > > On Nov 11, 1:09 am, Paul Rubin <no.em... at nospam.invalid> wrote:
> | > >> Cameron Simpson <c... at zip.com.au> writes:
> | > >> > | I'd prefer the original code ten times over this inaccessible beast.
> | > >> > Me too.
> | >
> | > >> Me, I like the itertools version better.  There's one chunk of data
> | > >> that goes through a succession of transforms each of which
> | > >> is very straightforward.
> | >
> | > > Thanks, Paul.
> | >
> | > > Even though I supplied the "inaccessible" itertools version, I can
> | > > understand why folks find it inaccessible.  As I said to the OP, there
> | > > was nothing wrong with the original imperative approach; I was simply
> | > > providing an alternative.
> | >
> | > > It took me a while to appreciate itertools, but the metaphor that
> | > > resonates with me is a Unix pipeline.
> [...]
> | > Actually you supplied the "accessible" itertools version. For reference,
> | > here's the inaccessible version:
> [...]
> | I know Peter's version is tongue in cheek, but I do think that it has
> | a certain expressive power, and it highlights three mind-expanding
> | Python modules.
> | Here's a re-flattened take on Peter's version ("Flat is better than
> | nested." -- PEP 20):
> [...]
>
> Ok, who's going to quiz the OP on his/her uptake of these techniques...

Cameron, with all due respect, I think you're missing the point.

Roy posted this code:

    page = 1
    while 1:
        r = api.GetSearch(term="foo", page=page)
        if not r:
            break
        for tweet in r:
            process(tweet)
        page += 1

In his own words, he described the loop as "gnarly" and the overall
code as "fidgety."

One way to eliminate the "while", the "if", and the "break" statements
is to use higher level constructs that are shipped with all modern
versions of Python, and which are well documented and well tested (and
fast, I might add):

    search = partial(api.GetSearch, "foo")
    paged_tweets = imap(search, count(1))
    paged_tweets = takewhile(bool, paged_tweets)
    tweets = chain.from_iterable(paged_tweets)
    for tweet in tweets:
        process(tweet)

The moral of the story is that you can avoid brittle loops by relying
on a well-tested library to work at a higher level of abstraction.

For this particular use case, the imperative version is fine, but for
more complex use cases, the loops are only gonna get more gnarly and
fidgety.