A gnarly little python loop

[Steve Howell]

On Nov 10, 2:58 pm, Roy Smith <r... at panix.com> wrote:
> I'm trying to pull down tweets with one of the many twitter APIs.  The
> particular one I'm using (python-twitter), has a call:
>
> data = api.GetSearch(term="foo", page=page)
>
> The way it works, you start with page=1.  It returns a list of tweets.
> If the list is empty, there are no more tweets.  If the list is not
> empty, you can try to get more tweets by asking for page=2, page=3, etc.
> I've got:
>
>     page = 1
>     while 1:
>         r = api.GetSearch(term="foo", page=page)
>         if not r:
>             break
>         for tweet in r:
>             process(tweet)
>         page += 1
>
> It works, but it seems excessively fidgety.  Is there some cleaner way
> to refactor this?

I think your code is perfectly readable and clean, but you can flatten
it like so:

    def get_tweets(term, get_page):
        page_nums = itertools.count(1)
        pages = itertools.imap(api.getSearch, page_nums)
        valid_pages = itertools.takewhile(bool, pages)
        tweets = itertools.chain.from_iterable(valid_pages)
        return tweets