A gnarly little python loop
Stefan Behnel
stefan_ml at behnel.de
Sun Nov 11 02:56:10 EST 2012
Steve Howell, 11.11.2012 04:03:
> On Nov 10, 2:58 pm, Roy Smith <r... at panix.com> wrote:
>> I'm trying to pull down tweets with one of the many twitter APIs. The
>> particular one I'm using (python-twitter), has a call:
>>
>> data = api.GetSearch(term="foo", page=page)
>>
>> The way it works, you start with page=1. It returns a list of tweets.
>> If the list is empty, there are no more tweets. If the list is not
>> empty, you can try to get more tweets by asking for page=2, page=3, etc.
>> I've got:
>>
>> page = 1
>> while 1:
>> r = api.GetSearch(term="foo", page=page)
>> if not r:
>> break
>> for tweet in r:
>> process(tweet)
>> page += 1
>>
>> It works, but it seems excessively fidgety. Is there some cleaner way
>> to refactor this?
>
> I think your code is perfectly readable and clean, but you can flatten
> it like so:
>
> def get_tweets(term, get_page):
> page_nums = itertools.count(1)
> pages = itertools.imap(api.getSearch, page_nums)
> valid_pages = itertools.takewhile(bool, pages)
> tweets = itertools.chain.from_iterable(valid_pages)
> return tweets
I'd prefer the original code ten times over this inaccessible beast.
Stefan
More information about the Python-list
mailing list