Generator slower than iterator?

[Arnaud Delobelle]

MRAB <google at mrabarnett.plus.com> writes:

> Federico Moreira wrote:
>> Great, 2min 34 secs with the open method =)
>>
>> but why?
>>
>> ip, sep, rest = line.partition(' ')
>>        match_counter[ip] += 1
>>
>> instead of
>>
>> match_counter[line.strip()[0]] += 1
>>
>> strip really takes more time than partition?
>>
>> I'm having the same results with both of them right now.
>>
> I think you meant split(), not strip().
>
> split() might split the string into many parts, but you want only the
> first part (at most 1 split), so the extra splits are unnecessary and
> waste time. split(None, 1) and partition(' ') do at most 1 split, so
> they don't do unnecessary work.

Here's a very quick test:

>>> timeit.Timer("a='fsdjflksjdf as fsd jkfl sdj'; b=a.partition(' ')[0]").timeit(5000000)
2.9831948280334473
>>> timeit.Timer("a='fsdjflksjdf as fsd jkfl sdj'; b=a[:a.find(' ')]").timeit(5000000)
3.0122888088226318
>>> timeit.Timer("a='fsdjflksjdf as fsd jkfl sdj'; b=a.split(None, 1)[0]").timeit(5000000)
4.5993850231170654
>>> timeit.Timer("a='fsdjflksjdf as fsd jkfl sdj'; b=a.split()[0]").timeit(5000000)
5.2067980766296387

So .split(None, 1) is quite a bit slower than .partition(' ') apparently.

-- 
Arnaud