enumerate overflow

[Steve Holden]

Tim Chase wrote:
>>> for lineNum, line in enumerate(f): ...
>>>
>>> However, lineNum soon overflows and starts counting backwards. How do
>>> i force enumerate to return long integer?
>>>
>> Just how "soon" exactly do you read sys.maxint lines from a file? I 
>> should have thought that it would take a significant amount of time to 
>> read 2,147,483,647 lines ...
> 
> A modestly (but not overwhelmingly) long time:
> 
> (defining our own xrange-ish generator that can handle things larger 
> than longs)
> 
>  >>> def xxrange(x):
> ...     i = 0
> ...     while i < x:
> ...             yield i
> ...             i += 1
> ...
>  >>> for i,j in enumerate(xxrange(2**33)): assert i==j
> ...
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> AssertionError
> 
> 
> It took me about an 60-90 minutes to hit the assertion on a dual-core 
> 2.8ghz machine under otherwise-light-load.  If batch-processing lengthy 
> log files or other large data such as genetic data, it's entirely 
> possible to hit this limit as the OP discovered.
> 
I wouldn't dream of suggesting it's impossible. I just regard "soon" as 
less than an hour in commuter's terms, I suppose.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC/Ltd           http://www.holdenweb.com
Skype: holdenweb      http://del.icio.us/steve.holden

Sorry, the dog ate my .sigline