enumerate overflow
Tim Chase
python.list at tim.thechases.com
Wed Oct 3 10:42:48 EDT 2007
>> for lineNum, line in enumerate(f): ...
>>
>> However, lineNum soon overflows and starts counting backwards. How do
>> i force enumerate to return long integer?
>>
> Just how "soon" exactly do you read sys.maxint lines from a file? I
> should have thought that it would take a significant amount of time to
> read 2,147,483,647 lines ...
A modestly (but not overwhelmingly) long time:
(defining our own xrange-ish generator that can handle things
larger than longs)
>>> def xxrange(x):
... i = 0
... while i < x:
... yield i
... i += 1
...
>>> for i,j in enumerate(xxrange(2**33)): assert i==j
...
Traceback (most recent call last):
File "<stdin>", line 1, in ?
AssertionError
It took me about an 60-90 minutes to hit the assertion on a
dual-core 2.8ghz machine under otherwise-light-load. If
batch-processing lengthy log files or other large data such as
genetic data, it's entirely possible to hit this limit as the OP
discovered.
-tkc
More information about the Python-list
mailing list