Maximum List Sizes

[Alex Martelli]

"bushi" <bushi at spiritone.com> wrote in message
news:39A76BF4.A369EE31 at spiritone.com...
    [snip]
> the vertices.  Can Python effectively handle a list this large (9500+
> items X 40-60 bytes / item) or would it be better to do the
> transformations using temp files?

Half a megabyte or so in a Python list is no problem on a typical machine
of today.  Just try to ensure your algorithms' performance goes about
linearly with list length, "O(N)" in normal jargon, rather than degrading
like N squared or whatever.  One trick that has made some difference
to me at times: rather than build a list with .append() one element at
a time, make it as big as it needs to be at once:
    biglist=[None]*100000
and then use biglist[index]=anelement to set its element; this seems
to me to make performance better for truly huge lists (just a subjective
impression: I haven't actually measured this effect!).


Alex