Huge Dicts and perfomance
Lucio Torre
lucio at movilogic.com
Thu Dec 20 14:32:10 EST 2001
Stephen D Evans wrote:
>Are you using a list for the first index ? e.g. using list comprehensions:
>
>dict = [ {} for i in range(2) ] # create a list of dictionaries
>dict[0][objectname] = object
>dict[1][(obj-a, obj-b)] = obj(a, b)
>
>This should give a slight performance increase.
>
Ill try it.
>
>Which version of python are you using? Some windows versions run noticeably
>slower than others with large amounts of data. (As an observation - the
>cygwin http://sources.redhat.com/cygwin/ version runs faster on the windows
>platform when using large dictionaries.)
>
I really dont know :) i have 1.5.2, and 2.0, and 2.1 installed in my
linux box, and cant remember now which one is the default (and i dont
have the machine with me, now). But i can pick anyone i like. I think i
prefer 1.5.2 because ZODB is available as a separate package. Is there a
zodb for 2.x? Any idea of the overhead of adding zodb? Or should i just
use the dbsomehting module?
Thanks!
Lucio.
>
>Stephen D Evans
>
>A picture speaks a thousand words.
>"Lucio Torre" <lucio at movilogic.com> wrote in message
>news:mailman.1008865945.12538.python-list at python.org...
>
>>Hi,
>>
>>I am making an application where i want to store links betwen nodes. So
>>i have a dictionary where i store the objects.
>>
>>dict[0][objectname] = object
>>
>>the [0] is because i to signify a link betwen obj-a and obj-b, i do:
>>
>>dict[1][(obj-a, obj-b)] = obj(a, b)
>>
>>and so on, to six levels deep.
>>
>>and i also store links betwen dict[x][y] and other nodes (the stored
>>object has those links)
>>
>>and i want to say for example, if i have a and b, what other nodes
>>usually come togheter? so i follow the links and find the root nodes (z,
>>x) and the groups (a, z) and (s, x).
>>
>>the main problem of course is that doing it this way, performance sucks.
>>I took me nothing to code it, but it takes to much to run (more than
>>coding). So, any ideas on how to achieve my goal in a better (faster) way?
>>
>>thanks,
>>
>>Lucio.
>>
>>
>>
>
>
More information about the Python-list
mailing list