How to use a 5 or 6 bit integer in Python?

Glen Wheeler adsl5lcq at tpg.com.au
Fri Dec 19 02:26:22 EST 2003


On Fri, 19 Dec 2003 06:27:59 GMT, "Rainer Deyke" <rainerd at eldwood.com>
wrote:

>Glen Wheeler wrote:
>>   I have one constantly changing dict with many millions of keys
>> (tuples of ints) which in turn associates itself with a tuple of ints
>> and two more dicts.
>
>That makes things a bit harder.  I expect that the big dict itself requires
>at least sixteen bytes per entry.  How big are your tuples?

  The key tuples, for the final calculation will range from 1 element
to 64 elements, with the majority at around 40-50 elements in size.

>  How big are the dicts in the data tuples?  What data do they contain?

  For the first dict:  Increasingly smaller as the calculation
continues (i.e. as the key for that entry increases).  Initially they
will have 64 keys (integers) with each integer having a 6-element
tuple of integers as it's data type.  All integers are from 0-63.
  For the second dict:  Trivially small, with a maximum of 32
elements, starts at 0 and an average of around 7.  Keys are kept as
integers from 0-63 and data ranges from -1-30, but the average case
will see the data at 7, as a single integer.

>   Are any of the tuples or dicts shared?
>

  Unfortunately, no.  This would be a major speedup for my program as
copying the dictionaries and tuples takes not only alot of memory but
alot of time too.
  But the truth is that I need these to be unique on a key-by-key
basis (context from the big dict) so I can't see any way around it.

>Encoding the key tuples as strings is easy (''.join([chr(x) for x in
>the_tuple]).  Encoding the ints in the data tuples is just as easy.  I'm not
>sure if those are enough to solve your problem.

  I'll be sure to do this.  Will it increase the hash speed for the
big dictionary significantly?  Will lookup times or memory storage
requirements decrease?

  Thanks,
  Glen





More information about the Python-list mailing list