Memory ?

[Scott Gilbert]

"Shagshag13" <shagshag13 at yahoo.fr> wrote:
> Hello,
> 
> i'm still looking for a way to check which are the best ways to save memory 
> because i work on many GB with float, etc. so is there a
> way to check how many bytes needs an object, a tuple, a list, a dict of 
> objects and so on ? (something like itemsize for array - in
> fact array should be great if they weren't boxed)
> 

Your best way to find out how much memory is being used is to measure
it using your native tools.  (something like ps on Unix, or somethig like 
Task Manager on Windows)

Probably your best way to figure out what is most efficient is to try
different strategies and measure the results.

The array module is going to give you very efficient storage as is Numeric.
If you're concerned about putting different types of data in one array (both
float and char for instance), you might consider creating 3 arrays: 1 for
type float, 1 for type char, and one that lets you know which of the other
two to use.  Wrap it all up in a class so that access to the data is
transparent.  Also, if you know that some values simply can't occur (char 0)
for instance, then you could just use 2 arrays etc...  All kinds of clever
hacks are possible, and if you hide them in a class they don't have to be
that ugly to use.

Finally, all of these are just going to get you some percentage of savings.
I think you can safely assume that all Python data structures use memory
proportional to how much data you are storing in them (a notable exception
being interned strings).  So if you can reorganize your problem to switch
from O(N*N) space to O(N*log(N)) space, you'll get much better savings.

Cheers,
    -Scott