Help with finding how much memory a variable is using

[Rich Somerfield]

<s713221 at student.gu.edu.au> wrote in message
news:3A986565.BAF4E05B at student.gu.edu.au...
> Rich Somerfield wrote:
> >
> > Hi,
> >
> > I am generating a huge, huge list of data (cant think of a decent way to
> > reduce the required storage size and have it in a usable form).
Everytime i
> > try to keep this list for future derivations I get a memory problem from
> > Windows (effectively terminating my python script).
> >
> > I presume this is because of the huge list.  Is it possible to find out
the
> > amount of memory a variable [not a type of the variable, the actual data
> > contained within the variable] is taking up?  This variable is local to
a
> > class and it would appear that it is when i go out of scope of that
class
> > that the memory error is being issued.  Would I be better making this
> > variable global (across all classes)? e.g. would this problem still
happen
> > if the variable was not just local to 1 class?
> >
> > Would a different python datatype solve this problem for me ?
> >
> > If I cant handle this amount of data then I will have to think of a way
of
> > better representing it, this will be unfortunate though as and
compression /
> > reduction of data will [as far as i can think] vastly reduce
performance -
> > which is already a concern.
> >
> > Regards
> > Rich
>
> Do you have to load all the list into memory, or is there a way you can
> load in parts and work on that sequentially. I haven't had a chance to
> use persistance modules yet, but their documentation may be a place to
> start.
>
> Joal Heagney/AncientHart

Unfortunately I need all of the data in memory at the same time.  The only
way I can see to solve this problem is to reduce the memory consumption of
my application.  And the only way I can see how to do that is by vastly
reducing the performance.

Do u know of any information that suggests how large a datatype can be in
Python (are there any upper limits?) ?

Regards
Rich