How to get memory size/usage of python object

[Steven D'Aprano]

On Thu, 10 Jan 2008 00:14:42 -0800, Santiago  Romero wrote:

>> Would you care to precisely define "REAL size" first? Consider:
>>
>> >>> atuple = (1, 2)
>> >>> mylist = [(0, 0), atuple]
>>
>> Should sizeof(mylist) include sizeof(atuple) ?
> 
>  No, I'm talking about "simple" lists, without REFERENCES to another
> objects into it.
> 
>  I mean:
> 
> lists = [ 0, 1, 2, 3, 4, (1,2), 3]

That list has 7 references to other objects. One of those objects has 2 
references to objects.

In total, depending on implementation, there could be as many as 9 
objects referenced by that list, or as few as 6 objects (both references 
to 3 could be to the same object).

In the current CPython implementation, that list will have 7 references 
to 6 objects. Including indirect references, there will be 9 references 
to 6 objects. (Or so I understand.)
 

>  or
> 
> array = [ [0,0,0,0,0,0,0], [1,1,1,1,2,1,2], ... ]

Ignoring the '...', there will be a total of 16 references to 5 objects 
in the current CPython implementation. Other Pythons (Jython, IronPython, 
PyPy, ...) may be different.

 
>  Maybe I can "pickle" the object to disk and see the filesize ... :-?

That would measure something very different.

Possibly you want something like this heuristic:

def sizeof(obj):
    """APPROXIMATE memory taken by some Python objects in 
    the current 32-bit CPython implementation.

    Excludes the space used by items in containers; does not
    take into account overhead of memory allocation from the
    operating system, or over-allocation by lists and dicts.
    """
    T = type(obj)
    if T is int:
        kind = "fixed"
        container = False
        size = 4
    elif T is list or T is tuple:
        kind = "variable"
        container = True
        size = 4*len(obj)
    elif T is dict:
        kind = "variable"
        container = True
        size = 144
        if len(obj) > 8:
            size += 12*(len(obj)-8)
    elif T is str:
        kind = "variable"
        container = False
        size = len(obj) + 1
    else:
        raise TypeError("don't know about this kind of object")
    if kind == "fixed":
        overhead = 8
    else: # "variable"
        overhead = 12
    if container:
        garbage_collector = 8
    else:
        garbage_collector = 0
    malloc = 8 # in most cases
    size = size + overhead + garbage_collector + malloc
    # Round to nearest multiple of 8 bytes
    x = size % 8
    if x != 0:
        size += 8-x
        size = (size + 8)
    return size


See:
http://mail.python.org/pipermail/python-list/2002-March/135223.html

to get you started.





-- 
Steven.