Memory usage per top 10x usage per heapy

[Oscar Benjamin]

On 26 September 2012 00:35, Tim Chase <python.list at tim.thechases.com> wrote:

> On 09/25/12 17:55, Oscar Benjamin wrote:
> > On 25 September 2012 23:10, Tim Chase <python.list at tim.thechases.com>
> wrote:
> >> If tuples provide a savings but you find them opaque, you might also
> >> consider named-tuples for clarity.
> >
> > Do they have the same memory usage?
> >
> > Since tuples don't have a per-instance __dict__, I'd expect them to be a
> > lot lighter. I'm not sure if I'm interpreting the results below properly
> > but they seem to suggest that a namedtuple can have a memory consumption
> > several times larger than an ordinary tuple.
>
> I think the "how much memory is $METHOD using" topic of the thread
> is the root of the problem.  From my testing of your question:
>
> >>> import collections, sys
> >>> A = collections.namedtuple('A', ['x', 'y'])
> >>> nt = A(1,3)
> >>> t = (1,3)
> >>> sys.getsizeof(nt)
> 72
> >>> sys.getsizeof(t)
> 72
> >>> nt_s = set(dir(nt))
> >>> t_s = set(dir(t))
> >>> t_s ^ nt_s
> set(['__module__', '_make', '_asdict', '_replace', '_fields',
> '__slots__', 'y', 'x'])
> >>> t_s - nt_s
> set([])
>

On my system these is an additional __dict__ attribute and it is bigger
than the original tuple:
$ python
Python 2.7.3 (default, Apr 20 2012, 22:39:59)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import collections, sys
>>> A = collections.namedtuple('A', ['x', 'y'])
>>> nt = A(1,3)
>>> t = (1,3)
>>> set(dir(nt)) - set(dir(t))
set(['__module__', '_replace', '_make', 'y', '__slots__', '_asdict',
'__dict__', 'x', '_fields'])
>>> sys.getsizeof(nt.__dict__)
280
>>> sys.getsizeof(t.__dict__)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'tuple' object has no attribute '__dict__'


>
> So a named-tuple has 6+n (where "n" is the number of fields) extra
> attributes, but it seems that namedtuples & tuples seem to occupy
> the same amount of space (72).
>
> Additionally, pulling up a second console and issuing
>
>   ps v | grep [p]ython
>
> shows the memory usage of the process as I perform these, and after
> them, and they both show the same usage (actual test was
>
> 1) pull up a fresh python
> 2) import sys, collections; A = collections.namedtuple('A',['x','y'])
> 3) check memory usage in other window
> 4a) x = (1,2)
> 4b) x = A(1,2)
> 5) check memory usage again in other window
> 6) quit python
>
> performing 4a on one run, and 4b on the second run.
>
> Both showed identical memory usage as well (Debian Linux (Stable),
> stock Python 2.6.6) at the system level.
>

Python uses memory pools for small memory allocations. I don't think it's
possible to tell from the outside how much memory is being used at such a
fine level.

Oscar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20120926/35e01392/attachment.html>