Useful? __slots__ for functions
Jean Brouwers
mrjean1 at comcast.net
Tue May 25 01:26:34 EDT 2004
[[ This message was both posted and mailed: see
the "To," "Cc," and "Newsgroups" headers for details. ]]
Nick,
I do not know the answer to your question, but the impact of __slots__
is very significant. Below is a message I posted a few weeks ago.
In addition to being faster (to create) and smaller, __slots__ objects
have another major advantage, at least in my view. They are 'frozen'
and can not be extended dynamically, not intentionally and -even more
importantly- not by accident. There are situations where the latter is
a major benefit, for example in applications where extensibility is not
a requirement.
/Jean Brouwers
ProphICy Semiconductor, Inc.
> Path: attbi_s04!attbi_s03!attbi_s01!attbi_s02!attbi_slave12!attbi_master11!wn14feed!worldnet.att.net!199.218.7.141!news.glorb.com!border1.nntp.dca.giganews.com!nntp.giganews.com!local1.nntp.dca.giganews.com!nntp.comcast.com!news.comcast.com.POSTED!not-for-mail
> NNTP-Posting-Date: Wed, 12 May 2004 15:05:16 -0500
> Subject: __slots__ vs __dict__
> Date: Wed, 12 May 2004 13:14:48 -0700
> From: Jean Brouwers <JBrouwers at ProphICy.com>
> Newsgroups: comp.lang.python
> Reply-To: JBrouwers at ProphICy.com
> Message-ID: <120520041314481389%JBrouwers at ProphICy.com>
> MIME-Version: 1.0
> Content-Type: text/plain; charset=ISO-8859-1
> Content-transfer-encoding: 8bit
> User-Agent: Thoth/1.7.2 (Carbon/OS X)
> Lines: 209
> NNTP-Posting-Host: 67.161.46.201
> X-Trace: sv3-IgpWl13JKku68lEaAxfUeuMDddQ2Hb+wpcL1Xy3D1hWFRMKqKkQK8oYKWO6fJa6w0ibSPg2HtHvHitY!pbEuqBO6sUAMUZawbd1zxH3XYX1WmCBcg/PRch8EUhprgUWBOELz1iGE9uwWRe93JDJDgO8=
> X-Complaints-To: abuse at comcast.net
> X-DMCA-Complaints-To: dmca at comcast.net
> X-Abuse-and-DMCA-Info: Please be sure to forward a copy of ALL headers
> X-Abuse-and-DMCA-Info: Otherwise we will be unable to process your complaint properly
> X-Postfilter: 1.1
> Xref: attbi_master11 comp.lang.python:158864
> X-Received-Date: Wed, 12 May 2004 20:05:16 GMT (attbi_s04)
>
>
> Classes using __slots__ seem to be quite a bit smaller and faster
> to instantiate than regular Python classes using __dict__.
>
> Below are the results for the __slots__ and __dict__ version of a
> specific class with 16 attributes. Each line in the tables shows the
> number of instances created so far, the total memory usage in Bytes,
> the CPU time in secs, the average size per instance in Bytes and the
> average CPU time per instance in micseconds.
>
> Instances of this particular class with __slots__ are almost 6x
> smaller and nearly 3x faster to create than intances of the __dict__
> version. Results for other classes will vary, obviously.
>
> Comments?
>
> /Jean Brouwers
> ProphICy Semiconductor, Inc.
>
>
> PS) The tests were run on a dual 2.4 GHz Xeon system with RedHat
> 8.0 and Python 2.3.2. The test script is attached but keep in mind
> that it only has been tested on Linux. It will not work elsewhere
> due to the implementation of the memory() function.
>
>
> testing __slots__ version ...
> 4096 insts so far: 3.0e+05 B 0.030 sec 73.0 B/i 7.3 usec/i
> 8192 insts so far: 8.8e+05 B 0.070 sec 107.5 B/i 8.5 usec/i
> 16384 insts so far: 1.5e+06 B 0.150 sec 92.2 B/i 9.2 usec/i
> 32768 insts so far: 3.3e+06 B 0.280 sec 101.0 B/i 8.5 usec/i
> 65536 insts so far: 6.6e+06 B 0.560 sec 101.2 B/i 8.5 usec/i
> 131072 insts so far: 1.4e+07 B 1.200 sec 103.4 B/i 9.2 usec/i
> 262144 insts so far: 2.7e+07 B 2.480 sec 103.4 B/i 9.5 usec/i
> 524288 insts so far: 5.5e+07 B 5.630 sec 104.0 B/i 10.7 usec/i
> 1048576 insts so far: 1.1e+08 B 13.980 sec 104.0 B/i 13.3 usec/i
> 1050000 insts total: 1.1e+08 B 14.000 sec 103.9 B/i 13.3 usec/i
>
>
> testing __dict__ version ...
> 4096 insts so far: 2.4e+06 B 0.050 sec 595.0 B/i 12.2 usec/i
> 8192 insts so far: 4.6e+06 B 0.090 sec 564.5 B/i 11.0 usec/i
> 16384 insts so far: 9.5e+06 B 0.180 sec 581.8 B/i 11.0 usec/i
> 32768 insts so far: 1.9e+07 B 0.370 sec 582.2 B/i 11.3 usec/i
> 65536 insts so far: 3.8e+07 B 0.830 sec 582.6 B/i 12.7 usec/i
> 131072 insts so far: 7.6e+07 B 1.760 sec 582.7 B/i 13.4 usec/i
> 262144 insts so far: 1.5e+08 B 4.510 sec 582.8 B/i 17.2 usec/i
> 524288 insts so far: 3.1e+08 B 12.820 sec 582.8 B/i 24.5 usec/i
> 1048576 insts so far: 6.1e+08 B 38.370 sec 583.1 B/i 36.6 usec/i
> 1050000 insts total: 6.1e+08 B 38.380 sec 583.1 B/i 36.6 usec/i
>
>
> -------------------------------slots.py-------------------------------
> <pre>
>
> from time import clock as time_clock
> def cputime(since=0.0):
> '''Return CPU in secs.
> '''
> return time_clock() - since
>
>
> import os
> _proc_status = '/proc/%d/status' % os.getpid() # Linux only
> _scale = {'kB': 1024.0, 'mB': 1024.0*1024.0,
> 'KB': 1024.0, 'MB': 1024.0*1024.0}
>
> def _VmB(VmKey):
> global _scale
> try: # get the /proc/<pid>/status pseudo file
> t = open(_proc_status)
> v = [v for v in t.readlines() if v.startswith(VmKey)]
> t.close()
> # convert Vm value to bytes
> if len(v) == 1:
> t = v[0].split() # e.g. 'VmRSS: 9999 kB'
> if len(t) == 3: ## and t[0] == VmKey:
> return float(t[1]) * _scale.get(t[2], 0.0)
> except:
> pass
> return 0.0
>
> def memory(since=0.0):
> '''Return process memory usage in bytes.
> '''
> return _VmB('VmSize:') - since
>
> def stacksize(since=0.0):
> '''Return process stack size in bytes.
> '''
> return _VmB('VmStk:') - since
>
>
>
> def slots(**kwds):
> '''Return the slots names as sequence.
> '''
> return tuple(kwds.keys())
>
> # __slots__ version
> class SlotsClass(object):
> __slots__ = slots(_attr1= False,
> _attr2= None,
> _attr3= None,
> _attr4= None,
> _attr5= None,
> _attr6= None,
> _attr7= 0,
> _attr8= None,
> _attr9= None,
> _attr10=None,
> _attr11=None,
> _attr12=None,
> _attr13=None,
> _attr14=None,
> _attr15=None,
> _attr16=None)
>
> def __init__(self, tuple4, parent):
> self._attr1 = False
> self._attr2 = None
> self._attr3 = None
> self._attr4 = None
> self._attr5 = None
> self._attr6 = None
> if parent:
> self._attr7 = parent._attr7 + 1
> self._attr8 = parent._attr8
> self._attr9 = parent._attr9
> self._attr10 = parent
> self._attr11 = parent._attr11
> self._attr12 = parent._attr12
> else:
> self._attr7 = 0
> self._attr8 = None
> self._attr9 = None
> self._attr10 = None
> self._attr11 = self
> self._attr12 = None
> self._attr13, self._attr14, self._attr15, self._attr16 = tuple4
>
>
> # __dict__ version
> class DictClass(object):
> _attr1 = None
> _attr2 = None
> _attr3 = None
> _attr4 = None
> _attr5 = None
> _attr6 = None
> _attr7 = 0
> _attr8 = None
> _attr9 = None
> _attr10 = None
> _attr11 = None
> _attr12 = None
> _attr13 = None
> _attr14 = None
> _attr15 = None
> _attr16 = None
>
> def __init__(self, tuple4, parent):
> if parent:
> self._attr7 = parent._attr7 + 1
> self._attr8 = parent._attr8
> self._attr9 = parent._attr9
> self._attr10 = parent
> self._attr11 = parent._attr11
> self._attr12 = parent._attr12
> else:
> self._attr11 = self
> self._attr13, self._attr14, self._attr15, self._attr16 = tuple4
>
>
> if __name__ == '__main__':
>
> import sys
>
> def report(txt, n, b0, c0):
> c = cputime(c0);
> b = memory(b0)
> print "%8d insts %s: %8.1e B %7.3f sec %6.1f B/i %6.1f usec/i" \
> % (n, txt, b, c, b/n, 1.0e6*c/n)
>
> if not sys.platform.startswith('linux'):
> raise NotImplementedError, "%r not supported" % sys.platform
>
> if 'dict' in sys.argv[1:]:
> print 'testing __dict__ version ...'
> testClass = DictClass
> else:
> print 'testing __slots__ version ...'
> testClass = SlotsClass
>
> t4 = ('', 0, 0, [])
> b0 = memory()
> c0 = cputime()
> p = testClass(t4, None)
> n, m = 1, 4096
> # generate 1+ M instances
> while n < 1050000: # 1048576:
> p = testClass(t4, p)
> n += 1
> if n >= m: # occasionally print stats
> m += m
> report('so far', n, b0, c0)
> report(' total', n, b0, c0)
>
> </pre>
In article <f8097096.0405241955.327e69dd at posting.google.com>, Nick
Jacobson <nicksjacobson at yahoo.com> wrote:
> The __slots__ attribute of new-style classes can reduce memory usage
> when there are millions of instantiations of a class.
>
> So would a __slots__ attribute for functions/methods have a similar
> benefit? i.e. could a function using __slots__ use significantly less
> memory, and therefore run faster, if called millions of times?
>
> If so, it will hopefully be in a future version of Python.
More information about the Python-list
mailing list