[Python-Dev] Limit free list of method and builtin function objects (was: [Python-checkins] r60614 - in python/trunk: Misc/NEWS Objects/classobject.c Objects/methodobject.c)
M.-A. Lemburg
mal at egenix.com
Wed Feb 6 14:00:04 CET 2008
Hi Christian,
could you explain how you came up with the 256 entry limit ?
It appears to be rather low and somehow arbitrary.
I understand that some limit is required, but since these
objects get created a lot (e.g. for bound methods), setting the
limit too low will significantly slow down the interpreter.
BTW: What does pybench have to say to this patch ?
To get an idea of how many objects are typically part of the
free list, I'd suggest running an application such as Zope for
a while and then check the maximum numfree value.
Thanks,
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, Feb 06 2008)
>>> Python/Zope Consulting and Support ... http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
________________________________________________________________________
:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
On 2008-02-06 13:44, christian.heimes wrote:
> Author: christian.heimes
> Date: Wed Feb 6 13:44:34 2008
> New Revision: 60614
>
> Modified:
> python/trunk/Misc/NEWS
> python/trunk/Objects/classobject.c
> python/trunk/Objects/methodobject.c
> Log:
> Limit free list of method and builtin function objects to 256 entries each.
>
> Modified: python/trunk/Misc/NEWS
> ==============================================================================
> --- python/trunk/Misc/NEWS (original)
> +++ python/trunk/Misc/NEWS Wed Feb 6 13:44:34 2008
> @@ -12,6 +12,9 @@
> Core and builtins
> -----------------
>
> +- Limit free list of method and builtin function objects to 256 entries
> + each.
> +
> - Patch #1953: Added ``sys._compact_freelists()`` and the C API functions
> ``PyInt_CompactFreeList`` and ``PyFloat_CompactFreeList``
> to compact the internal free lists of pre-allocted ints and floats.
>
> Modified: python/trunk/Objects/classobject.c
> ==============================================================================
> --- python/trunk/Objects/classobject.c (original)
> +++ python/trunk/Objects/classobject.c Wed Feb 6 13:44:34 2008
> @@ -4,10 +4,16 @@
> #include "Python.h"
> #include "structmember.h"
>
> +/* Free list for method objects to safe malloc/free overhead
> + * The im_self element is used to chain the elements.
> + */
> +static PyMethodObject *free_list;
> +static int numfree = 0;
> +#define MAXFREELIST 256
> +
> #define TP_DESCR_GET(t) \
> (PyType_HasFeature(t, Py_TPFLAGS_HAVE_CLASS) ? (t)->tp_descr_get : NULL)
>
> -
> /* Forward */
> static PyObject *class_lookup(PyClassObject *, PyObject *,
> PyClassObject **);
> @@ -2193,8 +2199,6 @@
> In case (b), im_self is NULL
> */
>
> -static PyMethodObject *free_list;
> -
> PyObject *
> PyMethod_New(PyObject *func, PyObject *self, PyObject *klass)
> {
> @@ -2207,6 +2211,7 @@
> if (im != NULL) {
> free_list = (PyMethodObject *)(im->im_self);
> PyObject_INIT(im, &PyMethod_Type);
> + numfree--;
> }
> else {
> im = PyObject_GC_New(PyMethodObject, &PyMethod_Type);
> @@ -2332,8 +2337,14 @@
> Py_DECREF(im->im_func);
> Py_XDECREF(im->im_self);
> Py_XDECREF(im->im_class);
> - im->im_self = (PyObject *)free_list;
> - free_list = im;
> + if (numfree < MAXFREELIST) {
> + im->im_self = (PyObject *)free_list;
> + free_list = im;
> + numfree++;
> + }
> + else {
> + PyObject_GC_Del(im);
> + }
> }
>
> static int
> @@ -2620,5 +2631,7 @@
> PyMethodObject *im = free_list;
> free_list = (PyMethodObject *)(im->im_self);
> PyObject_GC_Del(im);
> + numfree--;
> }
> + assert(numfree == 0);
> }
>
> Modified: python/trunk/Objects/methodobject.c
> ==============================================================================
> --- python/trunk/Objects/methodobject.c (original)
> +++ python/trunk/Objects/methodobject.c Wed Feb 6 13:44:34 2008
> @@ -4,7 +4,12 @@
> #include "Python.h"
> #include "structmember.h"
>
> +/* Free list for method objects to safe malloc/free overhead
> + * The m_self element is used to chain the objects.
> + */
> static PyCFunctionObject *free_list = NULL;
> +static int numfree = 0;
> +#define MAXFREELIST 256
>
> PyObject *
> PyCFunction_NewEx(PyMethodDef *ml, PyObject *self, PyObject *module)
> @@ -14,6 +19,7 @@
> if (op != NULL) {
> free_list = (PyCFunctionObject *)(op->m_self);
> PyObject_INIT(op, &PyCFunction_Type);
> + numfree--;
> }
> else {
> op = PyObject_GC_New(PyCFunctionObject, &PyCFunction_Type);
> @@ -125,8 +131,14 @@
> _PyObject_GC_UNTRACK(m);
> Py_XDECREF(m->m_self);
> Py_XDECREF(m->m_module);
> - m->m_self = (PyObject *)free_list;
> - free_list = m;
> + if (numfree < MAXFREELIST) {
> + m->m_self = (PyObject *)free_list;
> + free_list = m;
> + numfree++;
> + }
> + else {
> + PyObject_GC_Del(m);
> + }
> }
>
> static PyObject *
> @@ -346,14 +358,16 @@
> PyCFunctionObject *v = free_list;
> free_list = (PyCFunctionObject *)(v->m_self);
> PyObject_GC_Del(v);
> + numfree--;
> }
> + assert(numfree == 0);
> }
>
> /* PyCFunction_New() is now just a macro that calls PyCFunction_NewEx(),
> but it's part of the API so we need to keep a function around that
> existing C extensions can call.
> */
> -
> +
> #undef PyCFunction_New
> PyAPI_FUNC(PyObject *) PyCFunction_New(PyMethodDef *, PyObject *);
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
More information about the Python-Dev
mailing list