[pypy-dev] Py_DecRef() in cpyext

[Armin Rigo]

Hi Stefan,

On Sun, Feb 26, 2012 at 09:50, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Looking at Py_DecRef(), however, left me somewhat baffled. I would have
> expected this to be the most intensively tuned function in all of cpyext,
> but it even started with this comment: (...)

Indeed, it's an obvious starting place if we want to optimize cpyext
(which did not occur at all so far).  You are welcome to try.  Note
that the JIT has nothing to do here: we cannot JIT any code written in
C, and it makes no sense to apply the JIT on a short RPython callback
alone.  But because most of the code in module/cpyext/ is RPython
code, it means it gets turned into equivalent C code statically, with
(as you noticed) the debugging checks constant-folded away.

The first thing to try would be to rethink how the PyPy object and the
PyObject are linked together.  Right now it's done with two (possibly
weak) dictionaries, one for each direction.  We can at least improve
the situation by having a normal field in the PyObject pointing back
to the PyPy object.  This needs to be done carefully but can be done.

The issue is that the GC needs to know about this field.  It would
probably require something like: allocate some GcArrays of PyObject
structures (not pointers, directly PyObjects --- which have all the
same size here, so it works).  Use something like 100 PyObject
structures per GcArray, and collect all the GcArrays in a global list.
 Use a freelist for dead entries.  If you allocate each GcArray as
"non-movable", then you can take pointers to the PyObjects and pass
them to C code.  As they are inside the regular GcArrays, they are
GC-tracked and can contain a field that points back to the PyPy
object.


A bientôt,

Armin.