Accessing __slots__ from C

[Hrvoje Niksic]

Chris <ceball at users.sourceforge.net> writes:

>> PyObject_GetAttrString is convenient, but it creates a Python string
>> only so it can intern it (and in most cases throw away the freshly
>> created version).  For maximum efficiency, pre-create the string
>> object using PyString_InternFromString, and use that with
>> PyObject_GetAttr.
>
> Yes, we'd thought of that too, but it doesn't seem to be an
> important factor compared to the actual attribute lookup.

I'd like to see your test code.  In my experience, as long as you're
accessing simple slots, you should notice a difference.
PyObject_GetAttr with an interned string key executes a single dict
lookup of the type object followed by C code that simply and very
efficiently dereferences the PyObject from a constant offset described
by the slot.  The dict lookup involved is quite optimized, it doesn't
even have to calculate the string's hash, which remains cached in the
interned string object.

On the other hand, PyObject_GetAttrString creates a new string each
time, looks it up in the set of interned strings (which itself
requires a dict lookup complete with hash calculation), and then
executes PyObject_GetAttrString.  These additional operations should
have execution time of the same order of magnitude as
PyObject_GetAttr's execution time.

Here is a test program that shows a 4.6 time speedup simply by
switching from PyObject_GetAttrString to PyObject_GetAttr:

#include <Python.h>

/* gcc -shared -O2 -I/usr/include/python2.5 attr.c -lpython2.5 -o attr.so */

static PyObject *
bench1(PyObject *ignored, PyObject *obj)
{
  int i;
  for (i = 0; i < 1000000; i++) {
    PyObject *attr = PyObject_GetAttrString(obj, "abcdef");
    if (!attr)
      return NULL;
    Py_DECREF(attr);
  }
  Py_RETURN_NONE;
}

static PyObject *
bench2(PyObject *ignored, PyObject *obj)
{
  static PyObject *key;
  if (!key) {
    key = PyString_InternFromString("abcdef");
    if (!key)
      return NULL;
  }
  int i;
  for (i = 0; i < 1000000; i++) {
    PyObject *attr = PyObject_GetAttr(obj, key);
    if (!attr)
      return NULL;
    Py_DECREF(attr);
  }
  Py_RETURN_NONE;
}

static PyMethodDef attr_methods[] = {
  {"bench1", (PyCFunction) bench1, METH_O },
  {"bench2", (PyCFunction) bench2, METH_O },
  {NULL}
};

#ifndef PyMODINIT_FUNC  /* declarations for DLL import/export */
#define PyMODINIT_FUNC void
#endif
PyMODINIT_FUNC
initattr(void) 
{
  PyObject *module = Py_InitModule3("attr", attr_methods, NULL);
}

>>> import attr, time
>>> class MyClass(object):
...   __slots__ = 'abcdef',
... 
>>> o = MyClass()
>>> o.abcdef = 1
>>> t0 = time.time(); attr.bench1(o); t1 = time.time()
>>> t1-t0
0.28227686882019043
>>> t0 = time.time(); attr.bench2(o); t1 = time.time()
>>> t1-t0
0.060719013214111328

(Repeated runs show very similar timings.)