[Tutor] static PyObject * or non-static

[Danny Yoo]

On Tue, 14 Sep 2004, Rob Benton wrote:

> Alan Gauld wrote:
>
> >>static PyObject *
> >>Noddy_name(Noddy* self)
> >>{
> >>    static PyObject *format = NULL;
> >>    PyObject *args, *result;
> >>
> >>
> >>make everything static that can be?
> >
> >
> In that chunk of code why isn't the format object temporary?  I guess
> that's what confusing me.



[Note: skip if you're starting to learn Python.  This has more to do with
the Python/C API, which isn't relevant to most Python programmers.]


Hi Rob,


It's an optimization hack.  *grin*


Let's take a look again at the code.  This thread seems like it's been
running for a bit without a good resolution, so let's summarize what we're
talking about.


In the Extending and Embedding documentation, there's a function called
Noddy_name():

    http://docs.python.org/ext/node22.html

This Noddy_name() implements the name() method for a Noddy object, in C.


Part of this name() method involves making the format string:

   "%s %s"

so that it can later apply string formatting.  To generate that format
string, we need to do this:

    format = PyString_FromString("%s %s");


Remember, though, that we're in C, so as a C programmer, we try to
overoptimize everything, even if it's not appropriate.  *grin* In this
case, though, the optimization with static variables is sorta is nice to
do, and we'll see why in a moment.


What the code in Noddy_name is trying to do is cache the result of this
PyString_FromString() call in a static variable.  Static variables in C
live practically forever, so every time we enter Noddy_name(), the code
checks to see if the 'format' string has been initialized yet, using NULL
as a sentinel to mark that's something's uninitialized:

    if (format == NULL) {
        format = PyString_FromString("%s %s");
        if (format == NULL)
            return NULL;
    }


Why does the system check for (format == NULL) after
PyString_FromString()?  Because we could have run out of memory, trying to
construct a 5 character string.  Remember, this is C: we have to worry
about memory allocation and error trapping ALL THE TIME.  *grin*



In summary, the Noddy_name() code that looks like this:


/******/
static PyObject *
Noddy_name(Noddy* self)
{
    static PyObject *format = NULL;
    PyObject *args, *result;

    if (format == NULL) {
        format = PyString_FromString("%s %s");
        if (format == NULL)
            return NULL;
    }

    if (self->first == NULL) {
        PyErr_SetString(PyExc_AttributeError, "first");
        return NULL;
    }

    if (self->last == NULL) {
        PyErr_SetString(PyExc_AttributeError, "last");
        return NULL;
    }

    args = Py_BuildValue("OO", self->first, self->last);
    if (args == NULL)
        return NULL;

    result = PyString_Format(format, args);
    Py_DECREF(args);

    return result;
}
/******/



can be written like this:


/******/
static PyObject *
Noddy_name(Noddy* self)
{
    PyObject *format = NULL;
    PyObject *args = NULL;
    PyObject *result = NULL;

    if (self->first == NULL) {
        PyErr_SetString(PyExc_AttributeError, "first");
        goto error;
    }

    if (self->last == NULL) {
        PyErr_SetString(PyExc_AttributeError, "last");
        goto error;
    }

    args = Py_BuildValue("OO", self->first, self->last);
    if (args == NULL)
        goto error;

    format = PyString_FromString("%s %s");
    if (format == NULL)
        goto error;

    result = PyString_Format(format, args);

    Py_DECREF(args);
    Py_DECREF(format);
    return result;

    error:
    Py_XDECREF(args);
    Py_XDECREF(format);
    return NULL;
}
/******/


So here we've gotten rid of the static variable.  This should have same
functionality as the old code, although it will run slightly slower since
it constructs/destructs the format string on each function call.


But the code ends up being a little bit more ugly, because we have to go
through some hoops to make sure we don't leak memory.  That involves
things like making sure all the heap-allocated variables are freed, and to
make sure we don't try to free something that hasn't been initialized yet.


If anything "bad"  happens, we need to make sure to deallocate memory for
both the 'args' and 'format'.  And the revised code does this using a
'goto' to an error handling block.  Py_XDECREF is like Py_DECREF, but it
still works right even if the pointer given is NULL.

See:

    http://docs.python.org/api/countingRefs.html#l2h-68

and

    http://docs.python.org/api/exceptions.html#l2h-26

for another example of this Python/C idiom for handling exceptions.


In the original code, since we intend to make 'format' live forever
anyway, we can omit the code to DECREF it.  So we get two benefits: the
original program's a little faster, and ends up being shorter to type.


Aren't you glad you can program in Python and not in C?  *grin*


Hope this helps!