reference counting and PyTuple_SetItem

[Anne Wilson]

Tim,

Thanks much for responding!

Tim Peters wrote:
> 
> Apart from all that, you're cheating in ways that can hurt you:  tuples are
> supposed to be immutable objects, and it's not strictly legal to mutate them
> inside the loop.  You'd be much better off not trying to micro-optimize
> (i.e., build a new tuple each time you need one -- never use PyTuple_SetItem
> on the same index twice for a given tuple; the code as-is is too clever to
> ever work <0.9 wink>).
> 

Yeah, although I was resisting, I have come around to that conclusion. 
I'm testing this approach now.


First I tried this:

pyFiveMinArgs = PyTuple_New(PY_CALL_ARG_CNT);
PyTuple_SetItem(pyFiveMinArgs, 0, PyInt_FromLong(300L));
PyTuple_SetItem(pyFiveMinArgs, 1, PyInt_FromLong(1L));
PyTuple_SetItem(pyFiveMinArgs, 2, PyString_FromString("5min"));
PyTuple_SetItem(pyFiveMinArgs, 3, PyString_FromString(statsDir));
/* Field 4, rHost, we'll get from the article */


while(<some condition>)
{
     PyTuple_SetItem(pyFiveMinArgs, 4, PyString_FromString(rHost));
     PyTuple_SetItem(pyOneHourArgs, 4, PyString_FromString(rHost));
     ...
     pyValue = PyObject_CallObject(pyFunc, pyFiveMinArgs);
     Py_DECREF(pyValue);

     pyValue = PyObject_CallObject(pyFunc, pyOneHourArgs);
     Py_DECREF(pyValue);
}


but the program wasn't freeing memory properly - it got huge.  This 
makes sense to me as the tuples are never decrefed, so the rHost objects 
are never freed.

Now I'm testing this:


while(<some condition>)
{
     pyFiveMinArgs = PyTuple_New(PY_CALL_ARG_CNT);
     PyTuple_SetItem(pyFiveMinArgs, 0, PyInt_FromLong(300L));
     PyTuple_SetItem(pyFiveMinArgs, 1, PyInt_FromLong(1L));
     PyTuple_SetItem(pyFiveMinArgs, 2, PyString_FromString("5min"));
     PyTuple_SetItem(pyFiveMinArgs, 3, PyString_FromString(statsDir));
     PyTuple_SetItem(pyFiveMinArgs, 4, PyString_FromString(rHost));

     pyOneHourArgs = PyTuple_New(PY_CALL_ARG_CNT);
     PyTuple_SetItem(pyOneHourArgs, 0, PyInt_FromLong(3600L));
     PyTuple_SetItem(pyOneHourArgs, 1, PyInt_FromLong(15L));
     PyTuple_SetItem(pyOneHourArgs, 2, PyString_FromString("1hr"));
     PyTuple_SetItem(pyOneHourArgs, 3, PyString_FromString(statsDir));
     PyTuple_SetItem(pyOneHourArgs, 4, PyString_FromString(rHost));
     ...

     pyValue = PyObject_CallObject(pyFunc, pyFiveMinArgs);
     Py_DECREF(pyValue);
     pyValue = PyObject_CallObject(pyFunc, pyOneHourArgs);
     Py_DECREF(pyValue);

     ...
     Py_DECREF(pyFiveMinArgs);
     Py_DECREF(pyOneHourArgs);
}


But, OW!  It pains me to be so inefficient, creating the same damn 
PyObjects over and over and over and over and over again.  To me this is 
way beyond "micro" optimization.  In my own code I'm actually doing this 
for three more cases beyond the fiveMin and oneHour stuff shown above.

Is there a more efficient way to do this embedded call?  (... other than 
rewriting the Python code in C...)  Can I use a list instead of tuple?

Anne