reference counting and PyTuple_SetItem

[Anne Wilson]

Hi,

I'm embedding some Python code in my C code.  I've spent way too much 
time trying to ensure my code works properly.  I'm finding the docs and 
discussions very ambiguous.  The term "steal" is extremely ambiguous.

I want to thank Edward C. Jones for his reference counting document 
(http://members.tripod.com/~edcjones/refcount.html).  I found Edward's 
additional explanations very helpful, especially about "stealing a 
reference" and using PyTuple_SetItem().  It was especially enlightening 
to learn that PyTuple_SetItem will not incref the tuple's arguments, and 
will decref the tuple's arguments when the tuple is decrefed.

However, that method of reference count management seems to preclude two 
things I'm trying to do.  In one case, I'm reusing an item in more than 
one tuple, hence, more than one PyTuple_SetItem() call.  In the second 
case, in a loop I'm putting different items into the 4th argument of the 
same tuple.

Here's some pseudo code to illustrate what I'm trying to do (leaving out 
error checking, etc.):


     pyStatsDir = PyString_FromString(statsDir);
     ...

     /* ----------------------------------------
      * Set up arg list for 5 minute window size
      * ----------------------------------------
      */
     pyFiveMinArgs = PyTuple_New(PY_CALL_ARG_CNT);
     pyFiveMinWindowSize = PyInt_FromLong(300L);
     pyFiveMinBinSize = PyInt_FromLong(1L);
     pyFiveMinOutputFileStr = PyString_FromString("5min");
     /* ---> Note 1 <--- */
     PyTuple_SetItem(pyFiveMinArgs, 0, pyFiveMinWindowSize);
     PyTuple_SetItem(pyFiveMinArgs, 1, pyFiveMinBinSize);
     PyTuple_SetItem(pyFiveMinArgs, 2, pyFiveMinOutputFileStr);
     /* ---> Note 2 <--- */
     PyTuple_SetItem(pyFiveMinArgs, 3, pyStatsDir);
     /* Field 4, rHost, we'll get inside the loop */

     /* ----------------------------------------
      * Set up arg list for one hour window size
      * ----------------------------------------
      */
     pyOneHourArgs = PyTuple_New(PY_CALL_ARG_CNT);

     pyOneHourWindowSize = PyInt_FromLong(3600L);
     pyOneHourBinSize = PyInt_FromLong(15L);
     pyOneHourOutputFileStr = PyString_FromString("1hr");

     PyTuple_SetItem(pyOneHourArgs, 0, pyOneHourWindowSize);
     PyTuple_SetItem(pyOneHourArgs, 1, pyOneHourBinSize);
     PyTuple_SetItem(pyOneHourArgs, 2, pyOneHourOutputFileStr);
     /* ---> Note 2 <--- */
     PyTuple_SetItem(pyOneHourArgs, 3, pyStatsDir);
     /* Field 4, rHost, we'll get inside the loop */

     ...

     while (<some condition>)
     {
	pyRHost = PyString_FromString(rHost);

	Py_INCREF(pyRHost);  /* ---> Note 3 <--- */
	PyTuple_SetItem(pyFiveMinArgs, 4, pyRHost);
	PyTuple_SetItem(pyOneHourArgs, 4, pyRHost);

	pyValue = PyObject_CallObject(pyFunc, pyFiveMinArgs);
	Py_DECREF(pyValue);

	pyValue = PyObject_CallObject(pyFunc, pyOneDayArgs);
	Py_DECREF(pyValue);

	py_DECREF(pyRHost);   /* ---> Note 3 <--- */
     }
     Py_DECREF(pyStatsDir);
     Py_DECREF(pyFeedtype);
     Py_DECREF(pyFiveMinArgs);
     Py_DECREF(pyOneHourArgs);


---> Note 1: This looks straightforward - don't increment the ref count 
for pyFiveMinWindowSize, pyFiveMinBinSize, pyFiveMinOutputFileStr, as 
when pyFiveMinArgs is decref'ed it will handle the ref counting for 
those objects.

---> Note 2: Here I am reusing pyStatsDir in multiple tuples. Now, when 
the loop ends, each tuple will decref pyStatsDir??  Since the pyStatsDir 
isn't increfed, seems like that's too many decrefs!

---> Note 3: Here, pyRHost varies across each iteration of the while 
loop.  I *want* the pyRHost object space to be freed each time through 
the loop.  So, I thought I should keep a ref myself and then decrement 
it at the bottom of the loop, except for possibly the last iteration. 
Is this right???  (Also, not sure if I can determine if I'm in the last 
iteration of the loop or not.)

I'm still getting spurious crashes.  Maybe I just can't use 
PyTuple_SetItem?  If so, then what should I be using?

Any help or comments would be greatly appreciated.

Thanks!

Anne