PyObject_CallFunctionObjArgs segfaults

[Jen Kris]

Thanks very much for your detailed reply.  I have a few followup questions.  

You said, “Some functions return an object that has already been incref'ed ("new reference"). This occurs when it has either created a new object (the refcount will be 1) or has returned a pointer to an existing object (the refcount will be > 1 because it has been incref'ed).  Other functions return an object that hasn't been incref'ed. This occurs when you're looking up something, for example, looking at a member of a list or the value of an attribute.” 

In the official docs some functions show “Return value: New reference” and others do not.  Is there any reason why I should not just INCREF on every new object, regardless of whether it’s a new reference or not, and DECREF when I am finished with it?  The answer at https://stackoverflow.com/questions/59870703/python-c-extension-need-to-py-incref-a-borrowed-reference-if-not-returning-it-to says “With out-of-order execution, the INCREF/DECREF are basically free operations, so performance is no reason to leave them out.”  Doing so means I don’t have to check each object to see if it needs to be INCREF’d or not, and that is a big help. 

Also: 

What is a borrowed reference, and how does it effect reference counting?  According to https://jayrambhia.com/blog/pythonc-api-reference-counting, “Use Py_INCREF on a borrowed PyObject pointer you already have. This increments the reference count on the object, and obligates you to dispose of it properly.”  So I guess it’s yes, but I’m confused by “pointer you already have.” 

What does it mean to steal a reference?  If a function steals a reference does it have to decref it without incref (because it’s stolen)?

Finally, you said:

if (pMod_random == 0x0){
    PyErr_Print();
Leaks here because of the refcount

Assuming pMod_random is not null, why would this leak? 

Thanks again for your input on this question. 

Jen



Sep 29, 2022, 17:33 by python at mrabarnett.plus.com:

> On 2022-09-30 01:02, MRAB wrote:
>
>> On 2022-09-29 23:41, Jen Kris wrote:
>>
>>>
>>> I just solved this C API problem, and I’m posting the answer to help anyone else who might need it.
>>>
> [snip]
>
> What I like to do is write comments that state which variables hold a reference, followed by '+' if it's a new reference (incref'ed) and '?' if it could be null. '+?' means that it's probably a new reference but could be null. Once I know that it's not null, I can remove the '?', and once I've decref'ed it (if required) and no longer need it, I remobe it from the comment.
>
> Clearing up references, as soon as they're not needed, helps to keep the number of current references more manageable.
>
>
> int64_t Get_LibModules(int64_t * return_array) {
>  PyObject * pName_random = PyUnicode_FromString("random");
>  //> pName_random+?
>  if (!pName_random) {
>  PyErr_Print();
>  return 1;
>  }
>
>  //> pName_random+
>  PyObject * pMod_random = PyImport_Import(pName_random);
>  //> pName_random+ pMod_random+?
>  Py_DECREF(pName_random);
>  //> pMod_random+?
>  if (!pMod_random) {
>  PyErr_Print();
>  return 1;
>  }
>
>  //> pMod_random+
>  PyObject * pAttr_seed = PyObject_GetAttrString(pMod_random, "seed");
>  //> pMod_random+ pAttr_seed?
>  if (!pAttr_seed) {
>  Py_DECREF(pMod_random);
>  PyErr_Print();
>  return 1;
>  }
>
>  //> pMod_random+ pAttr_seed
>  PyObject * pAttr_randrange = PyObject_GetAttrString(pMod_random, "randrange");
>  //> pMod_random+ pAttr_seed pAttr_randrange?
>  Py_DECREF(pMod_random);
>  //> pAttr_seed pAttr_randrange?
>  if (!pAttr_randrange) {
>  PyErr_Print();
>  return 1;
>  }
>
>  //> pAttr_seed pAttr_randrange
>  return_array[0] = (int64_t)pAttr_seed;
>  return_array[1] = (int64_t)pAttr_randrange;
>
>  return 0;
> }
>
> int64_t C_API_2(PyObject * pAttr_seed, Py_ssize_t value_1) {
>  PyObject * value_ptr = PyLong_FromLong(value_1);
>  //> value_ptr+?
>  if (!!value_ptr) {
>  PyErr_Print();
>  return 1;
>  }
>
>  //> value_ptr+
>  PyObject * p_seed_calc = PyObject_CallFunctionObjArgs(pAttr_seed, value_ptr, NULL);
>  //> value_ptr+ p_seed_calc+?
>  Py_DECREF(value_ptr);
>  //> p_seed_calc+?
>  if (!p_seed_calc) {
>  PyErr_Print();
>  return 1;
>  }
>
>  //> p_seed_calc+
>  Py_DECREF(p_seed_calc);
>  return 0;
> }
>
> int64_t C_API_12(PyObject * pAttr_randrange, Py_ssize_t value_1) {
>  PyObject * value_ptr = PyLong_FromLong(value_1);
>  //> value_ptr+?
>  if (!value_ptr) {
>  PyErr_Print();
>  return 1;
>  }
>
>  //> value_ptr+
>  PyObject * p_randrange_calc = PyObject_CallFunctionObjArgs(pAttr_randrange, value_ptr, NULL);
>  //> value_ptr+ p_randrange_calc+?
>  Py_DECREF(value_ptr);
>  //> p_randrange_calc+?
>  if (!p_randrange_calc) {
>  PyErr_Print();
>  return 1;
>  }
>
>  //Prepare return values
>  //> p_randrange_calc+
>  return_val = PyLong_AsLong(p_randrange_calc);
>  Py_DECREF(p_randrange_calc);
>
>  return return_val;
> }
>
> -- 
> https://mail.python.org/mailman/listinfo/python-list
>