PyObject_CallFunctionObjArgs segfaults

Thu Sep 29 18:41:39 EDT 2022

I just solved this C API problem, and I’m posting the answer to help anyone else who might need it.  

The errors were:

(1) we must call Py_INCREF on each object when it’s created.

(2) in C_API_2 (see below) we don’t cast value_1 as I did before with PyObject * value_ptr = (PyObject * )value_1.  Instead we use PyObject * value_ptr = PyLong_FromLong(value_1);

(3) The command string to PyObject_CallFunctionObjArgs must be null terminated.

Here’s the revised code:

First we load the modules, and increment the reference to each object: 

int64_t Get_LibModules(int64_t * return_array)
{
PyObject * pName_random = PyUnicode_FromString("random");
PyObject * pMod_random = PyImport_Import(pName_random);

Py_INCREF(pName_random);
Py_INCREF(pMod_random);

if (pMod_random == 0x0){
PyErr_Print();
return 1;}

PyObject * pAttr_seed = PyObject_GetAttrString(pMod_random, "seed");
PyObject * pAttr_randrange = PyObject_GetAttrString(pMod_random, "randrange");

Py_INCREF(pAttr_seed);
Py_INCREF(pAttr_randrange);

return_array[0] = (int64_t)pAttr_seed;
return_array[1] = (int64_t)pAttr_randrange;

return 0;
}

Next we call a program to initialize the random number generator with random.seed(), and increment the reference to its return value p_seed_calc:

int64_t C_API_2(PyObject * pAttr_seed, Py_ssize_t value_1)
{
PyObject * value_ptr = PyLong_FromLong(value_1);
PyObject * p_seed_calc = PyObject_CallFunctionObjArgs(pAttr_seed, value_ptr, NULL);

// _________

if (p_seed_calc == 0x0){
    PyErr_Print();
    return 1;}

Py_INCREF(p_seed_calc);

return 0;
}

Now we call another program to get a random number:

int64_t C_API_12(PyObject * pAttr_randrange, Py_ssize_t value_1)
{
PyObject * value_ptr = PyLong_FromLong(value_1);
PyObject * p_randrange_calc = PyObject_CallFunctionObjArgs(pAttr_randrange, value_ptr, NULL);

if (p_randrange_calc == 0x0){
    PyErr_Print();
    return 1;}

//Prepare return values
long return_val = PyLong_AsLong(p_randrange_calc);

return return_val;
}

That returns 28, which is what I get from the Python command line. 

Thanks again to MRAB for helpful comments. 

Jen

Sep 29, 2022, 15:31 by python at mrabarnett.plus.com:

> On 2022-09-29 21:47, Jen Kris wrote:
>
>> To update my previous email, I found the problem, but I have a new problem.
>>
>> Previously I cast PyObject * value_ptr = (PyObject * )value_1 but that's not correct.  Instead I used PyObject * value_ptr = PyLong_FromLong(value_1) and that works.  HOWEVER, while PyObject_CallFunctionObjArgs does work now, it returns -1, which is not the right answer for random.seed.  I use "long return_val = PyLong_AsLong(p_seed_calc);" to convert it to a long.
>>
> random.seed returns None, so when you call PyObject_CallFunctionObjArgs it returns a new reference to Py_None.
>
> If you then pass to PyLong_AsLong a reference to something that's not a PyLong, it'll set an error and return -1.
>
>> So my question is why do I get -1 as return value?  When I query p_seed calc : get:
>>
>> (gdb) p p_seed_calc
>> $2 = (PyObject *) 0x7ffff69be120 <_Py_NoneStruct>
>>
> Exactly. It's Py_None, not a PyLong.
>
>> Thanks again.
>>
>> Jen
>>
>>
>>
>>
>> Sep 29, 2022, 13:02 by python-list at python.org:
>>
>>  Thanks very much to @MRAB for taking time to answer.  I changed my
>>  code to conform to your answer (as best I understand your comments
>>  on references), but I still get the same error.  My comments
>>  continue below the new code immediately below.
>>
>>  int64_t Get_LibModules(int64_t * return_array)
>>  {
>>  PyObject * pName_random = PyUnicode_FromString("random");
>>  PyObject * pMod_random = PyImport_Import(pName_random);
>>
>>  Py_INCREF(pName_random);
>>  Py_INCREF(pMod_random);
>>
>>  if (pMod_random == 0x0){
>>  PyErr_Print();
>>  return 1;}
>>
>>  PyObject * pAttr_seed = PyObject_GetAttrString(pMod_random, "seed");
>>  PyObject * pAttr_randrange = PyObject_GetAttrString(pMod_random,
>>  "randrange");
>>
>>  Py_INCREF(pAttr_seed);
>>  Py_INCREF(pAttr_randrange);
>>
>>  return_array[0] = (int64_t)pAttr_seed;
>>  return_array[1] = (int64_t)pAttr_randrange;
>>
>>  return 0;
>>  }
>>
>>  int64_t C_API_2(PyObject * pAttr_seed, Py_ssize_t value_1)
>>  {
>>  PyObject * value_ptr = (PyObject * )value_1;
>>  PyObject * p_seed_calc = PyObject_CallFunctionObjArgs(pAttr_seed,
>>  value_ptr, NULL);
>>
>>  if (p_seed_calc == 0x0){
>>      PyErr_Print();
>>      return 1;}
>>
>>  //Prepare return values
>>  long return_val = PyLong_AsLong(p_seed_calc);
>>
>>  return return_val;
>>  }
>>
>>  So I incremented the reference to all objects in Get_LibModules,
>>  but I still get the same segfault at
>>  PyObject_CallFunctionObjArgs.  Unfortunately, reference counting
>>  is not well documented so I’m not clear what’s wrong.
>>
>>
>>
>>
>>  Sep 29, 2022, 10:06 by python at mrabarnett.plus.com:
>>
>>  On 2022-09-29 16:54, Jen Kris via Python-list wrote:
>>
>>  Recently I completed a project where I used
>>  PyObject_CallFunctionObjArgs extensively with the NLTK
>>  library from a program written in NASM, with no problems.
>>  Now I am on a new project where I call the Python random
>>  library.  I use the same setup as before, but I am getting
>>  a segfault with random.seed.
>>
>>  At the start of the NASM program I call a C API program
>>  that gets PyObject pointers to “seed” and “randrange” in
>>  the same way as I did before:
>>
>>  int64_t Get_LibModules(int64_t * return_array)
>>  {
>>  PyObject * pName_random = PyUnicode_FromString("random");
>>  PyObject * pMod_random = PyImport_Import(pName_random);
>>
>>  Both PyUnicode_FromString and PyImport_Import return new
>>  references or null pointers.
>>
>>  if (pMod_random == 0x0){
>>  PyErr_Print();
>>
>>
>>  You're leaking a reference here (pName_random).
>>
>>  return 1;}
>>
>>  PyObject * pAttr_seed =
>>  PyObject_GetAttrString(pMod_random, "seed");
>>  PyObject * pAttr_randrange =
>>  PyObject_GetAttrString(pMod_random, "randrange");
>>
>>  return_array[0] = (int64_t)pAttr_seed;
>>  return_array[1] = (int64_t)pAttr_randrange;
>>
>>
>>  You're leaking 2 references here (pName_random and pMod_random).
>>
>>  return 0;
>>  }
>>
>>  Later in the same program I call a C API program to call
>>  random.seed:
>>
>>  int64_t C_API_2(PyObject * pAttr_seed, Py_ssize_t value_1)
>>  {
>>  PyObject * p_seed_calc =
>>  PyObject_CallFunctionObjArgs(pAttr_seed, value_1);
>>
>>
>>  It's expecting all of the arguments to be PyObject*, but
>>  value_1 is Py_ssize_t instead of PyObject* (a pointer to a
>>  _Python_ int).
>>
>>  The argument list must end with a null pointer.
>>
>>  It returns a new reference or a null pointer.
>>
>>
>>  if (p_seed_calc == 0x0){
>>      PyErr_Print();
>>      return 1;}
>>
>>  //Prepare return values
>>  long return_val = PyLong_AsLong(p_seed_calc);
>>
>>  You're leaking a reference here (p_seed_calc).
>>
>>  return return_val;
>>  }
>>
>>  The first program correctly imports “random” and gets
>>  pointers to “seed” and “randrange.”  I verified that the
>>  same pointer is correctly passed into C_API_2, and the
>>  seed value (1234) is passed as  Py_ssize_t value_1.  But I
>>  get this segfault:
>>
>>  Program received signal SIGSEGV, Segmentation fault.
>>  0x00007ffff64858d5 in _Py_INCREF (op=0x4d2) at
>>  ../Include/object.h:459
>>  459     ../Include/object.h: No such file or directory.
>>
>>  So I tried Py_INCREF in the first program:
>>
>>  Py_INCREF(pMod_random);
>>  Py_INCREF(pAttr_seed);
>>
>>  Then I moved Py_INCREF(pAttr_seed) to the second program. 
>>  Same segfault.
>>
>>  Finally, I initialized “random” and “seed” in the second
>>  program, where they are used.  Same segfault.
>>
>>  The segfault refers to Py_INCREF, so this seems to do with
>>  reference counting, but Py_INCREF didn’t solve it.
>>
>>  I’m using Python 3.8 on Ubuntu.
>>
>>  Thanks for any ideas on how to solve this.
>>
>>  Jen
>>
>>
>>  --         https://mail.python.org/mailman/listinfo/python-list
>>
>>
>>  --     https://mail.python.org/mailman/listinfo/python-list
>>
> -- 
> https://mail.python.org/mailman/listinfo/python-list
>