Calling python from C with OpenMP

[oysteijo at gmail.com]

Hi,

I have a framework written in C and I need to call Python from that framework. I have written the code, and it runs fine, however when I recompile with OpenMP enabled, I get segmentation faults and some times an error message: 

Fatal Python error: GC object already tracked

I'm able to reconstruct the bug with this simple code:

/* main.c */
#include <Python.h>
#include <omp.h>
int main()
{
    wchar_t *program = Py_DecodeLocale( "My_problem", NULL );
    if( !program ){
        fprintf(stderr, "cannot decode python program name\n");
        return -1;
    }

    Py_SetProgramName( program );
    Py_Initialize();

    PyObject *sys = PyImport_ImportModule("sys");
    PyObject *path = PyObject_GetAttrString(sys, "path");
    PyList_Append(path, PyUnicode_FromString("."));

    PyObject *module_filename = PyUnicode_FromString( "multiplier" );
    if(!module_filename){
        printf("Cannot create python module  multiplier.\n");
        return -1;
    }
    PyObject *module = PyImport_Import( module_filename );
    if(!module){
        printf("Cannot create python.\n");
        return -1;
    }

    PyObject *mult_obj = PyObject_CallMethod( module ,"multiplier", "i", 7);
    if(!mult_obj){
        printf("Cannot create python multiplier class instance\n");
        return -1;
    }

    Py_DECREF( module );
    Py_DECREF( module_filename );
    Py_DECREF( path );
    Py_DECREF( sys );
    /* Up to now we have actually done:
     * >>> import multiplier
     * >>> mult_obj = multipier.multiplier(7)
     */

    /* lets try something like:
     * >>> for x in range(10):
     * ...     printf(mult_obj.do_multiply(x))
     */

#pragma omp parallel for 
    for( int i = 0; i < 10; i++ ){
        PyObject *ret = PyObject_CallMethod( mult_obj, "do_multiply", "i", i );
        if( !ret ){
            printf("Cannot call 'do_multiply'\n");
            continue;
        }
        printf("The value calculated in Python was: %3d\n", (int) PyLong_AsLong(ret));
        Py_DECREF(ret);
    }

    Py_DECREF(mult_obj);
    Py_Finalize();

    return 0;
} 

Compile with:
gcc -std=gnu99 -O3 -Wall -Wextra -fopenmp `pkg-config --cflags --libs python3` -lgomp main.c -o main

Then you need the python code:

# multiplier.py
class multiplier(object):
    def __init__(self, factor):
        self.factor = factor
    def do_multiply(self, x):
        return self.factor * x

First question: Does my C code leak memory? Valgrind says it does, but the memory footprint of the executable is stable while looping?

Second and most important question: When I run this code it sometimes segementation faults, and sometimes some threads run normal and some other threads says "Cannot call 'do_multiply'". Sometimes I get the message: Fatal Python error: GC object already tracked. And some times it even runs normally...
I understand there is some kind of race condition here, where python tries to refer to some memory that has been already released. But how can I avoid this? What am I doing wrong? (or, less likely, is this a bug?)

Maybe needless to say, but the code works when compiled w/o OpenMP.

Using Python 3.5.1

Thanks,
-Øystein