Creating custom types from C code

[Eric Frederich]

Thanks for the reply.

I remember reading about named tuples when they were back-ported to
the 2.X series but I never played with them.
Is there a way to instantiate a named tuple from C code?

Maybe I'm over-thinking this whole thing.
Is there a simple way that I can define a class in Python and
instantiate that type from C?

On Sat, Dec 18, 2010 at 1:18 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Eric Frederich, 17.12.2010 23:58:
>>
>> I have an extension module for a 3rd party library in which I am
>> wrapping some structures.
>> My initial attempt worked okay on Windows but failed on Linux.
>> I was doing it in two parts.
>> The first part on the C side of things I was turning the entire
>> structure into a char array.
>> The second part in Python code I would unpack the structure.
>>
>> Anyway, I decided I should be doing things a little cleaner so I read
>> up on "Defining New Types"
>> http://docs.python.org/extending/newtypes.html
>>
>> I got it to work but I'm not sure how to create these new objects from C.
>
> You may want to take a look at Cython. It makes writing C extensions easy.
> For one, it will do all sorts of type conversions for you, and do them
> efficiently and safely (you get an exception on overflow, for example). It's
> basically Python, so creating classes and instantiating them is trivial.
>
> Also note that it's generally not too much work to rewrite an existing C
> wrapper in Cython, but it's almost always worth it. You immediately get more
> maintainable code that's much easier to extend and work on. It's also often
> faster than hand written code.
>
> http://cython.org
>
>
>> My setup is almost exactly like the example on that page except
>> instead of 2 strings and an integer I have 5 unsigned ints.
>>
>> I do not expect to ever be creating these objects in Python.  They
>> will only be created as return values from my wrapper functions to the
>> 3rd party library.
>
> In Cython 0.14, you can declare classes as "final" and "internal" using a
> decorator, meaning that they cannot be subtyped from Python and do not show
> up in the module dict. However, note that there is no way to prevent users
> from getting their hands at the type once you give them an instance.
>
>
>> I could return a tuple from those functions but I want to use dot
>> notation (e.g. somestruct.var1).
>
> Then __getattr__ or properties are your friend.
>
>
>> So, question number 1:
>>     Is defining my own type like that overkill just to have an object
>> to access using dots?
>
> Creating wrapper objects is totally normal.
>
> Also note that recent Python versions have named tuples, BTW.
>
>
>>     I'll never create those objects from Python.
>>     Is there a shortcut to creating objects and setting attributes
>> from within C?
>
> The Cython code for instantiating classes is identical to Python.
>
>
>> In any case, I was able to create my own custom object from C code like
>> so...
>>
>>     PyObject *foo(SomeCStruct bar){
>>         PyObject *ret;
>>         ret = _PyObject_New(&mymodule_SomeStructType);
>>         PyObject_SetAttrString(ret, "var1" , Py_BuildValue("I", bar.var1
>> ));
>>         PyObject_SetAttrString(ret, "var2" , Py_BuildValue("I", bar.var2
>> ));
>>         PyObject_SetAttrString(ret, "var3" , Py_BuildValue("I", bar.var3
>> ));
>>         PyObject_SetAttrString(ret, "var4" , Py_BuildValue("I", bar.var4
>> ));
>>         PyObject_SetAttrString(ret, "var5" , Py_BuildValue("I", bar.var5
>> ));
>>         return ret;
>>     }
>>
>> When using _PyObject_New I notice that neither my new or init function
>> are ever called.
>> I verified that they are getting called when creating the object from
>> Python
>
> Things often work a little different in Python and C. Directly calling
> _PyObject_New() is a lot less than what Python does internally. The
> canonical way is to PyObject_Call() the type (or to use one of the other
> call functions, depending on what your arguments are).
>
>
>> (which I would never do anyway).
>
> Your users could do it, though, so you should make sure that won't crash the
> interpreter that way by leaving internal data fields uninitialised.
>
>
>> Question number 2:
>>     Do I need to be calling PyObject_SetAttrString or is there a way
>> to set the unsigned ints on the structure direcly?
>>     It seems overkill to create a Python object for an unsigned int
>> just to set it as an attribute on a custom defined type.
>
> You will have to do it at some point, though, either at instantiation time
> or at Python access time. Depending on the expected usage, either of the two
> can be more wasteful.
>
>
>> Question number 3:
>>     In the above code, is there a memory leak?  Should I be
>> Py_DECREF'ing the return value from Py_BuildValue after I'm done using
>> it.
>
> You can look that up in the C-API docs. If a function doesn't say that it
> "steals" a reference, you still own the reference when it returns and have
> to manually decref it (again, a thing that you won't usually have to care
> about in Cython). So, yes, the above leaks one reference for each call to
> Py_BuildValue().
>
> Stefan
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>