Unable to subclass ctypes.c_uint64: was: Re: Battle of the garbage collectors, or ARGGHHHHHH!!!!

[CFK]

On Wed, Apr 26, 2017 at 10:38 PM, Cem Karan <cfkaran2 at gmail.com> wrote:

>
> On Apr 24, 2017, at 8:54 PM, Jon Ribbens <jon+usenet at unequivocal.eu>
> wrote:
>
> > On 2017-04-24, CFK <cfkaran2 at gmail.com> wrote:
> >> Long version: I'm trying to write bindings for python via ctypes to
> control
> >> a library written in C that uses the bdwgc garbage collector (
> >> http://www.hboehm.info/gc/).  The bindings mostly work, except for when
> >> either bdwgc or python's garbage collector decide to get into an
> argument
> >> over what is garbage and what isn't, in which case I get a segfault
> because
> >> one or the other collector has already reaped the memory.
> >
> > Make your Python C objects contain a pointer to a
> > GC_MALLOC_UNCOLLECTABLE block that contains a pointer to the
> > bwdgc object it's an interface to? And GC_FREE it in tp_dealloc?
> > Then bwdgc won't free any C memory that Python is referencing.
>
> OK, I realized today that there was a miscommunication somewhere.  My
> python code is all pure python, and the library is pure C, and it is not
> designed to be called by python (it's intended to be language neutral, so
> if someone wants to call it from a different language, they can).  That
> means that tp_dealloc (which is part of the python C API) is probably not
> going to work.
>
> I got interrupted (again) so I didn't have a chance to try the next trick
> and register the ctypes objects as roots from which to scan in bdwgc, but
> I'm hoping that roots aren't removed.  If that works, I'll post it to the
> list.
>
> Thanks,
> Cem Karan


I'm still working on fixing the battle of the garbage collectors, but as a
part of that work I've realized that it would be handy for me to subclass
various ctypes like so:

"""
from ctypes import *
class foo(c_uint64):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        #Additional setup work here, including marking this type as
something bdwgc should alone
    def __del__(self):
        #Allow anything not owned by python to be reclaimed by bdwgc
        super().__del__()
"""

Where the additional work would shift the type from being in the root set
to out of it, and (I hope) stopping the battle of the garbage collectors.
The issue is that while the above code works in python 3.4 and earlier, I
get the following from python 3.6.1:

"""
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __class__ set to <class '__main__.foo'> defining 'foo' as <class
'__main__.foo'>
"""

Is this the way of the future, or is this a bug that should be reported
appropriately?

Relevant info:

"""
$ python
Python 3.6.1 (default, Apr 24 2017, 08:00:07)
[GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.42)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()
$ uname -a
Darwin Mac-Pro.local 16.5.0 Darwin Kernel Version 16.5.0: Fri Mar  3
16:52:33 PST 2017; root:xnu-3789.51.2~3/RELEASE_X86_64 x86_64
$ sw_vers
ProductName:    Mac OS X
ProductVersion:    10.12.4
BuildVersion:    16E195
"""

Thanks,
Cem Karan