Battle of the garbage collectors, or ARGGHHHHHH!!!!

[CFK]

On Wed, Apr 26, 2017 at 10:38 PM, Cem Karan <cfkaran2 at gmail.com> wrote:

>
> On Apr 24, 2017, at 8:54 PM, Jon Ribbens <jon+usenet at unequivocal.eu>
> wrote:
>
> > On 2017-04-24, CFK <cfkaran2 at gmail.com> wrote:
> >> Long version: I'm trying to write bindings for python via ctypes to
> control
> >> a library written in C that uses the bdwgc garbage collector (
> >> http://www.hboehm.info/gc/).  The bindings mostly work, except for when
> >> either bdwgc or python's garbage collector decide to get into an
> argument
> >> over what is garbage and what isn't, in which case I get a segfault
> because
> >> one or the other collector has already reaped the memory.
> >
> > Make your Python C objects contain a pointer to a
> > GC_MALLOC_UNCOLLECTABLE block that contains a pointer to the
> > bwdgc object it's an interface to? And GC_FREE it in tp_dealloc?
> > Then bwdgc won't free any C memory that Python is referencing.
>
> OK, I realized today that there was a miscommunication somewhere.  My
> python code is all pure python, and the library is pure C, and it is not
> designed to be called by python (it's intended to be language neutral, so
> if someone wants to call it from a different language, they can).  That
> means that tp_dealloc (which is part of the python C API) is probably not
> going to work.
>
> I got interrupted (again) so I didn't have a chance to try the next trick
> and register the ctypes objects as roots from which to scan in bdwgc, but
> I'm hoping that roots aren't removed.  If that works, I'll post it to the
> list.
>
> Thanks,
> Cem Karan


Right, apparently I win at the 'late reply' game.  That said, I wanted to
give Jon Ribbens credit for his idea, because it was very close to what I
used in the end.  The only difference is that I also used weakref.finalize()
to tie a finalizer to the lifetime of the ctypes pointer that I was using.
The finalizer called GC_FREE() to free the uncollectable block, which
allowed the C allocator to cleanup memory.

The only thing I never figured out was how to get a C block to hold onto
python memory.  I didn't need it, but it felt like it would make for a nice
duality with the method above.

Thanks,
Cem Karan