ctypes, memory mapped files and context manager

[Hans-Peter Jansen]

On Mittwoch, 28. Dezember 2016 15:17:22 Hans-Peter Jansen wrote:
> On Mittwoch, 28. Dezember 2016 13:48:48 Peter Otten wrote:
> > Hans-Peter Jansen wrote:
> > > Dear Peter,
> > > 
> > > thanks for taking valuable time to look into my issue.
> > 
> > You're welcome!
> > 
> > > It might be related to my distinct silliness, but the problem persists
> > > with your code as well.
> > 
> > Unfortunately I posted the broken toy example rather than the fixed one.
> > Here's the latter. Basically you have to keep a reference in the context
> > manager (whether you implement it as a class or a generator doesn't
> > matter)
> > without giving another reference away to client code:
> > 
> > $ cat mmap_after.py
> > import ctypes
> > import mmap
> > import weakref
> > 
> > from contextlib import contextmanager
> > 
> > class T(ctypes.Structure):
> >     _fields = [("foo", ctypes.c_uint32)]
> > 
> > @contextmanager
> > 
> > def map_struct(m, n):
> >     m.resize(n * mmap.PAGESIZE)
> >     keep_me = T.from_buffer(m)
> >     yield weakref.proxy(keep_me)
> 
> Hooray, that did the trick. Great solution, thank you very much!

Sorry for bothering you again, Peter, but after applying it to the real 
project, that fails consistently similar to:

# first run, check explicitly disabled
$> python3 ctypes_mmap_ctx.py 
DEBUG: starting offset: 0x10
DEBUG: add_data: header
DEBUG: add_data: 5168
DEBUG: add_data: header
DEBUG: add_data: 5168
DEBUG: add_data: header
DEBUG: add_data: 5168
DEBUG: add_data: header
DEBUG: add_data: 5168
DEBUG: add_data: header
DEBUG: add_data: 5168
DEBUG: add_data: header
DEBUG: add_data: 5168
DEBUG: add_data: header
DEBUG: add_data: 5168
DEBUG: add_data: header
DEBUG: add_data: 5168
DEBUG: final offset: 0xa1d0

$> python3 ctypes_mmap_ctx.py 
Traceback (most recent call last):
  File "ctypes_mmap_ctx.py", line 163, in <module>
    mf = MapFile(mapfile)
  File "ctypes_mmap_ctx.py", line 109, in __init__
    if bytes(blk) != bytes(rest):
AttributeError: 'c_ubyte_Array_3632' object has no attribute '__bytes__'


The new issue appears in some consistency checking code. The mapfile is 
checked for zeroed out overhang on that line. The weakref proxy isn't behaving 
well at that point. Any idea, what could be going wrong with it?


Updated the gist to demonstrate the issue after switching to weakref.proxy().
(has grown even more code in that process, sorry).

 
It looks like a minor issue, but the code quality is really degraded from 
these ugly del statements.

Pete