ctypes, memory mapped files and context manager

[Peter Otten]

Hans-Peter Jansen wrote:

> On Mittwoch, 28. Dezember 2016 13:48:48 Peter Otten wrote:
>> Hans-Peter Jansen wrote:
>> > Dear Peter,
>> > 
>> > thanks for taking valuable time to look into my issue.
>> 
>> You're welcome!
>> 
>> > It might be related to my distinct silliness, but the problem persists
>> > with your code as well.
>> 
>> Unfortunately I posted the broken toy example rather than the fixed one.
>> Here's the latter. Basically you have to keep a reference in the context
>> manager (whether you implement it as a class or a generator doesn't
>> matter) without giving another reference away to client code:
>> 
>> $ cat mmap_after.py
>> import ctypes
>> import mmap
>> import weakref
>> 
>> from contextlib import contextmanager
>> 
>> class T(ctypes.Structure):
>>     _fields = [("foo", ctypes.c_uint32)]
>> 
>> 
>> @contextmanager
>> def map_struct(m, n):
>>     m.resize(n * mmap.PAGESIZE)
>>     keep_me = T.from_buffer(m)
>>     yield weakref.proxy(keep_me)
> 
> Hooray, that did the trick. Great solution, thank you very much!
> 
> If you don't mind, I will mention you and your solution at the various
> places, I placed this issue over the last weeks.
> 
> You made my day, Peter!
> 
> It leaves the question on why is Python2 acting as one would expect
> related to context managers, and Python3 needs this weakref juggling.
> Maybe something, that's worth to be placed in python-dev. What do you
> think?

Well, given

$ cat mmap_resize.py
import ctypes
import mmap

class T(ctypes.Structure):
    _fields_ = [("foo", ctypes.c_uint32)]

SIZE = 2 * mmap.PAGESIZE
f = open("tmp.dat", "w+b")
f.write(b"\0" * SIZE)
f.seek(0)

m = mmap.mmap(f.fileno(), SIZE)

a = T.from_buffer(m, mmap.PAGESIZE)
m.resize(mmap.PAGESIZE)
a.foo = 42
$ python3 mmap_resize.py
Traceback (most recent call last):
  File "mmap_resize.py", line 15, in <module>
    m.resize(mmap.PAGESIZE)
BufferError: mmap can't resize with extant buffers exported.
$ python2.7 mmap_resize.py
Segmentation fault

do you really prefer the Python 2 behaviour?