[Pythonmac-SIG] py2app apps codesigned with "--options runtime", i.e. any notorizable app, fails to launch with ctypes error (was Re: [Pyobjc-dev] ctypes MemoryError when attempting to launch notarized app)

[Glyph]

> On Oct 28, 2018, at 1:48 PM, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
> 
> 
> 
>> On 28 Oct 2018, at 19:47, Glyph <glyph at twistedmatrix.com> wrote:
>> 
>> 
>> 
>>> On Oct 28, 2018, at 11:20 AM, Glyph <glyph at twistedmatrix.com> wrote:
>>> 
>>> 
>>> 
>>>> On Oct 28, 2018, at 2:27 AM, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>>> 
>>>>> 
>>>>> Curiously, this is the same traceback that comes from https://forum.kodi.tv/showthread.php?tid=329171, which suggests it's something fundamental to strict shared-library sandboxing that ctypes trips over when trying to initialize itself.
>>>>> 
>>>>> Does anyone have experience with this, or ideas about what to do?
>>>> 
>>>> I’m afraid not. I currently get away with not signing apps at all, although properly supporting signing is on my way too long wish list for py2app.  
>>> 
>>> The ability to distribute unsigned apps is not-so-slowly going away; even the ability to distribute non-notarized apps has a very limited shelf-life at this point.  So this ought to be an alarming development for everyone - having Python apps effectively banned from macOS distribution is a big potential problem :-\.
>>> 
>>> The good news here is that aside from having to write a little for loop in shell (shown below) getting the app codesigned previously was easy, and my app *did* pass notarization, so nothing that py2app is doing is breaking things on apple's end.  It's just a matter of a ctypes bug.
>> 
>> On that note: more good news.  While I haven't round-tripped through notarization again yet, this is a bit less dire than it first appeared.  If I prevent the import of ctypes with an `import sys; sys.modules['ctypes'] = None`, and add a 'sed' script to my build process to prevent _setup_ctypes from running in __boot__, then the app launches again.
>> 
>> Apparently my app doesn't actually need ctypes.
> 
> Good to hear that. 
> 
>> 
>> The problem seems to be that Twisted includes a ctypes import; modulegraph sees this and thinks there is a hard dependency, and inserts the ctypes setup blob into __boot__.  However, this is a conditional import, and it's for Windows support anyway.
> 
> Hmm…. I wonder what’s the best way forward here. I could add on option to disable ctypes support, but that is a kludge.  A weak importing hook (something like the never withdrawn PEP 369) could execute this code only when actually needed, but I have no idea how hard it would be to implement this.
> 
> 
>> 
>> (There also seem to be problems with cffi-using libraries, but not other shared objects, so maybe this is a bug in libffi; however, these don't interfere with py2app itself starting up.)
> 
> Interesting…  I haven’t had complaints about PyObjC yet, and that also uses libffi.  
> 
> I wonder what the “hardened runtime” option actually does and enforces.   In 3.7 the line in ctypes/__init__.py that causes the exception is a call that creates a dummy C function, and likely triggers the first allocation for storing a libffi closure which could be something the hardened runtime doesn’t like (being writeable + executable memory). 

Interesting. Perhaps what I want is simply https://developer.apple.com/documentation/security/com_apple_security_cs_allow-unsigned-executable-memory then?  Any chance you know how to jam that into a `codesign` command line somehow? :-)

> P.S. I just noticed that the traceback in your initial message doesn’t include the actual exception, just the traceback. 

Oh; it’s “MemoryError”, no exception message.

> Ronald
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pythonmac-sig/attachments/20181028/4deae095/attachment.html>