[Pythonmac-SIG] Building an extensible bundle with py2app

[Ronald Oussoren]

On 15 Nov, 2012, at 21:00, Emanuele Santos <emanuelesantos at gmail.com> wrote:

> Hi, Ronald
> 
> Thanks for your suggestion. Your solution works great when I install the additional packages from outside my application. But as we want to make our users' life easier, we would like to install the additional packages into the user's provided folder from our bundle. 
> 
> As I was trying to implement this, I came across another problem that maybe you know a solution for it. If I try to run the system's easy_install it doesn't work. The error message was not very helpful but I think it's because I can't create a child process running a different interpreter. I am using subprocess.Popen() to start the process:
> {{{
> Traceback (most recent call last):
>   File "/usr/bin/easy_install-2.7", line 7, in <module>
>     from pkg_resources import load_entry_point
>   File "pkg_resources.pyc", line 698, in <module>
>   File "pkg_resources.pyc", line 701, in Environment
>   File "pkg_resources.pyc", line 99, in get_supported_platform
>   File "pkg_resources.pyc", line 209, in _macosx_vers
>   File "platform.pyc", line 804, in mac_ver
>   File "platform.pyc", line 781, in _mac_ver_xml
>   File "plistlib.pyc", line 78, in readPlist
>   File "plistlib.pyc", line 401, in parse
>   File "xml/parsers/expat.pyc", line 4, in <module>
> ImportError: dlopen(/Applications/MyApp.app/Contents/Resources/lib/python2.7/lib-dynload/pyexpat.so, 2): Symbol not found: __Py_HashSecret
>   Referenced from: /Applications/MyApp.app/Contents/Resources/lib/python2.7/lib-dynload/pyexpat.so
>   Expected in: flat namespace
>  in /Applications/MyApp.app/Contents/Resources/lib/python2.7/lib-dynload/pyexpat.so 
> }}}

PyHash_Secret is new in the latest 2.7 release (it was introduced by a fix for a security vulnerability). You problem use python 2.7.3 to build the plugin/extension bundle, but the hosting proces uses an older version of python. 

> 
> If I try to include setuptools in the bundle so I can run easy_install with the same interpreter, I get errors with missing variables in site.py. But after adding those variables just to see how far I would go, I get this error:
> 
> {{{
> Traceback (most recent call last):
>   File "/Applications/MyApp.app/Contents/Resources/lib/python2.7/gui/bundles/easy_install.py", line 10, in <module>
>     from setuptools.command.easy_install import main
>   File "/Applications/MyApp.app/Contents/Resources/lib/python2.7/setuptools/command/easy_install.py", line 21, in <module>
>     from setuptools.package_index import PackageIndex, parse_bdist_wininst
>   File "/Applications/VisTrails/VisTrails.app/Contents/Resources/lib/python2.7/setuptools/package_index.py", line 145, in <module>
>     urllib2.__version__, require('setuptools')[0].version
>   File "pkg_resources.pyc", line 666, in require
>   File "pkg_resources.pyc", line 565, in resolve
> pkg_resources.DistributionNotFound: setuptools
> }}}
> 
> Any ideas how I solve this?

That's issue #1 in the py2app tracker (<https://bitbucket.org/ronaldoussoren/py2app/issue/1/py2app-doesnt-support-egg-metadata>): py2app doesn't copy the setuptools metadata into the created bundles. This amongst others means that pkg_resources cannot find the information it needs to determine if a package is installed.

I haven't had time to add this feature to py2app yet, doing it cleanly is slightly more involved that I like. In particular, py2app needs to determine for every python module which distutils package it belongs to and that given multiple ways in which packages can be installed (setuptools zipped eggs, setuptools unzipped eggs, pip (setuptools --single-version-externally-managed with record file in the install tree), setuptools develop links, ...). For some installation methods (in particular --single-version-externally-managed without a record file) it might not be not possible to reliable detect which package a module belongs to.

Ronald

> 
> Thanks,
> 
> -- Emanuele.
> 
> 
> On Wed, Oct 17, 2012 at 11:45 AM, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
> 
> On 17 Oct, 2012, at 16:12, Emanuele Santos <emanuelesantos at gmail.com> wrote:
> 
> > Hi, all
> >
> > We build a bundle with py2app and we would like to make it extensible so users could install on demand other python packages into the python framework included in the bundle, for example by running easy_install.
> >
> > Is there a recommended way of doing that? I noticed that the site.py file included in the bundle is not complete so when I tried using the --user command line option or setting the $PYTHONHOME, I got an error.
> 
> I wouldn't change the application bundle itself, but would add an option to load python code from a user specified location. There are two reasons for not changing the application bundle itself: that would invalidate code signatures, and users cannot necessarily write into the bundle (I tend to install applications using a separate account with more privileges than my normal account).
> 
> One way to do this is adding something like this to your application's startup code:
> 
>     import os
>     import site
> 
>     site.addsitedir(os.path.expanduser("~/Library/Application Support/MyApp/Python"))
> 
> Users can then install additional modules in "~/Library/Application Support/MyApp/Python" and your application can then use them.
> 
> If you're using a semi-standalone build (for example because you use Apple's python build) you can also use the "--site-packages" option to add the default site-packages directory to your application's search path. However, when I look at the code that not only just works for semi-standalone builds, but I'm also also not sure if the code that adds ~/Library/Python/... to sys.path actually works.
> 
> Ronald
> >
> > Thanks,
> >
> > -- Emanuele.
> >
> > _______________________________________________
> > Pythonmac-SIG maillist  -  Pythonmac-SIG at python.org
> > http://mail.python.org/mailman/listinfo/pythonmac-sig
> > unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pythonmac-sig/attachments/20121116/8e73a538/attachment-0001.html>