[SciPy-dev] Issues "freezing" scipy using py2app

[Pearu Peterson]

On Wed, 21 Dec 2005, Josh Marshall wrote:

> I have an application that is using PyObjC, matplotlib and scipy to
> perform some image classification and feature extraction. I am now
> attempting to use py2app to bundle it up into a single OS X .app.
>
> This has led to a few questions regarding the architecture of scipy,
> since there are a few problems I have had to work around.
>
> 1) The first concerns __init.py__ and _import_utils.py. In
> get_info_modules() from _import_tools, the presence of info.py files
> is required. However, when distributing a condensed version, these
> files are not included while the compiled info.pyc is. I have made a
> few alterations to _import_tools.py, such that if the info.py-s don't
> exist, then the .pyc files are tried instead. What I have done is the
> following:
>
> +      use_pyc = False
>         if packages is None:
>             info_files = glob(os.path.join
> (self.parent_path,'*','info.py'))
> +           info_files = [f for f in info_files if os.path.exists(f)]
> +            if not info_files:
> +                use_pyc =True
> +                info_files = glob(os.path.join
> (self.parent_path,'*','info.pyc'))
>
> and then, when we try actually importing them using load_module:
>
>         for info_file in info_files:
>             package_name = os.path.basename(os.path.dirname(info_file))
>             fullname = self.parent_name +'.'+ package_name
>
>             if use_pyc:
>                 filedescriptor = ('.pyc','rb',2)
>             else:
>                 filedescriptor = ('.py','U',1)
>
>             try:
>                 info_module = imp.load_module(fullname+'.info',
>                                               open
> (info_file,filedescriptor[1]),
>                                                 info_file,
>                                               filedescriptor)

Thanks for the patch. I have applied it to new pkgload feature. scipy does 
not use _import_tools.py anymore.

> There must be a simpler way, such as using find_module, but I
> couldn't get this to work since it requires the parent package to be
> imported before the subpackages.

info.py files should be regarded as information files about packages, they 
are python modules only for convinience. And using imp.load_module is 
necessary to prevent importing the whole package, at the import time we 
need only the information from the info.py file.

> 2) The second issue was a problem involving get_config_vars being
> called from ppimport.py. distutils then requires the presence of the
> Python config file, which doesn't exist in the bundle.
>
>  This was easily fixed by adding
>
>         if sys.platform[:5]=='linux':
>             so_ext = '.so'
> +        elif sys.platform=='darwin':
> +            so_ext = '.so'

Hmm, what exception do you get when Python config file is not present?
ppimport._get_so_ext uses so_ext='.so' when ImportError appears.
I think the proper fix would be to catch proper exception.

> 3) Finally, I can't distribute my .app as-is, after being generated
> by py2app. py2app operates by all needed modules from the system site-
> packages being collected into a "site-packages.zip" within the
> application bundle. At runtime, some magic with import causes
> everything to work, and this is where scipy falls down. When trying
> to import the subpackages using the _import_utils.py magic, it cannot
> find them within the site-packages.zip. I can fix this by
> distributing the .app with the site-packages.zip expanded into a
> directory. However, I was wondering if anyone who knows the import
> magic better knows if we can fix this.

I'd like to learn more about the magic that Python does with .zip files.
Are there similar issues when using eggs?

Pearu