[SciPy-dev] Issues "freezing" scipy using py2app

[Josh Marshall]

I have an application that is using PyObjC, matplotlib and scipy to  
perform some image classification and feature extraction. I am now  
attempting to use py2app to bundle it up into a single OS X .app.

This has led to a few questions regarding the architecture of scipy,  
since there are a few problems I have had to work around.

1) The first concerns __init.py__ and _import_utils.py. In  
get_info_modules() from _import_tools, the presence of info.py files  
is required. However, when distributing a condensed version, these  
files are not included while the compiled info.pyc is. I have made a  
few alterations to _import_tools.py, such that if the info.py-s don't  
exist, then the .pyc files are tried instead. What I have done is the  
following:

+      use_pyc = False
         if packages is None:
             info_files = glob(os.path.join 
(self.parent_path,'*','info.py'))
+           info_files = [f for f in info_files if os.path.exists(f)]
+            if not info_files:
+                use_pyc =True
+                info_files = glob(os.path.join 
(self.parent_path,'*','info.pyc'))

and then, when we try actually importing them using load_module:

         for info_file in info_files:
             package_name = os.path.basename(os.path.dirname(info_file))
             fullname = self.parent_name +'.'+ package_name

             if use_pyc:
                 filedescriptor = ('.pyc','rb',2)
             else:
                 filedescriptor = ('.py','U',1)

             try:
                 info_module = imp.load_module(fullname+'.info',
                                               open 
(info_file,filedescriptor[1]),
                                                 info_file,
                                               filedescriptor)

There must be a simpler way, such as using find_module, but I  
couldn't get this to work since it requires the parent package to be  
imported before the subpackages.

2) The second issue was a problem involving get_config_vars being  
called from ppimport.py. distutils then requires the presence of the  
Python config file, which doesn't exist in the bundle.

  This was easily fixed by adding

         if sys.platform[:5]=='linux':
             so_ext = '.so'
+        elif sys.platform=='darwin':
+            so_ext = '.so'

I'm not sure if this is correct, since there can also be .dylibs, but  
on my system at least all the Python C extensions are .so-s.

3) Finally, I can't distribute my .app as-is, after being generated  
by py2app. py2app operates by all needed modules from the system site- 
packages being collected into a "site-packages.zip" within the  
application bundle. At runtime, some magic with import causes  
everything to work, and this is where scipy falls down. When trying  
to import the subpackages using the _import_utils.py magic, it cannot  
find them within the site-packages.zip. I can fix this by  
distributing the .app with the site-packages.zip expanded into a  
directory. However, I was wondering if anyone who knows the import  
magic better knows if we can fix this.

For reference, I am using current SVN scipy_core and scipy, current  
CVS (0.85.1) matplotlib, over Python 2.4.1 on OS X 10.4.3. I have a  
pretty nifty MatplotlibView which inherits from NSImageView, using  
some code from the CocoaAgg backend. (It only relies on the Agg  
backend). Works well for embedding matplotlib into a standalone  
application, if anyone is interested in it.

Thanks for the great work everyone's putting into scipy and  
matplotlib. It's completely replaced Matlab for me, and I have shown  
it can be used both for the prototyping stage and the final application.

Cheers,
Josh