Py2Exe, lessons from use...

[Mike C. Fletcher]

Disclaimer:
	The following is written very late at night, assume there are errors and
omissions. There is no guarantee, implicit or explicit that the methods
described below are, in any way, reliable, useful, or intelligible.  Use at
your own risk.

I have recently been experimenting with Py2EXE for packaging up a rather
large application including:


	wxPython
		large numbers of custom controls
		a few of the wxPython.lib controls
	PIL
	mxTextTools (SimpleParse)
	mcf.vrml
	mcf.utils
	Numeric python
	win32com (gen_py wrapping of an Active X control in wxPython)
	my application
		multiple plug-in types, with a plug-in registry scanning directories
(python packages within the application package, python modules within
plug-in packages, and disk files within the application package including
dynamically defined plug-ins).  The plug-ins are the primary users of most
of the above packages, the application framework itself really only uses
wxPython and standard Python modules.  The plug-in framework uses
__import__, so no automatic tracking can occur.
		large numbers of non-python resource files (vrml files and bitmaps
primarily, a few text and HTML files as well)


Now that it's over, figured I would let others know the tricks I've used in
hopes that it will save some time somewhere...


	The application:
		By far the most annoying thing about resources is trying to generate any
decent specification for their location.  Most of my plug-ins use the
__file__ attribute of the package __init__ module to determine the directory
to use for loading resources.  If these modules are packed into the exe's
archive, you need to is another mechanism.  For my purposes, I don't mind if
the main application is outside of the archive (and in fact, I came to
prefer this), so I simply excluded the entire application's package from the
archive.
		With that said, I did include a secondary package (the "large numbers of
custom controls") which I did not want to have distributed as a python
package on disk.  For this package's resources, I used relative filenames
from the application package's __file__ for specifying directories.  Note:
I've also used a mechanism in the past where sys.path[0] is used if the
__file__ attribute is not usable.  Works either way.


	Python Imaging Library

The python imaging library requires a number of plug-ins to be of any use.
Here's a simplified main script which seems to work for a simple PIL-based
application...

import thumbnail
import ArgImagePlugin, BmpImagePlugin, CurImagePlugin, DcxImagePlugin,
EpsImagePlugin,\
	FliImagePlugin, FpxImagePlugin, GbrImagePlugin, GifImagePlugin,
IcoImagePlugin,\
	ImImagePlugin, ImtImagePlugin, IptcImagePlugin, JpegImagePlugin,
McIdasImagePlugin, \
	MicImagePlugin, MpegImagePlugin, MspImagePlugin, PcdImagePlugin,
PcxImagePlugin, \
	PdfImagePlugin, PixarImagePlugin, PngImagePlugin, PpmImagePlugin,
PsdImagePlugin, \
	SgiImagePlugin, SunImagePlugin, TgaImagePlugin, TiffImagePlugin,
WmfImagePlugin, \
	XbmImagePlugin, XpmImagePlugin, XVThumbImagePlugin
thumbnail.test()

Where thumbnail is a simple script importing Image.  As you can see,
specifying all of those plug-ins on the command line would be somewhat
annoying.  I have not been able to figure out how to stop Py2EXE from
finding the TK and TCL directories using the above imports.  I just excluded
the tcl directory in my .tar command for packaging the application.  There's
probably a better approach.


	mcf.vrml, SimpleParse, mcf.utils
	Because these packages were only used by plug-ins and/or the excluded
application package, it was necessary to specify the modules directly for
import.  Because of the already large number of imports, I did this directly
in my main module, rather than using a command line:

	from simpleparse import generator
	from mcf.vrml import loader, prototype2, basenodes
	from mcf.vrml.walker import customwalker, vrmlwalker, attrref
	from mcf.walker import engine
	from mcf.utils import pathtools, collapse


	Numeric, mxTextTools
	Imported by mcf.vrml/simpleparse, worked without any effort :) .


	wxPython
	Again, wxPython was only used by the application (which was excluded from
scanning), so it was necessary to include those modules actually used by the
application and its plug-ins, again, I chose to import in the main module,
due to the number of imports required:

	from wxPython.wx import *
	from wxPython.lib import filebrowsebutton
	from wxPython.lib import activexwrapper
	from wxcontrols import colorcontrol, lazyvrmltree, magicimagelist,
mfeditor, numericcontrol

Note: activexwrapper actually includes the win 32 com subsystems as well as
the wxPython subsystems.


	win32com
	Basically, using the latest CVS version of py2exe, I specified a command
line argument -i win32com.gen_py.* which attempts to include all of the
modules in the gen_py directory.  During operation of the program, a new
gen_py directory will be created and the generated modules will be saved
there.  I spent some time trying to figure out what was going on when I got
errors reporting that an integer could not be properly converted when
loading from the gen_py directory.  Eventually I modified my distribution
archive to prevent the gen_py directory from being included, this seemed to
solve the problem, though I never did track down the reason it occurred.
Bottom line, don't include the gen_py directory in your distributions.  With
the activexwrapper included, this seemed to allow for proper operation.


General comments:
	The "command line" focus for the system is difficult to work with when
creating large applications.  I wound up creating batch files to automate
the publishing process, and including all of the dependencies into my main
script, which slows down startup time somewhat.
	I would really prefer to specify the files for scanning within the setup
module, leaving the command line for "end-user" options.  Using setup.cfg as
mentioned in the documentation didn't seem to work, I didn't have time to
track down why not.
	There needs to be a mechanism for differentiating critical and trivial
output while running distutils commands. A mode that printed (only) those
files actually copied into the archive and those copied to the destination
directory would be useful.  Similarly, one which suppresses the "X not
copied to directory y, source yada, yada" warnings so that build failures
and files actually copied can be seen would be useful.


Py2EXE is very useful (as was the McMillan installer which I was using for
previous projects), allowing executable python applications makes using
python in the workplace possible (I tried getting a colleague to set up his
machine for this application as a set of python modules, he ran into some
weird win 32 COM error, we never did get it fixed).  Thanks to all involved.

Enjoy yourselves,
Mike