app runs fine with interpreter, but not under py2exe

[Doug Morse]

Peter,

Genius!  You nailed it -- thanks!

py2exe is apparently getting confused by the fact that packages "Numeric" and
"numpy" both have files multiarray.pyd and umath.pyd.  It copies just one of
each -- from $PYTHONHOME/Lib/site-packages/numpy/core -- and puts both of them
into the top-level of the created "dist" directory.  Also, there are no such
files as multiarray.pyc and umath.pyc in my python installation, but py2exe
seems to create these (in the dist and dist/numpy/core directories) -- they
are small (e.g., 526 bytes) and I'm guessing that they are stubs that simply
point python to the matching .pyd that py2exe relocated.

So, by using the --skip-archive option to py2exe (i.e., python setup.py py2exe
--skip-archive) to create the dist and build directories, and then within the
dist directory hierarchy removing all the occurrances of multiarray.pyc and
umath.pyc, and then moving dist/multiarray.pyd and dist/umath.pyd to the
dist/numpy/core directory, and finally by copying multiarray.pyd and umath.pyd
from $PYTHONHOME/Lib/site-packages/Numeric to the dist directory, I am able to
get my application to run correctly and as a standalone executable.  HOORAH!!!

Just for completeness and perhaps a bit of additional clarity, here's a
limited directory listing of what the steps in the previous paragraph produce:

    morse> find dist | egrep "(multia|umath)" | xargs ls -l
    -rwxr-xr-x 1 morse None  26624 Nov 28  2006 dist/multiarray.pyd
    -rwxr-xr-x 1 morse None 348160 Nov  8 16:16 dist/numpy/core/multiarray.pyd
    -rwxr-xr-x 1 morse None 192512 Nov  8 16:16 dist/numpy/core/umath.pyd
    -rwxr-xr-x 1 morse None  54272 Nov 28  2006 dist/umath.pyd
    morse>

(This is still WinXP; I'm just using the Cygwin bash and fileutils here.
Also, note that there are NO multiarray.pyc or umath.pyc files.)

So, my next step will be to try to not use the --skip-archive option and then
make these same modifications regarding multiarray.pyd and umath.pyd to the
py2exe-generated library.zip file and see if I can get things running that way
as well (and in so doing reduce my "dist" directory by about 10mg).  I may
also try creating a dist/Numeric subdirectory and moving dist/multiarray.pyd
and dist/umath.pyd to this dist/Numeric subdirectory -- for the goal of more
accurately mirroring the actual file/directory structure found in
$PYTHONHOME/Lib/site-packages.

Again, thanks to everyone for their assistance, esp. Harald and Peter.

Is this something I should pursue getting the py2exe folks to fix / work with
them on fixing?  In investigating this issue, I noted that a lot of other
py2exe problems seem to revolve around confusions when duplicate files exist
in different modules.  I wouldn't thinking that getting py2exe to pay better
attention to the containing modules when building it's dependency tree would
be that difficult (or is it)?

Cheers,
Doug


On Fri, 14 Mar 2008 12:39:23 +0100, Peter Otten <__peter__ at web.de> wrote:
>  Doug Morse wrote:
> 
> > from multiarray import zeros
> > import string
> > 
> > typecodes = {'Character':'c', 'Integer':'1sil', 'UnsignedInteger':'bwu',
> > 'Float':'fd', 'Complex':'FD'}
> > 
> > def _get_precisions(typecodes):
> >     lst = []
> >     for t in typecodes:
> >         lst.append( (zeros( (1,), t ).itemsize()*8, t) )   <-- Line 18
> >     return lst
> > 
> > def _fill_table(typecodes, table={}):
> >     for key, value in typecodes.items():
> >         table[key] = _get_precisions(value)
> >     return table
> > 
> > _code_table = _fill_table(typecodes)
> >
> > I can't find any reason why line 18 is throwing a "data type not
> > understood" error.  It doesn't seem to have anything to do with dynamic
> > importing, but I can't be sure.  I would note that "zeros" is a built-in
> > function found in the "python dll" multiarray.pyd (in the Numeric module
> > directory).
> 
>  I don't know why, but your module seems to use numpy's multiarray instead of
>  Numeric's:
> 
> >>> from multiarray import zeros
> >>> zeros((1,), "1")
>  array([0], '1')
> >>> from numpy.core.multiarray import zeros
> >>> zeros((1,), "1")
>  Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
>  TypeError: data type not understood
> 
>  Peter