py2exe and library.zip

[David Bolen]

Timothy Smith <timothy at open-networks.net> writes:

> I've got this working now, and fyi it downloads the entire zip every
> time. and svn appears to be very slow at it to.

Hmm, not what I would have expected, and certainly unfortunate for
your desired use case.

I just tried some experiments with rsync (easier to test locally than
Subversion), and found that taking an existing zip, unpacking it and
then repacking it with some rearrangement was in fact sending
everything, even though the source files were unchanged.  Since py2exe
is effectively rebuilding that library.zip on each run, that probably
is a fair representation of the generation process.

I'm not familiar enough with zip file compression, but perhaps it
includes the use of something that is file specific to seed the
compression engine, which would mean that making a new zip file even
with the same files in it might not yield precisely the same internal
compressed storage.  Both versions would be proper and decompressible,
just not binary identical even for unchanged sources.

If I disabled compression for the zip files (just did a store only),
and rebuilt the zip even with a rearranged file order, rsync was
able to detect just the changes.

So you might want to try ensuring that your py2exe generated file is
not compressing the individual modules (a verbose zip listing of the
library.zip should just show that they were "Stored").  Your
library.zip will get larger, but it should become more efficient to
transfer - hopefully as well with Subversion as I was seeing with
rsync.

(In fact, I remember doing just something like this with a project of mine
that I was using py2exe with, and then using rsync to push out the resultant
files to remote sites - I had originally compressed the library.zip but
rsync was pushing the whole thing out, so I stopped using the compression)

-- David