quick n' dirty measurement of compression and byte-compilation

[Zooko]

[Please Cc: zooko at zooko.com in replies.  Thank you!]

 Michael Hudson <mwh at python.net> wrote:
>
> Zooko <zooko at zooko.com> writes:
> 
> > I noticed that the PyXML build script byte-compiles its .py files.
> > It seems like this potentially introduces incompatibility if the
> > version of Python used to build differs from the version used to run
> > the resulting package.
> 
> Depends where you install the files.  By default don't they end up in
> 
>     $(prefix)/lib/python$(VERSION)/site-packages

Mojo Nation currently gets distributed [1] either as a self-installing Windows 
package or as an "untar me and run in place" Unix tarball.  Even if we *did* 
install into `$(prefix)/lib/python$(VERSION)/site-packages', that wouldn't solve 
the problem of delivering only byte-compiled .pyo's for a version of Python that 
the user doesn't have.

I didn't include a measurement of including both .py's and .pyo's in my earlier 
post.  Here it is, appended.

The difference in package size can be substantial.  For Mojo Nation, if we were 
to transmit both .pyo's and .py's, using standard `gzip' compression, the 
resulting package would be 1,548,451 bytes.  If we were to transmit only .py's, 
using `bzip2 -9', the resulting package would be 601,511 bytes.

I'd like to repeat my suggestion to developers: transmit straight .py files and 
do byte-compilation on the end-user's computer, either in the installation 
script or simply at runtime.  (Also: use bzip2.)

Regards,

Zooko

http://zooko.com/
Security and Distributed Systems Engineering

[1] http://mojonation.net/download

Key:

py.  == not byte-compiled
pyc. == byte-compiled in normal mode
pyo. == byte-compiled in -OO mode
pyb. == both .py's *and* .pyo's

.tar     == uncompressed
.tar.gz3 == compressed with `gzip -3' (normal gzip compression)
.tar.gz9 == compressed with `gzip -9'
.tar.bz2 == compressed with `bzip2 -9'

files sorted by type:

 3512320 Jan  1 07:46 py.tar
 4003840 Jan  1 07:46 pyc.tar
 3317760 Jan  1 07:46 pyo.tar
 6819840 Jan  1 13:05 pyb.tar
  739409 Jan  1 07:43 py.tar.gz3
 1122935 Jan  1 07:43 pyc.tar.gz3
  808596 Jan  1 07:43 pyo.tar.gz3
 1548451 Jan  1 13:04 pyb.tar.gz3
  732414 Jan  1 07:43 py.tar.gz9
 1115887 Jan  1 07:43 pyc.tar.gz9
  799386 Jan  1 07:43 pyo.tar.gz9
 1531782 Jan  1 13:04 pyb.tar.gz9
  601511 Jan  1 07:44 py.tar.bz2
  846736 Jan  1 07:44 pyc.tar.bz2
  608945 Jan  1 07:44 pyo.tar.bz2
 1214033 Jan  1 13:05 pyb.tar.bz2

files sorted by size:

  601511 Jan  1 07:44 py.tar.bz2
  608945 Jan  1 07:44 pyo.tar.bz2
  732414 Jan  1 07:43 py.tar.gz
  739409 Jan  1 07:43 py.tgz
  799386 Jan  1 07:43 pyo.tar.gz
  808596 Jan  1 07:43 pyo.tgz
  846736 Jan  1 07:44 pyc.tar.bz2
 1115887 Jan  1 07:43 pyc.tar.gz
 1122935 Jan  1 07:43 pyc.tgz
 1214033 Jan  1 13:05 pyb.tar.bz2
 1531782 Jan  1 13:04 pyb.tar.gz9
 1548451 Jan  1 13:04 pyb.tar.gz3
 3317760 Jan  1 07:46 pyo.tar
 3512320 Jan  1 07:46 py.tar
 4003840 Jan  1 07:46 pyc.tar
 6819840 Jan  1 13:05 pyb.tar