Add .par functionality (like Java .jar)

[Paul Moore]

Johann Höchtl <big.john at bigfoot.com> writes:

> I think you mean package the main .py- file with all the other modules
> which are now in a zip file together and let the main.py file care
> about it's imports.
> 
> IMHO this is bad design as the startup.py has to care whether is is
> loading it's modules from the directory (and relying on standard
> python mechanism) or from a python-zip package.

No it doesn't. Just add the zip file to sys.path. That's easy, and can
be done in the driver code.

> The consequence is that already python(.exe) has to have the knowledge
> to inspect the archive and see if there is a startup-script in it.

As I said, to make this "built in" python.exe would need to know about
the concept of "executable archives", just like java.exe knows about
executable jarfiles. But it's trivial to write a module (which would
have to be installed for this to work, obviously...) which
encapsulates the process, and which can be run via the -c argument to
python.exe.

Here's a proof of concept implementation:

--- put this in par.__init__.py somewhere on sys.path...

"""Support Python Archive (par) files

This module provides basic support for executable "archives" of Python
code and associated data.

Python archive file format:
    Python archive files are basically just zip files. The only "special"
    aspect is that the file must have a "main.py" file which is the entry
    point to the archive.

Usage:
    python -c "import par; par.run()" parfile arg arg arg...
"""

import zipfile as _zipfile
import sys as _sys
import os as _os

def run(parfile = None):
    """Run a par file. Use sys.argv to get a filename if none is given."""

    # Get the name of the archive, and shift sys.argv one place
    # to remove the "-c" entry. Special case for run being passed
    # no filename. If a filename is passed, don't change sys.argv.
    if parfile is None:
        parfile = _sys.argv[1]
        del _sys.argv[0]

    # Register the parfile import hook (all the work is done on import)
    import par.importer

    # Install the parfile at the start of sys.path
    _sys.path.insert(0, parfile)

    # Get the entry point as metadata, and the content of the entry point
    # Then run the entry point in the top level namespace
    main = _zipfile.ZipFile(parfile).read("main.py")
    main = main.replace("\r", "")
    runcode(main, _os.path.join(parfile, "main.py"))

def runcode(code, filename):
    """Compile and run code, as if it came from filename, in the toplevel namespace"""
    toplevel = _sys._getframe()
    while toplevel.f_back:
        toplevel = toplevel.f_back
    codeobj = compile(code, filename, "exec")
    exec codeobj in toplevel.f_globals, toplevel.f_locals

----------------------------------------------------

This code relies on a par.importer module, which is just an import
hook (using Gordon McMillan's iu.py import hook package) to allow
zipfiles as part of sys.path. When this is supported natively (Python
2.3) the par.importer module is no longer needed, and you have a
simple 35-line (or so) module.

> I am a python beginner and i don't yet know if it would be a problem
> to name this file __init__.py like packages in a directory.

That sounds reasonable. As you can see, the name is hard-coded in the
package above. You could do all sorts of things, such as having a
MANIFEST file containing the name of the startup file, just like Java
does. But that's trivial details...

> That's already near at the goal, but it's not self-contained. You have
> to put this line into a unix semi-executable +x or create a shortcut
> in a GUI.

But you have to do that with JAR files, too!!! OS-level magic to make
particular filetypes executable, and the command line to run to
execute them, is highly system-dependent (filetype associations on
Windows, binfmt_misc magic on Linux, whatever else...)

If you care that much, write a custom executable and concatenate it
onto the front of the zipfile. The zip format copes with this, and
many executable formats (certainly Windows, and I believe Linux too)
don't mind. But that involves C coding, which is too hard for an
evening's quick hack :-)

> Thank's a lot for your explanations. I am still a Python
> beginner. After a year of language (re-)search I think I finally ended
> up with Python and I'm happy that I didn't get stuck with Java ....

No problem. Like many things with Python, the reason that there is no
"standard" way of doing this sort of thing is that it's pretty trivial
to put something together for your own use, but almost impossible to
persuade the rest of the community that your 10-line hack is so much
better than theirs as to be acceptable as a "standard" :-)

The PEP process should be good for this sort of thing, but it does
need a fairly committed champion to push anything through. Heck, maybe
I'll try pushing this one, just to see if I can work out how to make
the process smoother... :-)

Paul.