[Python-Dev] .pth files are evil

[P.J. Eby]

At 04:42 PM 5/9/2009 +0200, Martin v. Löwis wrote:
> >> If you always use --single-version-externally-managed with easy_install,
> >> it will stop editing .pth files on installation.
> >
> > It's --multi-version (-m) that does that.
> > --single-version-externally-managed is a "setup.py install" option.
> >
> > Both have the effect of not editing .pth files, but they do so in
> > different ways.  The "setup.py install" option causes it to install in a
> > distutils-compatible layout, whereas --multi-version simply drops .egg
> > files or directories in the target location and leaves it to the user
> > (or the generated script wrappers) to add them to sys.path.
>
>Ah, ok. Is there also an easy_install invocation that unpacks the zip
>file into some location of sys.path (which then wouldn't require
>editing sys.path)?

No; you'd have to use the -e option to easy_install to download and 
extract a source version of the package; then run that package's 
setup.py, e.g.:

    easy_install -eb /some/tmpdir SomeProject
    cd /some/tmpdir/someproject  # subdir is always lowercased/normalized
    setup.py install --single-version-externally-managed --record=...

I suspect that this is basically what pip is doing under the hood, as 
that would explain why it doesn't support .egg files.

I previously posted code to the distutils-sig that was an .egg 
unpacker with appropriate renaming, though.  It was untested, and 
assumes you already checked for collisions in the target directory, 
and that you're handling any uninstall manifest yourself.  It could 
probably be modified to take a filter function, though, something like:

def flatten_egg(egg_filename, extract_dir, filter=lambda s,d: d):
      eggbase = os.path.filename(egg_filename)+'-info'
      def file_filter(src, dst):
          if src.startswith('EGG-INFO/'):
              src = eggbase+s[8:]
              dst = os.path.join(extract_dir, *src.split('/'))
          return filter(src, dst)
      return unpack_archive(egg_filename, extract_dir, file_filter)

Then you could pass in a None-returning filter function to check and 
accumulate collisions and generate a manifest.  A second run with the 
default filter would do the unpacking.

(This function should work with either .egg files or .egg directories 
as input, btw, since unpack_archive treats a directory input as if it 
were an archive.)

Anyway, if you used "easy_install -mxd /some/tmpdir [specs]" to get 
your target eggs found/built, you could then run this flattening 
function (with appropriate filter functions) over the *.egg contents 
of /some/tmpdir to do the actual installation.

(The reason for using -mxd instead of -Zmaxd or -zmaxd is that we 
don't care whether the eggs are zipped or not, and we leave out the 
-a so that dependencies already present on sys.path aren't copied or 
re-downloaded to the target; only dependencies we don't already have 
will get dropped in /some/tmpdir.)

Of course, the devil of this is in the details; to handle conflicts 
and uninstalls properly you would need to know what namespace 
packages were in the eggs you are installing.  But if you don't care 
about blindly overwriting things (as the distutils does not), then 
it's actually pretty easy to make such an unpacker.

I mainly haven't made one myself because I *do* care about things 
being blindly overwritten.