[Distutils] Anyone stuck with easy_install / has .pth file issues -- this is for you!

[Garrett Cooper]

On Mon, May 4, 2009 at 1:11 PM, Garrett Cooper <yanegomi at gmail.com> wrote:
> Hi P.J.!
>
> On Mon, May 4, 2009 at 8:20 AM, P.J. Eby <pje at telecommunity.com> wrote:
>> At 02:43 AM 5/4/2009 -0700, Garrett Cooper wrote:
>>>
>>> Hi guys,
>>>    Just thought I'd might provide this script to fellow developers
>>> which fixes .pth files (easy-install.pth / .egg was the prime target
>>> -- see the comments for more details):
>>> <http://yaneurabeya.livejournal.com/3929.html>.
>>>    Comments are more than welcome.
>>
>> As far as I can tell, it doesn't do anything that "easy_install -mxN"
>> doesn't, although it appears to also convert paths of this form:
>>
>>   /foo/bar/baz/foo/bar/spam
>>
>> into:
>>
>>   ./baz./spam
>>
>> if I'm reading the code correctly.  It also seems to have no protection
>> against adding multiple versions of the same project to a .pth file
>
> You're right -- it doesn't protect against the following (><):
>
> /full/path/to/package.egg
> ./package.egg
>
> It does protect against ./package.egg in .pth not existing in the
> filesystem, and vice versa, depending on the glob of choice employed.
> I'll fix that gap tonight :).
>
> BTW, wasn't -m deprecated? I mean, -m's `delivered' functionality
> doesn't even work because python stores all loaded package / module
> entries in .pth files as keys in a dictionary _anyhow_ (hence the
> proposal I made a few weeks ago with __import__ to do version
> checking)..
>
> setuptools can't even fix this behavior (python does a BFS-like
> package search based on the module / package name and returns the
> first entry found), even though it's _sort_ of supposed to (look for
> "A more complete solution" on
> <http://mail.python.org/pipermail/distutils-sig/2006-March/006123.html>).
> Feel free to check sys.modules if you don't believe me :). Hence the
> need for virtualenv.
>
> I have no idea how -xN factor into this though -- maybe I'm missing an
> important easy-install usage note... either way, we're stuck with
> 0.6c3 at work because the folks we inherited it from hacked the code
> (in an ugly way... ew) and I don't have time to fix it now, so does
> this still apply ;\...?
>
>> and to ignore development eggs, whether or not their directories still exist.
>
> By default, no. If one wants to avoid a series of directories, they
> can omit entries with [a] relevant expression(s) via the -g / -r
> option(s).
>
>> In contrast, easy_install already removes non-existent files/directories
>> whenever it touches easy_install.pth, and if you gave it a command line
>> globbing the same files as this tool (i.e., just "easy_install [list of
>> eggs]"), you'd at least end up without any duplicates in the .pth file.
>>
>> In short, AFAICT, you could replace the entire tool with a short note on how
>> to accomplish the same things using easy_install, or by simply having it
>> invoke easy_install internally.
>
> - The duplicates erasure was just gravy, and purely an exercise on my
> part getting used to set's in python. The real point was the missing
> package in .pth functionality, caused by easy_install / setuptools
> lacking proper MP-safe logic.

I do need to check for commented entries though / expand the search
criteria or something... excellent point made P.J.

> - It's a simple tool (took a while to think up, but took 1.5 hours to
> code / validate) designed to function outside of easy_install /
> setuptools, even though that was the piece I was trying to target...
> so if someone hand edits a .pth file (heaven forbid) it would note
> that the file needed to be fixed / fix the file =].
>
> Thanks for the comments!
> -Garrett