[Patches] [ python-Patches-1174614 ] site enhancements
SourceForge.net
noreply at sourceforge.net
Fri Jun 3 02:52:28 CEST 2005
Patches item #1174614, was opened at 2005-04-01 00:24
Message generated for change (Comment added) made by etrepum
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1174614&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Library (Lib)
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Submitted By: Bob Ippolito (etrepum)
Assigned to: Nobody/Anonymous (nobody)
Summary: site enhancements
Initial Comment:
The current site.py has three major deficiencies:
(1) All site dirs must exist on the filesystem: Since PEP 302 (New
Import Hooks) was adopted, this is not necessarily true.
sys.meta_path and sys.path_hooks can have valid uses for non-
existent paths. Even the standard zipimport hook supports in-zip-
file paths (i.e. foo.zip/bar).
(2) The directories added to sys.path by .pth files are not scanned
for further .pth files. If they were, you could make life much easier
on developers and users of multi-user systems. For example, it
would be possible for an administrator to drop in a .pth file into the
system-wide site-packages to allow users to have their own local
site-packages folder. Currently, you could try this, but it wouldn't
work because many packages such as PIL, Numeric, and PyObjC
take advantage of .pth files during their installation.
(3) To support the above use case, .pth files should be allowed to
use os.path.expanduser(), so you can toss a tilde in front and do the
right thing. Currently, the only way to support (2) is to use an ugly
"import" pth hook.
Attached is a patch to CVS HEAD that:
(1) Removes the os.path.exists() / os.path.isdir() restrictions
(2) Changes the .pth reader to use addsitedir() (top down) rather than
sys.path.append()
(3) makepath() uses os.path.expanduser() after the os.path.join().
----------------------------------------------------------------------
>Comment By: Bob Ippolito (etrepum)
Date: 2005-06-02 20:52
Message:
Logged In: YES
user_id=139309
And if the listdir performance is the only killer, the "recursive site"
protocol could be changed slightly -- it could depend on the existence of
some file (e.g. "python-site"), so the os.listdir() turns into a false
os.path.exists() in the general case (where it is not intended to be added
as another site directory).
Alternatively, some token in the .pth file could say "this points to
another site dir". Currently that (undocumented) token actually exists,
but it looks like this:
import site; site.addsitedir("....")
----------------------------------------------------------------------
Comment By: Guido van Rossum (gvanrossum)
Date: 2005-06-02 20:49
Message:
Logged In: YES
user_id=6380
Do such shared web servers really exist? It sounds like an
invitation to being hacked to me. Given that this is a last
resort (it would make more sense to petition the ISP to
install Numeric) I don't see that patching sys.path at the
top of the cgi script is so bad.
----------------------------------------------------------------------
Comment By: Bob Ippolito (etrepum)
Date: 2005-06-02 20:44
Message:
Logged In: YES
user_id=139309
Did you read my whole comment? I did come up with an example that is
not Mac OS X based: CGIs on a web server.
----------------------------------------------------------------------
Comment By: Guido van Rossum (gvanrossum)
Date: 2005-06-02 20:43
Message:
Logged In: YES
user_id=6380
Sorry, I'm still -1, but I don't have the time to discuss it
further. All your examples seem to be Mac OS X based;
perhaps you can come up with a Mac OS X specific solution?
----------------------------------------------------------------------
Comment By: Bob Ippolito (etrepum)
Date: 2005-06-02 20:41
Message:
Logged In: YES
user_id=139309
Well in practice, .pth files aren't used a lot (most people will see maybe
wxPython, PIL, and Numeric), so those three os.listdir() probably aren't
going to be significant. If users are creating pth files themselves (which
they probably should be doing, during development), then they'd likely
take advantage of the new features.
As far as power for site admins goes.. why should they have power to
install Python at all? Lots of things an admin could do can cause
surprises.
PYTHONPATH is only useful when dealing with the command prompt
directly, which is probably not the case for users of Mac OS X and
Windows. It shouldn't have to be, anyway. Mac OS X doesn't even
ship with a GUI interface to edit login environment variables, and the
normal rc scripts aren't run in the context of LaunchServices. Also,
consider the situation where a user of some shared web server is writing
CGI scripts that depend on Numeric installed in their home directory.
They'd need to know to add the following to the top of *EVERY* cgi
script:
import site
import os
site.addsitedir(os.path.expanduser("~/lib/python" + sys.version[:3] + "/
site-packages"))
If the admin had configured a directory to put things by installing a .pth
file that pointed there, they simply would have to tell users what flags to
pass to distutils.
For Mac OS X, there is a workaround, in that ~/Library/Python/2.X/site-
packages is a second site dir, but this doesn't need to be a workaround
and should be available elsewhere.
----------------------------------------------------------------------
Comment By: Guido van Rossum (gvanrossum)
Date: 2005-06-02 20:28
Message:
Logged In: YES
user_id=6380
It still looks like a potentially considerable slowdown
(since every directory added by a .pth file is scanned
recursively), and I'm doubting the usefulness.
Why should a site admin have the power to make Python search
in an arbitrary place in my home directory? That could cause
surprises too!
When you are regularly switching between multiple Python
interpreters, wouldn't it make more sense to have a few
aliases (or shell scripts, .bat files, etc.) that set
PYTHONPATH without exporting it? Methinks that many people
have dealt with this issue *without* feeling the need to
patch site.py.
(BTW I'm okay with the other changes, from your description.)
----------------------------------------------------------------------
Comment By: Bob Ippolito (etrepum)
Date: 2005-06-02 20:21
Message:
Logged In: YES
user_id=139309
If you read closer, it only scans directories added by .pth files, not all
directories on sys.path. This is to facilitate better encapsulation, so an
admin user can create a .pth file that says "~/lib/python2.4/site-
packages" and any user that starts Python will have their own site-
packages dir, where .pth files work, without resorting to PYTHONPATH.
This way you can use distutils to install packages into your homedir,
and they will actually work as expected.
PYTHONPATH is utterly disastrous if you regularly deal with several
different Python interpreters.
----------------------------------------------------------------------
Comment By: Guido van Rossum (gvanrossum)
Date: 2005-06-02 20:14
Message:
Logged In: YES
user_id=6380
Hm. Looking for .pth files only in site-packages was
intentional; it will slow down startup considerably if you
have to do an os.listdir() of every directory on sys.path
(even worse if some are on NFS!). I'd recommend against that
part.
----------------------------------------------------------------------
Comment By: Just van Rossum (jvr)
Date: 2005-04-07 04:33
Message:
Logged In: YES
user_id=92689
Patch looks good, the new functionality is great to have, recommend apply.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1174614&group_id=5470
More information about the Patches
mailing list