[Python-ideas] Draft PEP for virtualenv in the stdlib

[Nick Coghlan]

On Wed, Oct 26, 2011 at 3:37 AM, Carl Meyer <carl at oddbird.net> wrote:
> Also, in earlier discussions on distutils-sig some people considered it
> a _feature_ to have the virtual environment's Python binary copied,
> making the virtual environment more isolated from system changes.
> Obviously, this is an area where programmers and sysadmins often don't
> see eye to eye ;)

I've been persauded that copying by default is reasonable - so long as
the option to use symlinks is there, sysadmins can put a policy for
their organisation in place that requires its use.

>> 'site' is *way* too overloaded already, let's not make it worse. I
>> suggest "sys.venv_prefix".
>
> My original thinking here was that sys.site_prefix is an attribute that
> should always exist, and always point to "where stuff should be
> installed to site-packages", whether or not you are in a venv (if you
> are not, it would have the same value as sys.prefix). It's a little odd
> to use an attribute named "sys.venv_prefix" in that way, even if your
> code doesn't know or care whether its actually in a venv (and in general
> we should be encouraging code that doesn't know or care). (The attribute
> doesn't currently always-exist in the reference implementation, but I'd
> like to change that).
>
> I agree that "site" is overloaded, though. Any ideas for a name that
> doesn't further overload that term, but still communicates "this
> attribute is a standard part of Python that always has the same meaning
> whether or not you are currently in a venv"?

I'd actually prefer that we use the explicit "sys.prefix" and
"sys.venv_prefix" naming (with the latter set to None when not in a
virtual env) and possibly offer a convenience API somewhere that hides
the "sys.prefix if sys.venv_prefix is None else sys.venv_prefix"
dance.

There's already a site.getsitepackage() API that gets the full list of
site package directories (it's not a given that there's only one).

>> I'm actually not seeing the rationale for the obfuscated FHS inspired
>> layout in the first place - why not dump the binaries adjacent to the
>> config file, with a simple "site-packages" directory immediately below
>> that? If there are reasons for a more complex default layout, they
>> need to be articulated in the PEP.
>
> The historical reason is that it emulates the layout found under
> sys.prefix in a regular Python installation (note that it's not actually
> Linux-specific, in virtualenv it matches the appropriate platform; i.e.
> on Windows it's "Lib\" rather than "lib\pythonX.X"). This was necessary
> for virtualenv because it couldn't make changes in distutils/sysconfig).
> I think there may be good reason to continue to follow this approach,
> simply because it makes the necessary changes to distutils/sysconfig
> less invasive, reducing the need for special-casing of the venv case.
> But I do need to look into this a bit more and update the PEP with
> further rationale in any case.
>
> Regardless, I would not be in favor of dumping binaries directly next to
> pyvenv.cfg. It feels cleaner to keep scripts and binaries in a directory
> specifically named and intended for that purpose, which can be added to
> the shell PATH.
>
> I also think there is some value, all else being roughly equal, in
> maintaining consistency with virtualenv's layout. This is not an
> overriding concern, but it will make a big difference in how much
> existing code that deals with virtual environments has to change.
>
>> If the problem is wanting to allow cross platform computation of
>> things like the site-packages directory location and other paths, then
>> the answer to that seems to lie in better helper methods (whether in
>> sysconfig, site, venv or elsewhere) rather than Linux specific layouts
>> inside language level virtual environments.

As Barry said, the rationale sounds reasonable, but the PEP needs to
explain it in terms those of us not fully versed in the details of
cross-platform virtual environments can understand (examples of both
*nix and Windows layouts with a simple example package installed would
probably be useful on that front)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia