[Python-Dev] Python install layout and the PATH on win32 (Rationale part 2: Moving the python.exe)

[Mark Hammond]

I'm responding to both of Van's recent messages in one:

On 23/03/2012 1:47 AM, VanL wrote:
> [PART 2: Moving the python binary]
...
> A regular complaint of those new to Python on windows (and new to
> programming generally) has been that one of the first things that
> they need to do is to edit the PATH to allow Python to be run. In
> particular, this is the difficult sequence:
>
> 1. Install python. 2. Open up a shell and run "python" 3. Use pip or
> easy_install to install regetron (a package that installs an
> executable file). 4. Run regetron.
...
> One possible response here is that the moving of the python.exe
> binary and the setting of the PATH would be tied to an
> unchecked-by-default installer option, making an explicit user choice
> needed to invoke the new functionality.

Given an off-by-default setting, I fail to see how it fixes your
"difficult sequence" above.  What would the instructions above now say?
That the user should re-install Python ensuring to set that checkbox?
Cover both cases, including how the user can tell if it is on the PATH
and how to fix it otherwise?  Something else?

> Breakage of existing tools: Mark Hammond, Paul Moore, and Tim Golden
> have all expressed that they have existing tools that would break
> and would need to be adjusted to match the new location of the
> python.exe, because that location is assumed to be at the root of the
> python install.
>
> A related issue is that this portion of the proposal has met with
> some resistance, but not much support here on Python-dev. The reason
> for that is selection bias: Those who are on Python-dev are much more
> likely to have tools that do advanced things with Python, such as
> introspect on the location of the binary, and are also much more
> likely to be comfortable with things like editing the PATH on
> windows. In contrast, the people that have trouble with this issue
> are those that are newest to Python and programming generally - those
> for whom editing the PATH is a challenge and whom are likely to be
> confused by the distinction between python.exe and a python program -
> and why, even after they add python to the path, the python program
> is not directly executable.

Here you are referring to the PATH, but that isn't really where the
objections are.  I would claim a selection bias on Python-dev, where
subscribers are less likely to use Windows regularly for development and
therefore less likely to have developed or use tools for finding and
launching Python.  IMO, the lack of objections on Python-dev to renaming
the binary directory is the same reason you aren't seeing overwhelming 
*support* for the change either.  Without the perspective of being 
regular Windows users, people are happy to agree "consistent is better". 
  All other things being equal, I'd agree too.

Really, we have just one anecdote from you about your process and as
Paul says, no attempt to outline other alternatives.  For example,
couldn't your "activate.bat" add both Scripts *and* bin to the PATH
whereas your "activate.sh" adds just "bin"?

> I have been running like this for several years across multiple
> Python versions, so I have experience with the "breakage" from this
> part of the proposal. I have found four packages that would need to
> be updated: Pip, virtualenv, PyPM, and Egginst would need 1-2 line
> patches.

With all due respect, I find this disingenuous.  Your lack of experience 
with the tools that are out there doesn't mean they don't exist and I've 
already offered a couple of examples. I certainly can't claim to know 
what most of them are; I expect that I am underestimating them.  IMO, 
your list is a fraction of the tools impacted.

> I have these patches, I would/could provide them. Generally these tools have something like:
>
> if platform == 'win32':
>   bin_dir = 'Scripts'
> else:
>   bin_dir = 'bin'
>
> The patches just remove the special casing - bin_dir just gets set to 'bin'.

So none of those tools need to work with previous Python versions?  But 
even if what you say is strictly true, I don't think a reasonable 
response to "but what about backwards compatibility and tool breakage" 
is "the breakage is simple and the fix is trivial" - the bar has never 
been that low for changes to the language itself.  I don't see why 
tooling around the language shouldn't be held to any less account.

So my summary of the situation is:

* There has been *exactly one* concrete case listed that would benefit 
from this, and I believe that one case can be mitigated by you having 2 
directories on the PATH in Windows and one on other platforms.

* You yourself listed 4 tools that would need to change to support this. 
  I've listed a further 2, and Paul and Tim both indicated they would be 
impacted.  ActiveState and Enthought haven't been canvassed.  I suspect 
this is the tip of the iceberg - although I concede it is probably a 
relatively small iceberg :)

Like Tim, I wont sulk if you can convince people to make this change 
anyway, but IMO it is completely clear the costs outweigh the benefits. 
  Thus, if it were my decision to make, it would not happen.  Paul and 
Tim have the same view best I can tell.  I think it would be a huge 
shame if it happens even in the face of these pragmatic objections.

Cheers,

Mark