[Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

[Ronald Oussoren]

On 16 Jul, 2013, at 15:08, Paul Moore <p.f.moore at gmail.com> wrote:

> On 16 July 2013 13:42, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
> > On the other hand, I'm missing something, as I don't see how the *current* exe wrappers avoid meaning that there need to be separate 32-bit and 64-bit versions of pip...
> 
> Couldn't you just ship both variants of the exe wrappers in a single distribution and then use the correct one for the current installation? That's what I'm doing in py2app.
> 
> That's OK for source-style installs (which is what setuptools does, and what pip mostly cares about right now). But for bundling with Python it needs to be considered (although it's just getting the right one in the right installer). But for wheels it's a pain, because instead of having just a single pip wheel, we need 32-bit and 64-bit wheels solely for the wrappers. That sucks. Hard. (And it's not just for pip, nose will have the same problem, as will many other projects that use exe wrappers). And the bdist_wheel command currently doesn't recognise this *at all*. So wheels using wrappers are potentially broken.

You could just have a wheel that contains two data files: wrapper-win32.exe and wrapper-win64.exe, then select the one that gets used as the wrapper for a specific script when you create that wrapper. 

That's assuming that the wrapper .exe gets "created" when a wheel is installed and is not included in the wheel.

> 
> I think the correct solution is to explicitly have declarative support for "console script entry point" metadata in PEP 426, as well as having tools like bdist_wheel and distil do some explicit backward compatibility hacking to remove legacy-style exe wrappers. The wheel install code should then explicitly install appropriate wrappers for the target platform (which may be exe wrappers similar to the current ones, but moving forward may be some other mechanism if one is found).

Yikes, that means my assumption is wrong. The section on "Recommended installer features" in the wheel spec[1] says that the wrapper executable should be created on installation, does pip not do this?

> 
> This is complex enough that I'm now concerned that we need reference "wheel install" code in the stdlib, just so that people don't make up their own on the basis that "wheel is simple to install manually" and screw it up. Also so that we only have one style of command line script wrapper to deal with going forward, not a multitude of mostly-compatible solutions.

I'd love to see comprehensive wheel support in the stdlib, but that may have to wait for 3.5 because the entire packaging systeem (wheels, metadata, ...) is moving forward quickly at the moment.  That said, it would be nice if distutils would grow support for creating wheels and modern metadata in sdists as that would mean I could drop usage of setuptools for most of my software (for python 3.4). 

> 
> Nick: See the above point re PEP 426 - do you agree that this needs addressing in Metadata 2.0?
> 
> Paul
> 
> PS There is still the proviso that I haven't tested my assumption that the separate 32 and 64 bit wrappers are *needed* (setuptools and distlib use them, so I think it's a valid assumption, but I need to test). I will try to get time to check that ASAP.

That depends on what the wrapper does, if it launches a regular python with the right command-line you might be able to get away with a single wrapper, if it loads python.dll and executes the script directory you do need separate wrappers for 32 and 64 bit.


[1] http://www.python.org/dev/peps/pep-0427/#recommended-installer-features