[Distutils] Other ideas from today's packaging meetup at EuroPython

[Steve Dower]

"Will that also allow me to put ‘macports’ or ‘homebrew’ in there you can create an Cython-0.20.1-cp27-none-lmacosx_x86_64-macports_10.10.whl distribution?"

Just how many wheels are people going to have to publish? Who has that many dev machines? Without a build farm, I can't see this being more helpful than frustrating (and the build farm is going to need a way to automatically get non-Python dependencies, which is not our business, so that's a long way off IMO).

An organisation building their own wheels in-house and configuring their machines to use an OS specific index/wheelhouse sounds like a much more feasible idea that works now and could do with more traction and encouragement. (Someone mentioned earlier that they do this - apologies for forgetting who.)

Cheers,
Steve

Top-posted from my Windows Phone
________________________________
From: Wichert Akkerman<mailto:wichert at wiggy.net>
Sent: ‎7/‎25/‎2014 22:27
To: Nick Coghlan<mailto:ncoghlan at gmail.com>
Cc: DistUtils mailing list<mailto:distutils-sig at python.org>
Subject: Re: [Distutils] Other ideas from today's packaging meetup at EuroPython


On 26 Jul 2014, at 00:08, Nick Coghlan <ncoghlan at gmail.com<mailto:ncoghlan at gmail.com>> wrote:


On 26 Jul 2014 05:56, "Donald Stufft" <donald at stufft.io<mailto:donald at stufft.io>> wrote:
>
> On July 25, 2014 at 3:50:30 PM, Wichert Akkerman (wichert at wiggy.net<mailto:wichert at wiggy.net>) wrote:
>> Will that guarantee the OS-provided Python was used? Or is there still a risk someone was using a custom compiled Python on an Ubuntu 14.04 system that is not binary compatible with the Ubuntu-provided Python?
>>
>> Wichert.
>>
>
> No It won’t guarantee the OS-provided Python is used. It doesn’t even guarantee that the OS provided libs are being linked to. However at that point you’ve more or less reached parity with Windows and OSX where Wheels (and Eggs before them) are generally built to target the “System” Python and if you’re not using the “System” Python you might end up having a bad time.

As Donald says, we haven't worked out the exact technical details yet, but the general idea is that just as a Windows or Mac OS X wheel published on PyPI should be binary compatible with the corresponding python.org<http://python.org/> installer, it should be possible to get the relevant info into wheel filenames for people to be able to indicate the expected contents of "/etc/os-release".

I suspect that for Linux you mean “system-provided Python”? Looking at https://www.python.org/downloads/release/python-341/ there is no python.org<http://python.org> binary installer for Linux. Even if there was I would expect only a small number of people to use that over the system-provided version, considering Linux distributions generally do a excellent job packaging Python.

An /etc/os-release approach does sound like a reasonable approach. Will that also allow me to put ‘macports’ or ‘homebrew’ in there you can create an Cython-0.20.1-cp27-none-lmacosx_x86_64-macports_10.10.whl distribution? That might be useful. It would be slightly abusing the spec though since macports does not have releases itself so we’re mixing an OSX version number in, but that does not feel any worse than the other binary compatibility guarantees.

For Linux why not use the existing /etc/lsb_release instead of creating a new file?

Another way to go, if softwarecollections.org<http://softwarecollections.org/> (aka "virtualenv for your distro package manager") gets any traction whatsoever on the Debian/Ubuntu side of the world, would be to publish wheels based on *software collection* compatibility, rather than distro compatibility.

I had never heard of that; I’ll need to take a look at it. A quick look at the website doesn’t really tell me what problem it is trying to solve or how it works. It might be in the same space as things like debootstrap, docke or PPSar. Or perhaps NixOS really is a better way to handle this.


That would be appealing to me wearing both my packaging hats - making prebuilt binaries for libraries more readily available to Python users in a way that maps to Python versions rather than distro variants, while at the same time encouraging them to get their end user applications the heck out of the system Python so they aren't tied so closely to the distro upgrade cycle :)

Isn’t that already handle in the Debian/Ubuntu world with PPAs? Those do make it completely trivial to install extra things on a system, and you could easily make PPAs that provide custom builds that don’t overlap the OS.

Wichert.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20140726/4a7d9617/attachment.html>