[Distutils] Library instability on PyPI and impact on OpenStack

[Donald Stufft]

On Monday, March 4, 2013 at 5:29 PM, Mark McLoughlin wrote:
> On Mon, 2013-03-04 at 12:44 -0500, Donald Stufft wrote:
> > On Monday, March 4, 2013 at 12:36 PM, Mark McLoughlin wrote:
> 
> 
> > > If parallel incompatible installs is a hopeless problem in Python,
> > > why
> > > the push to semantic versioning then rather than saying that
> > > incompatible API changes should mean a name change?
> > > 
> > 
> > Forcing a name change feels ugly as all hell. I don't really see what
> > parallel installs has much to do with anything. I don't bundle anything
> > and i'm ideologically opposed to it generally but I don't typically have
> > a need for parallel installs because I use virtual environments. Why
> > don't you utilize those? (Not being snarky, actually curious).
> > 
> 
> 
> It's a fair question.
> 
> To answer it with a question, how do you imagine Linux distributions
> using virtual environments such that:
> 
> $> yum install -y openstack-nova
> 
> uses a virtual environment? How does it differ from bundling? (Not being
> snarky, actually curious :)
> 
> 

Ah, See I don't install Python software via package managers so for
that use case yes it'd probably be a lot like bundling.

So the biggest problem with the setuptools style multi versioning that
I can remember is that it modified the global system state via pth files. 

I've got an idea for a system that would work for this use case however
it would result in app start up taking longer (meaninfully longer? I don't
know for sure). Although it would require either support from the linux
packagers or installation via a third party tool.
> 
> The approach that some Fedora folks are trying out is called "Software
> Collections". It's not Python specific, but it's basically the same as a
> virtual environment.
> 
> For OpenStack, I think we'd probably have all the Python libraries we
> require installed under e.g. /opt/rh/openstack-$version so that you
> could have programs from two different releases of OpenStack installed
> on the same system.
> 
> Long time packagers are usually horrified at this idea e.g.
> 
> http://lists.fedoraproject.org/pipermail/devel/2012-December/thread.html#174872
> 
> Some of the things to think about:
> 
> - Each of the Python libraries under /opt/rh/openstack-$version would 
> come from new packages like openstack-$version-python-eventlet.rpm -
> how many applications in Fedora would have a big stack of "bundled" 
> python packages like OpenStack? 5, 10, 50, 100? Let's say it's 10 
> and each one stack has 20 packages. That's 200 new packages which 
> need to be maintained by Fedora versus the current situation where 
> we (painfully) make a single stack of libraries work for all 
> applications.
> 
> - How many of these 200 new packages are essentially duplicates? Once 
> you go down the route of having applications bundle libraries like 
> this, there's going to basically be no sharing.
> 
> - What's the chance that that all of these 200 packages will be kept 
> up to date? If an application works with a given version of a 
> library and it can stick with that version, it will. As a Python 
> library maintainer, wow do you like the idea of 10 different
> versions of you library included in Fedora?
> 
> - The next time a security issue is found in a common Python library,
> does Fedora now have to rush out 10 parallel fixes for it?
> 
> You can see that reaction in mails like this:
> 
> http://lists.fedoraproject.org/pipermail/devel/2012-December/174944.html
> 
> and the "why can't these losers just maintain compatibility" view:
> 
> http://lists.fedoraproject.org/pipermail/devel/2012-December/175028.html
> http://lists.fedoraproject.org/pipermail/devel/2012-December/174929.html
> 
> Notice folks complaining about Ruby and Java here, not Python. I can see
> Python embracing semantic versioning and "just use venv" shortly leading
> to Python being included in the list of "heretics".
> 
> Thanks,
> Mark.
> 
> [1] - http://docs.fedoraproject.org/en-US/Fedora_Contributor_Documentation/1/html-single/Software_Collections_Guide/index.html 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20130304/52de34b2/attachment.html>