Python 2.5 incompatible with Fedora Core 6 - packaging problems again

[Paul Boddie]

John Nagle wrote:
> I've been installing Python and its supporting packages on
> a dedicated server with Fedora Core 6 for about a day now.
> This is a standard dedicated rackmount server in a colocation
> facility, controlled via Plesk control panel, and turned over
> to me with Fedora Core 6 in an empty state.  This is the
> standard way you get a server in a colo today.

You have my commiserations, and I don't mean that to be a flippant
remark.

>    Bringing Python up in this completely clean environment is
> a huge hassle, and it doesn't really work.

Once upon a time, I used to manage an entire Python environment with
lots of modules, additional libraries, and so on. I don't have the
patience for that any more.

>    Fedora Core 6 ships with Python 2.4, which appears to be a
> good decision on someone's part.  Of course, there's no
> standard RPM for a Python 2.5 install on Linux.  Packaging is
> someone else's problem, according to the Python developers.

I managed to find a site which gives some kind of searchable index of
packages:

http://www.fedoratracker.org/

It's a bit like the Debian and Ubuntu package sites but seemingly
unofficial and a lot slower. Once upon a time there were some
contributed RPMs available from python.org, but I don't know what
happened to them.

[FTP server, GCC, configuration]

>     But "make install" does some compiles, something "install" probably
> shouldn't be doing.

If the original build process didn't manage to compile everything,
make will revisit that part of the process. What should really happen
is that make should halt and not try and do anything else.

An aside on the subject of installing - something you may be aware of,
but worth repeating for others who have read this far. I guess that
"make altinstall" isn't necessary if you *don't* do something like
this when configuring:

./configure --prefix=/usr

The issue here is that if Fedora (or other distributions) has things
relying on /usr/bin/python, you have to make sure that you don't
overwrite it when installing. If you've been thinking that the new
version of Python should live alongside the others, you have to use
"make altinstall" to preserve the existing python program (the
python2.4 will survive, anyway) whilst installing python2.5 alongside
it.

Typically, the configure script behaves as if this were typed:

./configure --prefix=/usr/local

Thus, a "make install" will create /usr/local/bin/python and /usr/
local/bin/python2.5, hopefully not overwriting other things.

>  Some of these compiles fail, with error messages
> like
> "/var/www/vhosts/sitetruth.com/private/downloads/python/Python-2.5/Modules/_curses_panel.c:271:
> error: "PyCursesPanelObject" has no member named "pan"

This should stop the process, but as you note...

> But the install script plunges blindly on, presumably installing a broken
> Python 2.5.  Apparently Fedora Core 6 doesn't come with "curses" preinstalled,
> but the Python installer assumes it is there.  What's inexcusable is
> just plowing on after failed compiles, resulting in a broken install.

This might require a bug report.

> It's probably necessary to install something before building Python,
> but the README file is silent on this.
>
>      OK.  Plunging onward, and not really needing the "curses" library,
> we try to install MySQLdb.  We get that with FTP,
> unpack it, go to the approprate directory, and run "python setup.py".
> This fails because the MySQL headers aren't installed:
>
> "_mysql.c:35:23: error: my_config.h: No such file or directory"
>
> OK, looks like we need to install "mysql-devel", so we do that,
> using "yum".   This forces an update of OpenSSL, Kerberos, and
> "e2fsprogs-dev".  Which means we should reboot at some point.

Perhaps the system wasn't up-to-date anyway.

> But now we can build MySQLdb.  So we build and install it.
>
> So then we try "import MySQLdb" in Python 2.5.  And we get this:
>
> > /usr/local/lib/python2.5/site-packages/MySQL_python-1.2.2-py2.5-linux-i686.egg/_mysql.py:3: UserWarning: Module _mysql was already imported from /usr/local/lib/python2.5/site-packages/MySQL_python-1.2.2-py2.5-linux-i686.egg/_mysql.pyc, but /var/www/vhosts/sitetruth.com/private/downloads/MySQLdb/MySQL-python-1.2.2 is being added to sys.path
> >   import sys, pkg_resources, imp
> > Traceback (most recent call last):
> >   File "<stdin>", line 1, in <module>
> >   File "MySQLdb/__init__.py", line 19, in <module>
> >     import _mysql
> >   File "build/bdist.linux-i686/egg/_mysql.py", line 7, in <module>
> >   File "build/bdist.linux-i686/egg/_mysql.py", line 4, in __bootstrap__
> >   File "/usr/local/lib/python2.5/site-packages/setuptools-0.6c5-py2.5.egg/pkg_resources.py", line 800, in resource_filename
> >   File "/usr/local/lib/python2.5/site-packages/setuptools-0.6c5-py2.5.egg/pkg_resources.py", line 1228, in get_resource_filename
> >   File "/usr/local/lib/python2.5/site-packages/setuptools-0.6c5-py2.5.egg/pkg_resources.py", line 1250, in _extract_resource
> >   File "/usr/local/lib/python2.5/site-packages/setuptools-0.6c5-py2.5.egg/pkg_resources.py", line 880, in get_cache_path
> >   File "/usr/local/lib/python2.5/site-packages/setuptools-0.6c5-py2.5.egg/pkg_resources.py", line 846, in extraction_error
> > pkg_resources.ExtractionError: Can't extract file(s) to egg cache
> >
> > The following error occurred while trying to extract file(s) to the Python egg
> > cache:
> >
> >   [Errno 13] Permission denied: '/var/www/vhosts/sitetruth.com/.python-eggs'
> >
> > The Python egg cache directory is currently set to:
> >
> >   /var/www/vhosts/sitetruth.com/.python-eggs
> >
> > Perhaps your account does not have write access to this directory?  You can
> > change the cache directory by setting the PYTHON_EGG_CACHE environment
> > variable to point to an accessible directory.
>
> So what's going on?  We've run into a conflict between an assumption of Python
> and of the Plesk control panel.  Plesk doesn't let the user create files
> in their own home directory.  This installer assumes it can.  Oops.
> (Plesk sets up a very locked down environment, which is a good thing on
> a web server.)

This is probably a setuptools/easy_install FAQ. I only know this from
lurking on various mailing lists because I don't use setuptools/
easy_install.

> So, becoming the super-user, we create .python-eggs and change its ownership.
> Now we can import MySQLdb into Python.
>
> Now it turns out that we have Python 2.4 in /usr/bin, and Python 2.5
> in /usr/local/bin.  That's where the installer put it.  So the CGI
> scripts are invoking the wrong version of Python.  The scripts had to
> be changed.

That's possibly an environment issue, depending on what the first line
of the scripts are.

> At last, MySQLdb is connecting to MySQL properly.  However, the SSL module
> seems to be broken.  Tomorrow I'll deal with that, and try to get
> M2Crypto installed, always a tough job.
>
> This kind of nonsense is why hosting companies don't want to support Python.
> Perl and PHP may be dumb, but they just work.  Java has a company behind it.
> Python just isn't ready.  Which is embarassing, ten years on.

Do hosting companies support multiple versions of Perl or PHP in the
same environment? This isn't a provocative question, but I'd be
interested to know what they do.

> If the Python crowd wants market share, attention, and a career path,
> this has to be fixed.

Generally, if I were installing into a hosting environment - something
I may consider doing in the relatively near future - I'd firstly
choose a distribution I like working with (Debian-based, not Fedora)
and use the system packaging tools to their maximum extent. I'd even
create my own packages and repositories if necessary; I've done this
already for my own system. Managing dependencies, especially when you
get into dealing with lots of libraries from outside the Python
ecosystem, is a demanding task which can be sufficiently lightened if
you can make use of other people's packaging work.

Really, the "Python crowd" needs to involve the distribution makers
more, rather than (from time to time) calling them names, since most
installed versions of Python on non-Windows platforms are distributed
by those people to users, not obtained by users from python.org. One
exception may involve early adopters and the very latest Python
versions, however.

Paul