[TriZPUG] managing python versions as well as package versions, or "up-to-speed for the ecosystem"

[Chris Calloway]

On 8/3/2011 3:52 PM, Tom Roche wrote:
> summary: I'm having a particular problem involving docutils, python,
> ubuntu, and sloth; I'd most appreciate help with that problem. However
> my inability to solve my problem seems due to ignorance as well as
> sloth, so I'd appreciate pointers to doc regarding python package/
> project/version management best practices more generally, and preferably
> targeting folks who've coded a lot outside python and its ecosystem but
> not so much inside/with.
>
> details:
>
> I'm currently running
>
> $ lsb_release -ds
>> Ubuntu 10.04.3 LTS
>
> way downlevel I know, and I should be upgrading Sometime Next Week
> (famous last words :-) but at least it's up-to-date
>
> $ uname -rv
>> 2.6.32-33-generic #71-Ubuntu SMP Wed Jul 20 17:27:30 UTC 2011
>
> except that
>
> $ python --version
>> Python 2.6.5
>
> I'm trying to fix a doc problem on a site that uses yatiblog, but
>
> $ pip install yatiblog
> ...
>> VersionConflict: (docutils 0.6 (/usr/lib/pymodules/python2.6), Requirement.parse('docutils>=0.7'))
>
> which I take to mean that "yatiblog requires docutils>=0.7, and
> docutils>=0.7 requires python>2.6" (esp since I get the same dopeslap
> from
>
> $ pip install 'docutils>=0.7'
> ...
>> VersionConflict: (docutils 0.6 (/usr/lib/pymodules/python2.6), Requirement.parse('docutils>=0.7'))
>
> ). Is that correct? If so, then (transitively), to get yatiblog I need
> to get python>=2.7. So I'm wondering

Nothing I'm seeing here suggests that you need python>=2.7 in order to 
get docutils>=0.7.

There's couple of things that may be going on here.

One is Ubuntu. Your docutils may have been installed by Ubuntu package 
management rather than pythonic package management. This always leads to 
a big mess. I wish Ubuntu would just get out of the Python package 
management business and leave that to Python.

Another is the pip install command. Type "pip help install" or visit 
http://www.pip-installer.org/en/latest/index.html to read about the -U 
option for upgrading a package:

"pip also performs an automatic uninstall of an old version of a package 
before upgrading to a newer version, so outdated files (and egg-info 
data) from conflicting versions aren’t left hanging around to cause 
trouble. The old version of the package is automatically restored if the 
new version fails to download or install."

This suggests to me that pip doesn't want old versions hanging around 
(unlike easy_install which will let you keep up with multiple versions 
of a package). This suggest to me that pip wants to upgrade and not just 
install new versions of a package. Omitting the -U option may be pip's 
inelegant way of informing you that you are about to stomp on a 
previously installed version and do you really want to do that, yes or 
no. I haven't tested this. But you should.

> 1 What's the easiest or best way to globally install a more-uplevel
>    python on a given release of ubuntu, without hosing the global/
>    official/default/downlevel python, much less my debian/ubuntu/aptitude
>    packages and their managers?

The Ubuntu repository usually has multiple versions of Python avaialable 
to install through their packaging system.

Another way is to compile your own Python to someplace not in your $PATH 
using the --prefix option to the configure step.

> 2 Instead of doing global installs, I'd like to do (though I haven't
>    needed to, yet) isolated installs. But my impression is that (e.g.)
>    visualenv won't help with python-version problems, only
>    python-package-version problems. Am I missing something? If not,
>    should I use pythonbrew? or similar?

Correct. virtualenv is for managing the python packages of a particular 
python installation.

pythonbrew is for managing python version installs to your $HOME. I 
haven't used it so cannot advise if it's useful. I like just compiling 
my own Python. It's what professionals do and it gives you a great 
amount of control over your Python versions and what you are doing with 
them.

> 3 Periodically trizpug has threads on "getting started with python," but
>    they tend to be like "learning programming with python." I'd
>    appreciate pointers to doc more about the "python ecosystem"
>    (particularly, ... wait for it ... python-version and
>    python-package-version management :-) targeting folks who've grown up
>    programming elsewhere. FWIW I've scanned through
>
> http://trizpug.org/up-to-speed/
>
>    and am not seeing anything like that, but ...

http://www.packtpub.com/expert-python-programming/book

This covers virtualenv, which is great for python package management, 
and buildout, which is great for building Pythons (using Python). It's 
by the Python package management guru.

Python version management is done wholly through your $PATH environment 
variable. However, the awesome Doug Hellmann puts out this package which 
manages multiple virtualenvs, which I think is maybe what you want:

http://pypi.python.org/pypi/virtualenvwrapper

It's a bit meta-meta for me, though. I just make multiple virtualenvs 
and activate the one I want to work with at any given time. And I 
compile multiple Pythons, usually in /opt, to make those virtualenvs 
with whatever Python version I need.

You're the second person today I've talked to about managing multiple 
Python versions. I'm concluding it's a common newbie stumbling block. 
Actually, the hardest this anyone has to do at PyCamp is install Python. 
That's telling in and of itself. The number of things that can go wrong 
is just crazy and completely confusing for newbies.

The differences between installing on different OS platforms is itself 
hard to manage in a classroom environment where there all platforms are 
represented in multiple versions themselves (different procedures for 
Windows before and after 2000/Vista, different procedures for OSX before 
and after Leopand/Snow Leopard, different Python management on every 
flavor of Linux).

It's hard to tutor because it doesn't have much to do with Python 
itself. And it's a moving target.

-- 
Sincerely,

Chris Calloway http://nccoos.org/Members/cbc
office: 3313 Venable Hall   phone: (919) 599-3530
mail: Campus Box #3300, UNC-CH, Chapel Hill, NC 27599