[Pythonmac-SIG] PackageManager / Python 2.3.x - 2.4 ideas

[Jack Jansen]

First a comment: initially I thought that doing a PEP on the client 
side of
PackMan would be good enough, but now that I have had the experience of 
doing a
database three times, with months in between, I think that the scapegoat
side is important too. We also need to come up with tools or procedures
that make it easier for multiple people being responsible for a single 
database,
and also for a single person to keep information that is going to be 
helpful
the next time the database is updated.

Scapegoat-side could be integrated into distutils. One day I think this 
is
a very good idea, the next day I think it is a very bad idea. I will 
ponder
on this, and I would like input.

On woensdag, jul 30, 2003, at 23:01 Europe/Amsterdam, Bob Ippolito 
wrote:

> These are some ideas for future versions of Package Manager and 
> friends...
>
> Should remove shell executable dependencies because python does these
> 	tar
> 	unzip
> 	zip
> 	curl
> 	(not sure if it uses there or not)
> 	rm
> 	mv

Definitely.

> Should leverage metadata from PyPI in some way

This is scapegoat-side. If we go for a distutils-based solution we get 
it automatically,
as distutils has support for the PyPI metadata (I think?).

> Should read proxy settings from OS X rather than using environment 
> variables.. allow a user override, store this as a preference

Not a PackMan problem, this is a urllib problem. There's a bug filed 
already.

> Should allow user to override 'scratch area', they might not always 
> have space in /tmp .. store this as a preference

Definitely. Especially for people doing source installs you want to 
keep things around.
>

> We should write a cross-platform "preferences" module for python that:
> 	Uses the registry in win32 to determine a folders for global/user 
> plists, and then uses that :)
> 	Uses ~/Library/Preferences/plists and /Library/Preferences/plists in 
> OS X
> 	Uses somewhere/plists for global settings on unix and 
> ~/.dotfolder/plists in UNIX

While I agree we need it I think it isn't a PackMan problem. There are 
numerous
other packages that want this (bgen, for instance). This could be the 
subject of a
separate PEP.

> Not have separate plist files for each distutils.util.get_platform() 
> -- this is only semi-useful for binary OS X packages, and not useful 
> at all for other operating systems.

This one I disagree with, and I disagree with it strongly. The central 
philosophy of PackMan
is accountability (in other words: the scapegoat). We may need to 
structure things differently, with major/minor OS release in the 
filename and within the database entries a list of micro releases that 
are known to work, but we definitely need the functionality.

Binary packages are the reason for PackMan's existence, source releases 
are a service to the developer community. They already have PyPI.

> Have a cryptographically authenticated way of determining the 
> authenticity of 'stuff'. [...]

This is serious reading, I will come back to this later.

> Keep receipts for installed packages, even if we don't write an 
> uninstaller.

Parts are already in place, but they don't get written out to disk. It 
turned out to be a harder problem than I thought. But I agree we need 
to do something about this.

> Keep track of dependency versions.. for example, FooBar 2.0 might 
> require a minimum BarBaz of 1.0 and is tested compatible up to BarBaz 
> 1.5.  BarBaz might release a wonderful 1.6 version that breaks some 
> behavior that FooBar 2.0 or whatever other packages depended on.  
> Upgrading to BarBaz 1.6 should warn the user that "FooBar 2.0 and 
> SuperAlice 1.5 are dependent on BarBaz 1.0-1.5, but are not tested to 
> work with BarBaz 1.6.  This upgrade may require upgrades to these 
> dependencies".  Dependency graphs are a lot of fun, but we're all good 
> programmers here and we can't be lazy all the time.

This should all be in there already. You can depend on a package, a 
specific version of a package, or even a specific flavor 
(pil-1.1.4-source).

> Package Manager should be able to install distutils-based setup.py 
> source distributions that a user has on his harddrive or at any 
> arbitrary URL (i.e. not in the plist) either by dragging the tgz/zip 
> to PackageManager, picking a folder with a setup.py in it, or picking 
> a setup.py.

Definitely. And similarly for bdist_dumb distributions. And it needs a 
"recipy" file format, so system administrators can standardise large 
numbers of machines in one fell swoop. We can
even automate this for them: have a tool that takes a PackMan recipe 
and a Python distribution, and what comes out is a .mpkg that first 
installs Python and then feeds the recipe to PackMan.

> Package Manager should be organized categorically, like PyPI. (i.e. 
> they should be heavily integrated).

Yes, I think we do, indeed. On the other hand I want the database to 
remain manageable in size, both for the user and for the maintainer. 
The PyPI database has already become unmanageable:-)

> We should test upgrades to Package Manager :)

Definitely.

> All of Package Manager's functionality should really be in platform 
> independent "designed" modules (meaning they don't have to support 
> other platforms yet, just designed with that intent).

It's all there. Untested, but it's there.

> Package Manager should be a responsive application, either by using 
> nonblocking methods to speak with spawned processes.  Threading sucks 
> in python, let's not try and use it too much.

I agree with the responsive bit, not with the threading bit. I think 
switching to MVC should make threading a lot easier. Only the log 
remains a problem (but not a very big one, on unix).

> Speaking more with regard to separating the model and view of Package 
> Manager, we should think about making the Package Manager GUI speak 
> with Package Manager over sockets.. imagine managing python 
> installations for everyone on your LAN (that trusts you by SSH, or 
> whatever).

I'm not sure we need to go this far, see above.
--
- Jack Jansen        <Jack.Jansen at oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- Emma 
Goldman -