[Pythonmac-SIG] PackageManager / Python 2.3.x - 2.4 ideas

Wed Jul 30 17:04:37 EDT 2003

On Wednesday, July 30, 2003, at 03:25  PM, Jack Jansen wrote:

> First a comment: initially I thought that doing a PEP on the client 
> side of
> PackMan would be good enough, but now that I have had the experience 
> of doing a
> database three times, with months in between, I think that the 
> scapegoat
> side is important too. We also need to come up with tools or procedures
> that make it easier for multiple people being responsible for a single 
> database,
> and also for a single person to keep information that is going to be 
> helpful
> the next time the database is updated.

IMO, what we need here is to have a package metadata file for Python 
packages, possibly in setup.py, or whatever. Then, using a command 
(i.e. setup.py update_ppm), folks could "synch" that metadata with the 
main Package Manager database on Python.org. I've seen a PEP on 
something similar to this, but more limited - I think it was a way to 
specify trove classification data, description, etc. in setup.py and 
update it with PyPI. But this would be a simple and logical extension 
to that functionality.

> Scapegoat-side could be integrated into distutils. One day I think 
> this is
> a very good idea, the next day I think it is a very bad idea. I will 
> ponder
> on this, and I would like input.

I personally think it is a good idea, at least today. ;-) The core idea 
behind PyPI and other tools like it is that each package maintainer 
should be expected to promote and update their own package, and I think 
Package Manager should be the same way.

Thanks,

Kevin

> On woensdag, jul 30, 2003, at 23:01 Europe/Amsterdam, Bob Ippolito 
> wrote:
>
>> These are some ideas for future versions of Package Manager and 
>> friends...
>>
>> Should remove shell executable dependencies because python does these
>> 	tar
>> 	unzip
>> 	zip
>> 	curl
>> 	(not sure if it uses there or not)
>> 	rm
>> 	mv
>
> Definitely.
>
>> Should leverage metadata from PyPI in some way
>
> This is scapegoat-side. If we go for a distutils-based solution we get 
> it automatically,
> as distutils has support for the PyPI metadata (I think?).
>
>> Should read proxy settings from OS X rather than using environment 
>> variables.. allow a user override, store this as a preference
>
> Not a PackMan problem, this is a urllib problem. There's a bug filed 
> already.
>
>> Should allow user to override 'scratch area', they might not always 
>> have space in /tmp .. store this as a preference
>
> Definitely. Especially for people doing source installs you want to 
> keep things around.
>>
>
>> We should write a cross-platform "preferences" module for python that:
>> 	Uses the registry in win32 to determine a folders for global/user 
>> plists, and then uses that :)
>> 	Uses ~/Library/Preferences/plists and /Library/Preferences/plists in 
>> OS X
>> 	Uses somewhere/plists for global settings on unix and 
>> ~/.dotfolder/plists in UNIX
>
> While I agree we need it I think it isn't a PackMan problem. There are 
> numerous
> other packages that want this (bgen, for instance). This could be the 
> subject of a
> separate PEP.
>
>> Not have separate plist files for each distutils.util.get_platform() 
>> -- this is only semi-useful for binary OS X packages, and not useful 
>> at all for other operating systems.
>
> This one I disagree with, and I disagree with it strongly. The central 
> philosophy of PackMan
> is accountability (in other words: the scapegoat). We may need to 
> structure things differently, with major/minor OS release in the 
> filename and within the database entries a list of micro releases that 
> are known to work, but we definitely need the functionality.
>
> Binary packages are the reason for PackMan's existence, source 
> releases are a service to the developer community. They already have 
> PyPI.
>
>> Have a cryptographically authenticated way of determining the 
>> authenticity of 'stuff'. [...]
>
> This is serious reading, I will come back to this later.
>
>> Keep receipts for installed packages, even if we don't write an 
>> uninstaller.
>
> Parts are already in place, but they don't get written out to disk. It 
> turned out to be a harder problem than I thought. But I agree we need 
> to do something about this.
>
>> Keep track of dependency versions.. for example, FooBar 2.0 might 
>> require a minimum BarBaz of 1.0 and is tested compatible up to BarBaz 
>> 1.5.  BarBaz might release a wonderful 1.6 version that breaks some 
>> behavior that FooBar 2.0 or whatever other packages depended on.  
>> Upgrading to BarBaz 1.6 should warn the user that "FooBar 2.0 and 
>> SuperAlice 1.5 are dependent on BarBaz 1.0-1.5, but are not tested to 
>> work with BarBaz 1.6.  This upgrade may require upgrades to these 
>> dependencies".  Dependency graphs are a lot of fun, but we're all 
>> good programmers here and we can't be lazy all the time.
>
> This should all be in there already. You can depend on a package, a 
> specific version of a package, or even a specific flavor 
> (pil-1.1.4-source).
>
>> Package Manager should be able to install distutils-based setup.py 
>> source distributions that a user has on his harddrive or at any 
>> arbitrary URL (i.e. not in the plist) either by dragging the tgz/zip 
>> to PackageManager, picking a folder with a setup.py in it, or picking 
>> a setup.py.
>
> Definitely. And similarly for bdist_dumb distributions. And it needs a 
> "recipy" file format, so system administrators can standardise large 
> numbers of machines in one fell swoop. We can
> even automate this for them: have a tool that takes a PackMan recipe 
> and a Python distribution, and what comes out is a .mpkg that first 
> installs Python and then feeds the recipe to PackMan.
>
>> Package Manager should be organized categorically, like PyPI. (i.e. 
>> they should be heavily integrated).
>
> Yes, I think we do, indeed. On the other hand I want the database to 
> remain manageable in size, both for the user and for the maintainer. 
> The PyPI database has already become unmanageable:-)
>
>> We should test upgrades to Package Manager :)
>
> Definitely.
>
>> All of Package Manager's functionality should really be in platform 
>> independent "designed" modules (meaning they don't have to support 
>> other platforms yet, just designed with that intent).
>
> It's all there. Untested, but it's there.
>
>> Package Manager should be a responsive application, either by using 
>> nonblocking methods to speak with spawned processes.  Threading sucks 
>> in python, let's not try and use it too much.
>
> I agree with the responsive bit, not with the threading bit. I think 
> switching to MVC should make threading a lot easier. Only the log 
> remains a problem (but not a very big one, on unix).
>
>> Speaking more with regard to separating the model and view of Package 
>> Manager, we should think about making the Package Manager GUI speak 
>> with Package Manager over sockets.. imagine managing python 
>> installations for everyone on your LAN (that trusts you by SSH, or 
>> whatever).
>
> I'm not sure we need to go this far, see above.
> --
> - Jack Jansen        <Jack.Jansen at oratrix.com>        
> http://www.cwi.nl/~jack -
> - If I can't dance I don't want to be part of your revolution -- Emma 
> Goldman -
>
>
> _______________________________________________
> Pythonmac-SIG maillist  -  Pythonmac-SIG at python.org
> http://mail.python.org/mailman/listinfo/pythonmac-sig
>