[Distutils] Splitting distribute in several distributions

[Tarek Ziadé]

On Wed, Aug 12, 2009 at 3:00 AM, Sridhar
Ratnakumar<SridharR at activestate.com> wrote:
> On Tue, 11 Aug 2009 17:44:01 -0700, Tarek Ziadé <ziade.tarek at gmail.com>
> wrote:
>
>> 2009/8/12 Sridhar Ratnakumar <SridharR at activestate.com>:
>>>
>>> I am glad that you have started thinking about the module split.
>>>
>>> 1) Why are the distributions named in CamelCase? Why not
>>> 'distribute.resources' instead of 'DistributeResources' (like, for
>>> instance,
>>> zope.interface instead of ZopeInterface).
>>
>> Because the distribution name is not the packages names.
>> See for instance the Paste project : PasteScript, Paste, PasteDeploy.
>
> Ok. So there are apparently two conventions for naming distributions.
> However, zope.*, zc.*, z3c.* fall under more wide-namespaces (zope is a
> large namespace with many sub-packages).

Well, when Zope and Plone got splitted in multiple distributions, the convention
was : distribution name == package name because every distribution has
one single package.

The Zope community eventually followed, for the worse and the better (sometimes
it's better to distribute several packages in one distribution, or to
have modules
rathers than packages)


>
>
>>> 2) In your proposal below, version comparison is part of
>>> distribute.installer; this means distribute.resources (and other modules
>>> needing version comparison) will be depending on distribute.installer,
>>> correct? Is this intended, or should version comparison go to something
>>> like
>>> distribute.core?
>>
>> When would distribute.resources need version comparisons if its sole
>> role is to provide query APIs
>> for installed packages ? (pre-PEP 376, then PEP 376-compliant)
>
> When multiple versions of a same package are installed, then perhaps
> distribute.resources might have to return them in sorted order .. but,
> well, I am not aware about the querying API yet .. so cannot say for sure.
>
> Further, if you implement distribute.pypi .. then that too would require
> version comparison, would it not? In that case, distribute.pypi would
> depend upon distribute.installer .. and vice versa, I suppose, leading to
> circular dependencies.  Eg:

I am not sure because the only use case I have in mind is comparing versions
when you install, update or check installed distributions. What I propose
is to leave the version stuff in installer, and as soon as we realize
it's needed
elsewhere (that will happen if it's the case before 0.7 is out), we
split it on its own.

> distribute.pypi.Distribution.find('PasteScript').get_releases() may have
> to return the releases in sorted order (by version number).
>
> There must be a reason why PJE kept parts of setuptools in
> pkg_resources.py. What is it? We may have to consider it before splitting
> pkg_resources.py itself.

Sure. I would tend to think it has grown too quickly.

>
>
>>> 3) PyPM's backend uses a) pkg_resources' version comparison, b)
>>> package_index's download logic (not API-friendly). I'd be interested to
>>> see
>>> distribute.installer provide this download logic (finding URLs, tarballs
>>> and
>>> fetching them) as an API. I believe pip and zc.buildout too relies on
>>> this
>>> download logic.
>>
>> pakage_index would be included in distribute.installer, and we could
>> probably
>> add simpler APIs to use it to drive a download
>
> Note that package_index is not API-friendly. A few violations I can
> think of: 1) it prints warning messages to stderr without programmer
> control, 2) throws random network exceptions that should be handled in a
> custom exception (see the issues I reported in distribute tracker) 3) no
> ability to control URL types (eg: I want to fetch only tarballs/zipfiles,
> not svn/hg/etc..) and so on.

Yes there are a lot of stuff to improve there,


Cheers
Tarek
-- 
Tarek Ziadé | http://ziade.org