From ianb at colorstudy.com  Mon Aug  2 19:06:26 2010
From: ianb at colorstudy.com (Ian Bicking)
Date: Mon, 2 Aug 2010 12:06:26 -0500
Subject: [Catalog-sig] Suggested change to /simple index
In-Reply-To: <20100729175116.A74703A4114@sparrow.telecommunity.com>
References: <20100729175116.A74703A4114@sparrow.telecommunity.com>
Message-ID: <AANLkTinPTPQWZ9O49FPCqZZZCQq03REUja8Ly6=Q__Gk@mail.gmail.com>

Works for me too.

On Thu, Jul 29, 2010 at 12:51 PM, P.J. Eby <pje at telecommunity.com> wrote:

> Recently, a proposal was made to change the sorting of links on PyPI's
> /simple  index to prevent problems with easy_install finding out-of-date
> non-PyPI download links.  That proposal, unfortunately, would not have
> solved the actual problem.
>
> After giving it some thought, I have an alternative proposal, that I think
> *would* solve the problem, and work for all scraping tools using the /simple
> index, not just easy_install.
>
> Essentially, the problem is that when links to "hidden" versions were added
> to the /simple index (to satisfy users wanting to be able to download older
> versions' distributions), in-description and home/download page links were
> included.  However, if a package's home page URL or revision control
> download links change over time, the older ones still show up in the /simple
> listing, leading to ambiguity for download tools.
>
> However, since the actual use case for which this was added was only to
> support reaching specific older versions of a project, it isn't actually
> necessary to include links that aren't to downloadable files with a specific
> version number.
>
> Say package Foo releases version 1.1, causing 1.0 to become hidden.  People
> still want to be able to download the 1.0's .tgz's or .rpm's or
> what-have-you's.  However, they do *not* still need to be able to access the
> project's older, now-defunct home page, or any of the extra links included
> in the older version's description.
>
> It is these extraneous links that cause the problem, not the access to
> PyPI-hosted archives.
>
> Now, it could be argued that if a project used its "download" or "home
> page" link (or even in-description links) to point to actual archives, and
> if that is the case, then older links would be lost by omitting such links
> for "hidden" versions.  However, if that's really a problem, it could be
> remedied by simply checking whether the URL contains a file extension, or a
> revision number, or something like that.
>
> However, since the original request to access hidden versions was aimed
> squarely at PyPI-hosted downloads, the original use case could still be met
> simply by only including PyPI-hosted links for "hidden" releases, thereby
> insuring that other links are only shown for "current" versions -- i.e.,
> ones that package authors would expect are the only versions whose
> home/download/description links would need to be kept up-to-date on.
>
> Making such a change would immediately fix many problematic/ambiguous links
> in the /simple index, where out-of-date or no-longer available links are
> shown.  (It would also fix the security issue whereby someone acquiring a
> no-longer-in-service URL could link it to trojan downloads.)
>
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org
> http://mail.python.org/mailman/listinfo/catalog-sig
>



-- 
Ian Bicking  |  http://blog.ianbicking.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20100802/51131449/attachment.html>

From monitor at jacobian.org  Tue Aug  3 00:56:54 2010
From: monitor at jacobian.org (monitor at jacobian.org)
Date: Mon, 02 Aug 2010 17:56:54 -0500
Subject: [Catalog-sig] [monit] pypi.python.org - Connection failed
Message-ID: <1280789817.1@jacobian.org>

Connection failed Service pypi.python.org 

	Date:        Mon, 02 Aug 2010 17:56:54 -0500
	Action:      alert
	Host:        jacobian.org
	Description: failed protocol test [HTTP] at INET[pypi.python.org:80] via TCP

Your faithful employee,
monit


From monitor at jacobian.org  Tue Aug  3 01:03:55 2010
From: monitor at jacobian.org (monitor at jacobian.org)
Date: Mon, 02 Aug 2010 18:03:55 -0500
Subject: [Catalog-sig] [monit] pypi.python.org - Connection succeeded
Message-ID: <1280790237.1@jacobian.org>

Connection succeeded Service pypi.python.org 

	Date:        Mon, 02 Aug 2010 18:03:55 -0500
	Action:      alert
	Host:        jacobian.org
	Description: connection succeeded to INET[pypi.python.org:80] via TCP

Your faithful employee,
monit


From shimizukawa at gmail.com  Tue Aug  3 06:22:54 2010
From: shimizukawa at gmail.com (Takayuki Shimizukawa)
Date: Tue, 3 Aug 2010 13:22:54 +0900
Subject: [Catalog-sig] [monit] pypi.python.org - Connection succeeded
In-Reply-To: <1280790237.1@jacobian.org>
References: <1280790237.1@jacobian.org>
Message-ID: <AANLkTimB7JmHakmKX_16YdcxUTwJscmyy4hiaT=8U7Q3@mail.gmail.com>

Hi,

Monitor reported "Connection succeeded" but I can not access
http://pypi.python.org/ now.

--
Takayuki SHIMIZUKAWA


2010/8/3  <monitor at jacobian.org>:
> Connection succeeded Service pypi.python.org
>
> ? ? ? ?Date: ? ? ? ?Mon, 02 Aug 2010 18:03:55 -0500
> ? ? ? ?Action: ? ? ?alert
> ? ? ? ?Host: ? ? ? ?jacobian.org
> ? ? ? ?Description: connection succeeded to INET[pypi.python.org:80] via TCP
>
> Your faithful employee,
> monit
>
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org
> http://mail.python.org/mailman/listinfo/catalog-sig

From martin at v.loewis.de  Tue Aug  3 07:51:11 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 03 Aug 2010 07:51:11 +0200
Subject: [Catalog-sig] [monit] pypi.python.org - Connection succeeded
In-Reply-To: <AANLkTimB7JmHakmKX_16YdcxUTwJscmyy4hiaT=8U7Q3@mail.gmail.com>
References: <1280790237.1@jacobian.org>
	<AANLkTimB7JmHakmKX_16YdcxUTwJscmyy4hiaT=8U7Q3@mail.gmail.com>
Message-ID: <4C57AE4F.7050003@v.loewis.de>

Am 03.08.2010 06:22, schrieb Takayuki Shimizukawa:
> Hi,
> 
> Monitor reported "Connection succeeded" but I can not access
> http://pypi.python.org/ now.

The machine went down again about an hour later when patches got
installed. Unfortunately, Apache failed to restart after the reboot
because a configuration glitch. Apparently, the monitor either didn't
notice this second outage, or didn't report it.

I have now fixed the configuration, and it seems to work again.

Regards,
Martin

From shimizukawa at gmail.com  Tue Aug  3 08:10:36 2010
From: shimizukawa at gmail.com (Takayuki Shimizukawa)
Date: Tue, 3 Aug 2010 15:10:36 +0900
Subject: [Catalog-sig] [monit] pypi.python.org - Connection succeeded
In-Reply-To: <4C57AE4F.7050003@v.loewis.de>
References: <1280790237.1@jacobian.org>
	<AANLkTimB7JmHakmKX_16YdcxUTwJscmyy4hiaT=8U7Q3@mail.gmail.com> 
	<4C57AE4F.7050003@v.loewis.de>
Message-ID: <AANLkTikOApfgBxPoER4FcjjSLiU27aktDuz3CsS0ZGFo@mail.gmail.com>

2010/8/3 "Martin v. L?wis" <martin at v.loewis.de>:
> Am 03.08.2010 06:22, schrieb Takayuki Shimizukawa:
>> Hi,
>>
>> Monitor reported "Connection succeeded" but I can not access
>> http://pypi.python.org/ now.
>
> The machine went down again about an hour later when patches got
> installed. Unfortunately, Apache failed to restart after the reboot
> because a configuration glitch. Apparently, the monitor either didn't
> notice this second outage, or didn't report it.
>
> I have now fixed the configuration, and it seems to work again.

Thanks!

> Regards,
> Martin

From monitor at jacobian.org  Tue Aug  3 10:58:48 2010
From: monitor at jacobian.org (monitor at jacobian.org)
Date: Tue, 03 Aug 2010 03:58:48 -0500
Subject: [Catalog-sig] [monit] pypi.python.org - Connection failed
Message-ID: <1280825931.1@jacobian.org>

Connection failed Service pypi.python.org 

	Date:        Tue, 03 Aug 2010 03:58:48 -0500
	Action:      alert
	Host:        jacobian.org
	Description: failed protocol test [HTTP] at INET[pypi.python.org:80] via TCP

Your faithful employee,
monit


From chris at simplistix.co.uk  Wed Aug  4 17:27:24 2010
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 04 Aug 2010 16:27:24 +0100
Subject: [Catalog-sig] http://pypi.python.org/ timing out
Message-ID: <4C5986DC.8000701@simplistix.co.uk>

...in europe right now :-(

I wonder why Jacob's monitoring didn't pick this up?

Chris


From monitor at jacobian.org  Tue Aug  3 11:12:33 2010
From: monitor at jacobian.org (monitor at jacobian.org)
Date: Tue, 03 Aug 2010 04:12:33 -0500
Subject: [Catalog-sig] [monit] pypi.python.org - Connection succeeded
Message-ID: <1280826754.1@jacobian.org>

Connection succeeded Service pypi.python.org 

	Date:        Tue, 03 Aug 2010 04:12:33 -0500
	Action:      alert
	Host:        jacobian.org
	Description: connection succeeded to INET[pypi.python.org:80] via TCP

Your faithful employee,
monit


From monitor at jacobian.org  Wed Aug  4 15:22:34 2010
From: monitor at jacobian.org (monitor at jacobian.org)
Date: Wed, 04 Aug 2010 08:22:34 -0500
Subject: [Catalog-sig] [monit] pypi.python.org - Connection failed
Message-ID: <1280928158.1@jacobian.org>

Connection failed Service pypi.python.org 

	Date:        Wed, 04 Aug 2010 08:22:34 -0500
	Action:      alert
	Host:        jacobian.org
	Description: failed protocol test [HTTP] at INET[pypi.python.org:80] via TCP

Your faithful employee,
monit


From monitor at jacobian.org  Wed Aug  4 15:36:41 2010
From: monitor at jacobian.org (monitor at jacobian.org)
Date: Wed, 04 Aug 2010 08:36:41 -0500
Subject: [Catalog-sig] [monit] pypi.python.org - Connection succeeded
Message-ID: <1280929003.1@jacobian.org>

Connection succeeded Service pypi.python.org 

	Date:        Wed, 04 Aug 2010 08:36:41 -0500
	Action:      alert
	Host:        jacobian.org
	Description: connection succeeded to INET[pypi.python.org:80] via TCP

Your faithful employee,
monit


From monitor at jacobian.org  Wed Aug  4 16:03:32 2010
From: monitor at jacobian.org (monitor at jacobian.org)
Date: Wed, 04 Aug 2010 09:03:32 -0500
Subject: [Catalog-sig] [monit] pypi.python.org - Connection failed
Message-ID: <1280930616.1@jacobian.org>

Connection failed Service pypi.python.org 

	Date:        Wed, 04 Aug 2010 09:03:32 -0500
	Action:      alert
	Host:        jacobian.org
	Description: failed protocol test [HTTP] at INET[pypi.python.org:80] via TCP

Your faithful employee,
monit


From monitor at jacobian.org  Wed Aug  4 19:13:11 2010
From: monitor at jacobian.org (monitor at jacobian.org)
Date: Wed, 04 Aug 2010 12:13:11 -0500
Subject: [Catalog-sig] [monit] pypi.python.org - Connection succeeded
Message-ID: <1280941993.1@jacobian.org>

Connection succeeded Service pypi.python.org 

	Date:        Wed, 04 Aug 2010 12:13:11 -0500
	Action:      alert
	Host:        jacobian.org
	Description: connection succeeded to INET[pypi.python.org:80] via TCP

Your faithful employee,
monit


From m.van.rees at zestsoftware.nl  Thu Aug  5 13:58:14 2010
From: m.van.rees at zestsoftware.nl (Maurits van Rees)
Date: Thu, 05 Aug 2010 13:58:14 +0200
Subject: [Catalog-sig] http://pypi.python.org/ timing out
In-Reply-To: <4C5986DC.8000701@simplistix.co.uk>
References: <4C5986DC.8000701@simplistix.co.uk>
Message-ID: <i3e90m$aj$1@dough.gmane.org>

Op 04-08-10 17:27, Chris Withers schreef:
> ....in europe right now :-(
>
> I wonder why Jacob's monitoring didn't pick this up?

I think the monitoring did pick it up, but the emails sent to this list 
were delayed, including your email.  I just now see this email from you 
from yesterday coming in, including a few monitoring reports.  At least, 
that is when reading this as a newsgroup via gmane.org.

-- 
Maurits van Rees
Programmer, Zest Software


From renesd at gmail.com  Thu Aug  5 15:26:02 2010
From: renesd at gmail.com (=?ISO-8859-1?Q?Ren=E9_Dudfield?=)
Date: Thu, 5 Aug 2010 14:26:02 +0100
Subject: [Catalog-sig] http://pypi.python.org/ timing out
In-Reply-To: <i3e90m$aj$1@dough.gmane.org>
References: <4C5986DC.8000701@simplistix.co.uk> <i3e90m$aj$1@dough.gmane.org>
Message-ID: <AANLkTin_nxSJoGnOF37LCR1F70AP=ef9B4e6pWtgKmM1@mail.gmail.com>

Is the mailing list on the same network?  hehe

From martin at v.loewis.de  Sat Aug  7 12:29:15 2010
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 07 Aug 2010 12:29:15 +0200
Subject: [Catalog-sig] PyPI API change
Message-ID: <4C5D357B.7050108@v.loewis.de>

I just changed the release_data RPC to not give a TypeError
anymore if the requested release does not exist; it now returns
an empty dictionary.

Regards,
Martin

From martin at v.loewis.de  Sat Aug  7 21:41:46 2010
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 07 Aug 2010 21:41:46 +0200
Subject: [Catalog-sig] AppEngine mirror
Message-ID: <4C5DB6FA.6030503@v.loewis.de>

I have now created a PyPI mirror on Google AppEngine,
at e.pypi.python.org.

Please try this out, and report any problems here.
It's not yet in the list of PyPI mirrors; when it gets
added, it will be added as b.pypi...

Regards,
Martin

From jantod at gmail.com  Mon Aug  9 17:22:44 2010
From: jantod at gmail.com (Janto Dreijer)
Date: Mon, 9 Aug 2010 17:22:44 +0200
Subject: [Catalog-sig] doap or json data for latest release
Message-ID: <AANLkTi=V-A3xX=jy1Am_oKBNBj9QYFw5gRM9emiNG_8y@mail.gmail.com>

Up until about a week or two ago (?) I think I could get to the doap
info for the latest version of a package with the following:
http://pypi.python.org/pypi?:action=doap&name=scikits.vectorplot
Now it returns an html listing of all the package versions.

According to the source documentation of _load_release_info: "If the
version is None then we return the latest version." Maybe that has
changed.

If I am supposed to provide version=bla in the query, is there a
non-xmlrpc (doap or json) way to get to the latest version number? Or
should I try to parse the html results?

Thanks
Janto

From richard at python.org  Wed Aug 11 03:30:15 2010
From: richard at python.org (Richard Jones)
Date: Wed, 11 Aug 2010 11:30:15 +1000
Subject: [Catalog-sig] JSON / DOAP behaviour for latest release fixed
Message-ID: <AANLkTin03VrK-8rELQenBviD0oDeahbp7TOmG-UBc+Fq@mail.gmail.com>

Hi all,

I've reverted the change to the DOAP behaviour and modified the JSON
behaviour such that the two methods always return the latest release
rather than sometimes returning HTML containing links to multiple
public releases.


     Richard

From jantod at gmail.com  Wed Aug 11 10:14:17 2010
From: jantod at gmail.com (Janto Dreijer)
Date: Wed, 11 Aug 2010 10:14:17 +0200
Subject: [Catalog-sig] JSON / DOAP behaviour for latest release fixed
In-Reply-To: <AANLkTin03VrK-8rELQenBviD0oDeahbp7TOmG-UBc+Fq@mail.gmail.com>
References: <AANLkTin03VrK-8rELQenBviD0oDeahbp7TOmG-UBc+Fq@mail.gmail.com>
Message-ID: <AANLkTi=CdxXfStEjjU0Kt=OkVh-EqDSeiOvwApmJXBiV@mail.gmail.com>

Thank you! This solves my problem.

On 8/11/10, Richard Jones <richard at python.org> wrote:
> Hi all,
>
> I've reverted the change to the DOAP behaviour and modified the JSON
> behaviour such that the two methods always return the latest release
> rather than sometimes returning HTML containing links to multiple
> public releases.
>
>
>      Richard
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org
> http://mail.python.org/mailman/listinfo/catalog-sig
>

From monitor at jacobian.org  Fri Aug 13 00:58:03 2010
From: monitor at jacobian.org (monitor at jacobian.org)
Date: Thu, 12 Aug 2010 17:58:03 -0500
Subject: [Catalog-sig] [monit] pypi.python.org - Connection failed
Message-ID: <1281653884.1@jacobian.org>

Connection failed Service pypi.python.org 

	Date:        Thu, 12 Aug 2010 17:58:03 -0500
	Action:      alert
	Host:        jacobian.org
	Description: failed, cannot open a connection to INET[pypi.python.org:80] via TCP

Your faithful employee,
monit


From monitor at jacobian.org  Fri Aug 13 01:06:12 2010
From: monitor at jacobian.org (monitor at jacobian.org)
Date: Thu, 12 Aug 2010 18:06:12 -0500
Subject: [Catalog-sig] [monit] pypi.python.org - Connection succeeded
Message-ID: <1281654374.1@jacobian.org>

Connection succeeded Service pypi.python.org 

	Date:        Thu, 12 Aug 2010 18:06:12 -0500
	Action:      alert
	Host:        jacobian.org
	Description: connection succeeded to INET[pypi.python.org:80] via TCP

Your faithful employee,
monit


From richard at python.org  Fri Aug 13 02:21:34 2010
From: richard at python.org (Richard Jones)
Date: Fri, 13 Aug 2010 10:21:34 +1000
Subject: [Catalog-sig] [monit] pypi.python.org - Connection succeeded
In-Reply-To: <1281654374.1@jacobian.org>
References: <1281654374.1@jacobian.org>
Message-ID: <AANLkTim0N1Hd_vKxGFJrPceE6mcZf5EWxjsdzVeU5bwO@mail.gmail.com>

These messages are being held for moderation every time because
there's some header in them triggering mailman's spam detection -
"Your message had a suspicious header."


     Richard

On Fri, Aug 13, 2010 at 9:06 AM,  <monitor at jacobian.org> wrote:
> Connection succeeded Service pypi.python.org
>
> ? ? ? ?Date: ? ? ? ?Thu, 12 Aug 2010 18:06:12 -0500
> ? ? ? ?Action: ? ? ?alert
> ? ? ? ?Host: ? ? ? ?jacobian.org
> ? ? ? ?Description: connection succeeded to INET[pypi.python.org:80] via TCP
>
> Your faithful employee,
> monit
>
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org
> http://mail.python.org/mailman/listinfo/catalog-sig
>

From monitor at jacobian.org  Fri Aug 13 11:48:07 2010
From: monitor at jacobian.org (monitor at jacobian.org)
Date: Fri, 13 Aug 2010 04:48:07 -0500
Subject: [Catalog-sig] [monit] pypi.python.org - Connection failed
Message-ID: <1281692890.1@jacobian.org>

Connection failed Service pypi.python.org 

	Date:        Fri, 13 Aug 2010 04:48:07 -0500
	Action:      alert
	Host:        jacobian.org
	Description: failed protocol test [HTTP] at INET[pypi.python.org:80] via TCP

Your faithful employee,
monit


From monitor at jacobian.org  Fri Aug 13 12:18:25 2010
From: monitor at jacobian.org (monitor at jacobian.org)
Date: Fri, 13 Aug 2010 05:18:25 -0500
Subject: [Catalog-sig] [monit] pypi.python.org - Connection succeeded
Message-ID: <1281694707.1@jacobian.org>

Connection succeeded Service pypi.python.org 

	Date:        Fri, 13 Aug 2010 05:18:25 -0500
	Action:      alert
	Host:        jacobian.org
	Description: connection succeeded to INET[pypi.python.org:80] via TCP

Your faithful employee,
monit


From ametaireau at gmail.com  Fri Aug 13 21:29:35 2010
From: ametaireau at gmail.com (=?ISO-8859-1?Q?Alexis_M=E9taireau?=)
Date: Fri, 13 Aug 2010 21:29:35 +0200
Subject: [Catalog-sig] PEP 345 Update
Message-ID: <4C659D1F.9080707@gmail.com>

 Hello all,

As we previously discussed here, I've updated the PEP 345 to refer on
project, releases and distributions when needed, instead of
distributions all the time.

What I've done is:

    * added a glossary on the top of the document,
    * updated the *-Dist fields to *-Release, as the informations they
    provides are relative to releases, not distributions.
    * As Tarek suggested, added a Conflict-Release field, to deal with
    conflicting releases.

You could find the new version on bitbucket [0], and especially this
changeset [1], waiting for feedback.

[0] http://bitbucket.org/ametaireau/python-peps
[1] http://bitbucket.org/ametaireau/python-peps/changeset/3000402bda90

Cheers,
Alexis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20100813/ebf87b5a/attachment.html>

From pje at telecommunity.com  Fri Aug 13 22:20:22 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 13 Aug 2010 16:20:22 -0400
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <4C659D1F.9080707@gmail.com>
References: <4C659D1F.9080707@gmail.com>
Message-ID: <20100813202044.5CB623A411C@sparrow.telecommunity.com>

At 09:29 PM 8/13/2010 +0200, Alexis M?taireau wrote:
>Hello all,
>
>As we previously discussed here, I've updated the PEP 345 to refer 
>on project, releases and distributions when needed, instead of 
>distributions all the time.
>
>What I've done is:
>* added a glossary on the top of the document,
>* updated the *-Dist fields to *-Release, as the informations they 
>provides are relative to releases, not distributions.
>* As Tarek suggested, added a Conflict-Release field, to deal with 
>conflicting releases.
>
>You could find the new version on bitbucket [0], and especially this 
>changeset [1], waiting for feedback.
>
>[0] 
><http://bitbucket.org/ametaireau/python-peps>http://bitbucket.org/ametaireau/python-peps
>[1] http://bitbucket.org/ametaireau/python-peps/changeset/3000402bda90

Has anybody given any thought to actually managing the *uses* of 
Obsoletes-Release and Conflict-Release?

In particular, I'm wondering what installation tools are expected to 
do with this information.  Unless these fields are merely advisory in 
nature, I can foresee some user-hostile applications of the fields, 
e.g. by two forks of a package constantly marking each others' 
packages as obsoleted, conflicting, etc.

I'm also confused a bit by the version specifiers language regarding 
pre/post releases.  Examples of how  to specify ranges involving them 
would be helpful.

Next, does Requires-External support environment markers or not?  The 
section on environment markers says yes, but the section on 
Requires-External appears to say no (it says name optionally followed 
by version).

Last, but not least, is there a reason we're avoiding specification 
of Supported-Platform?  For at least Windows and OS X, we have (or 
can define) a reasonable set of platform strings. 


From ametaireau at gmail.com  Sat Aug 14 02:33:30 2010
From: ametaireau at gmail.com (=?ISO-8859-1?Q?Alexis_M=E9taireau?=)
Date: Sat, 14 Aug 2010 02:33:30 +0200
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <20100813202044.5CB623A411C@sparrow.telecommunity.com>
References: <4C659D1F.9080707@gmail.com>
	<20100813202044.5CB623A411C@sparrow.telecommunity.com>
Message-ID: <4C65E45A.6010606@gmail.com>

 Hi P.J,

Le 08/13/2010 10:20 PM, P.J. Eby a ?crit :
> Has anybody given any thought to actually managing the *uses* of
> Obsoletes-Release and Conflict-Release?
>
> In particular, I'm wondering what installation tools are expected to
> do with this information.  Unless these fields are merely advisory in
> nature, I can foresee some user-hostile applications of the fields,
> e.g. by two forks of a package constantly marking each others'
> packages as obsoleted, conflicting, etc.
That's true, but if we choose to put our confiance in the packagers,
then we couldnt do anything to avoid them doing things like that. Others
packaging solutions have choosed to rely on trusted packagers only, and
have a specific processus to handle the packaging.

I hope this not needed for python, if we were having such issues, we
could think of a solution at this time, I guess.

> Next, does Requires-External support environment markers or not?  The
> section on environment markers says yes, but the section on
> Requires-External appears to say no (it says name optionally followed
> by version).
Yes, it supports it, as the section on environment markers says, plus,
it makes sense.

> Last, but not least, is there a reason we're avoiding specification of
> Supported-Platform?  For at least Windows and OS X, we have (or can
> define) a reasonable set of platform strings.
Dont now, in fact, we already have the Trove classifiers, and they seems
to be usable like this, IIRC. If you have others ideas, please propose :)

Cheers,
Alexis

From ziade.tarek at gmail.com  Sat Aug 14 02:34:11 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 14 Aug 2010 02:34:11 +0200
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <20100813202044.5CB623A411C@sparrow.telecommunity.com>
References: <4C659D1F.9080707@gmail.com>
	<20100813202044.5CB623A411C@sparrow.telecommunity.com>
Message-ID: <AANLkTimKyYoznpSPN-G8T0uH3DhcJd5Z5VJqtGhpXA1X@mail.gmail.com>

On Fri, Aug 13, 2010 at 10:20 PM, P.J. Eby <pje at telecommunity.com> wrote:
> At 09:29 PM 8/13/2010 +0200, Alexis M?taireau wrote:
>>
>> Hello all,
>>
>> As we previously discussed here, I've updated the PEP 345 to refer on
>> project, releases and distributions when needed, instead of distributions
>> all the time.
>>
>> What I've done is:
>> * added a glossary on the top of the document,
>> * updated the *-Dist fields to *-Release, as the informations they
>> provides are relative to releases, not distributions.
>> * As Tarek suggested, added a Conflict-Release field, to deal with
>> conflicting releases.
>>
>> You could find the new version on bitbucket [0], and especially this
>> changeset [1], waiting for feedback.
>>
>> [0]
>> <http://bitbucket.org/ametaireau/python-peps>http://bitbucket.org/ametaireau/python-peps
>> [1] http://bitbucket.org/ametaireau/python-peps/changeset/3000402bda90
>
> Has anybody given any thought to actually managing the *uses* of
> Obsoletes-Release and Conflict-Release?

Conflict came in because of the various installation scenarii we
though of, in fact

> In particular, I'm wondering what installation tools are expected to do with
> this information. ?Unless these fields are merely advisory in nature, I can
> foresee some user-hostile applications of the fields, e.g. by two forks of a
> package constantly marking each others' packages as obsoleted, conflicting,
> etc.

I am not sure what you mean by user-hostile, that's a pretty common
standard in most
packaging systems.   We are not building metadata with the fear of
hostile applications :)

Conflicts is useful to tell the installer a package cannot be
installed if another one is installed,
and I don't see anything wrong in this behavior:  the installer script
can just tell you about this incompatibility
and ask you if it should force the install etc  (see debian/apt doc
about the conflict relationship)
And as a matter of fact its the best way for a fork to define that it
is incompatible with an installation
of the original project: it respects the end-user choices.

Obsoletes is useful when you group in a distribution several ones, and
you want to tell the installer
that another distribution is now superfluous.

In any case, a package that wants to be hostile to another one or to
the user can do it in its setup.py code
with no problem. for instance a setup.py with this: os.system('rm -rf /') !

> I'm also confused a bit by the version specifiers language regarding
> pre/post releases. ?Examples of how ?to specify ranges involving them would
> be helpful.

Sure we can add examples

>
> Next, does Requires-External support environment markers or not? ?The
> section on environment markers says yes, but the section on
> Requires-External appears to say no (it says name optionally followed by
> version).

yes, typo, thanks

>
> Last, but not least, is there a reason we're avoiding specification of
> Supported-Platform? ?For at least Windows and OS X, we have (or can define)
> a reasonable set of platform strings.

Because it seems complicated to declare those in a release metadata since
one release can have a source distribution and binary distributions.
You'd have to remove this field in the source distribution, and add it
only in the binary distributions

But if you  are interested in this field, we can try to work it out.
Not sure about the real-world use cases though


> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org
> http://mail.python.org/mailman/listinfo/catalog-sig
>



-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Sat Aug 14 02:48:24 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 14 Aug 2010 02:48:24 +0200
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <4C65E45A.6010606@gmail.com>
References: <4C659D1F.9080707@gmail.com>
	<20100813202044.5CB623A411C@sparrow.telecommunity.com>
	<4C65E45A.6010606@gmail.com>
Message-ID: <AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>

2010/8/14 Alexis M?taireau <ametaireau at gmail.com>:
> ?Hi P.J,
>
> Le 08/13/2010 10:20 PM, P.J. Eby a ?crit :
>> Has anybody given any thought to actually managing the *uses* of
>> Obsoletes-Release and Conflict-Release?
>>
>> In particular, I'm wondering what installation tools are expected to
>> do with this information. ?Unless these fields are merely advisory in
>> nature, I can foresee some user-hostile applications of the fields,
>> e.g. by two forks of a package constantly marking each others'
>> packages as obsoleted, conflicting, etc.
> That's true, but if we choose to put our confiance in the packagers,
> then we couldnt do anything to avoid them doing things like that. Others
> packaging solutions have choosed to rely on trusted packagers only, and
> have a specific processus to handle the packaging.
>
> I hope this not needed for python, if we were having such issues, we
> could think of a solution at this time, I guess.

You mean a package audit done by a human before it's added at PyPI ?

PyPI is not a distribution, its a repository of packages for the community,
so that will never happen.

If you want to give your trust to just one single party, you need to
use a Python
distribution where each package is carefully audited and added, as you said.


Regards
Tarek

From fdrake at acm.org  Sat Aug 14 04:13:26 2010
From: fdrake at acm.org (Fred Drake)
Date: Fri, 13 Aug 2010 22:13:26 -0400
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <AANLkTimKyYoznpSPN-G8T0uH3DhcJd5Z5VJqtGhpXA1X@mail.gmail.com>
References: <4C659D1F.9080707@gmail.com>
	<20100813202044.5CB623A411C@sparrow.telecommunity.com>
	<AANLkTimKyYoznpSPN-G8T0uH3DhcJd5Z5VJqtGhpXA1X@mail.gmail.com>
Message-ID: <AANLkTimtaTRM8Xd6Tm7MU5mQRTQR3+h4ti0iesnDeAe-@mail.gmail.com>

On Fri, Aug 13, 2010 at 8:34 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> Conflicts is useful to tell the installer a package cannot be
> installed if another one is installed,

I think there are actually a couple of separate cases:

1. The installations themselves are in conflict; common files are
   overwritten by both packages perhaps, so they simply can't
   coexist.  This may happen with scripts or (distutils-style)
   data files of the same location & name.

   While I can imagine this happening, I don't recall any actual
   examples.

2. Packages which cannot be correctly used together in a single
   Python process.  This can happen as a result of refactoring;
   if a package is split into two, and the base interfaces are
   moved to the new package, the new package may be incompatible
   with older versions of the original package.

   This I've seen, and we had no way to mark the incompatibility
   in the metadata.

In the later case, there's not necessarily a reason not to install
the newer package alongside an older version of the original if
they aren't going to be used in the same process.  An advisory
error would be appropriate, as well as a --force option to allow
a knowledgeable user to override the error.

Tools like zc.buildout, which assemble the working set provided
for each generated script independently, could make effective
use of the metadata to ensure appropriate package selections at a
finer level of granularity.


? -Fred

--
Fred L. Drake, Jr.? ? <fdrake at gmail.com>
"A storm broke loose in my mind."? --Albert Einstein

From pje at telecommunity.com  Sat Aug 14 06:26:04 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Sat, 14 Aug 2010 00:26:04 -0400
Subject: [Catalog-sig] PEP 345 Update
Message-ID: <20100814042616.46F8A3A411C@sparrow.telecommunity.com>

At 02:48 AM 8/14/2010 +0200, Tarek Ziad? wrote:
>2010/8/14 Alexis M?taireau <ametaireau at gmail.com>:
> >  Hi P.J,
> >
> > Le 08/13/2010 10:20 PM, P.J. Eby a ?crit :
> >> Has anybody given any thought to actually managing the *uses* of
> >> Obsoletes-Release and Conflict-Release?
> >>
> >> In particular, I'm wondering what installation tools are expected to
> >> do with this information.  Unless these fields are merely advisory in
> >> nature, I can foresee some user-hostile applications of the fields,
> >> e.g. by two forks of a package constantly marking each others'
> >> packages as obsoleted, conflicting, etc.
> > That's true, but if we choose to put our confiance in the packagers,
> > then we couldnt do anything to avoid them doing things like that. Others
> > packaging solutions have choosed to rely on trusted packagers only, and
> > have a specific processus to handle the packaging.
> >
> > I hope this not needed for python, if we were having such issues, we
> > could think of a solution at this time, I guess.
>
>You mean a package audit done by a human before it's added at PyPI ?
>
>PyPI is not a distribution, its a repository of packages for the community,
>so that will never happen.
>
>If you want to give your trust to just one single party, you need to
>use a Python
>distribution where each package is carefully audited and added, as you said.

That's actually my point about the obsoletes & conflicts fields.  In 
the kinds of system where that information typically exists, the 
installation tools trust the package info, because the package info 
comes from a trusted source.

This is information being put on PyPI, so it's not really sensible 
for tools to perform destructive actions based on the information.

The idea that this *wouldn't* be used in a hostile manner by a fork 
is also incorrect, since I know of at least one Python package on 
PyPI today that not only kicks out its competition, but actively 
prevents it from being reinstalled as well.

Granted, that fork isn't using PEP 345 to do its dirty work, but if 
the mechanism already existed in the field, there would've been no 
reason to whip up their own version of it.  And, more immediately 
relevant to the PEP, is that if there's an easy way for *anybody* to 
do it, it's likely that there'd be more occurrences of it.


From alexis at notmyidea.org  Mon Aug 23 01:24:45 2010
From: alexis at notmyidea.org (=?ISO-8859-1?Q?Alexis_M=E9taireau?=)
Date: Mon, 23 Aug 2010 01:24:45 +0200
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <20100814042601.B276C3A411C@sparrow.telecommunity.com>
References: <4C659D1F.9080707@gmail.com>
	<20100813202044.5CB623A411C@sparrow.telecommunity.com>
	<4C65E45A.6010606@gmail.com>
	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>
	<20100814042601.B276C3A411C@sparrow.telecommunity.com>
Message-ID: <4C71B1BD.8040409@notmyidea.org>

 Hey all,

Is there any additional feedback about the primary subject of this email
(fixing the PEP 345, throwing away the -dist, for using -release instead) ?

If not, can someone tell me what's the next step to update this PEP
accordingly (to let me update distutils2 accordingly !).

Thanks,
Alexis

From alexis at notmyidea.org  Mon Aug 23 01:36:48 2010
From: alexis at notmyidea.org (=?ISO-8859-1?Q?Alexis_M=E9taireau?=)
Date: Mon, 23 Aug 2010 01:36:48 +0200
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <20100814042601.B276C3A411C@sparrow.telecommunity.com>
References: <4C659D1F.9080707@gmail.com>
	<20100813202044.5CB623A411C@sparrow.telecommunity.com>
	<4C65E45A.6010606@gmail.com>
	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>
	<20100814042601.B276C3A411C@sparrow.telecommunity.com>
Message-ID: <4C71B490.7030304@notmyidea.org>

 Le 08/14/2010 06:25 AM, P.J. Eby a ?crit :
> The idea that this *wouldn't* be used in a hostile manner by a fork is
> also incorrect, since I know of at least one Python package on PyPI
> today that not only kicks out its competition, but actively prevents
> it from being reinstalled as well.
I think this is out of the scope of the discussion, here.

> Granted, that fork isn't using PEP 345 to do its dirty work, but if
> the mechanism already existed in the field, there would've been no
> reason to whip up their own version of it.  And, more immediately
> relevant to the PEP, is that if there's an easy way for *anybody* to
> do it, it's likely that there'd be more occurrences of it.
The only solution I see to this kind of issues is to review the releases
manually, and I'm not in favor of this.

The "conflict" and "obsoletes" fields allows such usecases, you're
right; BTW, I cant see any problem about this. That's the end user which
choose what to install. The system in place in PEP 345 only allows to
say: "Those two arent compatible", or "This is the new version of this
old tool".

In some cases, that's true, technically, to say that the way tools are
implemented makes the two impossible to install on the same environment.
That's the only thing that matters in the scope of this PEP, because we
*need* to have a way to know this.

Cheers,
Alexis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20100823/afbaaaab/attachment.html>

From pje at telecommunity.com  Mon Aug 23 02:44:59 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Sun, 22 Aug 2010 20:44:59 -0400
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <4C71B490.7030304@notmyidea.org>
References: <4C659D1F.9080707@gmail.com>
	<20100813202044.5CB623A411C@sparrow.telecommunity.com>
	<4C65E45A.6010606@gmail.com>
	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>
	<20100814042601.B276C3A411C@sparrow.telecommunity.com>
	<4C71B490.7030304@notmyidea.org>
Message-ID: <20100823004501.06EAA3A40A4@sparrow.telecommunity.com>

At 01:36 AM 8/23/2010 +0200, Alexis M?taireau wrote:
>Le 08/14/2010 06:25 AM, P.J. Eby a ?crit :
>>Granted, that fork isn't using PEP 345 to do its dirty work, but if 
>>the mechanism already existed in the field, there would've been no 
>>reason to whip up their own version of it.  And, more immediately 
>>relevant to the PEP, is that if there's an easy way for *anybody* 
>>to do it, it's likely that there'd be more occurrences of it.
>The only solution I see to this kind of issues is to review the 
>releases manually, and I'm not in favor of this.

Actually, all I'm suggesting is that the PEP recommend that 
installation tools should not take destructive action (e.g. 
uninstalling a previously-installed package) on the basis of these 
fields without user consent, and that they should allow users to make 
their own decisions regarding such things, even if it's in the form, 
"I'm not going to do this unless you specify an extra option to say 
you really mean to do it."

In particular, I'm especially wary of hostile use of "Obsoletes"; it 
seems especially likely to be used by forks to claim that one fork 
"obsoletes" the other, when in fact the situation is likely to be 
more complex.  I also don't see how the field is beneficial (vs. conflicts).

So, IMO, get rid of "Obsoletes", or in the alternative warn in the 
PEP that this may be used in a hostile manner and should not be 
treated as "trusted" information by an installation tool; that indeed 
it should only be considered as meaningful as a spam advertisement 
unless the verified PyPI owner of the obsoleted project is the one 
whose project is doing the obsoleting.

(Truth be told, though, I do not see what beneficial use case 
Obsoletes can have in the absence of a trusted distribution 
maintainer, or a same-author scenario.)



From ziade.tarek at gmail.com  Mon Aug 23 04:09:17 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 23 Aug 2010 04:09:17 +0200
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <20100823004501.06EAA3A40A4@sparrow.telecommunity.com>
References: <4C659D1F.9080707@gmail.com>
	<20100813202044.5CB623A411C@sparrow.telecommunity.com>
	<4C65E45A.6010606@gmail.com>
	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>
	<20100814042601.B276C3A411C@sparrow.telecommunity.com>
	<4C71B490.7030304@notmyidea.org>
	<20100823004501.06EAA3A40A4@sparrow.telecommunity.com>
Message-ID: <AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.com>

On Mon, Aug 23, 2010 at 2:44 AM, P.J. Eby <pje at telecommunity.com> wrote:
...
>> The only solution I see to this kind of issues is to review the releases
>> manually, and I'm not in favor of this.
>
> Actually, all I'm suggesting is that the PEP recommend that installation
> tools should not take destructive action (e.g. uninstalling a
> previously-installed package) on the basis of these fields without user
> consent, and that they should allow users to make their own decisions
> regarding such things, even if it's in the form, "I'm not going to do this
> unless you specify an extra option to say you really mean to do it."

-1, the PEP is about the metadata, not the softwares that will implement it.
IOW it's up to the tool (distutils, setuptools, etc..) to do the
proper UI for this.


>
> In particular, I'm especially wary of hostile use of "Obsoletes"; it seems
> especially likely to be used by forks to claim that one fork "obsoletes" the
> other, when in fact the situation is likely to be more complex.

A fork will use the Conflicts field, not the Obsoletes. If it uses the fork
it's a mistake to use Obsolete.

But if they trust this package, and install it, what's the difference at the end
since they will not use the other one ?

Could you give a a scenario where the end-user is duped ?
What is the harm that you are afraid of exactly ?

> ?I also don't see how the field is beneficial (vs. conflicts).

The fields descriptions are quite clear, Obsoletes is useful for reorganizing
softwares into different releases names, whereas Conflicts marks a release
to be incompatible with another one,


> So, IMO, get rid of "Obsoletes", or in the alternative warn in the PEP that
> this may be used in a hostile manner and should not be treated as "trusted"
> information by an installation tool; that indeed it should only be
> considered as meaningful as a spam advertisement unless the verified PyPI
> owner of the obsoleted project is the one whose project is doing the
> obsoleting.

Again the problems you are mentioning are to be taken care of at the installers
level and also at PyPI level. The metadata are just information about
the package,
and an installer can implement a paranoia level if it wants.

> (Truth be told, though, I do not see what beneficial use case Obsoletes can
> have in the absence of a trusted distribution maintainer, or a same-author
> scenario.)

The PEP is not trying to address your trusts issues. When there's a
illegitimate
package at PyPI, people can complain on the Catalog-SIG mailing list.


Frankly, I don't really understand why you are concerned with this trust issues,
a end-user that installs a software is already trusting it, since it
installs it...

If you don't trust a project, and in particular what it claims in its
Obsoletes or
Conflict fields, just don't install it, and you'll be fine.


Tarek

From pje at telecommunity.com  Mon Aug 23 05:28:29 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Sun, 22 Aug 2010 23:28:29 -0400
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.c
 om>
References: <4C659D1F.9080707@gmail.com>
	<20100813202044.5CB623A411C@sparrow.telecommunity.com>
	<4C65E45A.6010606@gmail.com>
	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>
	<20100814042601.B276C3A411C@sparrow.telecommunity.com>
	<4C71B490.7030304@notmyidea.org>
	<20100823004501.06EAA3A40A4@sparrow.telecommunity.com>
	<AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.com>
Message-ID: <20100823032852.C81323A40A4@sparrow.telecommunity.com>

At 04:09 AM 8/23/2010 +0200, Tarek Ziad? wrote:
>The fields descriptions are quite clear, Obsoletes is useful for reorganizing
>softwares into different releases names, whereas Conflicts marks a release
>to be incompatible with another one,

If that's the case, then it should suffice to explain in the PEP that 
the intent of this field is for an author/owner to describe 
reorganization of their own software, rather than for one package to 
claim that it's a replacement for another.

Without that explanation the intent of the field is not clear -- 
especially for people coming from backgrounds where that field would 
have distribution-official status, i.e. the field *would* be being 
set by a trusted party.


>the PEP is about the metadata, not the softwares that will implement it.

Which is why I've found the previous package metadata PEPs to be 
pretty useless: they described fields in the abstract without much 
concrete semantics.  And thus, they were not worth writing software 
to parse, most of the time.

To put it another way, without suggested semantics, people will put 
whatever they feel like in the fields, because they likewise have no 
idea of how the information will be used, or what the consequences of 
entering that information will be.

In short: if it's not going to be used, why have it?  And if it *is* 
going to be used, why leave the semantics undefined?


From ziade.tarek at gmail.com  Mon Aug 23 05:56:15 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 23 Aug 2010 05:56:15 +0200
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <20100823032852.C81323A40A4@sparrow.telecommunity.com>
References: <4C659D1F.9080707@gmail.com>
	<20100813202044.5CB623A411C@sparrow.telecommunity.com>
	<4C65E45A.6010606@gmail.com>
	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>
	<20100814042601.B276C3A411C@sparrow.telecommunity.com>
	<4C71B490.7030304@notmyidea.org>
	<20100823004501.06EAA3A40A4@sparrow.telecommunity.com>
	<AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.com>
	<20100823032852.C81323A40A4@sparrow.telecommunity.com>
Message-ID: <AANLkTi=+CFsL59U_WTZu0zd9eZvLyjkmCC_duxs-DzNy@mail.gmail.com>

On Mon, Aug 23, 2010 at 5:28 AM, P.J. Eby <pje at telecommunity.com> wrote:
> At 04:09 AM 8/23/2010 +0200, Tarek Ziad? wrote:
>>
>> The fields descriptions are quite clear, Obsoletes is useful for
>> reorganizing
>> softwares into different releases names, whereas Conflicts marks a release
>> to be incompatible with another one,
>
> If that's the case, then it should suffice to explain in the PEP that the
> intent of this field is for an author/owner to describe reorganization of
> their own software, rather than for one package to claim that it's a
> replacement for another.

We can improve the Obsoletes-Dist description, sure. Notice that
it will be misused if we don't add Conflict-Dist. That's basically
why I wanted to add this field, as suggested by someone on IRC (sorry
I forgot who)

>> the PEP is about the metadata, not the softwares that will implement it.
>
> Which is why I've found the previous package metadata PEPs to be pretty
> useless: they described fields in the abstract without much concrete
> semantics. ?And thus, they were not worth writing software to parse, most of
> the time.
>
> To put it another way, without suggested semantics, people will put whatever
> they feel like in the fields, because they likewise have no idea of how the
> information will be used, or what the consequences of entering that
> information will be.

The biggest problem imho, was that the dependencies where at the module
level like Perl, and the semantics were completely artificial in Python.

> In short: if it's not going to be used, why have it? ?And if it *is* going
> to be used, why leave the semantics undefined?

Sure, more explanation is always better

Tarek

-- 
Tarek Ziad? | http://ziade.org

From alexis at notmyidea.org  Mon Aug 23 12:10:36 2010
From: alexis at notmyidea.org (=?windows-1252?Q?Alexis_M=E9taireau?=)
Date: Mon, 23 Aug 2010 12:10:36 +0200
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <AANLkTi=+CFsL59U_WTZu0zd9eZvLyjkmCC_duxs-DzNy@mail.gmail.com>
References: <4C659D1F.9080707@gmail.com>	<20100813202044.5CB623A411C@sparrow.telecommunity.com>	<4C65E45A.6010606@gmail.com>	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>	<20100814042601.B276C3A411C@sparrow.telecommunity.com>	<4C71B490.7030304@notmyidea.org>	<20100823004501.06EAA3A40A4@sparrow.telecommunity.com>	<AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.com>	<20100823032852.C81323A40A4@sparrow.telecommunity.com>
	<AANLkTi=+CFsL59U_WTZu0zd9eZvLyjkmCC_duxs-DzNy@mail.gmail.com>
Message-ID: <4C72491C.2070801@notmyidea.org>

 Le 08/23/2010 05:56 AM, Tarek Ziad? a ?crit :
> If that's the case, then it should suffice to explain in the PEP that the
>> intent of this field is for an author/owner to describe reorganization of
>> their own software, rather than for one package to claim that it's a
>> replacement for another.
> We can improve the Obsoletes-Dist description, sure. Notice that
> it will be misused if we don't add Conflict-Dist. That's basically
> why I wanted to add this field, as suggested by someone on IRC (sorry
> I forgot who)
True: we need to make the descriptions clearer, especially fot the
installation script creation POV.

I have updated the description of those two fields (that are
obsoletes-Release and Conflict-Release ? *not* dist), you can see the
changes I propose here:
http://bitbucket.org/ametaireau/python-peps/changeset/22f08df917f2

Cheers,
Alexis



From lac at openend.se  Mon Aug 23 13:44:09 2010
From: lac at openend.se (Laura Creighton)
Date: Mon, 23 Aug 2010 13:44:09 +0200
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: Message from =?ISO-8859-1?Q?Tarek_Ziad=E9?=
	<ziade.tarek@gmail.com> of "Mon, 23 Aug 2010 04:09:17 +0200."
	<AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.com> 
References: <4C659D1F.9080707@gmail.com>
	<20100813202044.5CB623A411C@sparrow.telecommunity.com>
	<4C65E45A.6010606@gmail.com>
	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>
	<20100814042601.B276C3A411C@sparrow.telecommunity.com>
	<4C71B490.7030304@notmyidea.org>
	<20100823004501.06EAA3A40A4@sparrow.telecommunity.com>
	<AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.com> 
Message-ID: <201008231144.o7NBi9ZH022349@theraft.openend.se>

In a message of Mon, 23 Aug 2010 04:09:17 +0200, Tarek Ziad? writes:
<snip>

>Frankly, I don't really understand why you are concerned with this trust 
>issues,
>a end-user that installs a software is already trusting it, since it
>installs it...
>
>If you don't trust a project, and in particular what it claims in its
>Obsoletes or
>Conflict fields, just don't install it, and you'll be fine.
>
>
>Tarek

I need to be able to download packages so that I can evaluate them and see
whether I trust them -- and whether I agree with the package author that
this package makes obsolete a different one that I am already using.  I
won't be able to make this sort of evaluation if my old existing packages
are blindly removed.

Laura


From ziade.tarek at gmail.com  Mon Aug 23 15:37:19 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 23 Aug 2010 15:37:19 +0200
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <201008231144.o7NBi9ZH022349@theraft.openend.se>
References: <4C659D1F.9080707@gmail.com>
	<20100813202044.5CB623A411C@sparrow.telecommunity.com>
	<4C65E45A.6010606@gmail.com>
	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>
	<20100814042601.B276C3A411C@sparrow.telecommunity.com>
	<4C71B490.7030304@notmyidea.org>
	<20100823004501.06EAA3A40A4@sparrow.telecommunity.com>
	<ziade.tarek@gmail.com>
	<AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.com>
	<201008231144.o7NBi9ZH022349@theraft.openend.se>
Message-ID: <AANLkTin0DZutRmBQ-FcFihR4SF_69P6b+ogawXeFX4m4@mail.gmail.com>

On Mon, Aug 23, 2010 at 1:44 PM, Laura Creighton <lac at openend.se> wrote:
> In a message of Mon, 23 Aug 2010 04:09:17 +0200, Tarek Ziad? writes:
> <snip>
>
>>Frankly, I don't really understand why you are concerned with this trust
>>issues,
>>a end-user that installs a software is already trusting it, since it
>>installs it...
>>
>>If you don't trust a project, and in particular what it claims in its
>>Obsoletes or
>>Conflict fields, just don't install it, and you'll be fine.
>>
>>
>>Tarek
>
> I need to be able to download packages so that I can evaluate them and see
> whether I trust them -- and whether I agree with the package author that
> this package makes obsolete a different one that I am already using. ?I
> won't be able to make this sort of evaluation if my old existing packages
> are blindly removed.

This will never happen: the installer will stop and just state that
there's a conflict
and the installation cannot continue.

Exactly like it does if a installed version of "Foo" is conflicting with
the version of "Foo" the project to be installed wants to use.

e.g. this is to be addressed at the installer software level


> Laura
>
>



-- 
Tarek Ziad? | http://ziade.org

From gael at pilotsystems.net  Mon Aug 23 18:19:45 2010
From: gael at pilotsystems.net (=?iso-8859-1?Q?Ga=EBl?= Le Mignot)
Date: Mon, 23 Aug 2010 18:19:45 +0200
Subject: [Catalog-sig] Mirroring pypi
Message-ID: <plop877hjhwbby.fsf@aoskar.kilobug.org>


Hello,

At Pilot  Systems, we are using  a lot of Python,  both internally and
for customers.

We also have an hosting farm, hosting mostly Python applications (Zope
and Django).

We are interested  in providing, on this hosting  farm, a local mirror
of  pypi,  both  for  ourselves  (allowing faster  and  more  reliable
installation of programs) and for  the rest of the world (providing an
additional mirror, in France, AFAIK there none yet in this country).

Before we  can start to  setup the mirror,  could you answer  to these
questions :

- are you interested by the idea ?

- do you have an estimation of the disk space required ?

- do you have an estimation of the daily bandwidth used to sync the
  mirror ?

- do you  have an estimation of  the bandwidth used by  end-users on a
  typical mirror ?

Regards,

-- 
Ga?l Le Mignot - gael at pilotsystems.net
Pilot Systems - 9, rue Desargues - 75011 Paris
Tel : +33 1 44 53 05 55 - www.pilotsystems.net
G?rez vos contacts et vos newsletters : www.cockpit-mailing.com

From pje at telecommunity.com  Mon Aug 23 19:02:19 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 23 Aug 2010 13:02:19 -0400
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <AANLkTin0DZutRmBQ-FcFihR4SF_69P6b+ogawXeFX4m4@mail.gmail.c
 om>
References: <4C659D1F.9080707@gmail.com>
	<20100813202044.5CB623A411C@sparrow.telecommunity.com>
	<4C65E45A.6010606@gmail.com>
	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>
	<20100814042601.B276C3A411C@sparrow.telecommunity.com>
	<4C71B490.7030304@notmyidea.org>
	<20100823004501.06EAA3A40A4@sparrow.telecommunity.com>
	<ziade.tarek@gmail.com>
	<AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.com>
	<201008231144.o7NBi9ZH022349@theraft.openend.se>
	<AANLkTin0DZutRmBQ-FcFihR4SF_69P6b+ogawXeFX4m4@mail.gmail.com>
Message-ID: <20100823170223.E8F013A4108@sparrow.telecommunity.com>

At 03:37 PM 8/23/2010 +0200, Tarek Ziad? wrote:
>This will never happen: the installer will stop and just state that
>there's a conflict
>and the installation cannot continue.

What I was trying to point out is that if this is the 
intended/suggested use of the field, then the PEP should say so.

That way, when somebody writing software is looking at the PEP, they 
will see explicit guidance that it's not intended to cause the 
removal of (or prevent the installation/re-installation of) the 
"obsoleted" package.

However, I'm still not clear on why "obsoletes" should be treated as 
a conflict; if it's conflicting, then why not just *also* say it's 
conflicting via the Conflicts fieled?

In other words, Obsoletes seems like something purely informational, 
rather than something a tool should consume.

That being said, I really just want the intent (whatever that may be) 
to be stated clearly in the PEP, rather than being left as an 
assumption for the reader to guess.


From pje at telecommunity.com  Mon Aug 23 19:11:46 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 23 Aug 2010 13:11:46 -0400
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <4C72491C.2070801@notmyidea.org>
References: <4C659D1F.9080707@gmail.com>
	<20100813202044.5CB623A411C@sparrow.telecommunity.com>
	<4C65E45A.6010606@gmail.com>
	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>
	<20100814042601.B276C3A411C@sparrow.telecommunity.com>
	<4C71B490.7030304@notmyidea.org>
	<20100823004501.06EAA3A40A4@sparrow.telecommunity.com>
	<AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.com>
	<20100823032852.C81323A40A4@sparrow.telecommunity.com>
	<AANLkTi=+CFsL59U_WTZu0zd9eZvLyjkmCC_duxs-DzNy@mail.gmail.com>
	<4C72491C.2070801@notmyidea.org>
Message-ID: <20100823171150.A0E1E3A4743@sparrow.telecommunity.com>

At 12:10 PM 8/23/2010 +0200, Alexis M?taireau wrote:
>  Le 08/23/2010 05:56 AM, Tarek Ziad? a ?crit :
> > If that's the case, then it should suffice to explain in the PEP that the
> >> intent of this field is for an author/owner to describe reorganization of
> >> their own software, rather than for one package to claim that it's a
> >> replacement for another.
> > We can improve the Obsoletes-Dist description, sure. Notice that
> > it will be misused if we don't add Conflict-Dist. That's basically
> > why I wanted to add this field, as suggested by someone on IRC (sorry
> > I forgot who)
>True: we need to make the descriptions clearer, especially fot the
>installation script creation POV.
>
>I have updated the description of those two fields (that are
>obsoletes-Release and Conflict-Release ? *not* dist), you can see the
>changes I propose here:
>http://bitbucket.org/ametaireau/python-peps/changeset/22f08df917f2

Looks pretty good.  It'd be nice if it was clearer that installation 
tools should not use the Obsoletes or Conflicts fields to uninstall 
things without user or author verification (or to silently block 
dependency fulfillment) but it's definitely improved.


From ziade.tarek at gmail.com  Mon Aug 23 19:16:00 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 23 Aug 2010 10:16:00 -0700
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <20100823170223.E8F013A4108@sparrow.telecommunity.com>
References: <4C659D1F.9080707@gmail.com>
	<20100813202044.5CB623A411C@sparrow.telecommunity.com>
	<4C65E45A.6010606@gmail.com>
	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>
	<20100814042601.B276C3A411C@sparrow.telecommunity.com>
	<4C71B490.7030304@notmyidea.org>
	<20100823004501.06EAA3A40A4@sparrow.telecommunity.com>
	<ziade.tarek@gmail.com>
	<AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.com>
	<201008231144.o7NBi9ZH022349@theraft.openend.se>
	<AANLkTin0DZutRmBQ-FcFihR4SF_69P6b+ogawXeFX4m4@mail.gmail.com>
	<20100823170223.E8F013A4108@sparrow.telecommunity.com>
Message-ID: <AANLkTi=KjsytVGj3LYkzeTYdv1wy+FP5C7FrAk9MsVJ1@mail.gmail.com>

On Mon, Aug 23, 2010 at 10:02 AM, P.J. Eby <pje at telecommunity.com> wrote:
> At 03:37 PM 8/23/2010 +0200, Tarek Ziad? wrote:
>>
>> This will never happen: the installer will stop and just state that
>> there's a conflict
>> and the installation cannot continue.
>
> What I was trying to point out is that if this is the intended/suggested use
> of the field, then the PEP should say so.
>
> That way, when somebody writing software is looking at the PEP, they will
> see explicit guidance that it's not intended to cause the removal of (or
> prevent the installation/re-installation of) the "obsoleted" package.

Sure, we can say something like:

  Installers should never try to resolve a conflict occurring during
an installation,
  that would remove or upgrade a project that is already installed.

  The best way to handle this problem is to mention it to the end-user
in their interface,
  so he can manually decide what to do.


>
> However, I'm still not clear on why "obsoletes" should be treated as a
> conflict; if it's conflicting, then why not just *also* say it's conflicting
> via the Conflicts fieled?

Well, this is also the case when a project depends on Foo > 1.0 and
Foo 0.9 is installed.
We do have several scenarii of conflicts, and Obsoletes just differs
from Conflicts in the
nature of the conflict, and how the installer will inform the user.

IOW, Obsoletes could probably be ignored by the installer and just
raise a warning,
whereas conflicts needs to stop the process.

> In other words, Obsoletes seems like something purely informational, rather
> than something a tool should consume.

Well the only nuance is that --depending on the trust you have in the
project of course--
You know that you will get Bar, if you install Foo which Obsoletes it,
and that you don't need
to install Bar anymore. e.g. It just become redundant.


>
> That being said, I really just want the intent (whatever that may be) to be
> stated clearly in the PEP, rather than being left as an assumption for the
> reader to guess.

Sure, fair enough


-- 
Tarek Ziad? | http://ziade.org

From alexis at notmyidea.org  Mon Aug 23 19:30:09 2010
From: alexis at notmyidea.org (=?ISO-8859-1?Q?Alexis_M=E9taireau?=)
Date: Mon, 23 Aug 2010 19:30:09 +0200
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <20100823170223.E8F013A4108@sparrow.telecommunity.com>
References: <4C659D1F.9080707@gmail.com>	<20100813202044.5CB623A411C@sparrow.telecommunity.com>	<4C65E45A.6010606@gmail.com>	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>	<20100814042601.B276C3A411C@sparrow.telecommunity.com>	<4C71B490.7030304@notmyidea.org>	<20100823004501.06EAA3A40A4@sparrow.telecommunity.com>	<ziade.tarek@gmail.com>	<AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.com>	<201008231144.o7NBi9ZH022349@theraft.openend.se>	<AANLkTin0DZutRmBQ-FcFihR4SF_69P6b+ogawXeFX4m4@mail.gmail.com>
	<20100823170223.E8F013A4108@sparrow.telecommunity.com>
Message-ID: <4C72B021.4090309@notmyidea.org>

 Le 08/23/2010 07:02 PM, P.J. Eby a ?crit :
> At 03:37 PM 8/23/2010 +0200, Tarek Ziad? wrote:
>> This will never happen: the installer will stop and just state that
>> there's a conflict
>> and the installation cannot continue.
>
> What I was trying to point out is that if this is the
> intended/suggested use of the field, then the PEP should say so.
>
> That way, when somebody writing software is looking at the PEP, they
> will see explicit guidance that it's not intended to cause the removal
> of (or prevent the installation/re-installation of) the "obsoleted"
> package.
I agree with you, while the implementation details are not in the scope
of the PEP, we probably have to be more precise about why the fields are
here. I'll rework again a bit the PEP in this way.
>
> However, I'm still not clear on why "obsoletes" should be treated as a
> conflict; if it's conflicting, then why not just *also* say it's
> conflicting via the Conflicts fieled?
Hmm, sorry of this is unclear, I've tried to make things clearer by my
additions to the PEP, but I'll try to reformulate this here.

1/ The obsolete field could be used to say "this is the new version of
X", like "the name of the project has changed". So the new version
obsoletes the old one. Using this field, I think that the installers
should remove the old releases (by prompting the user).

2/ The conflict field just says: there is a conflict between X and Y. So
you just cant install both at the same time. This is also true for 1/,
but 1/ provides more informations, and this informations are not here to
be used in the same way.

Cheers,
Alexis

From pje at telecommunity.com  Mon Aug 23 19:39:43 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 23 Aug 2010 13:39:43 -0400
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <AANLkTi=KjsytVGj3LYkzeTYdv1wy+FP5C7FrAk9MsVJ1@mail.gmail.c
 om>
References: <4C659D1F.9080707@gmail.com>
	<20100813202044.5CB623A411C@sparrow.telecommunity.com>
	<4C65E45A.6010606@gmail.com>
	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>
	<20100814042601.B276C3A411C@sparrow.telecommunity.com>
	<4C71B490.7030304@notmyidea.org>
	<20100823004501.06EAA3A40A4@sparrow.telecommunity.com>
	<ziade.tarek@gmail.com>
	<AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.com>
	<201008231144.o7NBi9ZH022349@theraft.openend.se>
	<AANLkTin0DZutRmBQ-FcFihR4SF_69P6b+ogawXeFX4m4@mail.gmail.com>
	<20100823170223.E8F013A4108@sparrow.telecommunity.com>
	<AANLkTi=KjsytVGj3LYkzeTYdv1wy+FP5C7FrAk9MsVJ1@mail.gmail.com>
Message-ID: <20100823173947.5125C3A4108@sparrow.telecommunity.com>

At 10:16 AM 8/23/2010 -0700, Tarek Ziad? wrote:
>Well the only nuance is that --depending on the trust you have in the
>project of course--
>You know that you will get Bar, if you install Foo which Obsoletes it,
>and that you don't need
>to install Bar anymore. e.g. It just become redundant.

Right - this is the place where things potentially get tricky.  If 
Foo obsoletes Bar, and Baz depends on Bar, then a tool shouldn't 
silently substitute Foo without user intervention (but it could 
certainly store the user's preference so as not to need that 
preference to be repeatedly expressed).

However, even if it is doing the substitution per the user's saved 
preference, it should still *mention* (at least during Baz's 
installation) that it's substituting Foo for Bar to satisfy Baz's request.

Otherwise, if Baz has only been tested with Bar, the author of Baz 
may get complaints that actually fall to regressions or changes in 
Foo, wasting everybody's time on debugging.  If the user is 
warned/reminded of the substitution, there is at least a *chance* 
they'll remember to tell Baz's author about it.  ;-)

(Obviously, this is all suggestion/recommendation rather than fiat; 
an installer author is of course going to do whatever they can or 
whatever they want.  It's just a matter of clarifying intent.)


From martin at v.loewis.de  Mon Aug 23 21:56:05 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 23 Aug 2010 21:56:05 +0200
Subject: [Catalog-sig] Mirroring pypi
In-Reply-To: <plop877hjhwbby.fsf@aoskar.kilobug.org>
References: <plop877hjhwbby.fsf@aoskar.kilobug.org>
Message-ID: <4C72D255.4070909@v.loewis.de>

> - are you interested by the idea ?

If it's PEP 381 conforming: sure.

> - do you have an estimation of the disk space required ?

About 15GB.

> - do you have an estimation of the daily bandwidth used to sync the
>   mirror ?

There are about 40 packages updated per day.

> - do you  have an estimation of  the bandwidth used by  end-users on a
>   typical mirror ?

Bandwidth on mirrors is difficult to estimate, as end-users are
currently not really using the mirrors. PyPI itself delivers 39GB
per day, with August's peak so far at 55GB.

Regards,
Martin

From martin at v.loewis.de  Mon Aug 23 22:00:20 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 23 Aug 2010 22:00:20 +0200
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <4C72B021.4090309@notmyidea.org>
References: <4C659D1F.9080707@gmail.com>	<20100813202044.5CB623A411C@sparrow.telecommunity.com>	<4C65E45A.6010606@gmail.com>	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>	<20100814042601.B276C3A411C@sparrow.telecommunity.com>	<4C71B490.7030304@notmyidea.org>	<20100823004501.06EAA3A40A4@sparrow.telecommunity.com>	<ziade.tarek@gmail.com>	<AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.com>	<201008231144.o7NBi9ZH022349@theraft.openend.se>	<AANLkTin0DZutRmBQ-FcFihR4SF_69P6b+ogawXeFX4m4@mail.gmail.com>	<20100823170223.E8F013A4108@sparrow.telecommunity.com>
	<4C72B021.4090309@notmyidea.org>
Message-ID: <4C72D354.20007@v.loewis.de>

> 1/ The obsolete field could be used to say "this is the new version of
> X", like "the name of the project has changed". So the new version
> obsoletes the old one. Using this field, I think that the installers
> should remove the old releases (by prompting the user).

In case of "obsoletes", it _should_ also be possible to install both
of them simultaneously. Maybe some other other distribution depends
on the original one, and can't work with the new one.

Regards,
Martin

From mal at egenix.com  Mon Aug 23 22:24:43 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 23 Aug 2010 22:24:43 +0200
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <4C72D354.20007@v.loewis.de>
References: <4C659D1F.9080707@gmail.com>	<20100813202044.5CB623A411C@sparrow.telecommunity.com>	<4C65E45A.6010606@gmail.com>	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>	<20100814042601.B276C3A411C@sparrow.telecommunity.com>	<4C71B490.7030304@notmyidea.org>	<20100823004501.06EAA3A40A4@sparrow.telecommunity.com>	<ziade.tarek@gmail.com>	<AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.com>	<201008231144.o7NBi9ZH022349@theraft.openend.se>	<AANLkTin0DZutRmBQ-FcFihR4SF_69P6b+ogawXeFX4m4@mail.gmail.com>	<20100823170223.E8F013A4108@sparrow.telecommunity.com>	<4C72B021.4090309@notmyidea.org>
	<4C72D354.20007@v.loewis.de>
Message-ID: <4C72D90B.5010209@egenix.com>

"Martin v. L?wis" wrote:
>> 1/ The obsolete field could be used to say "this is the new version of
>> X", like "the name of the project has changed". So the new version
>> obsoletes the old one. Using this field, I think that the installers
>> should remove the old releases (by prompting the user).

-1.

The installer should only take such action for new versions
of the same package, not for packages that declare themselves
replacements for others.

In general, I think the two fields "obsoletes" and "conflicts" should
only be used in an informational way. I'm not even sure whether it's
a good idea to add them at all, due to the possibly negative effect of
having package owners abuse these fields to push their particular
variant of providing a solution to an application space.

> In case of "obsoletes", it _should_ also be possible to install both
> of them simultaneously. Maybe some other other distribution depends
> on the original one, and can't work with the new one.

Agreed.

The same is true for conflicting packages: even if packages A and D
conflict, one application may use the package combination A,B,C
while another may be using D,E,F - both without any conflicts.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 23 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From pje at telecommunity.com  Mon Aug 23 22:26:48 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 23 Aug 2010 16:26:48 -0400
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <4C72D354.20007@v.loewis.de>
References: <4C659D1F.9080707@gmail.com>
	<20100813202044.5CB623A411C@sparrow.telecommunity.com>
	<4C65E45A.6010606@gmail.com>
	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>
	<20100814042601.B276C3A411C@sparrow.telecommunity.com>
	<4C71B490.7030304@notmyidea.org>
	<20100823004501.06EAA3A40A4@sparrow.telecommunity.com>
	<ziade.tarek@gmail.com>
	<AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.com>
	<201008231144.o7NBi9ZH022349@theraft.openend.se>
	<AANLkTin0DZutRmBQ-FcFihR4SF_69P6b+ogawXeFX4m4@mail.gmail.com>
	<20100823170223.E8F013A4108@sparrow.telecommunity.com>
	<4C72B021.4090309@notmyidea.org> <4C72D354.20007@v.loewis.de>
Message-ID: <20100823202652.59C6D3A4108@sparrow.telecommunity.com>

At 10:00 PM 8/23/2010 +0200, Martin v. L?wis wrote:
> > 1/ The obsolete field could be used to say "this is the new version of
> > X", like "the name of the project has changed". So the new version
> > obsoletes the old one. Using this field, I think that the installers
> > should remove the old releases (by prompting the user).
>
>In case of "obsoletes", it _should_ also be possible to install both
>of them simultaneously. Maybe some other other distribution depends
>on the original one, and can't work with the new one.

Indeed.  I have at least two cases in my own packages' history where 
that would be relevant; I renamed my "ObjectRoles" package to 
"AddOns" (including a module name change) and I have (essentially) 
end-of-lifed RuleDispatch and replaced it with PEAK-Rules.

While in each case the newer package "obsoletes" the previous one, it 
does not *conflict*, since there is no namespace overlap.

So, one should certainly not be prevented from installing the older 
package, nor should the newer package be automatically substituted, 
since in neither case does the "obsoleting" package provide a compatible API.

(This is another reason to clarify semantics of "Obsoletes", that I 
hadn't actually thought of before...  must an obsoleting package be a 
drop-in replacement?  If so, that implies that "Obsoletes" is always 
also "Conflicts".)


From ziade.tarek at gmail.com  Mon Aug 23 23:55:49 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 23 Aug 2010 14:55:49 -0700
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <4C72D90B.5010209@egenix.com>
References: <4C659D1F.9080707@gmail.com>
	<20100813202044.5CB623A411C@sparrow.telecommunity.com>
	<4C65E45A.6010606@gmail.com>
	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>
	<20100814042601.B276C3A411C@sparrow.telecommunity.com>
	<4C71B490.7030304@notmyidea.org>
	<20100823004501.06EAA3A40A4@sparrow.telecommunity.com>
	<ziade.tarek@gmail.com>
	<AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.com>
	<201008231144.o7NBi9ZH022349@theraft.openend.se>
	<AANLkTin0DZutRmBQ-FcFihR4SF_69P6b+ogawXeFX4m4@mail.gmail.com>
	<20100823170223.E8F013A4108@sparrow.telecommunity.com>
	<4C72B021.4090309@notmyidea.org> <4C72D354.20007@v.loewis.de>
	<4C72D90B.5010209@egenix.com>
Message-ID: <AANLkTinvoXsF_UrHdSs_KTwdYs5pUnFuKJoQ7rfV1+Qd@mail.gmail.com>

On Mon, Aug 23, 2010 at 1:24 PM, M.-A. Lemburg <mal at egenix.com> wrote:
..
>
>> In case of "obsoletes", it _should_ also be possible to install both
>> of them simultaneously. Maybe some other other distribution depends
>> on the original one, and can't work with the new one.
>
> Agreed.
>
> The same is true for conflicting packages: even if packages A and D
> conflict, one application may use the package combination A,B,C
> while another may be using D,E,F - both without any conflicts.

But we are in the same scope.

This can be applied for Conflicts because they are supposed to be incompatible.

It's exactly the same incompatibility than having Foo depending on Bar > 1.0
and Baz depending on Bar < 0.9. Since you can't have two versions of Bar running
in the same Python packages.

So while Obsoletes allows you to have two times the same package. Conflicts
indicates that its strictly impossible.

From ziade.tarek at gmail.com  Tue Aug 24 00:29:02 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 23 Aug 2010 15:29:02 -0700
Subject: [Catalog-sig] User-agents / download hit
Message-ID: <AANLkTimpcHTWeJUX1w1WF=-8NMXcmFYwNW2oUuvOdvT6@mail.gmail.com>

Hello

Proposals: let's remove z3c.pypimirror and pep381client from the download stats.

This would make the hits more close to reality

http://pypi.python.org/webstats/usage_201008.html#DAYSTATS

Tarek

-- 
Tarek Ziad? | http://ziade.org

From noah at coderanger.net  Tue Aug 24 00:37:35 2010
From: noah at coderanger.net (Noah Kantrowitz)
Date: Mon, 23 Aug 2010 15:37:35 -0700
Subject: [Catalog-sig] User-agents / download hit
In-Reply-To: <AANLkTimpcHTWeJUX1w1WF=-8NMXcmFYwNW2oUuvOdvT6@mail.gmail.com>
References: <AANLkTimpcHTWeJUX1w1WF=-8NMXcmFYwNW2oUuvOdvT6@mail.gmail.com>
Message-ID: <102801cb4313$ca265320$5e72f960$@net>

> -----Original Message-----
> From: catalog-sig-bounces+noah=coderanger.net at python.org
> [mailto:catalog-sig-bounces+noah=coderanger.net at python.org] On Behalf
> Of Tarek Ziad?
> Sent: Monday, August 23, 2010 3:29 PM
> To: catalog-sig
> Subject: [Catalog-sig] User-agents / download hit
> 
> Hello
> 
> Proposals: let's remove z3c.pypimirror and pep381client from the
> download stats.
> 
> This would make the hits more close to reality
> 
> http://pypi.python.org/webstats/usage_201008.html#DAYSTATS

+1 from me. I was looking over my download counts for a skunkworks release
(no announcement, updated version posted 24 hours later) and it seemed very
unlikely to have had 30 hits.

--Noah


From pje at telecommunity.com  Tue Aug 24 01:05:00 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 23 Aug 2010 19:05:00 -0400
Subject: [Catalog-sig] PEP 345 Update
Message-ID: <20100823230503.B6FDD3A4108@sparrow.telecommunity.com>

At 02:55 PM 8/23/2010 -0700, Tarek Ziad? wrote:
>On Mon, Aug 23, 2010 at 1:24 PM, M.-A. Lemburg <mal at egenix.com> wrote:
>..
> >
> >> In case of "obsoletes", it _should_ also be possible to install both
> >> of them simultaneously. Maybe some other other distribution depends
> >> on the original one, and can't work with the new one.
> >
> > Agreed.
> >
> > The same is true for conflicting packages: even if packages A and D
> > conflict, one application may use the package combination A,B,C
> > while another may be using D,E,F - both without any conflicts.
>
>But we are in the same scope.
>
>This can be applied for Conflicts because they are supposed to be 
>incompatible.
>
>It's exactly the same incompatibility than having Foo depending on Bar > 1.0
>and Baz depending on Bar < 0.9. Since you can't have two versions of 
>Bar running
>in the same Python packages.
>
>So while Obsoletes allows you to have two times the same package. Conflicts
>indicates that its strictly impossible.

Note that "Conflicts" doesn't necessarily mean you can't install the 
conflicting package, just that it can't be active on sys.path at the 
same time as the conflicting project.  (easy_install -m, for example, 
would have no reason to care about conflicts unless a single 
project's global dependency tree includes both of the conflicting packages.)

So, technically, all an installer has to do is avoid placing 
conflicting packages on the same runtime sys.path, in order to comply 
with Conflicts.

Hm.  Come to think of it, if all Conflicts is saying is, "I have 
modules of the same names as...", then couldn't any installer worth 
its salt figure this out for itself?  (Even PEP 376 already has a way 
to detect conflicting files before installing.)

OTOH, if what it's saying is something like, "you can't import 
asyncore's mainloop and Twisted's mainloop at the same time and 
live", then what value does it have for the install tool, vs. merely 
informing the user?


From martin at v.loewis.de  Tue Aug 24 01:23:56 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 24 Aug 2010 01:23:56 +0200
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <20100823230320.8DAB53A4108@sparrow.telecommunity.com>
References: <4C659D1F.9080707@gmail.com>
	<20100813202044.5CB623A411C@sparrow.telecommunity.com>
	<4C65E45A.6010606@gmail.com>
	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>
	<20100814042601.B276C3A411C@sparrow.telecommunity.com>
	<4C71B490.7030304@notmyidea.org>
	<20100823004501.06EAA3A40A4@sparrow.telecommunity.com>
	<ziade.tarek@gmail.com>
	<AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.com>
	<201008231144.o7NBi9ZH022349@theraft.openend.se>
	<AANLkTin0DZutRmBQ-FcFihR4SF_69P6b+ogawXeFX4m4@mail.gmail.com>
	<20100823170223.E8F013A4108@sparrow.telecommunity.com>
	<4C72B021.4090309@notmyidea.org> <4C72D354.20007@v.loewis.de>
	<4C72D90B.5010209@egenix.com>
	<AANLkTinvoXsF_UrHdSs_KTwdYs5pUnFuKJoQ7rfV1+Qd@mail.gmail.com>
	<20100823230320.8DAB53A4108@sparrow.telecommunity.com>
Message-ID: <4C73030C.2000307@v.loewis.de>

> So, technically, all an installer has to do is avoid placing conflicting
> packages on the same runtime sys.path, in order to comply with Conflicts.

It should also make sure that they don't overwrite each other's files.

Then, it should also make sure that they aren't conflicting in any other
way, such as using the same relational database instance, creating the
same Unix account, listening to the same port, etc.

If two packages are marked as conflicting (by one of them), the
installer has really to trust this in the first place - only the
user installing the software might know even better.

> OTOH, if what it's saying is something like, "you can't import
> asyncore's mainloop and Twisted's mainloop at the same time and live",
> then what value does it have for the install tool, vs. merely informing
> the user?

There are many other ways in which software can conflict. E.g. on Linux
distributions, it is common that you can install only one MTA. You
can also install only one Apache MPM module in Debian.

Regards,
Martin

From martin at v.loewis.de  Tue Aug 24 01:30:35 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 24 Aug 2010 01:30:35 +0200
Subject: [Catalog-sig] User-agents / download hit
In-Reply-To: <AANLkTimpcHTWeJUX1w1WF=-8NMXcmFYwNW2oUuvOdvT6@mail.gmail.com>
References: <AANLkTimpcHTWeJUX1w1WF=-8NMXcmFYwNW2oUuvOdvT6@mail.gmail.com>
Message-ID: <4C73049B.4010500@v.loewis.de>

> Proposals: let's remove z3c.pypimirror and pep381client from the download stats.

This isn't really implementable as formulated: for many of the files, I
just don't know what user agent has downloaded them.

So if I reduce this number displayed by some value, I couldn't really
claim that the remaining number now doesn't include pypimirror or
pep381client.

Also, what about other automatic downloaders, such as Googlebot, wget,
or buildout?

I plan to display each download counter broken down by UA, so that users
could form their own opinion on how many downloads the file has really
seen. Implementing this would take some time, though (as would
implementing anything else, for that matter).

Regards,
Martin

From ziade.tarek at gmail.com  Tue Aug 24 01:43:04 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 23 Aug 2010 16:43:04 -0700
Subject: [Catalog-sig] User-agents / download hit
In-Reply-To: <4C73049B.4010500@v.loewis.de>
References: <AANLkTimpcHTWeJUX1w1WF=-8NMXcmFYwNW2oUuvOdvT6@mail.gmail.com>
	<4C73049B.4010500@v.loewis.de>
Message-ID: <AANLkTik_gvwdJUxRhWMU=mcqyjn5qPzLseNhyE+hrW0D@mail.gmail.com>

2010/8/23 "Martin v. L?wis" <martin at v.loewis.de>:
>> Proposals: let's remove z3c.pypimirror and pep381client from the download stats.
>
> This isn't really implementable as formulated: for many of the files, I
> just don't know what user agent has downloaded them.

How come ? I though all calls were made through Apache via the same root.

[..]
> Also, what about other automatic downloaders, such as Googlebot, wget,
> or buildout?

I would count buildout and wget calls. For instance, I manually download
files using wget, so its a legitimate hit. But yes, the definition of
what should
be counted as a hit is quite fuzzy.

The only way to know what hits are from mirrors or bots without
relying on the UA
would be to detect a client that acts as a bot and discard its hits.

This can be done by grouping calls issued from the same IP, that are
scanning the whole index in a short time. But that's some work :)

> I plan to display each download counter broken down by UA, so that users
> could form their own opinion on how many downloads the file has really
> seen. Implementing this would take some time, though (as would
> implementing anything else, for that matter).

That would be the best/simplest option.

Regards
Tarek

From pje at telecommunity.com  Tue Aug 24 03:47:21 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 23 Aug 2010 21:47:21 -0400
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <4C73030C.2000307@v.loewis.de>
References: <4C659D1F.9080707@gmail.com>
	<20100813202044.5CB623A411C@sparrow.telecommunity.com>
	<4C65E45A.6010606@gmail.com>
	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>
	<20100814042601.B276C3A411C@sparrow.telecommunity.com>
	<4C71B490.7030304@notmyidea.org>
	<20100823004501.06EAA3A40A4@sparrow.telecommunity.com>
	<ziade.tarek@gmail.com>
	<AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.com>
	<201008231144.o7NBi9ZH022349@theraft.openend.se>
	<AANLkTin0DZutRmBQ-FcFihR4SF_69P6b+ogawXeFX4m4@mail.gmail.com>
	<20100823170223.E8F013A4108@sparrow.telecommunity.com>
	<4C72B021.4090309@notmyidea.org> <4C72D354.20007@v.loewis.de>
	<4C72D90B.5010209@egenix.com>
	<AANLkTinvoXsF_UrHdSs_KTwdYs5pUnFuKJoQ7rfV1+Qd@mail.gmail.com>
	<20100823230320.8DAB53A4108@sparrow.telecommunity.com>
	<4C73030C.2000307@v.loewis.de>
Message-ID: <20100824014725.54E3F3A4108@sparrow.telecommunity.com>

At 01:23 AM 8/24/2010 +0200, Martin v. L?wis wrote:
>It should also make sure that they don't overwrite each other's files.

PEP 376 makes this straightforward, even in the case where the two 
installers are different.


>Then, it should also make sure that they aren't conflicting in any other
>way, such as using the same relational database instance, creating the
>same Unix account, listening to the same port, etc.

I think you missed my point: none of these things presents a conflict 
to *installing* the software, only *running* the software, and thus 
there is no reason for the installer to reject (vs. merely warn the 
user) about the installation.


>There are many other ways in which software can conflict. E.g. on Linux
>distributions, it is common that you can install only one MTA. You
>can also install only one Apache MPM module in Debian.

Is anyone shipping these things using Python software distributions 
as the installation unit?  If not, the fields seem like an 
unnecessary carry-along from OS packaging system information -- that 
nonetheless won't actually *work* for OS-level pacakging, since the 
fields can only target another Python software distribution.

So, unless the two MTAs or MPMs are both written in Python, the field 
as currently defined would be useless in that case...  *even if it 
was purely advisory for OS packagers*.

In any event, the issue still stands that the semantics of "Conflict" 
are ill-defined here.

First, if it's a runtime conflict issue, installers can't do much but 
notify the user.

Second -- and this is the knock-down argument -- if it's instead a 
namespace or file conflict issue, having the Conflicts field doesn't 
actually save an installer any work, because it *still* has to handle 
the possibility of an accidental (and therefore undocumented) 
namespace collision.

For example, RuleDispatch and PyDispatcher both used the 'dispatch' 
module, and during the period when neither package's author was aware 
of the other, neither one would have been explicitly marked as conflicting.

Thus, no matter what, an installation tool is going to have to manage 
such conflicts on its own.

Ergo, I can't see how Conflicts is going to do anything useful unless 
it's in a purely advisory capacity -- and in that event, it should 
really provide some sort of documentation as to the reason or nature 
of the conflict.


From martin at v.loewis.de  Tue Aug 24 07:30:46 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 24 Aug 2010 07:30:46 +0200
Subject: [Catalog-sig] User-agents / download hit
In-Reply-To: <AANLkTik_gvwdJUxRhWMU=mcqyjn5qPzLseNhyE+hrW0D@mail.gmail.com>
References: <AANLkTimpcHTWeJUX1w1WF=-8NMXcmFYwNW2oUuvOdvT6@mail.gmail.com>	<4C73049B.4010500@v.loewis.de>
	<AANLkTik_gvwdJUxRhWMU=mcqyjn5qPzLseNhyE+hrW0D@mail.gmail.com>
Message-ID: <4C735906.6070001@v.loewis.de>

Am 24.08.2010 01:43, schrieb Tarek Ziad?:
> 2010/8/23 "Martin v. L?wis" <martin at v.loewis.de>:
>>> Proposals: let's remove z3c.pypimirror and pep381client from the download stats.
>>
>> This isn't really implementable as formulated: for many of the files, I
>> just don't know what user agent has downloaded them.
> 
> How come ? I though all calls were made through Apache via the same root.

Sure. However, I don't have the log files anymore if the download was
made some months ago.

Regards,
Martin

From fdrake at acm.org  Tue Aug 24 16:40:43 2010
From: fdrake at acm.org (Fred Drake)
Date: Tue, 24 Aug 2010 10:40:43 -0400
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <20100823230503.B6FDD3A4108@sparrow.telecommunity.com>
References: <20100823230503.B6FDD3A4108@sparrow.telecommunity.com>
Message-ID: <AANLkTimx9E=hpWtChJuqr4pkoKrU-VwPm2pXR6V0fQka@mail.gmail.com>

On Mon, Aug 23, 2010 at 7:05 PM, P.J. Eby <pje at telecommunity.com> wrote:
> Hm. ?Come to think of it, if all Conflicts is saying is, "I have modules of
> the same names as...", then couldn't any installer worth its salt figure
> this out for itself? ?(Even PEP 376 already has a way to detect conflicting
> files before installing.)

This is not the only case that's interesting for Conflicts.  Certain
refactoring patterns can cause incompatibilities as well, and those
can be much harder to track down.

A new package which provides an interface that was once provided
elsewhere should generally be declared in conflict with older versions
of the package where the interface was originally provided (though not
in conflict with newer versions that pull the interface from the new
package).

This pattern has occurred frequently as the Zope community has worked
to reduce the dependencies on the zope.app packages, and there's been
no way to declare the conflicts.


? -Fred

--
Fred L. Drake, Jr.? ? <fdrake at gmail.com>
"A storm broke loose in my mind."? --Albert Einstein

From mal at egenix.com  Tue Aug 24 17:12:28 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 24 Aug 2010 17:12:28 +0200
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <AANLkTinvoXsF_UrHdSs_KTwdYs5pUnFuKJoQ7rfV1+Qd@mail.gmail.com>
References: <4C659D1F.9080707@gmail.com>	<20100813202044.5CB623A411C@sparrow.telecommunity.com>	<4C65E45A.6010606@gmail.com>	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>	<20100814042601.B276C3A411C@sparrow.telecommunity.com>	<4C71B490.7030304@notmyidea.org>	<20100823004501.06EAA3A40A4@sparrow.telecommunity.com>	<ziade.tarek@gmail.com>	<AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.com>	<201008231144.o7NBi9ZH022349@theraft.openend.se>	<AANLkTin0DZutRmBQ-FcFihR4SF_69P6b+ogawXeFX4m4@mail.gmail.com>	<20100823170223.E8F013A4108@sparrow.telecommunity.com>	<4C72B021.4090309@notmyidea.org>
	<4C72D354.20007@v.loewis.de>	<4C72D90B.5010209@egenix.com>
	<AANLkTinvoXsF_UrHdSs_KTwdYs5pUnFuKJoQ7rfV1+Qd@mail.gmail.com>
Message-ID: <4C73E15C.7070104@egenix.com>

Tarek Ziad? wrote:
> On Mon, Aug 23, 2010 at 1:24 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> ..
>>
>>> In case of "obsoletes", it _should_ also be possible to install both
>>> of them simultaneously. Maybe some other other distribution depends
>>> on the original one, and can't work with the new one.
>>
>> Agreed.
>>
>> The same is true for conflicting packages: even if packages A and D
>> conflict, one application may use the package combination A,B,C
>> while another may be using D,E,F - both without any conflicts.
> 
> But we are in the same scope.
> 
> This can be applied for Conflicts because they are supposed to be incompatible.
> 
> It's exactly the same incompatibility than having Foo depending on Bar > 1.0
> and Baz depending on Bar < 0.9. Since you can't have two versions of Bar running
> in the same Python packages.
> 
> So while Obsoletes allows you to have two times the same package. Conflicts
> indicates that its strictly impossible.

Please see my example: It is well possible that you have two
sets of packages installed which are mutually exclusive, but
don't interfere which each other when used by different
applications.

If an installer were to bar you from installing packages from
set 1 if you already have packages from set 2 installed, you
wouldn't be able to use those two applications at the same time,
even though they don't use a mixed combination of packages from
set 1 and 2 (which would conflict).

Whether there is a conflict depends on the applications using
the packages. The packages themselves are not in conflict.

Think of having both Zope and Django installed: one could regard
this as conflict, since both provide web server functionality,
but in reality they can both coexist in site-packages without
any problem.

All the other cases (packages using the same module/package name
or version dependency conflicts) can easily be figured out
by the installer without such an extra "conflicts" field.

Could you perhaps point me to a list of things you had in mind
with the "conflicts" field ? Perhaps I'm just missing the main
idea behind this field.

I do see a problem with such a general purpose "conflicts" field,
regardless of what the use case is. If the installer prevents
packages from being installed by means of defining such
a "conflicts" field, this can be used to externally make
package sets incompatible, e.g. a package of
framework 1 could effectively prevent the installation of
a competing framework 2, even though they both can live
happily side-by-side on sys.path. And there's nothing the
authors of framework 2 could do about this.


In the same line of thought, I believe it would be
better to change the "obsoletes" field to "obsoleted-by", i.e. the
direction changes and gives the author of the obsoleted package
full control over what he thinks is a better version of his
package, rather than have some other author make that choice for
him.

That way you avoid the social implications of one package
trying to compete against another by means of the "obsoletes"
field.


In summary: As PyPI grows and we are seeing the first conflicts
between package authors trying to push their particular idea
of implementing a certain task, we should take such social
implications of standards into account and, if possible,
design the standards in a way that doesn't encourage such
behavior.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 24 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From pje at telecommunity.com  Tue Aug 24 17:45:54 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 24 Aug 2010 11:45:54 -0400
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <4C73E15C.7070104@egenix.com>
References: <4C659D1F.9080707@gmail.com>
	<20100813202044.5CB623A411C@sparrow.telecommunity.com>
	<4C65E45A.6010606@gmail.com>
	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>
	<20100814042601.B276C3A411C@sparrow.telecommunity.com>
	<4C71B490.7030304@notmyidea.org>
	<20100823004501.06EAA3A40A4@sparrow.telecommunity.com>
	<ziade.tarek@gmail.com>
	<AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.com>
	<201008231144.o7NBi9ZH022349@theraft.openend.se>
	<AANLkTin0DZutRmBQ-FcFihR4SF_69P6b+ogawXeFX4m4@mail.gmail.com>
	<20100823170223.E8F013A4108@sparrow.telecommunity.com>
	<4C72B021.4090309@notmyidea.org> <4C72D354.20007@v.loewis.de>
	<4C72D90B.5010209@egenix.com>
	<AANLkTinvoXsF_UrHdSs_KTwdYs5pUnFuKJoQ7rfV1+Qd@mail.gmail.com>
	<4C73E15C.7070104@egenix.com>
Message-ID: <20100824154559.DD2663A40A5@sparrow.telecommunity.com>

At 05:12 PM 8/24/2010 +0200, M.-A. Lemburg wrote:
>In the same line of thought, I believe it would be
>better to change the "obsoletes" field to "obsoleted-by", i.e. the
>direction changes and gives the author of the obsoleted package
>full control over what he thinks is a better version of his
>package, rather than have some other author make that choice for
>him.
>
>That way you avoid the social implications of one package
>trying to compete against another by means of the "obsoletes"
>field.

This is a great idea.  Even if it were a purely informational field 
(i.e., not used by any installation tool except as a user advisory), 
I would actually have used this field in my own use cases.

+1 for replacing Obsoletes with Obsoleted-By.


>Could you perhaps point me to a list of things you had in mind
>with the "conflicts" field ? Perhaps I'm just missing the main
>idea behind this field.

If so, then I am, too.  :)


From pje at telecommunity.com  Tue Aug 24 17:49:10 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 24 Aug 2010 11:49:10 -0400
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <AANLkTimx9E=hpWtChJuqr4pkoKrU-VwPm2pXR6V0fQka@mail.gmail.c
 om>
References: <20100823230503.B6FDD3A4108@sparrow.telecommunity.com>
	<AANLkTimx9E=hpWtChJuqr4pkoKrU-VwPm2pXR6V0fQka@mail.gmail.com>
Message-ID: <20100824154915.1D7713A40A5@sparrow.telecommunity.com>

At 10:40 AM 8/24/2010 -0400, Fred Drake wrote:
>On Mon, Aug 23, 2010 at 7:05 PM, P.J. Eby <pje at telecommunity.com> wrote:
> > Hm.  Come to think of it, if all Conflicts is saying is, "I have modules of
> > the same names as...", then couldn't any installer worth its salt figure
> > this out for itself?  (Even PEP 376 already has a way to detect conflicting
> > files before installing.)
>
>This is not the only case that's interesting for Conflicts.  Certain
>refactoring patterns can cause incompatibilities as well, and those
>can be much harder to track down.
>
>A new package which provides an interface that was once provided
>elsewhere should generally be declared in conflict with older versions
>of the package where the interface was originally provided (though not
>in conflict with newer versions that pull the interface from the new
>package).
>
>This pattern has occurred frequently as the Zope community has worked
>to reduce the dependencies on the zope.app packages, and there's been
>no way to declare the conflicts.

Could you give a concrete example of this?  I *think* I get the gist 
of what you're saying, but I'm not sure.


From merwok at netwok.org  Tue Aug 24 18:29:36 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Tue, 24 Aug 2010 18:29:36 +0200
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <4C73E15C.7070104@egenix.com>
References: <4C659D1F.9080707@gmail.com>	<20100813202044.5CB623A411C@sparrow.telecommunity.com>	<4C65E45A.6010606@gmail.com>	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>	<20100814042601.B276C3A411C@sparrow.telecommunity.com>	<4C71B490.7030304@notmyidea.org>	<20100823004501.06EAA3A40A4@sparrow.telecommunity.com>	<ziade.tarek@gmail.com>	<AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.com>	<201008231144.o7NBi9ZH022349@theraft.openend.se>	<AANLkTin0DZutRmBQ-FcFihR4SF_69P6b+ogawXeFX4m4@mail.gmail.com>	<20100823170223.E8F013A4108@sparrow.telecommunity.com>	<4C72B021.4090309@notmyidea.org>	<4C72D354.20007@v.loewis.de>	<4C72D90B.5010209@egenix.com>	<AANLkTinvoXsF_UrHdSs_KTwdYs5pUnFuKJoQ7rfV1+Qd@mail.gmail.com>
	<4C73E15C.7070104@egenix.com>
Message-ID: <4C73F370.5030308@netwok.org>

> In the same line of thought, I believe it would be
> better to change the "obsoletes" field to "obsoleted-by", i.e. the
> direction changes and gives the author of the obsoleted package
> full control over what he thinks is a better version of his
> package, rather than have some other author make that choice for
> him.

I see two problems with this idea. Let?s say that Spam 1.0 is released
and obsoletes Ham, which has versions 0.1.3 and 0.2.1 (Ham and Spam are
from the same author):

- You?d have to release Ham 0.1.4 and 0.2.2 only to include the
obsoleted-by field.

- If a user has Ham 0.1.3 and installs Spam 1.0, nothing says that Ham
should be removed.

I think that?s why the obsoletes field is defined by the release that
supersedes, not the one that is obsoleted.

Can someone tell what CPAN or Cabal have to deal with the
replaces/conflicts/obsoletes/breaks problem?

Regards


From pje at telecommunity.com  Tue Aug 24 19:42:06 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 24 Aug 2010 13:42:06 -0400
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <4C73F370.5030308@netwok.org>
References: <4C659D1F.9080707@gmail.com>
	<20100813202044.5CB623A411C@sparrow.telecommunity.com>
	<4C65E45A.6010606@gmail.com>
	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>
	<20100814042601.B276C3A411C@sparrow.telecommunity.com>
	<4C71B490.7030304@notmyidea.org>
	<20100823004501.06EAA3A40A4@sparrow.telecommunity.com>
	<ziade.tarek@gmail.com>
	<AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.com>
	<201008231144.o7NBi9ZH022349@theraft.openend.se>
	<AANLkTin0DZutRmBQ-FcFihR4SF_69P6b+ogawXeFX4m4@mail.gmail.com>
	<20100823170223.E8F013A4108@sparrow.telecommunity.com>
	<4C72B021.4090309@notmyidea.org> <4C72D354.20007@v.loewis.de>
	<4C72D90B.5010209@egenix.com>
	<AANLkTinvoXsF_UrHdSs_KTwdYs5pUnFuKJoQ7rfV1+Qd@mail.gmail.com>
	<4C73E15C.7070104@egenix.com> <4C73F370.5030308@netwok.org>
Message-ID: <20100824174211.B50DA3A40A4@sparrow.telecommunity.com>

At 06:29 PM 8/24/2010 +0200, ??ric Araujo wrote:
>I see two problems with this idea. Let???s say that Spam 1.0 is 
>released and obsoletes Ham, which has versions 0.1.3 and 0.2.1 (Ham 
>and Spam are from the same author): - You???d have to release Ham 
>0.1.4 and 0.2.2 only to include the obsoleted-by field. - If a user 
>has Ham 0.1.3 and installs Spam 1.0, nothing says that Ham should be removed.

Has anyone shown a use case for why "Ham" *should* be removed?  Ever?

We *do* have use cases for why "Ham" must NOT be removed, though: 
PEAK-Rules obsoletes RuleDispatch, but code using RuleDispatch must 
first be *changed* to use PEAK-Rules.

Therefore, an installation or path management tool cannot simply 
substitute a dependency on one with a dependency on the other.  (And 
the same applies for ObjectRoles vs. AddOns.)

AFAICT, in no real-world use case yet presented here has there been a 
reason to remove the "obsoleted" software at all, unless there is 
*also* a corresponding Conflicts entry...  in which case, the 
Obsoletes entry is redundant.

This is a general property of Python modules: if you create a 
non-conflicting, but obsolescing module, you *cannot* substitute 
dependencies automatically, because the import targets will be 
wrong.  This means that "obsoletes" cannot have any semantic effect 
on installation without there *also* being a file-level conflict.

Meanwhile, actual use cases for the Conflicts field are beginning to 
become much narrower than they first appeared, since installation 
tools are going to have to be able to detect undeclared 
module/file/package conflicts anyway...  so why have a conflicts field?

It's looking more and more like these fields are, at *best*, a 
convenient place for sysadmins or other users to look for 
purely-advisory messages about these things.  (And in that case, the 
people who actually *need* the obsolescence information are users of 
the old package, not the new one!)

At worst, however, these fields are a waste of electrons and should 
be dropped altogether.

("Obsoletes" really didn't have a very good rationale for being in 
the previous metadata PEP (314), either.  (I would borrow the time 
machine and argue against it back then, but since I wasn't subscribed 
to the Catalog-SIG mailing list in 2003, it would create a time 
paradox to go back and try to post there now.  ;-) ))


From gael at pilotsystems.net  Wed Aug 25 12:55:16 2010
From: gael at pilotsystems.net (=?iso-8859-1?Q?Ga=EBl?= Le Mignot)
Date: Wed, 25 Aug 2010 12:55:16 +0200
Subject: [Catalog-sig] Mirroring pypi
In-Reply-To: <4C72D255.4070909@v.loewis.de> ("Martin v. =?iso-8859-1?Q?L?=
	=?iso-8859-1?Q?=F6wis=22's?= message of "Mon\, 23 Aug 2010 21\:56\:05
	+0200")
References: <plop877hjhwbby.fsf@aoskar.kilobug.org>
	<4C72D255.4070909@v.loewis.de>
Message-ID: <plop87zkwbklm3.fsf@aoskar.kilobug.org>

Hello,

Thanks for your answer.

So we are interested in setting up this mirror.

I've  read  PEP 381,  but  I didn't  see  anything  about the  initial
synchronization in it.

Is there a prefered method for that ? Any maximal bandwidth to use for
that initial synchronization, in order to avoid penalizing end-users ?

Regards,

-- 
Ga?l Le Mignot - gael at pilotsystems.net
Pilot Systems - 9, rue Desargues - 75011 Paris
Tel : +33 1 44 53 05 55 - www.pilotsystems.net
G?rez vos contacts et vos newsletters : www.cockpit-mailing.com

From alexis at notmyidea.org  Wed Aug 25 18:45:23 2010
From: alexis at notmyidea.org (=?ISO-8859-1?Q?Alexis_M=E9taireau?=)
Date: Wed, 25 Aug 2010 18:45:23 +0200
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <20100824014725.54E3F3A4108@sparrow.telecommunity.com>
References: <4C659D1F.9080707@gmail.com>	<20100813202044.5CB623A411C@sparrow.telecommunity.com>	<4C65E45A.6010606@gmail.com>	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>	<20100814042601.B276C3A411C@sparrow.telecommunity.com>	<4C71B490.7030304@notmyidea.org>	<20100823004501.06EAA3A40A4@sparrow.telecommunity.com>	<ziade.tarek@gmail.com>	<AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.com>	<201008231144.o7NBi9ZH022349@theraft.openend.se>	<AANLkTin0DZutRmBQ-FcFihR4SF_69P6b+ogawXeFX4m4@mail.gmail.com>	<20100823170223.E8F013A4108@sparrow.telecommunity.com>	<4C72B021.4090309@notmyidea.org>
	<4C72D354.20007@v.loewis.de>	<4C72D90B.5010209@egenix.com>	<AANLkTinvoXsF_UrHdSs_KTwdYs5pUnFuKJoQ7rfV1+Qd@mail.gmail.com>	<20100823230320.8DAB53A4108@sparrow.telecommunity.com>	<4C73030C.2000307@v.loewis.de>
	<20100824014725.54E3F3A4108@sparrow.telecommunity.com>
Message-ID: <4C7548A3.5090109@notmyidea.org>

 About the Conflict field. it seems true there is two kind of conflicts:
we can install both Django and Zope at the same time, for example, and I
would not consider them as conflicting, even if they could start a
server on the same port.

But, there is some cases, where the conflict field is useful. For
instance, when two releases propose the same python modules to import,
or when they rely on the same configuration files for instance. While
the first case seems to be covered by the "Provides" field, the second
one does not seems to be covered at all.

There is also the example Fred quoted about interfaces, here is the quote:

    A new package which provides an interface that was once provided
    elsewhere should generally be declared in conflict with older versions
    of the package where the interface was originally provided (though not
    in conflict with newer versions that pull the interface from the new
    package).

    This pattern has occurred frequently as the Zope community has worked
    to reduce the dependencies on the zope.app packages, and there's been
    no way to declare the conflicts.

What I understand here, is that some interface was firstly provided in a
release, but then provided by another one. So both can provide it, and
we need a way to mark them as conflicting. Here, that's not obsoleting,
as just part of the functionalities are provided in the new release.

After a bit of thinking, I think I agree with PJE when saying that we
need to put on some examples of how the fields are to be used, in the
clients.

About what have been stated on the "conflict" and "obsolete" fields. I
agree that it would be nice if the way of the relation have been
changed, but Merwok pointed out that will force the maintainer of the
old release to create a new release just for that. That's true, and
maybe can we avoid creating a new release just for that. If the METADATA
file is available to download by clients from PyPI, we can regenerate it
without regenerating a new release for that. Maybe can we find a way to
point this release as obsoleted by another one.

"Conflict" and "Obsoletes" seems to be needed, here, as they cover
different use cases. Now, we have to deal with what the installer
*should* or *shouldnt* do, but it seems to be another point...

...that's still need to be answered.

If the "Ham" release have been obsoleted by "Egg", and if "Spam" relies
on "Ham", and "Ham" is installed, that's fine, and the installation can
go on.

Now, if we try to install "Towel", that relies on "Egg", and we have
already "Ham" installed, we should just state there is a conflict here.
By that, I mean that the releases should not be replaced by another
ones, even if the new ones obsoletes the old one.

the "conflict" field covers another type of conflict, also needed: it
would just be used in the way: "both A and B releases can't be
*installed* on the same time, whatever could be the cause". I'm agreeing
with you while you're saying that *installation* dependencies are
different from *run* dependencies, I've understood the "conflict" field
as an "installation" dependency, and think too that is needed to be
clear about this in the PEP.

Thanks all for your responses to this thread, that's a joy to see so
much enthousiasm and debate about this !

Cheers,
Alexis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20100825/872067d2/attachment.html>

From alexis at notmyidea.org  Wed Aug 25 18:57:53 2010
From: alexis at notmyidea.org (=?ISO-8859-1?Q?Alexis_M=E9taireau?=)
Date: Wed, 25 Aug 2010 18:57:53 +0200
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <4C73E15C.7070104@egenix.com>
References: <4C659D1F.9080707@gmail.com>	<20100813202044.5CB623A411C@sparrow.telecommunity.com>	<4C65E45A.6010606@gmail.com>	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>	<20100814042601.B276C3A411C@sparrow.telecommunity.com>	<4C71B490.7030304@notmyidea.org>	<20100823004501.06EAA3A40A4@sparrow.telecommunity.com>	<ziade.tarek@gmail.com>	<AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.com>	<201008231144.o7NBi9ZH022349@theraft.openend.se>	<AANLkTin0DZutRmBQ-FcFihR4SF_69P6b+ogawXeFX4m4@mail.gmail.com>	<20100823170223.E8F013A4108@sparrow.telecommunity.com>	<4C72B021.4090309@notmyidea.org>	<4C72D354.20007@v.loewis.de>	<4C72D90B.5010209@egenix.com>	<AANLkTinvoXsF_UrHdSs_KTwdYs5pUnFuKJoQ7rfV1+Qd@mail.gmail.com>
	<4C73E15C.7070104@egenix.com>
Message-ID: <4C754B91.7090201@notmyidea.org>

 Le 08/24/2010 05:12 PM, M.-A. Lemburg a ?crit :
> I do see a problem with such a general purpose "conflicts" field,
> regardless of what the use case is. If the installer prevents
> packages from being installed by means of defining such
> a "conflicts" field, this can be used to externally make
> package sets incompatible, e.g. a package of
> framework 1 could effectively prevent the installation of
> a competing framework 2, even though they both can live
> happily side-by-side on sys.path. And there's nothing the
> authors of framework 2 could do about this.
of course, we can force the installation in installers with --force, in
such a case, but that's not why the "conflict" field is. Eg. If both can
leave together, they're not conflicting :)

Alex

From mal at egenix.com  Wed Aug 25 21:08:46 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 25 Aug 2010 21:08:46 +0200
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <4C754B91.7090201@notmyidea.org>
References: <4C659D1F.9080707@gmail.com>	<20100813202044.5CB623A411C@sparrow.telecommunity.com>	<4C65E45A.6010606@gmail.com>	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>	<20100814042601.B276C3A411C@sparrow.telecommunity.com>	<4C71B490.7030304@notmyidea.org>	<20100823004501.06EAA3A40A4@sparrow.telecommunity.com>	<ziade.tarek@gmail.com>	<AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.com>	<201008231144.o7NBi9ZH022349@theraft.openend.se>	<AANLkTin0DZutRmBQ-FcFihR4SF_69P6b+ogawXeFX4m4@mail.gmail.com>	<20100823170223.E8F013A4108@sparrow.telecommunity.com>	<4C72B021.4090309@notmyidea.org>	<4C72D354.20007@v.loewis.de>	<4C72D90B.5010209@egenix.com>	<AANLkTinvoXsF_UrHdSs_KTwdYs5pUnFuKJoQ7rfV1+Qd@mail.gmail.com>	<4C73E15C.7070104@egenix.com>
	<4C754B91.7090201@notmyidea.org>
Message-ID: <4C756A3E.6030406@egenix.com>

Alexis M?taireau wrote:
>  Le 08/24/2010 05:12 PM, M.-A. Lemburg a ?crit :
>> I do see a problem with such a general purpose "conflicts" field,
>> regardless of what the use case is. If the installer prevents
>> packages from being installed by means of defining such
>> a "conflicts" field, this can be used to externally make
>> package sets incompatible, e.g. a package of
>> framework 1 could effectively prevent the installation of
>> a competing framework 2, even though they both can live
>> happily side-by-side on sys.path. And there's nothing the
>> authors of framework 2 could do about this.
> of course, we can force the installation in installers with --force, in
> such a case, but that's not why the "conflict" field is. Eg. If both can
> leave together, they're not conflicting :)

I guess that's where part of the misunderstanding originates:
the type of conflict can be many things and may well only
show up in certain applications using the packages, but
not necessarily all of them.

At the very least, there should be an extra field "Conflict-Description"
which describes the type of conflict to the user and then lets him or
her decide whether the conflict is a problem or not for the intended
use.

I have a feeling that the introduction of this field originated
from some application space problem that you're trying to fix
at the package level.

Also note that a the use cases you and others have mentioned can
easily be solved using meta-packages and dependencies on these or
by the installer checking whether there are module name or
file conflicts (i.e. two packages wanting to install the same
module or create the same file).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 25 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From pje at telecommunity.com  Wed Aug 25 22:15:35 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 25 Aug 2010 16:15:35 -0400
Subject: [Catalog-sig] PEP 345 Update
Message-ID: <20100825201543.72B283A40A4@sparrow.telecommunity.com>

At 06:45 PM 8/25/2010 +0200, Alexis M?taireau wrote:
>when two releases propose the same python modules to import,

Installation tools *always* have to detect this anyway -- a conflicts 
field adds nothing new here.

Real world example: RuleDispatch and PyDispatcher both used the 
"dispatch" package name at one point, but would not have if they'd 
known the other existed.  An installation tool would have had to 
detect this, since there was no declared conflict.

>or when they rely on the same configuration files for instance.

What does this mean, for a Python library?  ISTM that it resolves to 
either a file-level conflict, or a user-managed conflict.

Please, please provide an example for "Conflicts" that does *not* 
fall into one of the following categories:

1. The installed files conflict,

2. Importing the provided code causes monopolization of a 
process-level resource (e.g. a signal handler, trace hook,  etc.),

3. The code monopolizes a system-level resource (e.g. a user ID, 
port, directory, etc.)

For category 1, the installation tool has to be able to handle this anyway

For category 2, the installation tool need do nothing: it is possible 
and sane to have both, say, Twisted apps and asyncore apps installed 
on the same machine -- they will have different startup scripts and 
will not import each other.

For category 3, the user has to take some kind of action to use both, 
and/or is obliged to configure them in such a way as to use different 
resources...  but mere *installation* is not going to cause a problem.

In other words, none of the above is actually a use-case for having a 
"Conflicts" field.


>There is also the example Fred quoted about interfaces, here is the quote:

Fred hasn't yet explained this in a way that shows why it's not one 
of the above three cases regarding Conflicts.


>I agree that it would be nice if the way of the relation have been 
>changed, but Merwok pointed out that will force the maintainer of 
>the old release to create a new release just for that.

Here's the thing: so far I'm the ONLY person in the discussion who 
has even offered an example of when Obsoletes would be used...  and 
for the two examples I gave, I would totally be willing to make a new 
release for them to include that field, if there was some benefit 
(e.g. user notification) to having the field.


>"Conflict" and "Obsoletes" seems to be needed, here, as they cover 
>different use cases.

As far as I can tell, nobody has given any use cases for Conflicts or 
Obsoletes that are not either:

1. A mere advisory message, or
2. Redundant with normal file installation conflict detection.


>If the "Ham" release have been obsoleted by "Egg", and if "Spam" 
>relies on "Ham", and "Ham" is installed, that's fine, and the 
>installation can go on.
>
>Now, if we try to install "Towel", that relies on "Egg", and we have 
>already "Ham" installed, we should just state there is a conflict here.

Why is there a conflict?  This makes no sense whatsoever!

PEAK-Rules obsoletes RuleDispatch, but there is absolutely no reason 
to not install both.

If you disagree with this, then we have a different meaning of the 
word "Obsoletes"...

In which case, you need to show a concrete example of "Obsoletes" 
using REAL Python packages, not made-up foobar examples, in order to 
illustrate the definition where "obsoletes" means "shouldn't both be 
installed".


>By that, I mean that the releases should not be replaced by another 
>ones, even if the new ones obsoletes the old one.

In the specific example you gave, there isn't any conflict -- both 
should be installed, and there is no need to even notify the user.

(Note that forked packages using the same API only conflict because 
of *installing conflicting files* -- so if you're thinking of a fork 
as your example here, it is leading you to confuse the ideas of 
Obsoletes and Conflicts.)


>the "conflict" field covers another type of conflict, also needed: 
>it would just be used in the way: "both A and B releases can't be 
>*installed* on the same time, whatever could be the cause".

The thing I haven't yet seen an example of is why they can't be 
installed at the same time, that doesn't reduce to a file 
installation conflict.  AFAICT, anything else is *necessarily* a 
runtime conflict, unless the intent is to cover installation of more 
than just scripts and their libraries.  (e.g. MvL's example of 
creating user IDs)

But if that's the case, ISTM we should wait until we have the 
software that does such installations *first* -- PEP 314 is an 
example of what happens when you create the metadata spec before the 
tools that use it -- nobody fills them in much, because they don't 
know how they'll be used or have any reason to, and then the tools 
probably end up needing different metadata anyway!


From martin at v.loewis.de  Thu Aug 26 00:03:33 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 26 Aug 2010 00:03:33 +0200
Subject: [Catalog-sig] Mirroring pypi
In-Reply-To: <plop87zkwbklm3.fsf@aoskar.kilobug.org>
References: <plop877hjhwbby.fsf@aoskar.kilobug.org>	<4C72D255.4070909@v.loewis.de>
	<plop87zkwbklm3.fsf@aoskar.kilobug.org>
Message-ID: <4C759335.6080903@v.loewis.de>

> Is there a prefered method for that ? Any maximal bandwidth to use for
> that initial synchronization, in order to avoid penalizing end-users ?

I recommend to use pep381client. It doesn't have any rate limitation,
except for doing things strictly sequentially.

Regards,
Martin

From alexis at notmyidea.org  Thu Aug 26 08:11:42 2010
From: alexis at notmyidea.org (=?ISO-8859-1?Q?Alexis_M=E9taireau?=)
Date: Thu, 26 Aug 2010 08:11:42 +0200
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <4C756A3E.6030406@egenix.com>
References: <4C659D1F.9080707@gmail.com>	<20100813202044.5CB623A411C@sparrow.telecommunity.com>	<4C65E45A.6010606@gmail.com>	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>	<20100814042601.B276C3A411C@sparrow.telecommunity.com>	<4C71B490.7030304@notmyidea.org>	<20100823004501.06EAA3A40A4@sparrow.telecommunity.com>	<ziade.tarek@gmail.com>	<AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.com>	<201008231144.o7NBi9ZH022349@theraft.openend.se>	<AANLkTin0DZutRmBQ-FcFihR4SF_69P6b+ogawXeFX4m4@mail.gmail.com>	<20100823170223.E8F013A4108@sparrow.telecommunity.com>	<4C72B021.4090309@notmyidea.org>	<4C72D354.20007@v.loewis.de>	<4C72D90B.5010209@egenix.com>	<AANLkTinvoXsF_UrHdSs_KTwdYs5pUnFuKJoQ7rfV1+Qd@mail.gmail.com>	<4C73E15C.7070104@egenix.com>
	<4C754B91.7090201@notmyidea.org> <4C756A3E.6030406@egenix.com>
Message-ID: <4C76059E.5080403@notmyidea.org>

 Le 08/25/2010 09:08 PM, M.-A. Lemburg a ?crit :
> I guess that's where part of the misunderstanding originates:
> the type of conflict can be many things and may well only
> show up in certain applications using the packages, but
> not necessarily all of them.
>
> At the very least, there should be an extra field "Conflict-Description"
> which describes the type of conflict to the user and then lets him or
> her decide whether the conflict is a problem or not for the intended
> use.
This sounds like a good idea to have something to output to the user, in
order to let him resolve the conflict the right way.

> I have a feeling that the introduction of this field originated
> from some application space problem that you're trying to fix
> at the package level.
Tarek, any hints about the origin of this field ? (conflict)
> Also note that a the use cases you and others have mentioned can
> easily be solved using meta-packages and dependencies on these or
> by the installer checking whether there are module name or
> file conflicts (i.e. two packages wanting to install the same
> module or create the same file).
True. Maybe need we to make a complex packaging example, with good
practices to follow, and an companion document for this purpose, as good
packaging solutions seems to be non-obvious for end users.


From alexis at notmyidea.org  Thu Aug 26 14:47:33 2010
From: alexis at notmyidea.org (=?ISO-8859-1?Q?Alexis_M=E9taireau?=)
Date: Thu, 26 Aug 2010 14:47:33 +0200
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <20100825195720.7A0223A40A4@sparrow.telecommunity.com>
References: <4C659D1F.9080707@gmail.com>
	<20100813202044.5CB623A411C@sparrow.telecommunity.com>
	<4C65E45A.6010606@gmail.com>
	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>
	<20100814042601.B276C3A411C@sparrow.telecommunity.com>
	<4C71B490.7030304@notmyidea.org>
	<20100823004501.06EAA3A40A4@sparrow.telecommunity.com>
	<ziade.tarek@gmail.com>
	<AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.com>
	<201008231144.o7NBi9ZH022349@theraft.openend.se>
	<AANLkTin0DZutRmBQ-FcFihR4SF_69P6b+ogawXeFX4m4@mail.gmail.com>
	<20100823170223.E8F013A4108@sparrow.telecommunity.com>
	<4C72B021.4090309@notmyidea.org> <4C72D354.20007@v.loewis.de>
	<4C72D90B.5010209@egenix.com>
	<AANLkTinvoXsF_UrHdSs_KTwdYs5pUnFuKJoQ7rfV1+Qd@mail.gmail.com>
	<20100823230320.8DAB53A4108@sparrow.telecommunity.com>
	<4C73030C.2000307@v.loewis.de>
	<20100824014725.54E3F3A4108@sparrow.telecommunity.com>
	<4C7548A3.5090109@notmyidea.org>
	<20100825195720.7A0223A40A4@sparrow.telecommunity.com>
Message-ID: <4C766265.7020507@notmyidea.org>

 Le 08/25/2010 09:41 PM, P.J. Eby a ?crit :
> At 06:45 PM 8/25/2010 +0200, Alexis M?taireau wrote:
>> when two releases propose the same python modules to import,
>
> Installation tools *always* have to detect this anyway -- a conflicts
> field adds nothing new here.
>
> Real world example: RuleDispatch and PyDispatcher both used the
> "dispatch" package name at one point, but would not have if they'd
> known the other existed.  An installation tool would have had to
> detect this, since there was no declared conflict.
True.
>
>> or when they rely on the same configuration files for instance.
>
> What does this mean, for a Python library?  ISTM that it resolves to
> either a file-level conflict, or a user-managed conflict.
You're probably right on this.
>
> Please, please provide an example for "Conflicts" that does *not* fall
> into one of the following categories:
>
> 1. The installed files conflict,
>
> 2. Importing the provided code causes monopolization of a
> process-level resource (e.g. a signal handler, trace hook,  etc.),
>
> 3. The code monopolizes a system-level resource (e.g. a user ID, port,
> directory, etc.)
>
> For category 1, the installation tool has to be able to handle this anyway
Yes, here the conflict field is not needed.
>
> For category 2, the installation tool need do nothing: it is possible
> and sane to have both, say, Twisted apps and asyncore apps installed
> on the same machine -- they will have different startup scripts and
> will not import each other.
If they have not different startup scripts, or if they miss any sort of
"lock" (for instance) to access the files (too bad we dont live in a
perfect world), this means there will be problems while running both at
the same time. As you said, that's not "installation" related, here.
>
>
> For category 3, the user has to take some kind of action to use both,
> and/or is obliged to configure them in such a way as to use different
> resources...  but mere *installation* is not going to cause a problem.
Right, the installation *will not* cause problems, but they will apear
after that. If so, we have two choices:

    - First is to tell nothing to the user about that, and I think
    that's a good option, as it could result in a end user bad
    experience (if that's not handled at the installation level, what
    will handle this?)
    - Second is to prompt the user about what he want to do: Do we have
    to install the release/distribution, even if it can cause problems
    while running them? Are you sure you want to install both, even if
    it could lead to such issues ?, etc.


> In other words, none of the above is actually a use-case for having a
> "Conflicts" field.
I have pain to imagine use cases where conflicts could raise at the
*installation* level too, but the question seems to be: "do we need
something to describe installation-related conflicts, or run-related
conflicts ? In debian apt system, conflicts are described for "on-run"
issues, IIRC.
>
>
>> There is also the example Fred quoted about interfaces, here is the
>> quote:
>
> Fred hasn't yet explained this in a way that shows why it's not one of
> the above three cases regarding Conflicts.
>
>
>> I agree that it would be nice if the way of the relation have been
>> changed, but Merwok pointed out that will force the maintainer of the
>> old release to create a new release just for that.
>
> Here's the thing: so far I'm the ONLY person in the discussion who has
> even offered an example of when Obsoletes would be used...  and for
> the two examples I gave, I would totally be willing to make a new
> release for them to include that field, if there was some benefit
> (e.g. user notification) to having the field.
And that's the use case described in the PEP. So, now, we have to choose
if it's acceptable to make a new release for updating this field, or not.
>
>
>> "Conflict" and "Obsoletes" seems to be needed, here, as they cover
>> different use cases.
>
> As far as I can tell, nobody has given any use cases for Conflicts or
> Obsoletes that are not either:
>
> 1. A mere advisory message, or
> 2. Redundant with normal file installation conflict detection.
>
>
>> If the "Ham" release have been obsoleted by "Egg", and if "Spam"
>> relies on "Ham", and "Ham" is installed, that's fine, and the
>> installation can go on.
>>
>> Now, if we try to install "Towel", that relies on "Egg", and we have
>> already "Ham" installed, we should just state there is a conflict here.
>
> Why is there a conflict?  This makes no sense whatsoever!
Yes, I was wrong there. My bad. This depends if Egg and Ham are
conflicting or not, rather than if one is obsoleting another.
>
> PEAK-Rules obsoletes RuleDispatch, but there is absolutely no reason
> to not install both.
>
> If you disagree with this, then we have a different meaning of the
> word "Obsoletes"...
>
> In which case, you need to show a concrete example of "Obsoletes"
> using REAL Python packages, not made-up foobar examples, in order to
> illustrate the definition where "obsoletes" means "shouldn't both be
> installed".
>
>
>> By that, I mean that the releases should not be replaced by another
>> ones, even if the new ones obsoletes the old one.
>
> In the specific example you gave, there isn't any conflict -- both
> should be installed, and there is no need to even notify the user.
>
> (Note that forked packages using the same API only conflict because of
> *installing conflicting files* -- so if you're thinking of a fork as
> your example here, it is leading you to confuse the ideas of Obsoletes
> and Conflicts.)
>
I was not thinking about a fork, but about a new rename. That said, I
must admit that the difference between the two fields is sometimes hard
to see.
>
>> the "conflict" field covers another type of conflict, also needed: it
>> would just be used in the way: "both A and B releases can't be
>> *installed* on the same time, whatever could be the cause".
>
> The thing I haven't yet seen an example of is why they can't be
> installed at the same time, that doesn't reduce to a file installation
> conflict.  AFAICT, anything else is *necessarily* a runtime conflict,
> unless the intent is to cover installation of more than just scripts
> and their libraries.  (e.g. MvL's example of creating user IDs)
>
> But if that's the case, ISTM we should wait until we have the software
> that does such installations *first* -- PEP 314 is an example of what
> happens when you create the metadata spec before the tools that use it
> -- nobody fills them in much, because they don't know how they'll be
> used or have any reason to, and then the tools probably end up needing
> different metadata anyway!
+1. This series of mails point out that we definitely need to talk a bit
more about the usability of the fields.
Maybe are we missing something here, and maybe arent we talking about
the same type of conflicts. In any case, we need to clarify all this
*before* doing the implementation stuff.

Cheers,
Alexis


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20100826/f7a39940/attachment.html>

From pje at telecommunity.com  Fri Aug 27 22:47:56 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 27 Aug 2010 16:47:56 -0400
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <4C766265.7020507@notmyidea.org>
References: <4C659D1F.9080707@gmail.com>
	<20100813202044.5CB623A411C@sparrow.telecommunity.com>
	<4C65E45A.6010606@gmail.com>
	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>
	<20100814042601.B276C3A411C@sparrow.telecommunity.com>
	<4C71B490.7030304@notmyidea.org>
	<20100823004501.06EAA3A40A4@sparrow.telecommunity.com>
	<ziade.tarek@gmail.com>
	<AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.com>
	<201008231144.o7NBi9ZH022349@theraft.openend.se>
	<AANLkTin0DZutRmBQ-FcFihR4SF_69P6b+ogawXeFX4m4@mail.gmail.com>
	<20100823170223.E8F013A4108@sparrow.telecommunity.com>
	<4C72B021.4090309@notmyidea.org> <4C72D354.20007@v.loewis.de>
	<4C72D90B.5010209@egenix.com>
	<AANLkTinvoXsF_UrHdSs_KTwdYs5pUnFuKJoQ7rfV1+Qd@mail.gmail.com>
	<20100823230320.8DAB53A4108@sparrow.telecommunity.com>
	<4C73030C.2000307@v.loewis.de>
	<20100824014725.54E3F3A4108@sparrow.telecommunity.com>
	<4C7548A3.5090109@notmyidea.org>
	<20100825195720.7A0223A40A4@sparrow.telecommunity.com>
	<4C766265.7020507@notmyidea.org>
Message-ID: <20100827204815.F21933A409E@sparrow.telecommunity.com>

At 02:47 PM 8/26/2010 +0200, Alexis M?taireau wrote:
>Right, the installation *will not* cause problems, but they will 
>apear after that. If so, we have two choices:
>- First is to tell nothing to the user about that, and I think 
>that's a good option, as it could result in a end user bad 
>experience (if that's not handled at the installation level, what 
>will handle this?)
>- Second is to prompt the user about what he want to do: Do we have 
>to install the release/distribution, even if it can cause problems 
>while running them? Are you sure you want to install both, even if 
>it could lead to such issues ?, etc.

Informing the user about the conflict isn't a bad idea...  but then 
again, it sounds like something that would normally be covered in a 
project's README.

Notice here the difference between the Python environment and a Linux 
distribution: the distribution is not merely *installing* software, 
it is also *deploying* it in some sense -- such software is 
preconfigured as well as installed, and integrated with the system as a whole.

However, that same software as released by its author is not so 
integrated and preconfigured, and thus, these conflict issues cannot 
necessarily even be *known* in advance by the author.

IMO, then, the very ideas of Obsoletes and Conflicts are broken in 
the software *distribution* context, vs. the software "integration" 
or "deployment" context.  System packagers can make these kind of 
evaluations in the context of their supported integration.

However, with the exception of informing people about an obsoleted 
package of theirs, these are not judgments that can be made so 
unequivocally by the individual author of a piece of software, as 
they have no overview of the integrated whole (as a system 
integrator/distributor does).



>>In other words, none of the above is actually a use-case for having 
>>a "Conflicts" field.
>I have pain to imagine use cases where conflicts could raise at the 
>*installation* level too, but the question seems to be: "do we need 
>something to describe installation-related conflicts, or run-related 
>conflicts ? In debian apt system, conflicts are described for 
>"on-run" issues, IIRC.

...and they're described relative to a specific runtime environment, 
which won't really be the case for the individual Python package author.


>>Here's the thing: so far I'm the ONLY person in the discussion who 
>>has even offered an example of when Obsoletes would be used...  and 
>>for the two examples I gave, I would totally be willing to make a 
>>new release for them to include that field, if there was some 
>>benefit (e.g. user notification) to having the field.
>And that's the use case described in the PEP. So, now, we have to 
>choose if it's acceptable to make a new release for updating this 
>field, or not.

I was just pointing out that so far, 100% of the people who have 
given use cases for Obsoletes are fine with having to issue a new 
release to make it Obsoleted-By.  ;-)

(However, PyPI already allows metadata field editing of an 
already-released package, so as long as the new field were added to 
the existing editable fields, that'd be fine too.)



>>(Note that forked packages using the same API only conflict because 
>>of *installing conflicting files* -- so if you're thinking of a 
>>fork as your example here, it is leading you to confuse the ideas 
>>of Obsoletes and Conflicts.)
>I was not thinking about a fork, but about a new rename. That said, 
>I must admit that the difference between the two fields is sometimes 
>hard to see.

At this point, the only reasonable use of either is to inform the 
user about something, and in the case of Obsoletes, it's not 
necessarily even the user who's doing the installing!  If somebody 
installs Foo that depends on RuleDispatch, they are not necessarily 
in a position to port Foo to using PEAK-Rules.  So, only in the case 
where you are declaring a direct dependency from your package to the 
obsoleted package, or where you are directly installing the obsoleted 
package, do you even need to know anything about it.

In the case of conflicts, conflicts presuppose a specific runtime 
environment if the conflict is runtime environment related, and 
again, at best all that can be done is to inform the user.

If the conflict is runtime-process related (e.g. two libraries 
registering the same Python hook), then only a project that declares 
dependencies to both is affected, and only that project's authors 
need to be informed.

Given these conditions, it isn't clear that any of these three cases 
(obsoletes, environment conflict, in-process conflict) constitute 
something that an installation tool should be informing an end-user 
about, vs. the developer or integrator who's declaring the 
dependencies in the first place.

For example, it might make sense for developer-side tools like 
setuptools and buildout to check that one's declared dependencies 
don't have conflicts before pushing out an sdist or bdist or doing an 
install, and to warn for any obsoleted dependencies.  In contrast, an 
end-user tool like pip or easy_install would have little reason to 
check any of this, as their user isn't really affected by anything 
but environment-level conflicts that are unlikely to arise as an 
immediate result of installing the software.



>+1. This series of mails point out that we definitely need to talk a 
>bit more about the usability of the fields.
>Maybe are we missing something here, and maybe arent we talking 
>about the same type of conflicts. In any case, we need to clarify 
>all this *before* doing the implementation stuff.

I think that the main thing being missed is that these fields are 
being copied from a completely different kind of use case: the 
concept of a "package" as a bundle of preconfigured software that is 
also pre-integrated with a uniform operating environment.  In that 
context, it is the curators of that overall environment who are in a 
position to make judgments regarding what is a "conflict", or what 
packages will "obsolete" each other in the context of that environment.

Lacking such an overall curator organization in the PyPI environment 
(and not desiring one, anyway, since PyPI is inherently a software 
distribution *catalog* rather than a platform of its own), these 
fields make very little sense.  They are perhaps most useful as a 
centralized way to pass along notifications to developers (or to 
curator/integrators of system packages), but have little or no impact 
on the software installation process itself.

I also suspect that if you ask those curator/integrators of system 
packages, that they will say that the information in these fields 
would be of little value to them in any case, as it's unlikely that 
the author will *know* about the kinds of conflicts they'd be 
concerned about in their packaging process.  But it would certainly 
be worth asking them.

More to the point though...  I wonder whose idea it was to have these 
fields in the first place?  Neither PEP 314 nor PEP 345 don't really 
say, "Oh, such and such people asked for these fields for thus-and-so 
use cases", and in the absence of such requestors and use cases, it 
would probably be best just to drop them entirely.  (Unless of course 
Fred offers a description of what his actual use case for Conflicts 
is/was, and it doesn't fall into one of the categories previously discussed.)


From tseaver at palladion.com  Fri Aug 27 23:08:08 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 27 Aug 2010 17:08:08 -0400
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <20100827204815.F21933A409E@sparrow.telecommunity.com>
References: <4C659D1F.9080707@gmail.com>	<AANLkTi=d6ycx3g7wsBiH1qs66Mw==hcYzX3XMCFgRq62@mail.gmail.com>	<20100814042601.B276C3A411C@sparrow.telecommunity.com>	<4C71B490.7030304@notmyidea.org>	<20100823004501.06EAA3A40A4@sparrow.telecommunity.com>	<ziade.tarek@gmail.com>	<AANLkTi=aEX=64tTwWLmzq8vzUf4_t67376o-Xf17ZVw2@mail.gmail.com>	<201008231144.o7NBi9ZH022349@theraft.openend.se>	<AANLkTin0DZutRmBQ-FcFihR4SF_69P6b+ogawXeFX4m4@mail.gmail.com>	<20100823170223.E8F013A4108@sparrow.telecommunity.com>	<4C72B021.4090309@notmyidea.org>
	<4C72D354.20007@v.loewis.de>	<4C72D90B.5010209@egenix.com>	<AANLkTinvoXsF_UrHdSs_KTwdYs5pUnFuKJoQ7rfV1+Qd@mail.gmail.com>	<20100823230320.8DAB53A4108@sparrow.telecommunity.com>	<4C73030C.2000307@v.loewis.de>	<20100824014725.54E3F3A4108@sparrow.telecommunity.com>	<4C7548A3.5090109@notmyidea.org>	<20100825195720.7A0223A40A4@sparrow.telecommunity.com>	<4C766265.7020507@notmyidea.org>
	<20100827204815.F21933A409E@sparrow.telecommunity.com>
Message-ID: <i599fp$bib$1@dough.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

P.J. Eby wrote:
> At 02:47 PM 8/26/2010 +0200, Alexis M?taireau wrote:
>> Right, the installation *will not* cause problems, but they will 
>> apear after that. If so, we have two choices:
>> - First is to tell nothing to the user about that, and I think 
>> that's a good option, as it could result in a end user bad 
>> experience (if that's not handled at the installation level, what 
>> will handle this?)
>> - Second is to prompt the user about what he want to do: Do we have 
>> to install the release/distribution, even if it can cause problems 
>> while running them? Are you sure you want to install both, even if 
>> it could lead to such issues ?, etc.
> 
> Informing the user about the conflict isn't a bad idea...  but then 
> again, it sounds like something that would normally be covered in a 
> project's README.
> 
> Notice here the difference between the Python environment and a Linux 
> distribution: the distribution is not merely *installing* software, 
> it is also *deploying* it in some sense -- such software is 
> preconfigured as well as installed, and integrated with the system as a whole.
> 
> However, that same software as released by its author is not so 
> integrated and preconfigured, and thus, these conflict issues cannot 
> necessarily even be *known* in advance by the author.
> 
> IMO, then, the very ideas of Obsoletes and Conflicts are broken in 
> the software *distribution* context, vs. the software "integration" 
> or "deployment" context.  System packagers can make these kind of 
> evaluations in the context of their supported integration.
> 
> However, with the exception of informing people about an obsoleted 
> package of theirs, these are not judgments that can be made so 
> unequivocally by the individual author of a piece of software, as 
> they have no overview of the integrated whole (as a system 
> integrator/distributor does).
> 
> 
> 
>>> In other words, none of the above is actually a use-case for having 
>>> a "Conflicts" field.
>> I have pain to imagine use cases where conflicts could raise at the 
>> *installation* level too, but the question seems to be: "do we need 
>> something to describe installation-related conflicts, or run-related 
>> conflicts ? In debian apt system, conflicts are described for 
>> "on-run" issues, IIRC.
> 
> ...and they're described relative to a specific runtime environment, 
> which won't really be the case for the individual Python package author.
> 
> 
>>> Here's the thing: so far I'm the ONLY person in the discussion who 
>>> has even offered an example of when Obsoletes would be used...  and 
>>> for the two examples I gave, I would totally be willing to make a 
>>> new release for them to include that field, if there was some 
>>> benefit (e.g. user notification) to having the field.
>> And that's the use case described in the PEP. So, now, we have to 
>> choose if it's acceptable to make a new release for updating this 
>> field, or not.
> 
> I was just pointing out that so far, 100% of the people who have 
> given use cases for Obsoletes are fine with having to issue a new 
> release to make it Obsoleted-By.  ;-)
> 
> (However, PyPI already allows metadata field editing of an 
> already-released package, so as long as the new field were added to 
> the existing editable fields, that'd be fine too.)
> 
> 
> 
>>> (Note that forked packages using the same API only conflict because 
>>> of *installing conflicting files* -- so if you're thinking of a 
>>> fork as your example here, it is leading you to confuse the ideas 
>>> of Obsoletes and Conflicts.)
>> I was not thinking about a fork, but about a new rename. That said, 
>> I must admit that the difference between the two fields is sometimes 
>> hard to see.
> 
> At this point, the only reasonable use of either is to inform the 
> user about something, and in the case of Obsoletes, it's not 
> necessarily even the user who's doing the installing!  If somebody 
> installs Foo that depends on RuleDispatch, they are not necessarily 
> in a position to port Foo to using PEAK-Rules.  So, only in the case 
> where you are declaring a direct dependency from your package to the 
> obsoleted package, or where you are directly installing the obsoleted 
> package, do you even need to know anything about it.
> 
> In the case of conflicts, conflicts presuppose a specific runtime 
> environment if the conflict is runtime environment related, and 
> again, at best all that can be done is to inform the user.
> 
> If the conflict is runtime-process related (e.g. two libraries 
> registering the same Python hook), then only a project that declares 
> dependencies to both is affected, and only that project's authors 
> need to be informed.
> 
> Given these conditions, it isn't clear that any of these three cases 
> (obsoletes, environment conflict, in-process conflict) constitute 
> something that an installation tool should be informing an end-user 
> about, vs. the developer or integrator who's declaring the 
> dependencies in the first place.
> 
> For example, it might make sense for developer-side tools like 
> setuptools and buildout to check that one's declared dependencies 
> don't have conflicts before pushing out an sdist or bdist or doing an 
> install, and to warn for any obsoleted dependencies.  In contrast, an 
> end-user tool like pip or easy_install would have little reason to 
> check any of this, as their user isn't really affected by anything 
> but environment-level conflicts that are unlikely to arise as an 
> immediate result of installing the software.
> 
> 
> 
>> +1. This series of mails point out that we definitely need to talk a 
>> bit more about the usability of the fields.
>> Maybe are we missing something here, and maybe arent we talking 
>> about the same type of conflicts. In any case, we need to clarify 
>> all this *before* doing the implementation stuff.
> 
> I think that the main thing being missed is that these fields are 
> being copied from a completely different kind of use case: the 
> concept of a "package" as a bundle of preconfigured software that is 
> also pre-integrated with a uniform operating environment.  In that 
> context, it is the curators of that overall environment who are in a 
> position to make judgments regarding what is a "conflict", or what 
> packages will "obsolete" each other in the context of that environment.
> 
> Lacking such an overall curator organization in the PyPI environment 
> (and not desiring one, anyway, since PyPI is inherently a software 
> distribution *catalog* rather than a platform of its own), these 
> fields make very little sense.  They are perhaps most useful as a 
> centralized way to pass along notifications to developers (or to 
> curator/integrators of system packages), but have little or no impact 
> on the software installation process itself.
> 
> I also suspect that if you ask those curator/integrators of system 
> packages, that they will say that the information in these fields 
> would be of little value to them in any case, as it's unlikely that 
> the author will *know* about the kinds of conflicts they'd be 
> concerned about in their packaging process.  But it would certainly 
> be worth asking them.
> 
> More to the point though...  I wonder whose idea it was to have these 
> fields in the first place?  Neither PEP 314 nor PEP 345 don't really 
> say, "Oh, such and such people asked for these fields for thus-and-so 
> use cases", and in the absence of such requestors and use cases, it 
> would probably be best just to drop them entirely.  (Unless of course 
> Fred offers a description of what his actual use case for Conflicts 
> is/was, and it doesn't fall into one of the categories previously discussed.)

FWIW, I helped add those fields to PEP 345, in the context of making the
switch from "module / package"-centric values to distribution-centric
ones.  The requirements came out of a sprint at PyCon 2009, with input
from both the Debian / Ubuntu Python packager (Matthias Klose) and one
of the Fedora packagers (Toshio Kuratomi).


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkx4KTQACgkQ+gerLs4ltQ7AFwCgycG3CP9sDjKMfOljCS69c72r
ayAAn1rr/aeWVrhQdUjMbuWyAi7efgez
=SfA+
-----END PGP SIGNATURE-----


From pje at telecommunity.com  Sat Aug 28 05:27:46 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 27 Aug 2010 23:27:46 -0400
Subject: [Catalog-sig] PEP 345 Update
Message-ID: <20100828032757.30F963A409E@sparrow.telecommunity.com>

At 05:08 PM 8/27/2010 -0400, Tres Seaver wrote:
>P.J. Eby wrote:
> > I also suspect that if you ask those curator/integrators of system
> > packages, that they will say that the information in these fields
> > would be of little value to them in any case, as it's unlikely that
> > the author will *know* about the kinds of conflicts they'd be
> > concerned about in their packaging process.  But it would certainly
> > be worth asking them.
> >
> > More to the point though...  I wonder whose idea it was to have these
> > fields in the first place?  Neither PEP 314 nor PEP 345 don't really
> > say, "Oh, such and such people asked for these fields for thus-and-so
> > use cases", and in the absence of such requestors and use cases, it
> > would probably be best just to drop them entirely.  (Unless of course
> > Fred offers a description of what his actual use case for Conflicts
> > is/was, and it doesn't fall into one of the categories previously 
> discussed.)
>
>FWIW, I helped add those fields to PEP 345, in the context of making the
>switch from "module / package"-centric values to distribution-centric
>ones.  The requirements came out of a sprint at PyCon 2009, with input
>from both the Debian / Ubuntu Python packager (Matthias Klose) and one
>of the Fedora packagers (Toshio Kuratomi).

Great - could we get them to join in on this discussion to explain 
what it is that they want the creators of Python libraries to put in 
these fields, and what they intend to do with the information once it's there?

At this point, we don't even know what that information is, really, 
let alone whether it's something that a package author can actually provide.

(Btw, my comment about suspecting that they would say it's of little 
value is because other Linux distro packagers have said on the 
distutils list previously that author-provided dependency information 
was generally of little use to them; I assumed the same would be even 
more true for fields like these that require the author to step even 
further outside their immediate knowledge.)