From alexis at notmyidea.org  Tue Oct  5 03:26:17 2010
From: alexis at notmyidea.org (=?ISO-8859-1?Q?Alexis_M=E9taireau?=)
Date: Tue, 05 Oct 2010 02:26:17 +0100
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <20100828032757.30F963A409E@sparrow.telecommunity.com>
References: <20100828032757.30F963A409E@sparrow.telecommunity.com>
Message-ID: <4CAA7EB9.2020406@notmyidea.org>

Hey all,

Sorry for this long silence; I'm now back on track.

Le 08/28/2010 04:27 AM, P.J. Eby a ?crit :
> At 05:08 PM 8/27/2010 -0400, Tres Seaver wrote:
>> P.J. Eby wrote:
>> FWIW, I helped add those fields to PEP 345, in the context of making the
>> switch from "module / package"-centric values to distribution-centric
>> ones.  The requirements came out of a sprint at PyCon 2009, with input
>> from both the Debian / Ubuntu Python packager (Matthias Klose) and one
>> of the Fedora packagers (Toshio Kuratomi).
> 
> Great - could we get them to join in on this discussion to explain what
> it is that they want the creators of Python libraries to put in these
> fields, and what they intend to do with the information once it's there?
+1 on that. We need to have inputs on that: that's good to have fields,
but it's more important to know how they are supposed to be considered.

Any news about that ? Do someone have email adresses of Matthias Klose
and Toshio Kuratomi ?

Also, it's important to notice there is a difference between the new
field (release-conflict IIRC), and I assume it's important to have a
discussion about it know, and the others fields that are just being
renamed. (*-release fields). Maybe could we update the PEP a first time
about the renaming changes, in order to use them in the first version of
Distutils2, and them continue the discussion about the conflict field.

Alexis


From pnasrat at google.com  Tue Oct  5 18:26:57 2010
From: pnasrat at google.com (Paul Nasrat)
Date: Tue, 5 Oct 2010 09:26:57 -0700
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <4CAA7EB9.2020406@notmyidea.org>
References: <20100828032757.30F963A409E@sparrow.telecommunity.com>
	<4CAA7EB9.2020406@notmyidea.org>
Message-ID: <AANLkTikYguzQ==_56xhz6_=WqBZkzrbbFBu9JmNoh9YL@mail.gmail.com>

On Mon, Oct 4, 2010 at 6:26 PM, Alexis M?taireau <alexis at notmyidea.org> wrote:
> Hey all,
>
> Sorry for this long silence; I'm now back on track.
>
> Le 08/28/2010 04:27 AM, P.J. Eby a ?crit :
>> At 05:08 PM 8/27/2010 -0400, Tres Seaver wrote:
>>> P.J. Eby wrote:
>>> FWIW, I helped add those fields to PEP 345, in the context of making the
>>> switch from "module / package"-centric values to distribution-centric
>>> ones. ?The requirements came out of a sprint at PyCon 2009, with input
>>> from both the Debian / Ubuntu Python packager (Matthias Klose) and one
>>> of the Fedora packagers (Toshio Kuratomi).
>>
>> Great - could we get them to join in on this discussion to explain what
>> it is that they want the creators of Python libraries to put in these
>> fields, and what they intend to do with the information once it's there?
> +1 on that. We need to have inputs on that: that's good to have fields,
> but it's more important to know how they are supposed to be considered.
>
> Any news about that ? Do someone have email adresses of Matthias Klose
> and Toshio Kuratomi ?

Fedora python people are listed here:

http://fedoraproject.org/wiki/SIGs/Python

http://fedoraproject.org/wiki/ToshioKuratomi should have enough for
you to email him.

Paul

From pje at telecommunity.com  Tue Oct  5 19:57:59 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 05 Oct 2010 13:57:59 -0400
Subject: [Catalog-sig] PEP 345 Update
Message-ID: <20101005175753.713233A4105@sparrow.telecommunity.com>

At 02:26 AM 10/5/2010 +0100, Alexis M?taireau wrote:
>Hey all,
>
>Sorry for this long silence; I'm now back on track.
>
>Le 08/28/2010 04:27 AM, P.J. Eby a ?crit :
> > At 05:08 PM 8/27/2010 -0400, Tres Seaver wrote:
> >> P.J. Eby wrote:
> >> FWIW, I helped add those fields to PEP 345, in the context of making the
> >> switch from "module / package"-centric values to distribution-centric
> >> ones.  The requirements came out of a sprint at PyCon 2009, with input
> >> from both the Debian / Ubuntu Python packager (Matthias Klose) and one
> >> of the Fedora packagers (Toshio Kuratomi).
> >
> > Great - could we get them to join in on this discussion to explain what
> > it is that they want the creators of Python libraries to put in these
> > fields, and what they intend to do with the information once it's there?
>+1 on that. We need to have inputs on that: that's good to have fields,
>but it's more important to know how they are supposed to be considered.
>
>Any news about that ? Do someone have email adresses of Matthias Klose
>and Toshio Kuratomi ?
>
>Also, it's important to notice there is a difference between the new
>field (release-conflict IIRC), and I assume it's important to have a
>discussion about it know, and the others fields that are just being
>renamed. (*-release fields). Maybe could we update the PEP a first time
>about the renaming changes, in order to use them in the first version of
>Distutils2, and them continue the discussion about the conflict field.

If a field doesn't have clear semantics, there is no point in 
allowing people to specify it - it is literally adding a new field to 
a database and inviting the entire world to put whatever they want in 
it...  which means you end up with a worthless database (at least in 
that column).

While this isn't the first metadata PEP with this problem, it would 
be nice if the previous one were the last.  ;-)


From alexis at notmyidea.org  Tue Oct  5 22:30:58 2010
From: alexis at notmyidea.org (=?ISO-8859-1?Q?Alexis_M=E9taireau?=)
Date: Tue, 05 Oct 2010 21:30:58 +0100
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <20101005170519.491213A4105@sparrow.telecommunity.com>
References: <20100828032757.30F963A409E@sparrow.telecommunity.com>
	<4CAA7EB9.2020406@notmyidea.org>
	<20101005170519.491213A4105@sparrow.telecommunity.com>
Message-ID: <4CAB8B02.7080503@notmyidea.org>

Le 10/05/2010 06:05 PM, P.J. Eby a ?crit :
> If a field doesn't have clear semantics, there is no point in allowing
> people to specify it - it is literally adding a new field to a database
> and inviting the entire world to put whatever they want in it...  which
> means you end up with a worthless database (at least in that column).

That's true.

BTW, we dont agree on the utility of this field because we're not sure
how it's needed to be considered. It could be simple, for packagers, to
say: "okay, I *know* that my release is not compatible with another
specific one".

We have to think about this conlict scope, because we do not agree on
that too: are the fields made for "installation scope", or for "running
scope".

I mean: is that "conflict" field means that "it's impossible to
*install* the both distributions at the same time", or that "it's
impossible to *run* them at the same time" ?

And, maybe do we need a way to specify those two cases.

After reading a bit again this thread, it comes that we have another
proposition, of an "obsoleted-by" field, which can replace the
"obsoletes" one (for renames).

To resume the pro and the cons of each:

1/ obsoleted-by:
   - Pro: It allows the package owners to manage what will obsolete they
releases; And avoid malicious usages.
   - Cons: It will need package owners to re-issue a distribution with
this changes.

2/ obsoletes:
   - Pro: Don't need to re-issue a distribution.
   - Cons: Anyone can tell he obsoletes anyone else package.

What do you think we should do with that ? That's a fact that this PEP
need to be revamped a bit, with the questions we have introduced in mind.

> While this isn't the first metadata PEP with this problem, it would be
> nice if the previous one were the last.  ;-)
Yep, so let's work on this problems to fix them.

Alexis

From pje at telecommunity.com  Wed Oct  6 00:35:57 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 05 Oct 2010 18:35:57 -0400
Subject: [Catalog-sig] PEP 345 Update
Message-ID: <20101005223550.84F223A4105@sparrow.telecommunity.com>

At 09:30 PM 10/5/2010 +0100, Alexis M?taireau wrote:
>Le 10/05/2010 06:05 PM, P.J. Eby a ?crit :
> > If a field doesn't have clear semantics, there is no point in allowing
> > people to specify it - it is literally adding a new field to a database
> > and inviting the entire world to put whatever they want in it...  which
> > means you end up with a worthless database (at least in that column).
>
>That's true.
>
>BTW, we dont agree on the utility of this field because we're not sure
>how it's needed to be considered. It could be simple, for packagers, to
>say: "okay, I *know* that my release is not compatible with another
>specific one".
>
>We have to think about this conlict scope, because we do not agree on
>that too: are the fields made for "installation scope", or for "running
>scope".
>
>I mean: is that "conflict" field means that "it's impossible to
>*install* the both distributions at the same time", or that "it's
>impossible to *run* them at the same time" ?
>
>And, maybe do we need a way to specify those two cases.


Quick summary of my previous comments on this point:

* In the simplest case, installation conflict equals conflicting 
files -- which any installation tool *must* be able to handle on its 
own. (Otherwise, it will break in the event of an undeclared conflict.)

* The only way for other installation conflicts to exist is if the 
installation tools is going to set up user IDs, cronjobs, etc., which 
are properly *application configuration*, rather than software.

* The same is true for runtime conflicts: they once again represent 
*configuration* conflicts -- i.e., in general, to have a runtime 
conflict between two pieces of code, they must have overlapping 
configurations, or be non-configurable.

In short, the only way to specify that a piece of software *does* 
conflict with another one at runtime, is if you *fully specify the 
configuration* of both pieces of software; e.g., if neither package 
allows any configuration whatsoever of ports used, user IDs, 
etc.  All other conflicts can be reduced to file-level conflicts.

Because of this, the conflict field can have no function except to 
notify a user of the *possibility* of a conflict.  An installation 
tool MUST still check for file conflicts, even if they are not 
declared, and MUST still allow the user to install software that is 
declared conflicting, but which does not include any file-level 
install-time conflicts.

The reason that existing OS-level packaging tools have enforced 
"conflict" fields is because they are *system integration* packaging 
tools, not generic software installation.  A distro like Fedora or 
Ubuntu represents a set of predefined application *configurations*, 
not just raw software.  Because of this, they can (and should) 
enforce conflict declarations, because they know precisely how each 
package is configured.

However, two packages that conflict in the Fedora configuration, do 
not necessarily conflict in the Ubuntu configuration, nor vice 
versa.  This means that a Python-supplied "conflicts" field can only 
be advisory at best.


>After reading a bit again this thread, it comes that we have another
>proposition, of an "obsoleted-by" field, which can replace the
>"obsoletes" one (for renames).
>
>To resume the pro and the cons of each:
>
>1/ obsoleted-by:
>    - Pro: It allows the package owners to manage what will obsolete they
>releases; And avoid malicious usages.
>    - Cons: It will need package owners to re-issue a distribution with
>this changes.
>
>2/ obsoletes:
>    - Pro: Don't need to re-issue a distribution.
>    - Cons: Anyone can tell he obsoletes anyone else package.
>
>What do you think we should do with that ? That's a fact that this PEP
>need to be revamped a bit, with the questions we have introduced in mind.

"Obsoleted-By" is a field with actual use cases; i.e., there are 
packages I have right now that I wouldn't mind pushing a new 
distribution of to specify their "Obsoleted-By" fields.  (Assuming I 
were using a sufficient distutils version, of course.)

"Obsoletes", on the other hand, has not received any similar support 
in this thread.

Of course, in either case the field must still be for human 
consumption (and tool warning messages) only.  If a developer 
declares a dependency on an obsolete package, they should be 
notified; but an end-user who's merely installing the package in 
order to satisfy another package's requirements doesn't need to be warned.


From jeremy.kloth at gmail.com  Wed Oct  6 00:50:08 2010
From: jeremy.kloth at gmail.com (Jeremy Kloth)
Date: Tue, 5 Oct 2010 16:50:08 -0600
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <20101005223550.84F223A4105@sparrow.telecommunity.com>
References: <20101005223550.84F223A4105@sparrow.telecommunity.com>
Message-ID: <201010051650.09263.jeremy.kloth@gmail.com>

On Tuesday, October 05, 2010 04:35:57 pm P.J. Eby wrote:
> At 09:30 PM 10/5/2010 +0100, Alexis M?taireau wrote:
> >We have to think about this conlict scope, because we do not agree on
> >that too: are the fields made for "installation scope", or for "running
> >scope".
> 
> Quick summary of my previous comments on this point:
> 
> * In the simplest case, installation conflict equals conflicting
> files -- which any installation tool *must* be able to handle on its
> own. (Otherwise, it will break in the event of an undeclared conflict.)

Here you are assuming that a file list exists for the given distribution. 
sdists, for one, do not contain this.

What about tools that operate just on the metadata?  For example, determining 
what distributions to download.  It would be impossible for that to work.  It 
seems to me that you are seeing this problem only from an *installing* 
standpoint.

-- 
Jeremy Kloth

From pje at telecommunity.com  Wed Oct  6 01:45:07 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 05 Oct 2010 19:45:07 -0400
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <201010051650.09263.jeremy.kloth@gmail.com>
References: <20101005223550.84F223A4105@sparrow.telecommunity.com>
	<201010051650.09263.jeremy.kloth@gmail.com>
Message-ID: <20101005234501.E89CA3A4108@sparrow.telecommunity.com>

At 04:50 PM 10/5/2010 -0600, Jeremy Kloth wrote:
>Here you are assuming that a file list exists for the given distribution.
>sdists, for one, do not contain this.

They will once they're being installed.


>What about tools that operate just on the metadata?  For example, determining
>what distributions to download.  It would be impossible for that to work.

So?  If nobody noticed or declared the conflict, you'll still have a 
problem once you *do* download.

Meanwhile, you can't *trust* the information in these fields to mean 
that you actually shouldn't attempt to download or install the 
package, so all you can do with the information ahead of time is 
alert the user of the *possibility* of a conflict.

IOW, merely having a "Conflicts" field will not stop you from 
sometimes downloading things that conflict with each other, nor does 
the presence of the Conflicts field actually mean you shouldn't 
download and install it anyway. (Because there might not really be 
any conflict, especially if the field gets used by people who don't 
read specifications.)


From alexis at notmyidea.org  Wed Oct  6 09:39:03 2010
From: alexis at notmyidea.org (=?ISO-8859-1?Q?Alexis_M=E9taireau?=)
Date: Wed, 06 Oct 2010 08:39:03 +0100
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <20101005223550.84F223A4105@sparrow.telecommunity.com>
References: <20101005223550.84F223A4105@sparrow.telecommunity.com>
Message-ID: <4CAC2797.3040800@notmyidea.org>

Le 10/05/2010 11:35 PM, P.J. Eby a ?crit :
> Quick summary of my previous comments on this point:
Thanks for that, it helps.

> * In the simplest case, installation conflict equals conflicting files
> -- which any installation tool *must* be able to handle on its own.
> (Otherwise, it will break in the event of an undeclared conflict.) Foo
Even if we check for this kind of conflicts, a "conflict" field allows
us to do less operations, and to be quicker to find wrong cases, and
print something to the user.

If "foo 1.0" and "bar 1.0" are well known conflicting releases, we could
avoid the check of all the files to output an error.

Then, if the user really wants to install the two conflicting releases,
he could do so using some "--force" parameter, with installers, to avoid
the check of this "conflict" field.

> * The only way for other installation conflicts to exist is if the
> installation tools is going to set up user IDs, cronjobs, etc., which
> are properly *application configuration*, rather than software.
> 
> * The same is true for runtime conflicts: they once again represent
> *configuration* conflicts -- i.e., in general, to have a runtime
> conflict between two pieces of code, they must have overlapping
> configurations, or be non-configurable.
> 
> In short, the only way to specify that a piece of software *does*
> conflict with another one at runtime, is if you *fully specify the
> configuration* of both pieces of software; e.g., if neither package
> allows any configuration whatsoever of ports used, user IDs, etc.  All
> other conflicts can be reduced to file-level conflicts.

So we agree there is conflicts that cannot be find a simple way at the
installation time (thanks for providing those examples). I agree that
the conflict field have to stand for a *possibility* of a conflict.
> 
> Because of this, the conflict field can have no function except to
> notify a user of the *possibility* of a conflict.  An installation tool
> MUST still check for file conflicts, even if they are not declared, and
> MUST still allow the user to install software that is declared
> conflicting, but which does not include any file-level install-time
> conflicts.
+1.
> 
> The reason that existing OS-level packaging tools have enforced
> "conflict" fields is because they are *system integration* packaging
> tools, not generic software installation.  A distro like Fedora or
> Ubuntu represents a set of predefined application *configurations*, not
> just raw software.  Because of this, they can (and should) enforce
> conflict declarations, because they know precisely how each package is
> configured.

Yep. I've discussed yesterday with Toshio Kuratomi, who have clearly
indicated to me that at the distro level, they are packaging with
configuration files, and that the first way to avoid a conflict, when
possible, will be to configure it in a way it does not conflict anymore.

> However, two packages that conflict in the Fedora configuration, do not
> necessarily conflict in the Ubuntu configuration, nor vice versa.  This
> means that a Python-supplied "conflicts" field can only be advisory at
> best.

This lead to another question: Should we use exactly the same fields
with the same meanings, depending the context ? I think the answer is no.

For instance, we could interpret the "conflicts" field differently if we
want to install the software "as is" (in which case I think it's better
to trust it), than if we want to use that field in a distro context (as
the distros can provide configuration for the python distributions, they
can resolve those conflicts). BTW, if we provide a "conflict" field, it
can help them to detect such the simple way.

>> After reading a bit again this thread, it comes that we have another
>> proposition, of an "obsoleted-by" field, which can replace the
>> "obsoletes" one (for renames).
>>
>> To resume the pro and the cons of each:
>>
>> 1/ obsoleted-by:
>>    - Pro: It allows the package owners to manage what will obsolete they
>> releases; And avoid malicious usages.
>>    - Cons: It will need package owners to re-issue a distribution with
>> this changes.
>>
>> 2/ obsoletes:
>>    - Pro: Don't need to re-issue a distribution.
>>    - Cons: Anyone can tell he obsoletes anyone else package.
>>
>> What do you think we should do with that ? That's a fact that this PEP
>> need to be revamped a bit, with the questions we have introduced in mind.
> 
> "Obsoleted-By" is a field with actual use cases; i.e., there are
> packages I have right now that I wouldn't mind pushing a new
> distribution of to specify their "Obsoleted-By" fields.  (Assuming I
> were using a sufficient distutils version, of course.)
> 
> "Obsoletes", on the other hand, has not received any similar support in
> this thread.

Any other ideas in favor of obsoletes, or in defavor of obsoleted-by,
before I start editing the PEP again ?

> Of course, in either case the field must still be for human consumption
> (and tool warning messages) only.  If a developer declares a dependency
> on an obsolete package, they should be notified; but an end-user who's
> merely installing the package in order to satisfy another package's
> requirements doesn't need to be warned.

I think it's a good deal, and we could this way add such a check in the
distutils2 "check" command. That way, it'll allow people who want to
rely on "obsoleted" dists to do so.

Great, things goes on!

Alexis

From pje at telecommunity.com  Wed Oct  6 19:17:56 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 06 Oct 2010 13:17:56 -0400
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <4CAC2797.3040800@notmyidea.org>
References: <20101005223550.84F223A4105@sparrow.telecommunity.com>
	<4CAC2797.3040800@notmyidea.org>
Message-ID: <20101006171752.A31743A4100@sparrow.telecommunity.com>

At 08:39 AM 10/6/2010 +0100, Alexis M?taireau wrote:
>So we agree there is conflicts that cannot be find a simple way at the
>installation time (thanks for providing those examples). I agree that
>the conflict field have to stand for a *possibility* of a conflict.

I would suggest, therefore, that the conflict field format be changed 
to include a brief human-readable description of the nature of the 
conflict(s), and a URL where the user can obtain more information.


>If "foo 1.0" and "bar 1.0" are well known conflicting releases, we could
>avoid the check of all the files to output an error.

Of course.  But if we are just caching file-conflict information, 
then perhaps we should just list the files that will be installed, 
and then there is no need for the developer of either package to:

1. figure out that the conflict exists,
2. re-release their package with the information, and
3. keep it up-to-date (and re-release it) when the files change *in 
either package*!

Let's once again take an actual (historically-occuring) use case: 
PyDispatcher and RuleDispatch.  When both packages were released, 
they included a package named "dispatch", and so somebody downloading 
both would've only detected the problem at installation time.

Let's say that I responded to this situation by adding a 
"Conflicts-Release: PyDispatcher" to RuleDispatch's 
distribution.  Well, not too long after we both knew about the 
conflict, the author of PyDispatcher actually changed his package 
name, alleviating the conflict...  but (historically) I didn't find 
out about this until much later.

So then, the Conflicts-Release in RuleDispatch would have been 
incorrect: a user with the newer PyDispatcher installed would be 
warned to *not* install RuleDispatch, even though there's no conflict any more.

Again, this is a case where the lack of an authoritative third-party 
administering the metadata makes the whole thing a mess.

Now instead, consider what happens if we simply list the files to be 
installed, or some portion thereof.  Now, the release of a new 
version of PyDispatcher resolves the conflict immediately and 
transparently, without any explicit action by either author, or by 
the user.  (And it's not teaching users to force installation due to 
widespread inaccuracies in the metadata.)

So, if you review this scenario, what you will see is that using a 
Conflicts-Release field to manage file-level conflicts is actually 
*worse* for EVERYBODY in this scenario, users and developers alike!

IOW, if you want to gain the benefit of being able to pre-detect 
file-level conflicts, the only safe solution is to have a way to e.g. 
query PyPI for installation manifests, or include some sort of 
metadata for that in the spec.  Expecting the developers to 
*manually* maintain file conflict information (and especially to 
correct it when the problem is resolved!) is not a good tradeoff vs. 
the minor extra time needed to download the file and notice the conflict.

If for some reason it can't be done this way, the alternative would 
be to manually specify something like a Conflicts-Files field that 
lists some filenames known to conflict with certain other 
packages.  The tool doing the fetching can then check whether *those* 
files are already installed.  Then, if I changed RuleDispatch to say 
there were potential conflicts on dispatch/__init__.py, then tools 
would be able to notice that yes, there's a problem, or no, there's 
not.  And I wouldn't need to say *which* other project releases might 
be conflicting -- once the conflict was declared, then any project 
including those files would conflict, and any project that ended its 
conflict would become nonconflicting automatically.

Or, you know, we could just let people find out at installation time, 
and complain to the package author(s), like they do now.  ;-)

(Btw: part of the rationale for creating .egg files in the first 
place was that they *can't* have installation conflicts with each 
other, except for scripts...  and scripts weren't part of the 
original use case.)


>This lead to another question: Should we use exactly the same fields
>with the same meanings, depending the context ? I think the answer is no.
>
>For instance, we could interpret the "conflicts" field differently if we
>want to install the software "as is" (in which case I think it's better
>to trust it), than if we want to use that field in a distro context (as
>the distros can provide configuration for the python distributions, they
>can resolve those conflicts). BTW, if we provide a "conflict" field, it
>can help them to detect such the simple way.

I don't think I understood either of these paragraphs.  Could you elaborate?


> > Of course, in either case the field must still be for human consumption
> > (and tool warning messages) only.  If a developer declares a dependency
> > on an obsolete package, they should be notified; but an end-user who's
> > merely installing the package in order to satisfy another package's
> > requirements doesn't need to be warned.
>
>I think it's a good deal, and we could this way add such a check in the
>distutils2 "check" command. That way, it'll allow people who want to
>rely on "obsoleted" dists to do so.

Makes sense to me.


From monitor at jacobian.org  Thu Oct  7 06:44:27 2010
From: monitor at jacobian.org (monitor at jacobian.org)
Date: Wed, 06 Oct 2010 23:44:27 -0500
Subject: [Catalog-sig] [monit] pypi.python.org - Connection succeeded
Message-ID: <1286426673.1@jacobian.org>

Connection succeeded Service pypi.python.org 

	Date:        Wed, 06 Oct 2010 23:44:27 -0500
	Action:      alert
	Host:        jacobian.org
	Description: connection succeeded to INET[pypi.python.org:80] via TCP

Your faithful employee,
monit


From monitor at jacobian.org  Thu Oct  7 02:32:11 2010
From: monitor at jacobian.org (monitor at jacobian.org)
Date: Wed, 06 Oct 2010 19:32:11 -0500
Subject: [Catalog-sig] [monit] pypi.python.org - Connection failed
Message-ID: <1286411534.1@jacobian.org>

Connection failed Service pypi.python.org 

	Date:        Wed, 06 Oct 2010 19:32:11 -0500
	Action:      alert
	Host:        jacobian.org
	Description: failed protocol test [HTTP] at INET[pypi.python.org:80] via TCP

Your faithful employee,
monit


From monitor at jacobian.org  Thu Oct  7 00:26:04 2010
From: monitor at jacobian.org (monitor at jacobian.org)
Date: Wed, 06 Oct 2010 17:26:04 -0500
Subject: [Catalog-sig] [monit] pypi.python.org - Connection succeeded
Message-ID: <1286403966.1@jacobian.org>

Connection succeeded Service pypi.python.org 

	Date:        Wed, 06 Oct 2010 17:26:04 -0500
	Action:      alert
	Host:        jacobian.org
	Description: connection succeeded to INET[pypi.python.org:80] via TCP

Your faithful employee,
monit


From monitor at jacobian.org  Thu Oct  7 00:02:31 2010
From: monitor at jacobian.org (monitor at jacobian.org)
Date: Wed, 06 Oct 2010 17:02:31 -0500
Subject: [Catalog-sig] [monit] pypi.python.org - Connection failed
Message-ID: <1286402556.1@jacobian.org>

Connection failed Service pypi.python.org 

	Date:        Wed, 06 Oct 2010 17:02:31 -0500
	Action:      alert
	Host:        jacobian.org
	Description: failed protocol test [HTTP] at INET[pypi.python.org:80] via TCP

Your faithful employee,
monit


From alexis at notmyidea.org  Fri Oct  8 17:49:06 2010
From: alexis at notmyidea.org (=?ISO-8859-1?Q?Alexis_M=E9taireau?=)
Date: Fri, 08 Oct 2010 16:49:06 +0100
Subject: [Catalog-sig] PEP 345 Update
In-Reply-To: <20101006171752.A31743A4100@sparrow.telecommunity.com>
References: <20101005223550.84F223A4105@sparrow.telecommunity.com>
	<4CAC2797.3040800@notmyidea.org>
	<20101006171752.A31743A4100@sparrow.telecommunity.com>
Message-ID: <4CAF3D72.60406@notmyidea.org>

Le 10/06/2010 06:17 PM, P.J. Eby a ?crit :
> At 08:39 AM 10/6/2010 +0100, Alexis M?taireau wrote:
>> So we agree there is conflicts that cannot be find a simple way at the
>> installation time (thanks for providing those examples). I agree that
>> the conflict field have to stand for a *possibility* of a conflict.
> 
> I would suggest, therefore, that the conflict field format be changed to
> include a brief human-readable description of the nature of the
> conflict(s), and a URL where the user can obtain more information.
+1 on that.

>> If "foo 1.0" and "bar 1.0" are well known conflicting releases, we could
>> avoid the check of all the files to output an error.
> 
> Of course.  But if we are just caching file-conflict information, then
> perhaps we should just list the files that will be installed, and then
> there is no need for the developer of either package to:
> 
> 1. figure out that the conflict exists,
> 2. re-release their package with the information, and
> 3. keep it up-to-date (and re-release it) when the files change *in
> either package*!
> 
> Let's once again take an actual (historically-occuring) use case:
> PyDispatcher and RuleDispatch.  When both packages were released, they
> included a package named "dispatch", and so somebody downloading both
> would've only detected the problem at installation time.
> 
> Let's say that I responded to this situation by adding a
> "Conflicts-Release: PyDispatcher" to RuleDispatch's distribution.  Well,
> not too long after we both knew about the conflict, the author of
> PyDispatcher actually changed his package name, alleviating the
> conflict...  but (historically) I didn't find out about this until much
> later.
> 
> So then, the Conflicts-Release in RuleDispatch would have been
> incorrect: a user with the newer PyDispatcher installed would be warned
> to *not* install RuleDispatch, even though there's no conflict any more.
> 
> Again, this is a case where the lack of an authoritative third-party
> administering the metadata makes the whole thing a mess.
> 
> Now instead, consider what happens if we simply list the files to be
> installed, or some portion thereof.  Now, the release of a new version
> of PyDispatcher resolves the conflict immediately and transparently,
> without any explicit action by either author, or by the user.  (And it's
> not teaching users to force installation due to widespread inaccuracies
> in the metadata.)
> 
> So, if you review this scenario, what you will see is that using a
> Conflicts-Release field to manage file-level conflicts is actually
> *worse* for EVERYBODY in this scenario, users and developers alike!
> 
> IOW, if you want to gain the benefit of being able to pre-detect
> file-level conflicts, the only safe solution is to have a way to e.g.
> query PyPI for installation manifests, or include some sort of metadata
> for that in the spec.  Expecting the developers to *manually* maintain
> file conflict information (and especially to correct it when the problem
> is resolved!) is not a good tradeoff vs. the minor extra time needed to
> download the file and notice the conflict.
You convinced me :) +1 on that too: add a way to query PyPI for
metadatas manifest, and then check on the system if the files are
already present ot not at installation time. This easy too the detection
of the conflicts when not installing distributions.

> 
> If for some reason it can't be done this way, the alternative would be
> to manually specify something like a Conflicts-Files field that lists
> some filenames known to conflict with certain other packages.  The tool
> doing the fetching can then check whether *those* files are already
> installed.  Then, if I changed RuleDispatch to say there were potential
> conflicts on dispatch/__init__.py, then tools would be able to notice
> that yes, there's a problem, or no, there's not.  And I wouldn't need to
> say *which* other project releases might be conflicting -- once the
> conflict was declared, then any project including those files would
> conflict, and any project that ended its conflict would become
> nonconflicting automatically.
So you mean adding a list of files that could be possibly conflicting ?
I dont get the added value of this way to think over the manual
detection system.

> Or, you know, we could just let people find out at installation time,
> and complain to the package author(s), like they do now.  ;-)
Hmm, that will always be the case, anyway :)

> (Btw: part of the rationale for creating .egg files in the first place
> was that they *can't* have installation conflicts with each other,
> except for scripts...  and scripts weren't part of the original use case.)
> 
> 
>> This lead to another question: Should we use exactly the same fields
>> with the same meanings, depending the context ? I think the answer is no.
>>
>> For instance, we could interpret the "conflicts" field differently if we
>> want to install the software "as is" (in which case I think it's better
>> to trust it), than if we want to use that field in a distro context (as
>> the distros can provide configuration for the python distributions, they
>> can resolve those conflicts). BTW, if we provide a "conflict" field, it
>> can help them to detect such the simple way.
> 
> I don't think I understood either of these paragraphs.  Could you
> elaborate?
Sure,

I'm talking about the fact that the meanings of the fields change
depending on which context they are. For instance, we have to be sure
that the context for the "conflict" field is at *installation* time, not
at *runtime*.

So, I guess, we need to change a bit the PEP to explain the "scope" of
such fields.

I'll try to do that in the next days.

Alexis

From jim at zope.com  Fri Oct  8 20:03:29 2010
From: jim at zope.com (Jim Fulton)
Date: Fri, 8 Oct 2010 14:03:29 -0400
Subject: [Catalog-sig] Request for trove classifier: Framework :: ZODB
Message-ID: <AANLkTi=A8TONgQFAxHJx-EP3xAGY0a0WgscCL3fZm2fi@mail.gmail.com>

Could we get a trove classifier for:

  Framework :: ZODB

?

Jim

-- 
Jim Fulton

From jim at zope.com  Fri Oct  8 20:28:37 2010
From: jim at zope.com (Jim Fulton)
Date: Fri, 8 Oct 2010 14:28:37 -0400
Subject: [Catalog-sig] Request for trove classifier: Framework :: ZODB
In-Reply-To: <AANLkTi=A8TONgQFAxHJx-EP3xAGY0a0WgscCL3fZm2fi@mail.gmail.com>
References: <AANLkTi=A8TONgQFAxHJx-EP3xAGY0a0WgscCL3fZm2fi@mail.gmail.com>
Message-ID: <AANLkTimZVLLBEcyP4e2MHQZCV9mnRnTaSLFTCKixpud5@mail.gmail.com>

Oops. There already was one. That was embarrassing. :)

Jim

On Fri, Oct 8, 2010 at 2:03 PM, Jim Fulton <jim at zope.com> wrote:
> Could we get a trove classifier for:
>
> ?Framework :: ZODB
>
> ?
>
> Jim
>
> --
> Jim Fulton
>



-- 
Jim Fulton

From fuzzyman at gmail.com  Fri Oct 15 12:40:15 2010
From: fuzzyman at gmail.com (Michael Foord)
Date: Fri, 15 Oct 2010 11:40:15 +0100
Subject: [Catalog-sig] Syntax highlighting for code examples on PyPI
Message-ID: <AANLkTikAv6uc1Yc3XUKnSQfrq6pHN9-Zc1pZHBmDi=75@mail.gmail.com>

Hello guys,

Can I make a feature request for PyPI. Please allow syntax highlighting for
code examples using pygments and the rest code-block directive.

Pygments includes a rest directive:

    http://pygments.org/docs/rstdirective/

It would require the addition of pygments css (not big) to the pages.

All the best,

Michael Foord

-- 
http://www.voidspace.org.uk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20101015/dc1d9017/attachment.html>

From g.brandl at gmx.net  Fri Oct 15 22:56:18 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 15 Oct 2010 22:56:18 +0200
Subject: [Catalog-sig] Syntax highlighting for code examples on PyPI
In-Reply-To: <AANLkTikAv6uc1Yc3XUKnSQfrq6pHN9-Zc1pZHBmDi=75@mail.gmail.com>
References: <AANLkTikAv6uc1Yc3XUKnSQfrq6pHN9-Zc1pZHBmDi=75@mail.gmail.com>
Message-ID: <i9af6d$ie4$1@dough.gmane.org>

Am 15.10.2010 12:40, schrieb Michael Foord:
> Hello guys,
> 
> Can I make a feature request for PyPI. Please allow syntax highlighting for code
> examples using pygments and the rest code-block directive.
> 
> Pygments includes a rest directive:
> 
>     http://pygments.org/docs/rstdirective/
> 
> It would require the addition of pygments css (not big) to the pages.

Rather than requiring a custom directive (which most people won't know about,
and probably don't want to have in their "pure" reST), I'd suggest the strategy
that Sphinx follows: for any code block, if it parses as Python, highlight it
as Python.  If it is a doctest block, highlight it as such.  Otherwise, don't
highlight it.  Since most code blocks are expected to be Python, I think that
should be sufficient.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From martin at v.loewis.de  Sun Oct 17 19:44:55 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 17 Oct 2010 19:44:55 +0200
Subject: [Catalog-sig] Syntax highlighting for code examples on PyPI
In-Reply-To: <AANLkTikAv6uc1Yc3XUKnSQfrq6pHN9-Zc1pZHBmDi=75@mail.gmail.com>
References: <AANLkTikAv6uc1Yc3XUKnSQfrq6pHN9-Zc1pZHBmDi=75@mail.gmail.com>
Message-ID: <4CBB3617.5090700@v.loewis.de>

> Can I make a feature request for PyPI. Please allow syntax highlighting
> for code examples using pygments and the rest code-block directive.

Unfortunately, I don't even understand what the request means
(I don't know what pygments is, for example), so I won't be able
to do much about such a request.

Regards,
Martin

From g.brandl at gmx.net  Sun Oct 17 19:50:31 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 17 Oct 2010 19:50:31 +0200
Subject: [Catalog-sig] Syntax highlighting for code examples on PyPI
In-Reply-To: <4CBB3617.5090700@v.loewis.de>
References: <AANLkTikAv6uc1Yc3XUKnSQfrq6pHN9-Zc1pZHBmDi=75@mail.gmail.com>
	<4CBB3617.5090700@v.loewis.de>
Message-ID: <i9fd22$ca7$1@dough.gmane.org>

Am 17.10.2010 19:44, schrieb "Martin v. L?wis":
>> Can I make a feature request for PyPI. Please allow syntax highlighting
>> for code examples using pygments and the rest code-block directive.
> 
> Unfortunately, I don't even understand what the request means
> (I don't know what pygments is, for example), so I won't be able
> to do much about such a request.

I can help out -- I happen to know a bit about Pygments, and if there
are no objections I'll present you a patch soon.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From martin at v.loewis.de  Sun Oct 17 20:06:16 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 17 Oct 2010 20:06:16 +0200
Subject: [Catalog-sig] Syntax highlighting for code examples on PyPI
In-Reply-To: <i9fd22$ca7$1@dough.gmane.org>
References: <AANLkTikAv6uc1Yc3XUKnSQfrq6pHN9-Zc1pZHBmDi=75@mail.gmail.com>	<4CBB3617.5090700@v.loewis.de>
	<i9fd22$ca7$1@dough.gmane.org>
Message-ID: <4CBB3B18.9010602@v.loewis.de>

Am 17.10.2010 19:50, schrieb Georg Brandl:
> Am 17.10.2010 19:44, schrieb "Martin v. L?wis":
>>> Can I make a feature request for PyPI. Please allow syntax highlighting
>>> for code examples using pygments and the rest code-block directive.
>>
>> Unfortunately, I don't even understand what the request means
>> (I don't know what pygments is, for example), so I won't be able
>> to do much about such a request.
> 
> I can help out -- I happen to know a bit about Pygments, and if there
> are no objections I'll present you a patch soon.

Sure, go ahead.

Regards,
Martin

From fuzzyman at gmail.com  Sun Oct 17 23:26:27 2010
From: fuzzyman at gmail.com (Michael Foord)
Date: Sun, 17 Oct 2010 22:26:27 +0100
Subject: [Catalog-sig] Syntax highlighting for code examples on PyPI
In-Reply-To: <4CBB3617.5090700@v.loewis.de>
References: <AANLkTikAv6uc1Yc3XUKnSQfrq6pHN9-Zc1pZHBmDi=75@mail.gmail.com>
	<4CBB3617.5090700@v.loewis.de>
Message-ID: <AANLkTi=X-BNM2oHj8d8Zsno-09aTnnBHb5q1hcHAZkgs@mail.gmail.com>

On 17 October 2010 18:44, "Martin v. L?wis" <martin at v.loewis.de> wrote:

> > Can I make a feature request for PyPI. Please allow syntax highlighting
> > for code examples using pygments and the rest code-block directive.
>
> Unfortunately, I don't even understand what the request means
>


I would like the Python code examples that appear on many PyPI summary pages
to be "syntax highlighted", such as you might have in any competent Python
IDE.

Pygments is a (even "the") popular Python library for syntax highlighting of
*many* programming languages and it integrates very well with the
reStructured Text used in PyPI descriptions.

http://pygments.org/

A few lines of code in PyPI hooking up Pygments into reStructured Text
(Pygments provides a ready made directive for this purpose) and including
the pygments css in PyPI pages would allow this.

All the best,

Michael Foord



> (I don't know what pygments is, for example), so I won't be able
> to do much about such a request.
>
> Regards,
> Martin
>



-- 
http://www.voidspace.org.uk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20101017/5ccdd0ae/attachment.html>

From martin at v.loewis.de  Mon Oct 18 00:12:36 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 18 Oct 2010 00:12:36 +0200
Subject: [Catalog-sig] Syntax highlighting for code examples on PyPI
In-Reply-To: <AANLkTi=X-BNM2oHj8d8Zsno-09aTnnBHb5q1hcHAZkgs@mail.gmail.com>
References: <AANLkTikAv6uc1Yc3XUKnSQfrq6pHN9-Zc1pZHBmDi=75@mail.gmail.com>	<4CBB3617.5090700@v.loewis.de>
	<AANLkTi=X-BNM2oHj8d8Zsno-09aTnnBHb5q1hcHAZkgs@mail.gmail.com>
Message-ID: <4CBB74D4.5010904@v.loewis.de>

> A few lines of code in PyPI hooking up Pygments into reStructured Text
> (Pygments provides a ready made directive for this purpose) and
> including the pygments css in PyPI pages would allow this.

Unfortunately, I still have no idea what those few lines might look
like, or what possible undesirable consequences of such hooking
might be. Fortunately, Georg has volunteered to write them (or do
whatever is necessary to implement the original request).

Regards,
Martin

From fuzzyman at gmail.com  Mon Oct 18 00:19:01 2010
From: fuzzyman at gmail.com (Michael Foord)
Date: Sun, 17 Oct 2010 23:19:01 +0100
Subject: [Catalog-sig] Syntax highlighting for code examples on PyPI
In-Reply-To: <4CBB74D4.5010904@v.loewis.de>
References: <AANLkTikAv6uc1Yc3XUKnSQfrq6pHN9-Zc1pZHBmDi=75@mail.gmail.com>
	<4CBB3617.5090700@v.loewis.de>
	<AANLkTi=X-BNM2oHj8d8Zsno-09aTnnBHb5q1hcHAZkgs@mail.gmail.com>
	<4CBB74D4.5010904@v.loewis.de>
Message-ID: <AANLkTimo_KuvQqwy1Yq0wpXWYn4t+YDB2pPsE-34_NDz@mail.gmail.com>

On 17 October 2010 23:12, "Martin v. L?wis" <martin at v.loewis.de> wrote:

> > A few lines of code in PyPI hooking up Pygments into reStructured Text
> > (Pygments provides a ready made directive for this purpose) and
> > including the pygments css in PyPI pages would allow this.
>
> Unfortunately, I still have no idea what those few lines might look
> like, or what possible undesirable consequences of such hooking
> might be. Fortunately, Georg has volunteered to write them (or do
> whatever is necessary to implement the original request).
>
>
Right. Georg wrote pygments, so he has *some* expertise in the area... :-)

Michael


> Regards,
> Martin
>



-- 
http://www.voidspace.org.uk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20101017/08aadd01/attachment.html>

From merwok at netwok.org  Mon Oct 18 14:33:50 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Mon, 18 Oct 2010 14:33:50 +0200
Subject: [Catalog-sig] Syntax highlighting for code examples on PyPI
In-Reply-To: <AANLkTi=X-BNM2oHj8d8Zsno-09aTnnBHb5q1hcHAZkgs@mail.gmail.com>
References: <AANLkTikAv6uc1Yc3XUKnSQfrq6pHN9-Zc1pZHBmDi=75@mail.gmail.com>	<4CBB3617.5090700@v.loewis.de>
	<AANLkTi=X-BNM2oHj8d8Zsno-09aTnnBHb5q1hcHAZkgs@mail.gmail.com>
Message-ID: <4CBC3EAE.60007@netwok.org>

> A few lines of code in PyPI hooking up Pygments into reStructured Text
> (Pygments provides a ready made directive for this purpose) and including
> the pygments css in PyPI pages would allow this.

FTR, there are two ways to integrate reST and Sphinx: The sourcecode
pure-reST directive provided by Pygments and the code-block directive
available in Sphinx.  For long_description values built from external
files that are also part of a Sphinx-made documentation, code-block can
be used but not sourcecode.

Of course, Georg knows that, being the author of both, and his plan to
use heuristics and not force any directive sounds good.  On the other
hand, the directives allow one to specify the language, which can be
useful for config file examples on PyPI pages.

Regards


From fuzzyman at gmail.com  Mon Oct 18 14:59:23 2010
From: fuzzyman at gmail.com (Michael Foord)
Date: Mon, 18 Oct 2010 13:59:23 +0100
Subject: [Catalog-sig] Syntax highlighting for code examples on PyPI
In-Reply-To: <4CBC3EAE.60007@netwok.org>
References: <AANLkTikAv6uc1Yc3XUKnSQfrq6pHN9-Zc1pZHBmDi=75@mail.gmail.com>
	<4CBB3617.5090700@v.loewis.de>
	<AANLkTi=X-BNM2oHj8d8Zsno-09aTnnBHb5q1hcHAZkgs@mail.gmail.com>
	<4CBC3EAE.60007@netwok.org>
Message-ID: <AANLkTi=5o0y=ddqcwr4UqoHB597bcTA7bvjKdgNo3wro@mail.gmail.com>

On 18 October 2010 13:33, ?ric Araujo <merwok at netwok.org> wrote:

> > A few lines of code in PyPI hooking up Pygments into reStructured Text
> > (Pygments provides a ready made directive for this purpose) and including
> > the pygments css in PyPI pages would allow this.
>
> FTR, there are two ways to integrate reST and Sphinx: The sourcecode
> pure-reST directive provided by Pygments and the code-block directive
> available in Sphinx.  For long_description values built from external
> files that are also part of a Sphinx-made documentation, code-block can
> be used but not sourcecode.
>
> Of course, Georg knows that, being the author of both, and his plan to
> use heuristics and not force any directive sounds good.  On the other
> hand, the directives allow one to specify the language, which can be
> useful for config file examples on PyPI pages.
>


Also for C code examples which some PyPI packages use. I don't think "sphinx
style" *precludes* specifying the language though?

Michael


>
> Regards
>
>


-- 
http://www.voidspace.org.uk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20101018/3517c639/attachment.html>

From merwok at netwok.org  Mon Oct 18 15:02:24 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Mon, 18 Oct 2010 15:02:24 +0200
Subject: [Catalog-sig] Syntax highlighting for code examples on PyPI
In-Reply-To: <AANLkTi=5o0y=ddqcwr4UqoHB597bcTA7bvjKdgNo3wro@mail.gmail.com>
References: <AANLkTikAv6uc1Yc3XUKnSQfrq6pHN9-Zc1pZHBmDi=75@mail.gmail.com>	<4CBB3617.5090700@v.loewis.de>	<AANLkTi=X-BNM2oHj8d8Zsno-09aTnnBHb5q1hcHAZkgs@mail.gmail.com>	<4CBC3EAE.60007@netwok.org>
	<AANLkTi=5o0y=ddqcwr4UqoHB597bcTA7bvjKdgNo3wro@mail.gmail.com>
Message-ID: <4CBC4560.3000602@netwok.org>

> Also for C code examples which some PyPI packages use.
Good point.

> I don't think "sphinx style" *precludes* specifying the language though?
Depends what you mean by that.  The Shinx-provided code-block directive
takes an argument, so it?s good, but the implicit style (using a syntax
code block introduced by ?::?, not a directive) does not TTBOMK.

Regards


From ziade.tarek at gmail.com  Tue Oct 19 21:21:36 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 19 Oct 2010 21:21:36 +0200
Subject: [Catalog-sig] Getting a last modified date for the trove classifiers
Message-ID: <AANLkTinkSm1_nFvvezZ75rCwnRiMWHsq0ozp65E7+_Zk@mail.gmail.com>

Hello,

We are keeping a copy of
http://pypi.python.org/pypi?:action=list_classifiers in distutils2 for
our browsing needs and we would like to update that cache only if it
has changed. Instead of doing a custom and inefficient diff in the
client code, could we add at PyPI a Last-Modified or an ETag header
for that page, and check for a If-Last-Modified or Etag on requests ?
and return a 304 in case the content has not changed

If Martin agrees I can write the patch

Tarek

-- 
Tarek Ziad? | http://ziade.org

From alexis at notmyidea.org  Tue Oct 19 23:35:34 2010
From: alexis at notmyidea.org (=?ISO-8859-1?Q?Alexis_M=E9taireau?=)
Date: Tue, 19 Oct 2010 22:35:34 +0100
Subject: [Catalog-sig] distutils2 pysetup UI
Message-ID: <4CBE0F26.7040207@notmyidea.org>

Hi there,

I'm now working on the pysetup script, and would like to have some
inputs from you about the user interface it will have.

As you may know, pysetup will be an entry point for both distutils2
commands (the ones in distutils2.command) and scripts (such as
"depgraph", "search", "info" etc.)

I was thinking about something on the form:

	$ pysetup COMMAND|SCRIPTNAME [options]

I wanted to add too a "help" command, to refer on the help of different
commands, eg.

	$ pysetup help command

Internally, I would like to have a list of all the existing commands
(including the custom ones as well), plus the list of existing "scripts"
we want to use directly from this "pysetup", and then just forward the
options to them, directly, maybe with the exception of the "--help" one.

Of course, it will check if it's possible to rely on distribution based
commands before trying to invoke them.

I've started implementing that using argparse. what do you think about
that? I do know that ?ric is a bit reluctant to the idea, but I don't
get why, so that's time to express yourself.

Cheers,
Alexis

From martin at v.loewis.de  Wed Oct 20 00:32:43 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 20 Oct 2010 00:32:43 +0200
Subject: [Catalog-sig] Getting a last modified date for the trove
	classifiers
In-Reply-To: <AANLkTinkSm1_nFvvezZ75rCwnRiMWHsq0ozp65E7+_Zk@mail.gmail.com>
References: <AANLkTinkSm1_nFvvezZ75rCwnRiMWHsq0ozp65E7+_Zk@mail.gmail.com>
Message-ID: <4CBE1C8B.4000002@v.loewis.de>

> We are keeping a copy of
> http://pypi.python.org/pypi?:action=list_classifiers in distutils2 for
> our browsing needs and we would like to update that cache only if it
> has changed. Instead of doing a custom and inefficient diff in the
> client code, could we add at PyPI a Last-Modified or an ETag header
> for that page, and check for a If-Last-Modified or Etag on requests ?
> and return a 304 in case the content has not changed
> 
> If Martin agrees I can write the patch

Sure, go ahead. I recommend to make journal entries for added
classifiers, and then select order by submitted_date desc limit 1
to find out when the last classifier was added. With that, all
change notification machineries will also learn about new classifiers.

If you are ambitious, you can also add if-modified-since support.

Regards,
Martin

From aljosa.mohorovic at gmail.com  Thu Oct 28 20:31:53 2010
From: aljosa.mohorovic at gmail.com (=?UTF-8?B?QWxqb8WhYSBNb2hvcm92acSH?=)
Date: Thu, 28 Oct 2010 20:31:53 +0200
Subject: [Catalog-sig] pypi statistics
Message-ID: <AANLkTinLknK=yS0X7X2H1P_Du6PQbw+12B3auJo0pGY2@mail.gmail.com>

is it possible to get information about pypi.python.org usage?
stuff like:
 - number of users
 - used storage for packages
 - avg. daily bandwidth
 - avg. monthly bandwidth
 - similar info

Aljosa Mohorovic

From martin at v.loewis.de  Thu Oct 28 21:23:30 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 28 Oct 2010 21:23:30 +0200
Subject: [Catalog-sig] pypi statistics
In-Reply-To: <AANLkTinLknK=yS0X7X2H1P_Du6PQbw+12B3auJo0pGY2@mail.gmail.com>
References: <AANLkTinLknK=yS0X7X2H1P_Du6PQbw+12B3auJo0pGY2@mail.gmail.com>
Message-ID: <4CC9CDB2.7070408@v.loewis.de>

Am 28.10.2010 20:31, schrieb Aljo?a Mohorovi?:
> is it possible to get information about pypi.python.org usage?
> stuff like:
>  - number of users

Accounts: 19120

Some people have multiple accounts, and some account holders
don't use PyPI, so "number of users" is difficult to say.

>  - used storage for packages

17GiB

>  - avg. daily bandwidth

5 MBits/s on average in the last month.

>  - avg. monthly bandwidth

Isn't that the same?

>  - similar info

It depends. If you are not just asking whether it's possible
to get it, but also where, in addition to asking me: look at

http://pypi.python.org/webstats/
http://pypi.python.org/munin/localdomain/localhost.localdomain.html

Regards,
Martin

From aljosa.mohorovic at gmail.com  Fri Oct 29 00:44:12 2010
From: aljosa.mohorovic at gmail.com (=?UTF-8?B?QWxqb8WhYSBNb2hvcm92acSH?=)
Date: Fri, 29 Oct 2010 00:44:12 +0200
Subject: [Catalog-sig] pypi statistics
In-Reply-To: <4CC9CDB2.7070408@v.loewis.de>
References: <AANLkTinLknK=yS0X7X2H1P_Du6PQbw+12B3auJo0pGY2@mail.gmail.com>
	<4CC9CDB2.7070408@v.loewis.de>
Message-ID: <AANLkTinC9gKzx8xSD2EZRf=+K-PTN-u1RqbNqgNXK7cJ@mail.gmail.com>

On Thu, Oct 28, 2010 at 9:23 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> ?- avg. daily bandwidth
>
> 5 MBits/s on average in the last month.
>
>> ?- avg. monthly bandwidth
>
> Isn't that the same?

my mistake, i was actually asking about bandwidth usage/traffic and
daily/monthly peeks.
~1.5TB monthly traffic with daily avg. ~50GB and ~90GB peek.

webstats/munin reports are more than i needed.
thanks.

Aljosa