From pje at telecommunity.com  Mon Aug  1 17:41:47 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 01 Aug 2005 11:41:47 -0400
Subject: [Catalog-sig] PyPI update broke setuptools
In-Reply-To: <5.1.1.6.0.20050605180352.02208810@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050801113700.026e1cf8@mail.telecommunity.com>

The new version of PyPI changed the third assumption below, breaking 
setuptools' ability to do fallback searches for a request whose 
upper/lowercase didn't precisely match the cases used by the package 
author.  Is there any chance someone could revert this, since this breaks a 
ton of software in the field?

At 06:22 PM 6/5/2005 -0400, Phillip J. Eby wrote:
>For the benefit of the PyPI maintainers, here is a summary of EasyInstall's
>fairly minimal assumptions about PyPI's current behavior, assuming that
>"base-url" is the root URL of the package index:
>
>     * Going to "base-url/SomePackage" produces an HTML page that either has
>a title containing "Index of Packages" and links to zero or more pages for
>specific versions, or else it is a single-version package page.
>
>     * Single-version package pages may have a home page and download URL
>link, each of which occurs after '<th>Home Page' and '<th>Download URL'
>respectively, if present.
>
>     * Going to "base-url/" (note trailing '/') produces an HTML page
>containing links to all active versions of registered packages
>
>     * Links to package pages always have URLs of the form
>"base-url/SomePackage/itsVersion" - i.e., exactly two path parts following
>the base URL, with no query strings, parameters, fragments, etc.
>
>EasyInstall should continue to work with PyPI if these assumptions continue
>to hold.


From richardjones at optushome.com.au  Tue Aug  2 00:00:11 2005
From: richardjones at optushome.com.au (Richard Jones)
Date: Tue, 2 Aug 2005 08:00:11 +1000
Subject: [Catalog-sig] PyPI update broke setuptools
In-Reply-To: <5.1.1.6.0.20050801113700.026e1cf8@mail.telecommunity.com>
References: <5.1.1.6.0.20050801113700.026e1cf8@mail.telecommunity.com>
Message-ID: <200508020800.11913.richardjones@optushome.com.au>

On Tue, 2 Aug 2005 01:41 am, Phillip J. Eby wrote:
> >     * Going to "base-url/" (note trailing '/') produces an HTML page
> >containing links to all active versions of registered packages

Sorry, this didn't register when you first posted. You were relying on a 
side-effect that I removed while we were trying to remove the "pypi" part of 
the URL. It seemed wrong to me that PyPI would return totally different 
results for:

    http://cheeseshop.python.org/pypi
    http://cheeseshop.python.org/pypi/

Is there any chance the software could be changed to use the more correct 
"?:action=search" which is what the "/" side-effect was effectively doing?


     Richard
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/catalog-sig/attachments/20050802/45443e22/attachment.pgp

From richardjones at optushome.com.au  Tue Aug  2 00:24:39 2005
From: richardjones at optushome.com.au (Richard Jones)
Date: Tue, 2 Aug 2005 08:24:39 +1000
Subject: [Catalog-sig] PyPI update broke setuptools
In-Reply-To: <200508020800.11913.richardjones@optushome.com.au>
References: <5.1.1.6.0.20050801113700.026e1cf8@mail.telecommunity.com>
	<200508020800.11913.richardjones@optushome.com.au>
Message-ID: <200508020824.39254.richardjones@optushome.com.au>

On Tue, 2 Aug 2005 08:00 am, Richard Jones wrote:
> On Tue, 2 Aug 2005 01:41 am, Phillip J. Eby wrote:
> > >     * Going to "base-url/" (note trailing '/') produces an HTML page
> > >containing links to all active versions of registered packages
>
> Sorry, this didn't register when you first posted. You were relying on a
> side-effect that I removed while we were trying to remove the "pypi" part
> of the URL. It seemed wrong to me that PyPI would return totally different
> results for:
>
>     http://cheeseshop.python.org/pypi
>     http://cheeseshop.python.org/pypi/

I can't revert the change as currently the web server redirects 
http://cheeseshop.python.org to the latter, so people going directly to that 
URL will get the not-so-useful list of packages, rather than the home page. I 
don't have root access to fix it either, but we'll work on it.


    Richard
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/catalog-sig/attachments/20050802/33d640fa/attachment.pgp

From pje at telecommunity.com  Tue Aug  2 02:42:17 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 01 Aug 2005 20:42:17 -0400
Subject: [Catalog-sig] PyPI update broke setuptools
Message-ID: <5.1.1.6.0.20050801202252.02591430@mail.telecommunity.com>


>Sorry, this didn't register when you first posted. You were relying on a
>side-effect that I removed while we were trying to remove the "pypi" part of
>the URL. It seemed wrong to me that PyPI would return totally different
>results for:
>
>     http://cheeseshop.python.org/pypi
>     http://cheeseshop.python.org/pypi/

Well, it seemed quite *right* to me, with "pypi" being the front page, and 
pypi/ being the directory listing of packages.  It fit perfectly with the 
rest of the URL scheme, and so I assumed it was an intentional feature, not 
just a desirable one.  :)


>Is there any chance the software could be changed to use the more correct
>"?:action=search" which is what the "/" side-effect was effectively doing?

Certainly it could be changed, but because the PyPI change breaks the 
versions that are already in the field, it would require a new release, 
which would be rather awkward at the moment, because I don't want to force 
people to upgrade to the heavily refactored 0.6 series from the relatively 
stable 0.5 series.  I was planning to make a release of 0.6a1 this coming 
weekend, but not upgrade ez_setup, so that people would need to make a 
conscious decision to upgrade.  Thus, I'm somewhat loathe to put an 
upgraded version into wide release in order to fix an unrelated problem.

Also, the reason I didn't use the uglier search API in the first place was 
because the more REST-like interface using a / seemed more compatible with 
manually-created catalogs.  i.e., somebody can simulate a catalog of their 
own just by using static files and directories, whereas the :action=search 
thing is pretty heavily tied to a single protocol version of PyPI and 
requires the index to be implemented using some kind of dynamic 
technology.  I deliberately wanted to make EasyInstall's protocol 
expectations modest enough that a wide variety of technologies could 
fulfill them, so as to allow for small private or specialty indexes to 
exist without needing to set up databases and such running the same 
software as PyPI.

PyPI's existing URL-based API (including the '/' trick) seemed *perfect* 
for this, and both python.org/pypi and cheeseshop.python.org/ seemed to 
obey it.

If the software has to change, it has to change.  But I'd much prefer it if 
we could restore service to existing versions of the software, and it would 
also be nice if I could continue to allow people to create interoperable 
indexes based on static files, without EasyInstall having to do more 
sophisticated (and therefore more fragile) sniffing to tell what version of 
PyPI (or not) it's running against, or having to introduce command line 
options to tell EasyInstall what PyPI protocol version to use, or whatever.


From richardjones at optushome.com.au  Fri Aug  5 04:07:57 2005
From: richardjones at optushome.com.au (Richard Jones)
Date: Fri, 5 Aug 2005 12:07:57 +1000
Subject: [Catalog-sig] PyPI enhancement doc
In-Reply-To: <42941CE4.7070206@colorstudy.com>
References: <42941CE4.7070206@colorstudy.com>
Message-ID: <200508051207.57610.richardjones@optushome.com.au>

On Wed, 25 May 2005 04:36 pm, Ian Bicking wrote:
> This is an initial list of proposed methods for PyPI to support:
> search(field_specifiers, [operator='and']):
> package_releases(package_name):
> package_stable_version(package_name):
> package_urls(package_name, version):
> package_data(package_name, version):

I have implemented as per your spec except as follows:


>      All keys are required.  None/NULL is translated to ''.

Missing values are marked "UNKNOWN" at the moment. This is a hysterical oddity 
introduced by the PKG-INFO format, IIRC.


>      Open issues: will emails be obscured?  Is keywords turned into a list?

No and no at the moment.


> I'm a little soft on these, since I don't know if specifiers and the
> necessary metadata is really ready:
>
> providing_packages(specifier):
>      A list of (name, version) from release_provides.  E.g.,
>      providing_packages('PageTemplate>=1.0') == [('zpt', '1.0')].  This
>      will only return non-hidden packages.
>
> requiring_packages(specifier):
>      A list of (name, version) from release_requires.

I haven't implemented these since we don't have code that handles dependency 
lookup. I believe the data and implementation spec are all ready to go 
though.



     Richard
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/catalog-sig/attachments/20050805/7a379c44/attachment.pgp

From amk at amk.ca  Fri Aug  5 14:28:30 2005
From: amk at amk.ca (A.M. Kuchling)
Date: Fri, 5 Aug 2005 08:28:30 -0400
Subject: [Catalog-sig] PyPI enhancement doc
In-Reply-To: <200508051207.57610.richardjones@optushome.com.au>
References: <42941CE4.7070206@colorstudy.com>
	<200508051207.57610.richardjones@optushome.com.au>
Message-ID: <20050805122830.GB8968@rogue.amk.ca>

On Fri, Aug 05, 2005 at 12:07:57PM +1000, Richard Jones wrote:
> > This is an initial list of proposed methods for PyPI to support:

How should we document this API?  DocXMLRPCserver.py could be used to
present a simple web page that used pydoc on the RPC methods, but we
should also have a few web pages describing it.  Should it be a wiki
page, or an "official" page inside the pydotorg SVN tree?

(Describing the API would make a good oreillynet.com article.  If no
one else wants to write one, I will...)

--amk

From pje at telecommunity.com  Mon Aug 15 01:06:38 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 14 Aug 2005 19:06:38 -0400
Subject: [Catalog-sig] File integrity checking and host blocking for
	EasyInstall
Message-ID: <5.1.1.6.0.20050814183910.02704078@mail.telecommunity.com>

After thinking over the last week's distutils-sig discussion about 
security, signatures, etc., I think I have a plan for handling basic file 
integrity checking and (non-cryptographic) trust management for 
EasyInstall.  It is not a high-security end-to-end solution, but I think it 
will allow security-conscious persons to take a more "locked down" approach 
if they want to, while providing everyone else with some baseline 
protection against corrupted files.

The first part of the plan is to add md5 digest checking to 
EasyInstall.  Because one of EasyInstall's design goals is to make it easy 
for anybody to publish links to packages, we need to be able to include the 
md5 signature in a package's URL.  I'm thinking we could achieve this via 
an '#md5=...' fragment identifier.  For example, a setuptools source 
archive URL might be:

http://www.python.org/packages/source/s/setuptools/setuptools-0.5a13.zip#md5=91f31a9058330174640a867cf5d4de57

The advantage of this approach is that it allows anyone to assert what the 
md5 of the targeted file is, and it can be asserted in any web page, just 
by pointing an HREF at the file.  EasyInstall could detect the '#md5=' 
marker, and then use this to verify the file during download.

The disadvantage, of course, is that PyPI doesn't currently support this; 
it creates a separate link to a page that displays the md5, and that URL 
doesn't contain anything that connects it back to the distribution file it 
refers to.  I could probably create some kind of parsing hack to fix that 
for PyPI, but it seems it might be worth adding the #md5 trick to PyPI to 
support this.

EasyInstall would also need to grow a --require-md5 option, which would 
refuse to install anything from a Subversion checkout or a distribution 
without a known md5 signature.

In addition to md5 support in EasyInstall, I propose to also add it to 
ez_setup; there, however, the md5 values for various distributions will be 
hardcoded into ez_setup.py itself.  (I'll make my "release" script append 
the md5 signatures for new distributions to the end of ez_setup.py.)  In 
this way, the bootstrap installation of setuptools can also be reasonably 
secured, as long as you trust a particular version of ez_setup.py.

The next part of the plan would be to add an --allow-hosts option to 
EasyInstall.  This would be a list of host wildcards that EasyInstall would 
be allowed to contact.  For example, --allow-hosts=*.python.org would let 
EasyInstall download or scan pages from PyPI or www.python.org, but not 
anywhere else.  The default, if not specified, would be '*', meaning that 
any host may be accessed.  If EasyInstall finds itself about to download a 
page or distribution from a host that isn't allowed, it will abort with a 
message explaining the problem.

This would allow folks like Paul Moore to configure a default --allow-hosts 
list in their pydistutils.cfg, to prevent EasyInstall from downloading 
things from just any old place on the Internet.  Once he's verified that he 
trusts a particular site, he can edit pydistutils.cfg and add it, or else 
manually download the blocked URL, publish it on a trusted intranet host, etc.

So, this is not a complete security solution, as it doesn't deal with 
end-to-end file integrity, and could easily be subverted by taking over a 
site somewhere in the middle (e.g. python.org).  But until we have more of 
the cryptographic infrastructure in place, I think this plan could provide 
us with a good starting point.  Comments, anyone?


From richardjones at optushome.com.au  Mon Aug 15 01:37:31 2005
From: richardjones at optushome.com.au (Richard Jones)
Date: Mon, 15 Aug 2005 09:37:31 +1000
Subject: [Catalog-sig] File integrity checking and host blocking for
	EasyInstall
In-Reply-To: <5.1.1.6.0.20050814183910.02704078@mail.telecommunity.com>
References: <5.1.1.6.0.20050814183910.02704078@mail.telecommunity.com>
Message-ID: <200508150937.35777.richardjones@optushome.com.au>

On Mon, 15 Aug 2005 09:06 am, Phillip J. Eby wrote:
> After thinking over the last week's distutils-sig discussion about
> security, signatures, etc., I think I have a plan for handling basic file
> integrity checking and (non-cryptographic) trust management for
> EasyInstall.

I'm not on the distutils-sig any more (not sure why - I thought I was and 
there just wasn't any discussion going on ;) so I've missed that discussion.


> The first part of the plan is to add md5 digest checking to
> EasyInstall.  Because one of EasyInstall's design goals is to make it easy
> for anybody to publish links to packages, we need to be able to include the
> md5 signature in a package's URL.  I'm thinking we could achieve this via
> an '#md5=...' fragment identifier.  For example, a setuptools source
> archive URL might be:
>
> http://www.python.org/packages/source/s/setuptools/setuptools-0.5a13.zip#md
>5=91f31a9058330174640a867cf5d4de57

Any idea what various "normal" browsers do when they encounter something like 
this?


> The advantage of this approach is that it allows anyone to assert what the
> md5 of the targeted file is, and it can be asserted in any web page, just
> by pointing an HREF at the file.  EasyInstall could detect the '#md5='
> marker, and then use this to verify the file during download.
>
> The disadvantage, of course, is that PyPI doesn't currently support this;

As with all things that "PyPI doesn't currently support" I'm happy to add 
features, and even happier for others to add features :)


> So, this is not a complete security solution, as it doesn't deal with
> end-to-end file integrity, and could easily be subverted by taking over a
> site somewhere in the middle (e.g. python.org).  But until we have more of
> the cryptographic infrastructure in place, I think this plan could provide
> us with a good starting point.  Comments, anyone?

I presume the distutils discussion included the signing of packages that PyPI 
supports?


     Richard
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/catalog-sig/attachments/20050815/5c3133f4/attachment.pgp

From ehs at pobox.com  Mon Aug 15 03:30:37 2005
From: ehs at pobox.com (Edward Summers)
Date: Sun, 14 Aug 2005 20:30:37 -0500
Subject: [Catalog-sig] File integrity checking and host blocking for
	EasyInstall
In-Reply-To: <200508150937.35777.richardjones@optushome.com.au>
References: <5.1.1.6.0.20050814183910.02704078@mail.telecommunity.com>
	<200508150937.35777.richardjones@optushome.com.au>
Message-ID: <FE2650C0-B6BC-4F5E-8980-43C928DC04A8@pobox.com>

On Aug 14, 2005, at 6:37 PM, Richard Jones wrote:
> As with all things that "PyPI doesn't currently support" I'm happy  
> to add
> features, and even happier for others to add features :)

Speaking of adding features, is the PyPI CheeseShop source code  
available in svn somewhere? How about the leatest db snapshot? I've  
been playing around with the xmlrpc interface a bit and wondered how  
a few things were working. Consider this volunteering to help out if  
any hands are needed :-)

//Ed

From richardjones at optushome.com.au  Mon Aug 15 03:40:36 2005
From: richardjones at optushome.com.au (Richard Jones)
Date: Mon, 15 Aug 2005 11:40:36 +1000
Subject: [Catalog-sig] File integrity checking and host blocking for
	EasyInstall
In-Reply-To: <FE2650C0-B6BC-4F5E-8980-43C928DC04A8@pobox.com>
References: <5.1.1.6.0.20050814183910.02704078@mail.telecommunity.com>
	<200508150937.35777.richardjones@optushome.com.au>
	<FE2650C0-B6BC-4F5E-8980-43C928DC04A8@pobox.com>
Message-ID: <200508151140.43818.richardjones@optushome.com.au>

On Mon, 15 Aug 2005 11:30 am, Edward Summers wrote:
> On Aug 14, 2005, at 6:37 PM, Richard Jones wrote:
> > As with all things that "PyPI doesn't currently support" I'm happy
> > to add
> > features, and even happier for others to add features :)
>
> Speaking of adding features, is the PyPI CheeseShop source code
> available in svn somewhere?

The code is in the pydotorg svn repos:

   https://svn.python.org/packages/trunk/pypi


> How about the leatest db snapshot?

We shouldn't really make db snapshots available as the db contains contact 
info for all the submitters. If you'd like to write up a script that 
generates dummy packages for testing, that'd be cool :)


    Richard
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/catalog-sig/attachments/20050815/c160b0d7/attachment.pgp

From ehs at pobox.com  Thu Aug 18 17:43:32 2005
From: ehs at pobox.com (Edward Summers)
Date: Thu, 18 Aug 2005 10:43:32 -0500
Subject: [Catalog-sig] File integrity checking and host blocking for
	EasyInstall
In-Reply-To: <200508151140.43818.richardjones@optushome.com.au>
References: <5.1.1.6.0.20050814183910.02704078@mail.telecommunity.com>
	<200508150937.35777.richardjones@optushome.com.au>
	<FE2650C0-B6BC-4F5E-8980-43C928DC04A8@pobox.com>
	<200508151140.43818.richardjones@optushome.com.au>
Message-ID: <83560D0C-3BF2-431D-9C34-FFD7B80549CD@pobox.com>

> We shouldn't really make db snapshots available as the db contains  
> contact
> info for all the submitters. If you'd like to write up a script that
> generates dummy packages for testing, that'd be cool :)

That sounds like a good idea, and I'd be willing to take a stab at  
it. Perhaps it would be easier to dump the existing db, and write a  
program to obliterate the passwords in the snapshot? Are you worried  
about people grabbing email addresses? Aren't those already available  
via HTTP now?

It should be fairly trivial to write a program that copies the latest  
text pgdump, obscures what needs to be obscured, and makes it  
available for download no?

//Ed

From amk at amk.ca  Fri Aug 19 00:22:18 2005
From: amk at amk.ca (A.M. Kuchling)
Date: Thu, 18 Aug 2005 18:22:18 -0400
Subject: [Catalog-sig] 'Hidden' field display bug
Message-ID: <20050818222218.GA23547@rogue.amk.ca>

For my packages on PyPI, the package editing page shows them as all
hidden; the 'Hide?' column is all "Yes".  I went into the database and
toggled the hidden field for some of my packages, but that didn't make
any difference to what was displayed

Here's the relevant bit of the template.  I don't know ZPT; is the
following excerpt correct?

    <select tal:attributes="name string:hid_${release/version}">
     <option value="0" tal:attributes="selected release/_pypi_hidden">No</option
>
     <option value="1" tal:attributes="selected not:release/_pypi_hidden">Yes</o
ption>
    </select>

Things like the version field are displayed correctly, so the variable
'release' seems to be defined.  Is that attribute syntax correct?

--amk


From pje at telecommunity.com  Fri Aug 19 01:08:23 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 18 Aug 2005 19:08:23 -0400
Subject: [Catalog-sig] File integrity checking and host blocking for
 EasyInstall
Message-ID: <5.1.1.6.0.20050818190358.01b3eeb0@mail.telecommunity.com>

On Mon, 15 Aug 2005, Richard Jones wrote:
>On Mon, 15 Aug 2005 09:06 am, Phillip J. Eby wrote:
> > The first part of the plan is to add md5 digest checking to
> > EasyInstall.  Because one of EasyInstall's design goals is to make it easy
> > for anybody to publish links to packages, we need to be able to include the
> > md5 signature in a package's URL.  I'm thinking we could achieve this via
> > an '#md5=...' fragment identifier.  For example, a setuptools source
> > archive URL might be:
> >
> > http://www.python.org/packages/source/s/setuptools/setuptools-0.5a13.zip#md
> >5=91f31a9058330174640a867cf5d4de57
>
>Any idea what various "normal" browsers do when they encounter something like
>this?

Well, Firefox works fine.  I haven't tried IE or Mozilla or anything 
else.  That is a valid link above, so if you still have the original email 
you can click on it and see what happens.  :)


> > So, this is not a complete security solution, as it doesn't deal with
> > end-to-end file integrity, and could easily be subverted by taking over a
> > site somewhere in the middle (e.g. python.org).  But until we have more of
> > the cryptographic infrastructure in place, I think this plan could provide
> > us with a good starting point.  Comments, anyone?
>
>I presume the distutils discussion included the signing of packages that PyPI
>supports?

Yes; I have yet to encounter a signed package yet, though, and I also don't 
want to try to implement signature checking in the 0.6 series of 
setuptools.  I'll first need to learn to use GPG and get a real 
understanding of how it deals with trust chains, and understand what 
EasyInstall should do with them.  Unless of course somebody already knows 
what the design should be and can explain it well enough for me to 
implement.  :)

Even then though, I'd like to get 0.6a1 out the door first.  I'm not 100% 
sure 0.6a1 will even do any of the md5 stuff I proposed.


From richardjones at optushome.com.au  Fri Aug 19 01:15:29 2005
From: richardjones at optushome.com.au (Richard Jones)
Date: Fri, 19 Aug 2005 09:15:29 +1000
Subject: [Catalog-sig] File integrity checking and host blocking for
	EasyInstall
In-Reply-To: <5.1.1.6.0.20050818190358.01b3eeb0@mail.telecommunity.com>
References: <5.1.1.6.0.20050818190358.01b3eeb0@mail.telecommunity.com>
Message-ID: <200508190915.33852.richardjones@optushome.com.au>

On Fri, 19 Aug 2005 09:08 am, Phillip J. Eby wrote:
> On Mon, 15 Aug 2005, Richard Jones wrote:
> > On Mon, 15 Aug 2005 09:06 am, Phillip J. Eby wrote:
> > > http://www.python.org/packages/source/s/setuptools/setuptools-0.5a13.zi
> > >p#md 5=91f31a9058330174640a867cf5d4de57
> >
> >Any idea what various "normal" browsers do when they encounter something
> > like this?
>
> Well, Firefox works fine.  I haven't tried IE or Mozilla or anything
> else.  That is a valid link above, so if you still have the original email
> you can click on it and see what happens.  :)

I'll be wanting to hear feedback from someone who has IE before I make the 
jump. I just worry too much, probably :)


> > > So, this is not a complete security solution, as it doesn't deal with
> > > end-to-end file integrity, and could easily be subverted by taking over
> > > a site somewhere in the middle (e.g. python.org).  But until we have
> > > more of the cryptographic infrastructure in place, I think this plan
> > > could provide us with a good starting point.  Comments, anyone?
> >
> >I presume the distutils discussion included the signing of packages that
> > PyPI supports?
>
> Yes; I have yet to encounter a signed package yet

http://www.python.org/pypi/roundup :)


     Richard
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/catalog-sig/attachments/20050819/57b9350b/attachment.pgp

From pje at telecommunity.com  Fri Aug 19 02:12:35 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 18 Aug 2005 20:12:35 -0400
Subject: [Catalog-sig] File integrity checking and host blocking for
 EasyInstall
In-Reply-To: <200508190915.33852.richardjones@optushome.com.au>
References: <5.1.1.6.0.20050818190358.01b3eeb0@mail.telecommunity.com>
	<5.1.1.6.0.20050818190358.01b3eeb0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050818201200.01b34878@mail.telecommunity.com>

At 09:15 AM 8/19/2005 +1000, Richard Jones wrote:
>On Fri, 19 Aug 2005 09:08 am, Phillip J. Eby wrote:
> > Yes; I have yet to encounter a signed package yet
>
>http://www.python.org/pypi/roundup :)

FYI, the source package's signature file is empty.



From pje at telecommunity.com  Fri Aug 19 03:11:21 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 18 Aug 2005 21:11:21 -0400
Subject: [Catalog-sig] File integrity checking and host blocking for
 EasyInstall
In-Reply-To: <200508191022.52333.richardjones@optushome.com.au>
References: <5.1.1.6.0.20050818201200.01b34878@mail.telecommunity.com>
	<5.1.1.6.0.20050818190358.01b3eeb0@mail.telecommunity.com>
	<5.1.1.6.0.20050818201200.01b34878@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050818210951.01b349b0@mail.telecommunity.com>

At 10:22 AM 8/19/2005 +1000, Richard Jones wrote:
>On Fri, 19 Aug 2005 10:12 am, you wrote:
> > At 09:15 AM 8/19/2005 +1000, Richard Jones wrote:
> > >On Fri, 19 Aug 2005 09:08 am, Phillip J. Eby wrote:
> > > > Yes; I have yet to encounter a signed package yet
> > >
> > >http://www.python.org/pypi/roundup :)
> >
> > FYI, the source package's signature file is empty.
>
>Not for me it isn't.

How odd; my Firefox shows it consistently empty, but not for the binary 
package, and if I try to access the file from a different machine, it's not 
empty.

<sigh> Browsers.  Can't live with 'em, can't live without 'em.


From westside_indie at yahoo.com  Sun Aug 21 18:27:32 2005
From: westside_indie at yahoo.com (John Walton)
Date: Sun, 21 Aug 2005 09:27:32 -0700 (PDT)
Subject: [Catalog-sig] networking programming
Message-ID: <20050821162732.30206.qmail@web31007.mail.mud.yahoo.com>

Hello, everyone.  I just began school, and they
already assigned us science fair.  Since I'm in 8th
grade, I get to do demonstrations for our projects. 
I'm probably going to demonstrate Python's networking
capabilities by writing a simple instant messenger
program.  I only have a few problems:

1. I know squat about Python network Programming

2. I know nothing about networks

  So if any of you know of a good Python Networking
Tutorial or a website with lots of information on
networks and networking, please reply.  Thanks!


		
__________________________________ 
Yahoo! Mail for Mobile 
Take Yahoo! Mail with you! Check email on your mobile phone. 
http://mobile.yahoo.com/learn/mail 

From ianb at colorstudy.com  Sun Aug 21 21:20:39 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Sun, 21 Aug 2005 14:20:39 -0500
Subject: [Catalog-sig] networking programming
In-Reply-To: <20050821162732.30206.qmail@web31007.mail.mud.yahoo.com>
References: <20050821162732.30206.qmail@web31007.mail.mud.yahoo.com>
Message-ID: <4308D407.9080200@colorstudy.com>

John Walton wrote:
> Hello, everyone.  I just began school, and they
> already assigned us science fair.  Since I'm in 8th
> grade, I get to do demonstrations for our projects. 
> I'm probably going to demonstrate Python's networking
> capabilities by writing a simple instant messenger
> program.  I only have a few problems:
> 
> 1. I know squat about Python network Programming
> 
> 2. I know nothing about networks
> 
>   So if any of you know of a good Python Networking
> Tutorial or a website with lots of information on
> networks and networking, please reply.  Thanks!

It would probably be better to ask on the tutor or general python lists:

http://mail.python.org/mailman/listinfo/tutor
http://mail.python.org/mailman/listinfo/python-list

-- 
Ian Bicking  /  ianb at colorstudy.com  / http://blog.ianbicking.org

From westside_indie at yahoo.com  Mon Aug 22 05:49:51 2005
From: westside_indie at yahoo.com (John Walton)
Date: Sun, 21 Aug 2005 20:49:51 -0700 (PDT)
Subject: [Catalog-sig] Networking Information
Message-ID: <20050822034951.38527.qmail@web31011.mail.mud.yahoo.com>

Hello. It's me again.  Thanks for all the help with
the Python Networking Resources, but does anyone know
what I'll need to know to write a paper on Network
Programming and Python.  Like terminology and all
that.  Maybe I'll have a section on socketets, TCP,
Clients (half of the stuff I don't even know what it
means).  So, does anyone know any good websites with
Network Programming information.  Thanks!

John


		
____________________________________________________
Start your day with Yahoo! - make it your home page 
http://www.yahoo.com/r/hs 
 

From pje at telecommunity.com  Mon Aug 22 06:03:10 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 22 Aug 2005 00:03:10 -0400
Subject: [Catalog-sig] setuptools/EasyInstall 0.6a1 candidate 1 snapshot
Message-ID: <5.1.1.6.0.20050821235458.01afafd8@mail.telecommunity.com>

This thing's been in limbo too long, so I've made a prerelease 
snapshot.  You can get it with:

     easy_install -f http://peak.telecommunity.com/snapshots/ setuptools

I have fixed all reported bugs, and implemented quite a few new features 
like MD5 checking (which works even with PyPI's md5 data) and an enhanced 
"build_ext --inplace" that lets you switch between different Python 
interpreters without having to recompile your extensions.  (The extensions 
are built in build/, then copied to the source tree whenever you 
"build_ext" using a different interpreter than the last.)

If you have any projects using setuptools, I encourage you to give this 
snapshot a try, so I can find out if there are still any lurking bugs.  If 
there are no bug reports by next weekend, this will get pushed out as 0.6a1.

If you want to look at the documentation, use CVS or browse the 
EasyInstall.txt, setuptools.txt, and pkg_resources.txt files found at:

    http://cvs.sourceforge.net/viewcvs.py/python/python/nondist/sandbox/setuptools/

(I just made some changes to these, however, that may take a few hours to 
update in SourceForge's anonymous CVS.)


From richardjones at optushome.com.au  Mon Aug 22 06:30:09 2005
From: richardjones at optushome.com.au (Richard Jones)
Date: Mon, 22 Aug 2005 14:30:09 +1000
Subject: [Catalog-sig] setuptools/EasyInstall 0.6a1 candidate 1 snapshot
In-Reply-To: <5.1.1.6.0.20050821235458.01afafd8@mail.telecommunity.com>
References: <5.1.1.6.0.20050821235458.01afafd8@mail.telecommunity.com>
Message-ID: <200508221430.10167.richardjones@optushome.com.au>

On Mon, 22 Aug 2005 02:03 pm, Phillip J. Eby wrote:
> I have fixed all reported bugs, and implemented quite a few new features
> like MD5 checking (which works even with PyPI's md5 data)

Glad you could get things going in spite of PyPI :)

I'm going to be busy with PyWeek for the next few weeks. Once that's out of 
the way, I can do some PyPI stuff if need be (of course, anyone else is free 
to work on PyPI too... :)


   Richard
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/catalog-sig/attachments/20050822/bd2ba436/attachment.pgp

From pje at telecommunity.com  Mon Aug 22 14:59:41 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 22 Aug 2005 08:59:41 -0400
Subject: [Catalog-sig] setuptools/EasyInstall 0.6a1 candidate 1 snapshot
In-Reply-To: <200508221430.10167.richardjones@optushome.com.au>
References: <5.1.1.6.0.20050821235458.01afafd8@mail.telecommunity.com>
	<5.1.1.6.0.20050821235458.01afafd8@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050822023224.01b22720@mail.telecommunity.com>

At 02:30 PM 8/22/2005 +1000, Richard Jones wrote:
>On Mon, 22 Aug 2005 02:03 pm, Phillip J. Eby wrote:
> > I have fixed all reported bugs, and implemented quite a few new features
> > like MD5 checking (which works even with PyPI's md5 data)
>
>Glad you could get things going in spite of PyPI :)

I didn't mean it *that* way.  In fact, one of the other changes in this 
release is that I took out the workarounds for .egg uploads not being 
supported.


From ianb at colorstudy.com  Mon Aug 22 20:26:27 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Mon, 22 Aug 2005 13:26:27 -0500
Subject: [Catalog-sig] setuptools or PyPI problem...?
Message-ID: <430A18D3.5040700@colorstudy.com>

I don't know which side this belongs to, but I had a problem when I 
tried to create a package with a "-" in it ("Paste-Deploy").  setup.py 
register worked fine, and created a "Paste-Deploy" project; however, 
when I did an upload it created a "Paste_Deploy-0.1.tar.gz" file, and 
PyPI wouldn't accept it, I believe because it thought it belonged to the 
(nonexistant) Paste_Deploy project.  In the end I just smushed the words 
together, but I figure this really should work.  I don't know if it's 
setuptools (or distutils) that is uploading to PyPI incorrectly, or PyPI 
that is mismatching projects.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From pje at telecommunity.com  Mon Aug 22 20:43:44 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 22 Aug 2005 14:43:44 -0400
Subject: [Catalog-sig] [Distutils] setuptools or PyPI problem...?
In-Reply-To: <430A18D3.5040700@colorstudy.com>
Message-ID: <5.1.1.6.0.20050822143617.01b224a0@mail.telecommunity.com>

At 01:26 PM 8/22/2005 -0500, Ian Bicking wrote:
>I don't know which side this belongs to, but I had a problem when I
>tried to create a package with a "-" in it ("Paste-Deploy").  setup.py
>register worked fine, and created a "Paste-Deploy" project; however,
>when I did an upload it created a "Paste_Deploy-0.1.tar.gz" file, and
>PyPI wouldn't accept it, I believe because it thought it belonged to the
>(nonexistant) Paste_Deploy project.  In the end I just smushed the words
>together, but I figure this really should work.  I don't know if it's
>setuptools (or distutils) that is uploading to PyPI incorrectly, or PyPI
>that is mismatching projects.

Try the latest setuptools; you shouldn't have this problem as I changed it 
not to create anything other than eggs with escaped '-' characters.

However, it would be very nice to have PyPI support escaped 
nonalphanumerics in filenames, as they are a bitch to deal with 
otherwise.  Eggs absolutely have to have an unambiguously parseable 
filename, and the only way to do that is by escaping '-' to '_'.  This 
means that you can't upload eggs for a project named e.g. 'Paste-Deploy', 
if PyPI rejects a Paste_Deploy-whatever.egg file.

On the broader scope of things, I'd like to see PyPI smash all 
non-alphanumeric runs in project names to a single '-', and use 
case-insensitive project name comparison.  I'd attempt to try my hand at 
PyPI patches but at the moment don't have any obvious way to test them.


From martin at v.loewis.de  Tue Aug 23 00:01:00 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 23 Aug 2005 00:01:00 +0200
Subject: [Catalog-sig] setuptools or PyPI problem...?
In-Reply-To: <430A18D3.5040700@colorstudy.com>
References: <430A18D3.5040700@colorstudy.com>
Message-ID: <430A4B1C.8020900@v.loewis.de>

Ian Bicking wrote:
> I don't know which side this belongs to, but I had a problem when I 
> tried to create a package with a "-" in it ("Paste-Deploy").  setup.py 
> register worked fine, and created a "Paste-Deploy" project; however, 
> when I did an upload it created a "Paste_Deploy-0.1.tar.gz" file, and 
> PyPI wouldn't accept it, I believe because it thought it belonged to the 
> (nonexistant) Paste_Deploy project.  In the end I just smushed the words 
> together, but I figure this really should work.  I don't know if it's 
> setuptools (or distutils) that is uploading to PyPI incorrectly, or PyPI 
> that is mismatching projects.

Sounds like a PyPI problem to me.

Regards,
Martin

From pje at telecommunity.com  Tue Aug 23 00:03:53 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 22 Aug 2005 18:03:53 -0400
Subject: [Catalog-sig] setuptools/EasyInstall 0.6a1 candidate 1 snapshot
In-Reply-To: <200508230752.25321.richardjones@optushome.com.au>
References: <5.1.1.6.0.20050822023224.01b22720@mail.telecommunity.com>
	<5.1.1.6.0.20050821235458.01afafd8@mail.telecommunity.com>
	<5.1.1.6.0.20050822023224.01b22720@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050822180055.01b22d78@mail.telecommunity.com>

At 07:52 AM 8/23/2005 +1000, Richard Jones wrote:
>On Mon, 22 Aug 2005 10:59 pm, you wrote:
> > At 02:30 PM 8/22/2005 +1000, Richard Jones wrote:
> > >On Mon, 22 Aug 2005 02:03 pm, Phillip J. Eby wrote:
> > > > I have fixed all reported bugs, and implemented quite a few new
> > > > features like MD5 checking (which works even with PyPI's md5 data)
> > >
> > >Glad you could get things going in spite of PyPI :)
> >
> > I didn't mean it *that* way.
>
>I was being silly, mostly :)
>
>
> > In fact, one of the other changes in this
> > release is that I took out the workarounds for .egg uploads not being
> > supported.
>
>I thought they were supported now....

They are!  That's why I was able to remove the 
workarounds!  :)  Previously, setuptools' "upload" command renamed egg 
uploads to .egg.zip and pretended they were source distributions.  I've 
removed those hacks, because PyPI supports egg uploads just fine now.


From richardjones at optushome.com.au  Tue Aug 23 00:29:31 2005
From: richardjones at optushome.com.au (Richard Jones)
Date: Tue, 23 Aug 2005 08:29:31 +1000
Subject: [Catalog-sig] [Distutils] setuptools or PyPI problem...?
In-Reply-To: <5.1.1.6.0.20050822143617.01b224a0@mail.telecommunity.com>
References: <5.1.1.6.0.20050822143617.01b224a0@mail.telecommunity.com>
Message-ID: <200508230829.31592.richardjones@optushome.com.au>

On Tue, 23 Aug 2005 04:43 am, Phillip J. Eby wrote:
> On the broader scope of things, I'd like to see PyPI smash all
> non-alphanumeric runs in project names to a single '-', and use
> case-insensitive project name comparison.  I'd attempt to try my hand at
> PyPI patches but at the moment don't have any obvious way to test them.

I'm not sure I'm comfortable with this. The only key of uniqueness for 
packages is <package name>/<version> ... if we start messing with the package 
name...


     Richard
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/catalog-sig/attachments/20050823/0bc65d7b/attachment.pgp

From pje at telecommunity.com  Tue Aug 23 01:19:09 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 22 Aug 2005 19:19:09 -0400
Subject: [Catalog-sig] [Distutils] setuptools or PyPI problem...?
In-Reply-To: <200508230829.31592.richardjones@optushome.com.au>
References: <5.1.1.6.0.20050822143617.01b224a0@mail.telecommunity.com>
	<5.1.1.6.0.20050822143617.01b224a0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050822190414.01b19d98@mail.telecommunity.com>

At 08:29 AM 8/23/2005 +1000, Richard Jones wrote:
>On Tue, 23 Aug 2005 04:43 am, Phillip J. Eby wrote:
> > On the broader scope of things, I'd like to see PyPI smash all
> > non-alphanumeric runs in project names to a single '-', and use
> > case-insensitive project name comparison.  I'd attempt to try my hand at
> > PyPI patches but at the moment don't have any obvious way to test them.
>
>I'm not sure I'm comfortable with this. The only key of uniqueness for
>packages is <package name>/<version> ... if we start messing with the package
>name...

I'm just pointing out that these changes make the packaging system *more* 
robust, not less -- as long as it's done relatively soon.

Right now, we have people creating packages with names like "Python Super 
Whatchamacallit!" because they're not using the distutils and therefore 
don't notice that it's impossible to build a sane distribution with that 
name.  At the opposite extreme, we have packages with VERYMixedCase that 
nobody will ever remember how to type correctly except the package author.  :)

Neither of these is a big problem when EasyInstall isn't a part of the mix, 
because if the data's crap, who cares?  However, for EasyInstall it's not 
practical to try to type "easy_install 'Python Super Whatchamacallit!'" or 
"easy_install VERYMixedCase" and get it right.  It would be better if these 
names were canonicalized (internal to PyPI) as 
"python_super_whatchamacallit_" and "verymixedcase", with those values 
being used as database keys.  This would prevent having both a 
"VeryMixedCase" and a "VERYMixedCase" at the same time (e.g. 'sqlobject' 
and 'SQLObject').

Case-insensitivity is needed because some important platforms (Windows, Mac 
OS X) have case-insensitive filesystems.  Ergo, 'SQLObject-1.0-py2.3.egg' 
and 'sqlobject-1.0-py2.3.egg' are the *same file* on such systems, which is 
just not going to work.

Similarly, escaping non-alphanumerics as '_' (in both the project name and 
version) is necessary to ensure that egg filenames are unambiguously 
machine-readable.  Currently the distutils happily produces distributions 
with names like 'Paste-Deploy-1.2-prerelease.tar.gz', which are 
ambiguous.  A program can't tell with any certainty whether this is package 
"Paste", version "Deploy-1.2-prerelease", or package "Paste-Deploy-1.2", 
version "prerelease", or whatever else.  Any rule you can think of that 
would sort out the ambiguity, I can probably find an *actual* 
counterexample for you in the PyPI database!  (For example, there are 
packages with numbers in their names, so you can't use that to figure out 
where the version starts.)

Currently, I work around case-sensitivity and ambiguity on PyPI by doing a 
full index search if there's no correct case match.  I also build an index 
that contains all possible interpretations(!) of ambiguous filenames, so 
that I can match against what a user asked for.  Luckily, the odds of 
somebody mistakenly requesting version "Deploy-1.2-prerelease" of "Paste" 
and thereby getting Paste-Deploy version 1.2-prerelease instead are fairly 
low.  :)

However, while I can do that workaround for indexing PyPI's contents, it 
doesn't work for installed eggs; the names *must* be unambiguous in order 
to avoid the overhead of opening or extracting PKG-INFO in order to find 
out the distribution's name.  So egg files at least need to turn '-' to '_' 
in their filenames.