From stephen at thorne.id.au  Tue Mar  3 00:21:44 2009
From: stephen at thorne.id.au (Stephen Thorne)
Date: Tue, 3 Mar 2009 09:21:44 +1000
Subject: [Catalog-sig] pypi xmlrpc interface
Message-ID: <20090302232144.GA1604@thorne.id.au>

G'day,

I get a traceback from the server when I try to list releases of a
package using the xmlrpc interface.

The specific example used on the wiki reproduced here does this:

>>> import xmlrpclib
>>> server = xmlrpclib.ServerProxy('http://pypi.python.org/pypi')
>>> server.package_urls('roundup', '1.1.2')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.5/xmlrpclib.py", line 1150, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib64/python2.5/xmlrpclib.py", line 1440, in __request
    verbose=self.__verbose
  File "/usr/lib64/python2.5/xmlrpclib.py", line 1204, in request
    return self._parse_response(h.getfile(), sock)
  File "/usr/lib64/python2.5/xmlrpclib.py", line 1343, in _parse_response
    return u.close()
  File "/usr/lib64/python2.5/xmlrpclib.py", line 790, in close
    raise Fault(**self._stack[0])
xmlrpclib.Fault: <Fault 1: 'Traceback (most recent call last):\n ...'>
>>> 

That traceback-from-the-server-on-a-stick-in-a-string looks like this when
rendered nicely:

Traceback (most recent call last):
  File "/data/pypi/src/pypi/rpc.py", line 23, in handle_request
    xml = xmlrpclib.dumps((response,), methodresponse=True, allow_none=True)
  File "/usr/lib/python2.5/xmlrpclib.py", line 1080, in dumps
    data = m.dumps(params)
  File "/usr/lib/python2.5/xmlrpclib.py", line 623, in dumps
    dump(v, write)
  File "/usr/lib/python2.5/xmlrpclib.py", line 635, in __dump
    f(self, value, write)
  File "/usr/lib/python2.5/xmlrpclib.py", line 695, in dump_array
    dump(v, write)
  File "/usr/lib/python2.5/xmlrpclib.py", line 635, in __dump
    f(self, value, write)
  File "/usr/lib/python2.5/xmlrpclib.py", line 716, in dump_struct
    dump(v, write)
  File "/usr/lib/python2.5/xmlrpclib.py", line 633, in __dump
    raise TypeError, "cannot marshal %s objects" % type(value)
TypeError: cannot marshal <type 'mx.DateTime.DateTime'> objects

-- 
Regards,
Stephen Thorne
Development Engineer
NetBox Blue - 1300 737 060

Find out why NetBox Blue's solutions are growing in popularity in 
today's economic environment: http://netboxblue.com/news/creditcrunch 


Scanned by the NetBox from NetBox Blue
(http://netboxblue.com/)


From martin at v.loewis.de  Sat Mar  7 11:27:15 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 07 Mar 2009 11:27:15 +0100
Subject: [Catalog-sig] pypi xmlrpc interface
In-Reply-To: <20090302232144.GA1604@thorne.id.au>
References: <20090302232144.GA1604@thorne.id.au>
Message-ID: <49B24C03.2010604@v.loewis.de>

> I get a traceback from the server when I try to list releases of a
> package using the xmlrpc interface.

Thanks for the report. This is now fixed.

Regards,
Martin

P.S. Please don't call the method package_urls, but release_urls;
the former is deprecated.

From ziade.tarek at gmail.com  Mon Mar  9 06:43:29 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 9 Mar 2009 06:43:29 +0100
Subject: [Catalog-sig] Packaging Survey
Message-ID: <94bdd2610903082243o7a7fb445keef03357c062fe5e@mail.gmail.com>

The Python Langage Summit is coming up. To prepare this event, I have
put online a survey you can take to tell us a bit more about you and
how you package your Python applications.

    * Who should take the survey : any Python developer that packages
and distributes his code, no matter how.
    * Take the survey: http://tinyurl.com/package-survey

Thanks to all the people that helped building the survey, and a
special thanks to Massimo Di Pierro who created the application that
runs the Survey and helped me set up the survey.

Regards
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From robertwb at math.washington.edu  Sat Mar 21 01:01:21 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 20 Mar 2009 17:01:21 -0700
Subject: [Catalog-sig] A Trove classifier for Cython?
In-Reply-To: <e7ba66e40903201600y27145d56va771dd9d3d5aa6aa@mail.gmail.com>
References: <e7ba66e40903201204l13e954c8ua252f38c6984ac69@mail.gmail.com>
	<49C40348.3000104@behnel.de>
	<e7ba66e40903201600y27145d56va771dd9d3d5aa6aa@mail.gmail.com>
Message-ID: <04ED9E4A-1030-44A0-B5AB-AA2EEE0B4D29@math.washington.edu>

Could we get a trove classifier for the programing language Cython?

Programming Language :: Cython

Thanks,
Robert Bradshaw


From martin at v.loewis.de  Sat Mar 21 07:26:30 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 21 Mar 2009 07:26:30 +0100
Subject: [Catalog-sig] A Trove classifier for Cython?
In-Reply-To: <04ED9E4A-1030-44A0-B5AB-AA2EEE0B4D29@math.washington.edu>
References: <e7ba66e40903201204l13e954c8ua252f38c6984ac69@mail.gmail.com>	<49C40348.3000104@behnel.de>	<e7ba66e40903201600y27145d56va771dd9d3d5aa6aa@mail.gmail.com>
	<04ED9E4A-1030-44A0-B5AB-AA2EEE0B4D29@math.washington.edu>
Message-ID: <49C48896.5040308@v.loewis.de>

Robert Bradshaw wrote:
> Could we get a trove classifier for the programing language Cython?
> 
> Programming Language :: Cython

What packages would be classified under this classifier (both
specific, and in principle)?

Regards,
Martin

From tseaver at palladion.com  Sat Mar 21 14:00:52 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Sat, 21 Mar 2009 09:00:52 -0400
Subject: [Catalog-sig] A Trove classifier for Cython?
In-Reply-To: <49C48896.5040308@v.loewis.de>
References: <e7ba66e40903201204l13e954c8ua252f38c6984ac69@mail.gmail.com>	<49C40348.3000104@behnel.de>	<e7ba66e40903201600y27145d56va771dd9d3d5aa6aa@mail.gmail.com>	<04ED9E4A-1030-44A0-B5AB-AA2EEE0B4D29@math.washington.edu>
	<49C48896.5040308@v.loewis.de>
Message-ID: <gq2oe4$ntv$2@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Martin v. L?wis wrote:
> Robert Bradshaw wrote:
>> Could we get a trove classifier for the programing language Cython?
>>
>> Programming Language :: Cython
> 
> What packages would be classified under this classifier (both
> specific, and in principle)?

One example:

- - 'lxml' uses Cython to implement its wrappers of libxml2 and libxslt;
  it can be installed without Cython (the generated C code is part of
  the released package), but requires Cython for a build after a
  'distclean'.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJxOUE+gerLs4ltQ4RAl8SAKCth7L6S1lKwhZ0tYtfkhc2nNLuSwCg1zmb
2p997QN0+m3ePdVevpo8Oto=
=bugK
-----END PGP SIGNATURE-----


From dalcinl at gmail.com  Sat Mar 21 15:36:24 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Sat, 21 Mar 2009 11:36:24 -0300
Subject: [Catalog-sig] [Cython] A Trove classifier for Cython?
In-Reply-To: <gq2oe4$ntv$2@ger.gmane.org>
References: <e7ba66e40903201204l13e954c8ua252f38c6984ac69@mail.gmail.com>
	<49C40348.3000104@behnel.de>
	<e7ba66e40903201600y27145d56va771dd9d3d5aa6aa@mail.gmail.com>
	<04ED9E4A-1030-44A0-B5AB-AA2EEE0B4D29@math.washington.edu>
	<49C48896.5040308@v.loewis.de> <gq2oe4$ntv$2@ger.gmane.org>
Message-ID: <e7ba66e40903210736p1f21a4bdmdef40e8ccb006a9a@mail.gmail.com>

On Sat, Mar 21, 2009 at 10:00 AM, Tres Seaver <tseaver at palladion.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Martin v. L?wis wrote:
>> Robert Bradshaw wrote:
>>> Could we get a trove classifier for the programing language Cython?
>>>
>>> Programming Language :: Cython
>>
>> What packages would be classified under this classifier (both
>> specific, and in principle)?
>

More examples, from my side (all of them at googlecode.comm):

1) mpi4py
2) petsc4py
3) slepc4py
4) tao4py



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From martin at v.loewis.de  Sat Mar 21 19:22:45 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 21 Mar 2009 19:22:45 +0100
Subject: [Catalog-sig] [Cython] A Trove classifier for Cython?
In-Reply-To: <e7ba66e40903210736p1f21a4bdmdef40e8ccb006a9a@mail.gmail.com>
References: <e7ba66e40903201204l13e954c8ua252f38c6984ac69@mail.gmail.com>	<49C40348.3000104@behnel.de>	<e7ba66e40903201600y27145d56va771dd9d3d5aa6aa@mail.gmail.com>	<04ED9E4A-1030-44A0-B5AB-AA2EEE0B4D29@math.washington.edu>	<49C48896.5040308@v.loewis.de>
	<gq2oe4$ntv$2@ger.gmane.org>
	<e7ba66e40903210736p1f21a4bdmdef40e8ccb006a9a@mail.gmail.com>
Message-ID: <49C53075.2020504@v.loewis.de>

>>>> Could we get a trove classifier for the programing language Cython?
>>>>
>>>> Programming Language :: Cython
>>> What packages would be classified under this classifier (both
>>> specific, and in principle)?
> 
> More examples, from my side (all of them at googlecode.comm):
> 
> 1) mpi4py
> 2) petsc4py
> 3) slepc4py
> 4) tao4py

Thanks, I have added this classifier now.

Martin

From robertwb at math.washington.edu  Sat Mar 21 20:31:41 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Sat, 21 Mar 2009 12:31:41 -0700
Subject: [Catalog-sig] [Cython] A Trove classifier for Cython?
In-Reply-To: <e7ba66e40903210736p1f21a4bdmdef40e8ccb006a9a@mail.gmail.com>
References: <e7ba66e40903201204l13e954c8ua252f38c6984ac69@mail.gmail.com>
	<49C40348.3000104@behnel.de>
	<e7ba66e40903201600y27145d56va771dd9d3d5aa6aa@mail.gmail.com>
	<04ED9E4A-1030-44A0-B5AB-AA2EEE0B4D29@math.washington.edu>
	<49C48896.5040308@v.loewis.de> <gq2oe4$ntv$2@ger.gmane.org>
	<e7ba66e40903210736p1f21a4bdmdef40e8ccb006a9a@mail.gmail.com>
Message-ID: <0EA97A4E-D300-4698-AAFC-85E1AF0D65DD@math.washington.edu>

On Mar 21, 2009, at 7:36 AM, Lisandro Dalcin wrote:

> On Sat, Mar 21, 2009 at 10:00 AM, Tres Seaver  
> <tseaver at palladion.com> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Martin v. L?wis wrote:
>>> Robert Bradshaw wrote:
>>>> Could we get a trove classifier for the programing language Cython?
>>>>
>>>> Programming Language :: Cython
>>>
>>> What packages would be classified under this classifier (both
>>> specific, and in principle)?
>>
>
> More examples, from my side (all of them at googlecode.comm):
>
> 1) mpi4py
> 2) petsc4py
> 3) slepc4py
> 4) tao4py

Note that these are also indexed on pypi. Some other packages using  
Cython are

http://pypi.python.org/pypi/PyAMF
http://pypi.python.org/pypi/cogent
http://pypi.python.org/pypi/PyYAML
http://pypi.python.org/pypi/jwp_ri
http://pypi.python.org/pypi/line_profiler
http://pypi.python.org/pypi/python-ctags
http://pypi.python.org/pypi/python-ecore
http://pypi.python.org/pypi/python-edje
http://pypi.python.org/pypi/python-emotion
http://pypi.python.org/pypi/python-epsilon
http://pypi.python.org/pypi/python-evas
http://pypi.python.org/pypi/scikits.audiolab
http://pypi.python.org/pypi/mwlib
http://pypi.python.org/pypi/TailSpin
http://pypi.python.org/pypi/BIP
http://pypi.python.org/pypi/WorldMill

The project I'm most involved in, http://sagemath.org also has 200,000 
+ lines of Cython code. In principle, any project could use Cython,  
but the most likely candidates are wrappers of external C/C++  
libraries (Cython makes this really easy) and scientific code (where  
speed is highly valued).

- Robert


From ziade.tarek at gmail.com  Thu Mar 26 04:58:51 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 26 Mar 2009 04:58:51 +0100
Subject: [Catalog-sig] Packaging Survey first results + Summit schedule
Message-ID: <94bdd2610903252058n1a15daa7ic87cd32c7570ba81@mail.gmail.com>

Hi,

Sorry for the cross-post, but it seemed appropriate since packaging is
being discussed in python-dev tonight,

- Here are the first results for the packaging survey:
http://tarekziade.wordpress.com/2009/03/26/packaging-survey-first-results/
- And tomorrow's Summit schedule for the packaging part :
http://tarekziade.wordpress.com/2009/03/26/pycon-language-summit-is-tomorrow/

Please comment (in the appropriate list or in my blog) if you have
something you would like to say or see addressed
during the Summit and you are unable to be present.

(I am already trying to summarize what has been said in python-dev
today but I am not sure I'll be able to read everything
 before tomorrow)

Regards
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From kgmuller at xs4all.nl  Thu Mar 26 18:43:27 2009
From: kgmuller at xs4all.nl (kgmuller)
Date: Thu, 26 Mar 2009 10:43:27 -0700 (PDT)
Subject: [Catalog-sig] Can not submit new packages
In-Reply-To: <49774B14.90109@jcea.es>
References: <49774B14.90109@jcea.es>
Message-ID: <22727321.post@talk.nabble.com>


The same happened to me today (March 26 09). I tried to upload a
new release for SimPy and got the same 401 message, although I 
had identified myself.

What is going on?

Thanks for any help!

Klaus Muller
Lead Developer SimPy, (http://simpy.sourceforge.net)


Jesus Cea-2 wrote:
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> I am trying to upload a new package to PYPI, and the sending is failing
> with an authentication failure. I already register again with "python
> setup.py register" (using my old credentials), and the server gives "200
> OK".. But trying "python setup.py sdist upload --sign --show-response"
> gives this error:
> 
> Submitting dist/bsddb3-4.7.4.tar.gz to http://pypi.python.org/pypi
> Upload failed (401): You must be identified to edit package information
> -
> ---------------------------------------------------------------------------
> <strong>Login required</strong><br /><br />
> 
> You must be identified to edit package information<br /><br />
> 
> <p>If you are a new user,  /pypi?:action=register_form please
> register .</p>
> <p>If you have forgotten your password, you can have it
>  /pypi?:action=forgotten_password_form reset for you .</p>
> -
> ---------------------------------------------------------------------------
> 
> - --
> Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
> jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
> jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
> .                              _/_/  _/_/    _/_/          _/_/  _/_/
> "Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
> "My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
> "El amor es poner tu felicidad en la felicidad de otro" - Leibniz
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.8 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
> 
> iQCVAwUBSXdLD5lgi5GaxT1NAQKFsgP/R3djp/2pwEsGJ2T0KTqv2xDEEaRzZLkb
> A4agXQqQ6VXM1Zd5KPR+z+/jjUAEIsb/Glih6QIrMS2dyNbE4C8w9i1ktqOCHpjQ
> h78mbzoWwJ9GaEwlb1vZIjazFdewIYyCsAXjZaB3VGiHSOStGUiPXG1X70eAyujo
> VWoSqV/GIbs=
> =Bi7t
> -----END PGP SIGNATURE-----
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org
> http://mail.python.org/mailman/listinfo/catalog-sig
> 
> 

-- 
View this message in context: http://www.nabble.com/Can-not-submit-new-packages-tp21586515p22727321.html
Sent from the Python - catalog-sig mailing list archive at Nabble.com.


From denis-bz at t-online.de  Fri Mar 27 16:29:01 2009
From: denis-bz at t-online.de (denis)
Date: Fri, 27 Mar 2009 16:29:01 +0100
Subject: [Catalog-sig] are quotes and blanks in package names a good idea ?
Message-ID: <49CCF0BD.1000002@t-online.de>

Folks,

  are quotes ' and blanks in package names a good idea ?
"abo's pysync" broke BeautifulSoup on pypi /simple this week;
Martin patched that quickly (thanks Martin)
but other package tools, python or distis, will surely break.

Agree / disagree / dont-care ?


(A simple lesson from engineering, whether Toyotas or software, is:
if you want quality, you have to work at it --
define what you'll accept or not, post it, enforce it, keep after 
people, /improve/;
a thankless task.
end-of-sermon).

cheers
    -- denis


p.s. a trivia question for oldtimers: what languages allow(ed) blanks in 
identifiers ?

From martin at v.loewis.de  Fri Mar 27 20:43:39 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 27 Mar 2009 14:43:39 -0500
Subject: [Catalog-sig] are quotes and blanks in package names a good
 idea ?
In-Reply-To: <49CCF0BD.1000002@t-online.de>
References: <49CCF0BD.1000002@t-online.de>
Message-ID: <49CD2C6B.5010102@v.loewis.de>

>  are quotes ' and blanks in package names a good idea ?
> "abo's pysync" broke BeautifulSoup on pypi /simple this week;
> Martin patched that quickly (thanks Martin)
> but other package tools, python or distis, will surely break.
> 
> Agree / disagree / dont-care ?

Disagree. PyPI and setuptools perform a package name normalization,
which is file-system-safe. I don't think anything else needs to
be done.

Regards,
Martin

From robertwb at math.washington.edu  Fri Mar 27 22:31:26 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 27 Mar 2009 14:31:26 -0700
Subject: [Catalog-sig] [Cython] A Trove classifier for Cython?
In-Reply-To: <0EA97A4E-D300-4698-AAFC-85E1AF0D65DD@math.washington.edu>
References: <e7ba66e40903201204l13e954c8ua252f38c6984ac69@mail.gmail.com>
	<49C40348.3000104@behnel.de>
	<e7ba66e40903201600y27145d56va771dd9d3d5aa6aa@mail.gmail.com>
	<04ED9E4A-1030-44A0-B5AB-AA2EEE0B4D29@math.washington.edu>
	<49C48896.5040308@v.loewis.de> <gq2oe4$ntv$2@ger.gmane.org>
	<e7ba66e40903210736p1f21a4bdmdef40e8ccb006a9a@mail.gmail.com>
	<0EA97A4E-D300-4698-AAFC-85E1AF0D65DD@math.washington.edu>
Message-ID: <9866BAA7-8BB2-4D60-8EBD-B61C3614459F@math.washington.edu>

On Mar 21, 2009, at 12:31 PM, Robert Bradshaw wrote:

> On Mar 21, 2009, at 7:36 AM, Lisandro Dalcin wrote:
>
>> On Sat, Mar 21, 2009 at 10:00 AM, Tres Seaver
>> <tseaver at palladion.com> wrote:
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA1
>>>
>>> Martin v. L?wis wrote:
>>>> Robert Bradshaw wrote:
>>>>> Could we get a trove classifier for the programing language  
>>>>> Cython?
>>>>>
>>>>> Programming Language :: Cython
>>>>
>>>> What packages would be classified under this classifier (both
>>>> specific, and in principle)?
>>>
>>

To follow up, looks like it's been added

http://pypi.python.org/pypi?%3Aaction=list_classifiers

Thanks.

- Robert

From renesd at gmail.com  Sat Mar 28 03:59:15 2009
From: renesd at gmail.com (=?ISO-8859-1?Q?Ren=E9_Dudfield?=)
Date: Sat, 28 Mar 2009 13:59:15 +1100
Subject: [Catalog-sig] trove classifier for pygame - 'Library :: PyGame' or
	'Framework :: PyGame'
Message-ID: <64ddb72c0903271959u5b1f3d7bpba94283d69f98de6@mail.gmail.com>

Hello,

I think it'd be good if there was a trove classifier for pygame.  I'm not
sure what it would be exactly...

I think this could be best:
    'Library :: PyGame'
or this:
    'Framework :: PyGame'


There's 1000s of projects which use it.  Many of them applications, games,
or libraries.

Some specific libraries are listed here:
    http://pygame.org/tags/libraries



cheers,
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20090328/b3029b5c/attachment.htm>

From martin at v.loewis.de  Sat Mar 28 14:16:42 2009
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 28 Mar 2009 08:16:42 -0500
Subject: [Catalog-sig] Mirror authenticity
Message-ID: <49CE233A.8090900@v.loewis.de>

At the language summit, there was a request that PyPI mirrors
should get authenticated through some kind of digital signature
that is generated by the master server, and can be verified by
clients using the mirror. This addresses the threat of somebody
taking over a mirror and injecting false packages. Attacks
against the master are not addressed; authors should use the
existing PGP signing of packages to guarantee authenticity.

I propose the following structure to provide the ability
of verification at the clients (i.e. setuptools and friends).
At the server, the following URLs are available:

/serverkey   Public DSA key of the server, in the PEM format
              as generated by "openssl dsa -pubout" (i.e. RFC 3280
              SubjectPublicKeyInfo, with the algorithm 1.3.14.3.2.12).
              This URL must *not* be mirrored, and clients must fetch
              the official serverkey from PyPI directly. The serverkey
              will change roughly once every year. Clients should cache
              the serverkey, and refetch it if it is
              a) more than one month old, or
              b) a signature failed to verify (which might be because
                 the serverkey has changed)
/serversig/<package>
              DSA signature of the parallel URL /simple/<package>,
              in DER form, using SHA-1 with DSA (i.e. as a RFC 3279
              Dsa-Sig-Value, created by algorithm 1.2.840.10040.4.3)
              These URLs must be mirrored.

Signing the individual package pages is necessary because an
attacker might inject an additional download URL to a package,
tricking the client to download from a different location.
With the individual pages signed, signing the actual package
data is not necessary anymore, since each page contains md5 checksums
of the individual files.

Clients should only verify keys when they download from a mirror of
their (respective) central repository. Signing will cause
overhead (both for the server and the client), which is unnecessary
when the master server is contacted. In addition, the client might
be pointed to a master server which doesn't provide signatures
(and consequentially, doesn't provide mirrors, either).

Clients which do verify need to
1. compute SHA1 of the of the /simple page
2. compute the DSA signature of that hash
3. compare it with the /serversig data (byte-for-byte)
4. compute and verify md5 sums for all the files that they
    then download from mirror. Verification of files downloaded
    from other URLs is not possible with this approach.

I will try to provide a pure-Python implementation of
the page verification, based on AMK's python-crypto code.

Comments on this proposal are appreciated.

Regards,
Martin


From amk at amk.ca  Sat Mar 28 15:51:43 2009
From: amk at amk.ca (A.M. Kuchling)
Date: Sat, 28 Mar 2009 09:51:43 -0500
Subject: [Catalog-sig] Mirror authenticity
In-Reply-To: <49CE233A.8090900@v.loewis.de>
References: <49CE233A.8090900@v.loewis.de>
Message-ID: <20090328145143.GA1353@amk.local>

On Sat, Mar 28, 2009 at 08:16:42AM -0500, "Martin v. L?wis" wrote:
> I will try to provide a pure-Python implementation of
> the page verification, based on AMK's python-crypto code.

I suspect python-crypto is too low-level; OpenSSL uses PEM-encoding
and supports S/MIME signatures, but pycrypto doesn't implement PEM at
all.  It might be better to rely on having the 'openssl' executable
available and figuring out the right switches to generate a signature.

(BTW, I'm not maintaining python-crypto any longer; Dwayne
Litzenberger has taken it over and has a new site at www.pycrypto.org.
I don't know what his plans are for a new release.)

--amk




From martin at v.loewis.de  Sat Mar 28 19:22:04 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 28 Mar 2009 13:22:04 -0500
Subject: [Catalog-sig] Mirror authenticity
In-Reply-To: <20090328145143.GA1353@amk.local>
References: <49CE233A.8090900@v.loewis.de> <20090328145143.GA1353@amk.local>
Message-ID: <49CE6ACC.3050300@v.loewis.de>

> I suspect python-crypto is too low-level; OpenSSL uses PEM-encoding
> and supports S/MIME signatures, but pycrypto doesn't implement PEM at
> all.  It might be better to rely on having the 'openssl' executable
> available and figuring out the right switches to generate a signature.

Unfortunately, using the openssl command line isn't good enough.
It doesn't support DSA signing or verifying (the PyPI client would
need verification, not signing).

On the server, I have now M2Crypto working.

One option would have been to use gpg signing, however that would
break on systems that don't normally have a gpg binary available
(similar to relying on the openssl binary)

> (BTW, I'm not maintaining python-crypto any longer; Dwayne
> Litzenberger has taken it over and has a new site at www.pycrypto.org.
> I don't know what his plans are for a new release.)

I really only need the algorithm that does the signature verification.
I'll do the PEM support myself; I find DER not too difficult.

Regards,
Martin

From jafo at tummy.com  Sun Mar 29 07:06:57 2009
From: jafo at tummy.com (Sean Reifschneider)
Date: Sat, 28 Mar 2009 23:06:57 -0600
Subject: [Catalog-sig] Mirror authenticity
In-Reply-To: <49CE6ACC.3050300@v.loewis.de>
References: <49CE233A.8090900@v.loewis.de> <20090328145143.GA1353@amk.local>
	<49CE6ACC.3050300@v.loewis.de>
Message-ID: <49CF01F1.70705@tummy.com>

Martin v. L?wis wrote:
> Unfortunately, using the openssl command line isn't good enough.
> It doesn't support DSA signing or verifying (the PyPI client would
> need verification, not signing).

Are you sure?  Doesn't the "dgst" message digest sub-command do what you're
looking for, given a DSA public/private key pair?

   openssl dgst -sign private-key-file -out signature-file <file-to-verify
   openssl dgst -verify public-key-file -signature signature-file <file-to-verify

Sean
-- 
Sean Reifschneider, Member of Technical Staff <jafo at tummy.com>
tummy.com, ltd. - Linux Consulting since 1995: Ask me about High Availability

From martin at v.loewis.de  Sun Mar 29 14:07:39 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 29 Mar 2009 07:07:39 -0500
Subject: [Catalog-sig] Mirror authenticity
In-Reply-To: <49CF01F1.70705@tummy.com>
References: <49CE233A.8090900@v.loewis.de>
	<20090328145143.GA1353@amk.local>	<49CE6ACC.3050300@v.loewis.de>
	<49CF01F1.70705@tummy.com>
Message-ID: <49CF648B.6080308@v.loewis.de>

> Are you sure?  Doesn't the "dgst" message digest sub-command do what you're
> looking for, given a DSA public/private key pair?
> 
>    openssl dgst -sign private-key-file -out signature-file <file-to-verify
>    openssl dgst -verify public-key-file -signature signature-file <file-to-verify

Interesting - I missed that. However, I can't get it to work, either:

$ openssl dgst -sign privkey -sha1 /etc/passwd
Error Signing Data
5216:error:0606B06E:digital envelope routines:EVP_SignFinal:wrong public
key type:p_sign.c:103:

where privkey is a PEM "DSA PRIVATE KEY". I'm puzzled about the error
message - *of course* I'm not passing a public key. This is with Apple's
openssl 0.9.7l.

In any case, I have now completed a mixed M2Crypto/pure-python signature
verification procedure, so I don't need to rely on an openssl binary
(which typically wouldn't be available on Windows, anyway). If you are
curious, please review the attached code. The __main__ should actually
work for the live pypi.python.org; the server key is in /serverkey.

Regards,
Martin

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: verify.py
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20090329/f33f2d6a/attachment.txt>

From martin at v.loewis.de  Sun Mar 29 21:18:27 2009
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 29 Mar 2009 14:18:27 -0500
Subject: [Catalog-sig] Reordering links on simple pages
Message-ID: <49CFC983.8080503@v.loewis.de>

I have changed the order of links on the /simple
pages, to put the links to uploaded files first.
Supposedly, this will have setuptools consider
those files first before going to off-site links.

Please let me know if there are any problem.

Regards,
Martin

From martin at v.loewis.de  Sun Mar 29 21:20:25 2009
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 29 Mar 2009 14:20:25 -0500
Subject: [Catalog-sig] Generating relative /packages links
Message-ID: <49CFC9F9.1090903@v.loewis.de>

I would like to change the links to the uploaded
files to relative links, pointing to /packages.
This will simplify mirroring.

Can anybody see problems with such a change?

Also, if you can't see problems: can anybody
confirm specifically that setuptools would process
these fine?

Regards,
Martin

From jafo at tummy.com  Sun Mar 29 23:44:49 2009
From: jafo at tummy.com (Sean Reifschneider)
Date: Sun, 29 Mar 2009 15:44:49 -0600
Subject: [Catalog-sig] Mirror authenticity
In-Reply-To: <49CF648B.6080308@v.loewis.de>
References: <49CE233A.8090900@v.loewis.de>
	<20090328145143.GA1353@amk.local>	<49CE6ACC.3050300@v.loewis.de>
	<49CF01F1.70705@tummy.com> <49CF648B.6080308@v.loewis.de>
Message-ID: <49CFEBD1.5060300@tummy.com>

Martin v. L?wis wrote:
> $ openssl dgst -sign privkey -sha1 /etc/passwd
> Error Signing Data
> 5216:error:0606B06E:digital envelope routines:EVP_SignFinal:wrong public
> key type:p_sign.c:103:

openssl dsaparam 2048 < /dev/urandom > dsaparam.pem
openssl gendsa dsaparam.pem -out dsapriv.pem
openssl dsa -in dsapriv.pem -pubout -out dsapub.pem
openssl dgst -dss1 -sign dsapriv.pem </etc/services >services.sig

Then:

   guin:/tmp$ openssl dgst -dss1 -verify dsapub.pem -signature services.sig
</etc/services
   Verified OK
   guin:/tmp$ openssl dgst -dss1 -verify dsapub.pem -signature services.sig
</etc/passwd
   Verification Failure
   zsh: exit 1     openssl dgst -dss1 -verify dsapub.pem -signature
services.sig < /etc/passwd
   guin:/tmp$

> where privkey is a PEM "DSA PRIVATE KEY". I'm puzzled about the error
> message - *of course* I'm not passing a public key. This is with Apple's
> openssl 0.9.7l.

It's kind of a funny work-flow to set up a public/private key pair because
of the way DSA works.

Sean
-- 
Sean Reifschneider, Member of Technical Staff <jafo at tummy.com>
tummy.com, ltd. - Linux Consulting since 1995: Ask me about High Availability

From pje at telecommunity.com  Mon Mar 30 01:17:59 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Sun, 29 Mar 2009 19:17:59 -0400
Subject: [Catalog-sig] Generating relative /packages links
In-Reply-To: <49CFC9F9.1090903@v.loewis.de>
References: <49CFC9F9.1090903@v.loewis.de>
Message-ID: <20090329231538.33DAC3A406A@sparrow.telecommunity.com>

At 02:20 PM 3/29/2009 -0500, Martin v. L?wis wrote:
>I would like to change the links to the uploaded
>files to relative links, pointing to /packages.
>This will simplify mirroring.
>
>Can anybody see problems with such a change?
>
>Also, if you can't see problems: can anybody
>confirm specifically that setuptools would process
>these fine?

Setuptools uses urlparse.urljoin() on all URLs that it pulls out of HTML.


From ziade.tarek at gmail.com  Mon Mar 30 06:25:45 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 29 Mar 2009 23:25:45 -0500
Subject: [Catalog-sig] [Python-checkins] r70699 - peps/trunk/pep-0381.txt
In-Reply-To: <49D00050.8020300@v.loewis.de>
References: <20090329213727.6EEA81E4002@bag.python.org>
	<49D00050.8020300@v.loewis.de>
Message-ID: <94bdd2610903292125y38856111m43ddca582ee43c7b@mail.gmail.com>

RIght I am fixing this right now.

There's something we didn't talk about yet : since there will be other
package indexes
out there (not PyPI mirrors) that might have their own mirrors, we do
need to provide
somewhere the hostname that holds the mirrors IP for the client software to work
the same way in all case.

That is, mirrors.pypi.python.org for PyPI, but maybe
mirrors.packages.plone.org for
another index.

I'd go for a /mirror-hostname unique page at PyPI (and its mirrors)
even if this page is unreachable when PyPI is down.

This enforces that the other indexes also use the DNS technique, but I
think it's fine

Tarek


On Sun, Mar 29, 2009 at 6:12 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>
>> +A mirror has to be a hostname. For example:
>> +
>> +- http://pypi.my-company.com
>> +- http://mirror-pypi.somewhere.org
>> +
>
> That actually doesn't work - the mirrors have to be
> IP addresses (and I need a commitment from the mirror
> operator to not change it)
>
> Regards,
> Martin
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From lists at zopyx.com  Mon Mar 30 15:18:28 2009
From: lists at zopyx.com (Andreas Jung)
Date: Mon, 30 Mar 2009 08:18:28 -0500
Subject: [Catalog-sig] Issue with XMLRPC API: server.changelog
Message-ID: <49D0C6A4.9080201@zopyx.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Martin,

the z3c.pypimirror script throws an error since yesterday or so:

(Pdb) server.changelog(int(time.time() - fetch_since_days*24*3600))
*** Fault: <Fault 1: 'Traceback (most recent call last):\n  File
"/data/pypi/src/pypi/rpc.py", line 25, in handle_request\n    response =
globals()[methodName](webui_obj.store, *methodArgs)\n  File
"/data/pypi/src/pypi/rpc.py", line 91, in changelog\n    for row in
result]\nAttributeError: \'datetime.datetime\' object has no attribute
\'gmticks\'\n'>

Any ideas?

Andreas


- -- 
ZOPYX Ltd. & Co. KG - Charlottenstr. 37/1 - 72070 T?bingen - Germany
Web: www.zopyx.com - Email: info at zopyx.com - Phone +49 - 7071 - 793376
Registergericht: Amtsgericht Stuttgart, Handelsregister A 381535
Gesch?ftsf?hrer/Gesellschafter: ZOPYX Limited, Birmingham, UK
- ------------------------------------------------------------------------
E-Publishing, Python, Zope & Plone development, Consulting

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAknQxqMACgkQCJIWIbr9KYykjwCg0CH9KrmVNzqB2XjUeIJqETed
/fUAn2K+iHlThOA+MLmWiptB0Tfz7B89
=0Fpv
-----END PGP SIGNATURE-----
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lists.vcf
Type: text/x-vcard
Size: 316 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20090330/70528bcc/attachment.vcf>

From ianb at colorstudy.com  Mon Mar 30 18:42:16 2009
From: ianb at colorstudy.com (Ian Bicking)
Date: Mon, 30 Mar 2009 11:42:16 -0500
Subject: [Catalog-sig] [Python-checkins] r70699 - peps/trunk/pep-0381.txt
In-Reply-To: <94bdd2610903292125y38856111m43ddca582ee43c7b@mail.gmail.com>
References: <20090329213727.6EEA81E4002@bag.python.org>
	<49D00050.8020300@v.loewis.de>
	<94bdd2610903292125y38856111m43ddca582ee43c7b@mail.gmail.com>
Message-ID: <b654cd2e0903300942x21190fe4ob799386040f93d47@mail.gmail.com>

2009/3/29 Tarek Ziad? <ziade.tarek at gmail.com>:
> RIght I am fixing this right now.
>
> There's something we didn't talk about yet : since there will be other
> package indexes
> out there (not PyPI mirrors) that might have their own mirrors, we do
> need to provide
> somewhere the hostname that holds the mirrors IP for the client software to work
> the same way in all case.
>
> That is, mirrors.pypi.python.org for PyPI, but maybe
> mirrors.packages.plone.org for
> another index.
>
> I'd go for a /mirror-hostname unique page at PyPI (and its mirrors)
> even if this page is unreachable when PyPI is down.
>
> This enforces that the other indexes also use the DNS technique, but I
> think it's fine

Another approach is some way of detecting the mirror index (e.g., a
<link> on the index front page), and strongly suggest that clients
cache that mirror index location.

The most reliable way for a tool like pip to use the mirror, I think,
would be to try the main index always to get metadata, then it could
use a mirror for fetching the actual packages.  There's less
synchronization issues in that case, and only when an index is down
would pip need to fall back entirely on the mirror.  Anyway, this
would fit the mirror index detection pattern.

-- 
Ian Bicking  |  http://blog.ianbicking.org

From martin at v.loewis.de  Mon Mar 30 20:40:38 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 30 Mar 2009 13:40:38 -0500
Subject: [Catalog-sig] Issue with XMLRPC API: server.changelog
In-Reply-To: <49D0C6A4.9080201@zopyx.com>
References: <49D0C6A4.9080201@zopyx.com>
Message-ID: <49D11226.1020100@v.loewis.de>

> Any ideas?

Oops. I upgraded from psycopg to psycopg2, which now
uses datetime, which doesn't have mx' .gmticks anymore.

This is now fixed; please reply.

Regards,
Martin


From lists at zopyx.com  Mon Mar 30 20:55:48 2009
From: lists at zopyx.com (Andreas Jung)
Date: Mon, 30 Mar 2009 13:55:48 -0500
Subject: [Catalog-sig] Issue with XMLRPC API: server.changelog
In-Reply-To: <49D11226.1020100@v.loewis.de>
References: <49D0C6A4.9080201@zopyx.com> <49D11226.1020100@v.loewis.de>
Message-ID: <42d8a3d10903301155o42edbdcbi3ffcd2e88790f9ed@mail.gmail.com>

thanks, it works again.

Andreas

On Mon, Mar 30, 2009 at 13:40, "Martin v. L?wis" <martin at v.loewis.de> wrote:

> Any ideas?
>>
>
> Oops. I upgraded from psycopg to psycopg2, which now
> uses datetime, which doesn't have mx' .gmticks anymore.
>
> This is now fixed; please reply.
>
> Regards,
> Martin
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20090330/491d5f0b/attachment.htm>

From jelleferinga at gmail.com  Tue Mar 31 17:05:41 2009
From: jelleferinga at gmail.com (Jelle Feringa)
Date: Tue, 31 Mar 2009 17:05:41 +0200
Subject: [Catalog-sig] missing category on pypi
Message-ID: <A778BE2B-6DBF-449D-ACD4-6AE4B875CD78@gmail.com>

Hi there,

I have a request for category for pypi categories:

CAD	( Computer Aided Design )
CAE	( Computer Aided Engineering )
KBE	( Knowledge Based Engineering )

These are the categories that define the pythonOCC project, which  
provides wrappers for OpenCASCADE, the sole open source CAD kernel out  
there.

http://pypi.python.org/pypi/pythonOCC/0.1

Thanks so much in advance,

-jelle