From ziade.tarek at gmail.com  Mon Jun  1 00:08:59 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 1 Jun 2009 00:08:59 +0200
Subject: [Distutils] Mutual recursion between setuptools and distutils
	with Python 2.7
In-Reply-To: <loom.20090531T180210-628@post.gmane.org>
References: <loom.20090531T180210-628@post.gmane.org>
Message-ID: <94bdd2610905311508u9edbc2cyf0fee4ae75f50139@mail.gmail.com>

Hello Antoine,

Yes, I have already mentioned the problem to Phillip a few months ago
(in February). Setuptools no longer works with Distutils trunk.

it happens because setuptools build_py implementation has a
__getattr__ that computes data_file:

  def __getattr__(self,attr):
        if attr=='data_files':  # lazily compute data files
            self.data_files = files = self._get_data_files(); return files
        return _build_py.__getattr__(self,attr)

But Distutils sdist command loops over build_py.data_files to add them
in the MANIFEST file,
because this attribute should be computed by finalize_command.

And this is done when sdist.add_default is called, so it loops
recursively and dies

see "for pkg, src_dir, build_dir, filenames in build_py.data_files: in"

http://svn.python.org/view/python/trunk/Lib/distutils/command/sdist.py?r1=68951&r2=69692

I think Setuptools needs to change its strategy when it builds the
file lists to include. Probably by splitting
the work in two distinct phases and remove the getattr hack.

Now the major problem with this is : the last setuptools commit is 7
months old, and right now
people are trying to obtain a new setuptools version for svn 1.6
compatiblity for quite a time now.

So I fear that setuptools might end up broken when 2.7 and 3.1 is out.

I can try to help on that, but I need two things from Phillip:

1/ what he wants to do on setuptools side to fix that problem
2/ blessing me to do it,  and do a release with that change + the
patches that are waiting for some time,
     that he is OK to add.


++
Tarek

-- 
Tarek Ziad? | http://ziade.org

From solipsis at pitrou.net  Mon Jun  1 00:41:50 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 31 May 2009 22:41:50 +0000 (UTC)
Subject: [Distutils] Mutual recursion between setuptools and distutils
	with Python 2.7
References: <loom.20090531T180210-628@post.gmane.org>
	<94bdd2610905311508u9edbc2cyf0fee4ae75f50139@mail.gmail.com>
Message-ID: <loom.20090531T223807-235@post.gmane.org>

Tarek Ziad? <ziade.tarek <at> gmail.com> writes:
> 
> Now the major problem with this is : the last setuptools commit is 7
> months old, and right now
> people are trying to obtain a new setuptools version for svn 1.6
> compatiblity for quite a time now.

I don't have any take in this, but perhaps a solution would be a friendly fork
of setuptools. Apparently, it's not like there would be much competition between
both branches. :)

Not being able to test setuptools-requiring software with 2.7 is rather annoying.

Regards

Antoine.



From ziade.tarek at gmail.com  Mon Jun  1 01:01:48 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 1 Jun 2009 01:01:48 +0200
Subject: [Distutils] Mutual recursion between setuptools and distutils
	with Python 2.7
In-Reply-To: <loom.20090531T223807-235@post.gmane.org>
References: <loom.20090531T180210-628@post.gmane.org>
	<94bdd2610905311508u9edbc2cyf0fee4ae75f50139@mail.gmail.com>
	<loom.20090531T223807-235@post.gmane.org>
Message-ID: <94bdd2610905311601y32277d6bs906861c78c6470f1@mail.gmail.com>

On Mon, Jun 1, 2009 at 12:41 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Tarek Ziad? <ziade.tarek <at> gmail.com> writes:
>>
>> Now the major problem with this is : the last setuptools commit is 7
>> months old, and right now
>> people are trying to obtain a new setuptools version for svn 1.6
>> compatiblity for quite a time now.
>
> I don't have any take in this, but perhaps a solution would be a friendly fork
> of setuptools. Apparently, it's not like there would be much competition between
> both branches. :)

If I fork setuptools to fix those problem, I will have to release it
under another name,
and ask people to change their setup.py file.

This can lead to a messy situation in the Python community imho.

People are already quite disturbed by the fact that we have distutils
*and* setuptools

I think the solution is that Phillip bless someone that have time to
spend on it,
to work on fixing those problems in setuptools.

I have volunteered to maintain distutils, and I am currently pushing
the good bits of setuptools
in Distutils. I can work a bit on setuptools side too since it's
located on the same Subversion.

But I need Phillip insights and blessing for that.

>
> Not being able to test setuptools-requiring software with 2.7 is rather annoying.

Yes, I agree.

Regards
Tarek



-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Mon Jun  1 04:26:06 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 1 Jun 2009 04:26:06 +0200
Subject: [Distutils] PEP 376 up-to-date
In-Reply-To: <20090526035707.03D2F3A404D@sparrow.telecommunity.com>
References: <94bdd2610905250329n21fc744bu52327e2ed0c15440@mail.gmail.com>
	<20090526035707.03D2F3A404D@sparrow.telecommunity.com>
Message-ID: <94bdd2610905311926p1e25cab7qd56072616faecd2b@mail.gmail.com>

I have reworked the APIs,

see http://bitbucket.org/tarek/pep376/src/tip/pkgutil.py

Now there's an EggInfoDirectory collection, that contains EggInfo
instances, and an EggInfoDirectories that contains EggInfoDirectory
instances.

While there's a global instance for the EggInfoDirectories class, it
doesn't load the sys.path implictely like setuptools does with its
master working set. This is done only if one of the global function is used.

One more thing:

2009/5/26 P.J. Eby <pje at telecommunity.com>:
> * Project name normalization and case insensitive comparison is still not
> implemented
>
> * File path normalization (absolutizing, case-normalization,
> de-cross-platforming, etc.) is not implemented

The only thing done here is:

- the egg_info_dirname APIs, which returns the egg-info directory name
of a project.
- the de-cross-platforming and absolutizing done in the EggInfo class
when you use its methods

I can see the need of more APIs in the egg_info and install_egg_info
commands, but I don't
see what else would be required in pkgutil


> (At this point, I have only briefly skimmed the updated PEP; I'll take a
> closer look at it once the API/implementation gets more settled.)

ok then: I didn't update the PEP yet to reflect the code: I'll do it
once we all find the API more settled

Tarek

From jeff at taupro.com  Mon Jun  1 06:49:16 2009
From: jeff at taupro.com (Jeff Rush)
Date: Sun, 31 May 2009 23:49:16 -0500
Subject: [Distutils] Python svn builds
In-Reply-To: <200905312057.18783.a.cavallo@mailsnare.com>
References: <200905312057.18783.a.cavallo@mailsnare.com>
Message-ID: <4A235DCC.4010605@taupro.com>

A. Cavallo wrote:
> Hi all,
> 
> I've just published few python rpms for the trunk source. You can download and 
> install the packages from (either by hand or using yum/yast):
> 
> Module developer can use these build to test against the more up to date 
> python source and anticipating bugs.  Developer can use it to run
> regression/integration tests.

Thanks Antonio.  Having RPMs for the trunk is useful.

It is not well known but you can also build custom versions of Python using
zc.buildout, which can be run with the system version of Python but use a
different Python version inside for any or all of the parts.

My system Python is 2.5 but here is a buildout I use to test stuff with 2.4.

The magic is the python= assignment in the [buildout] section which sets
globally the executable to use for all parts which do not specify their onw
python= in their part definition.

The other magic is the executable= assignment in the part that creates an
instance of the Python interpreter.  It points to the actual executable binary.

Using this you can create one buildout with a handful of Python versions and
run your module test suites against all of them.

--- cut here ---
[buildout]
python = Python-2.4.3

parts =
  Python-2.4.3
  pyprompt-2.4

[Python-2.4.3]
recipe = zc.recipe.cmmi
url =
  http://www.python.org/ftp/python/2.4.3/Python-2.4.3.tgz
extra_options   =
  --prefix ${buildout:parts-directory}/python243
  --enable-unicode=ucs2
executable =
  ${buildout:parts-directory}/python243/bin/python

[pyprompt-2.4]
recipe = zc.recipe.egg:scripts
interpreter = python2.4
eggs =
extra-paths =

# example of setting up yet another prompt
[pyprompt-2.6]
recipe = zc.recipe.egg:scripts
interpreter = python2.6
python = Python-2.6.2
eggs =
extra-paths =
--- cut here ---


From a.cavallo at mailsnare.com  Mon Jun  1 09:24:43 2009
From: a.cavallo at mailsnare.com (A. Cavallo)
Date: Mon, 1 Jun 2009 08:24:43 +0100
Subject: [Distutils] Python svn builds
In-Reply-To: <4A235DCC.4010605@taupro.com>
References: <200905312057.18783.a.cavallo@mailsnare.com>
	<4A235DCC.4010605@taupro.com>
Message-ID: <200906010824.44064.a.cavallo@mailsnare.com>

Hi,
the scope for these svn builds is slightly different.

Firstly they are done on a xen virtual machine: this means they are rebuilt as 
in a "fresh" standard distro install.

Buldbot don't do that, normally, so there's no guarantee libraries present on 
the buildbot host are going to be binary compatible with the one installed on 
the destination machine (yes people do tamper their build systems).


Secondly they're mean to be "easily" deployable: in fact the opensuse build 
server provides the yum/yast hosting as well.

In business time is money but for an opensource/freeware developer time is 
life and, as humble contributor, I don't want to waste my time on playing the 
rebuilding game.
I hope to save this time to other as well making possible for everyone to 
focus more on the important things (in life and development).
(IMHO This is one of the main reasons for the never ending "The year of linux 
on the desktop" joke).


Thirdly they're provided with a (simple) smoke test: it will check if the 
binary provides at least a minimum set of modules.

Fourth they're drop in package and won't interfere with normal system 
administration (a fourth important point in my opinion): they can be installed 
side by side with each other and with the system python.


Fifth they use the latest version of the svn python and they can be rebuild 
for any upcoming revision as well.

Sixth sorry for this very long mail,

Regards,
Antonio 



 
> Thanks Antonio.  Having RPMs for the trunk is useful.
>
> It is not well known but you can also build custom versions of Python using
> zc.buildout, which can be run with the system version of Python but use a
> different Python version inside for any or all of the parts.
>
> My system Python is 2.5 but here is a buildout I use to test stuff with
> 2.4.


From solipsis at pitrou.net  Mon Jun  1 16:37:50 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 1 Jun 2009 14:37:50 +0000 (UTC)
Subject: [Distutils]
	=?utf-8?q?Mutual_recursion_between_setuptools_and_dis?=
	=?utf-8?q?tutils=09with_Python_2=2E7?=
References: <loom.20090531T180210-628@post.gmane.org>
	<94bdd2610905311508u9edbc2cyf0fee4ae75f50139@mail.gmail.com>
	<loom.20090531T223807-235@post.gmane.org>
	<94bdd2610905311601y32277d6bs906861c78c6470f1@mail.gmail.com>
Message-ID: <loom.20090601T143301-881@post.gmane.org>

Tarek Ziad? <ziade.tarek <at> gmail.com> writes:
> 
> If I fork setuptools to fix those problem, I will have to release it
> under another name,
> and ask people to change their setup.py file.

Obviously I'm playing devil's advocate a bit, but a distribution doesn't have to
be named "setuptools" to provide a Python package named "setuptools" ;)

> I think the solution is that Phillip bless someone that have time to
> spend on it,
> to work on fixing those problems in setuptools.

It seems to be the issue here: PJE accepting to delegate maintenance to another
person or group of people. This would be a reasonable attitude.

Otherwise someone *will* have to fork it eventually... There's no third
alternative, too many people are relying on it right now.

Regards

Antoine.



From ziade.tarek at gmail.com  Mon Jun  1 17:39:12 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 1 Jun 2009 17:39:12 +0200
Subject: [Distutils] Mutual recursion between setuptools and distutils
	with Python 2.7
In-Reply-To: <loom.20090601T143301-881@post.gmane.org>
References: <loom.20090531T180210-628@post.gmane.org>
	<94bdd2610905311508u9edbc2cyf0fee4ae75f50139@mail.gmail.com>
	<loom.20090531T223807-235@post.gmane.org>
	<94bdd2610905311601y32277d6bs906861c78c6470f1@mail.gmail.com>
	<loom.20090601T143301-881@post.gmane.org>
Message-ID: <94bdd2610906010839v1671386co78dc834c16c0fa8f@mail.gmail.com>

On Mon, Jun 1, 2009 at 4:37 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Tarek Ziad? <ziade.tarek <at> gmail.com> writes:
>>
>> If I fork setuptools to fix those problem, I will have to release it
>> under another name,
>> and ask people to change their setup.py file.
>
> Obviously I'm playing devil's advocate a bit, but a distribution doesn't have to
> be named "setuptools" to provide a Python package named "setuptools" ;)

Yes, so this will somehow overwrite an existing installation of setuptools

>
>> I think the solution is that Phillip bless someone that have time to
>> spend on it,
>> to work on fixing those problems in setuptools.
>
> It seems to be the issue here: PJE accepting to delegate maintenance to another
> person or group of people. This would be a reasonable attitude.

+1

>
> Otherwise someone *will* have to fork it eventually... There's no third
> alternative, too many people are relying on it right now.
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | http://ziade.org

From pje at telecommunity.com  Mon Jun  1 17:49:41 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 01 Jun 2009 11:49:41 -0400
Subject: [Distutils] Mutual recursion between setuptools and distutils
 with Python 2.7
In-Reply-To: <94bdd2610906010839v1671386co78dc834c16c0fa8f@mail.gmail.co
 m>
References: <loom.20090531T180210-628@post.gmane.org>
	<94bdd2610905311508u9edbc2cyf0fee4ae75f50139@mail.gmail.com>
	<loom.20090531T223807-235@post.gmane.org>
	<94bdd2610905311601y32277d6bs906861c78c6470f1@mail.gmail.com>
	<loom.20090601T143301-881@post.gmane.org>
	<94bdd2610906010839v1671386co78dc834c16c0fa8f@mail.gmail.com>
Message-ID: <20090601154655.ABDC03A40D9@sparrow.telecommunity.com>

At 05:39 PM 6/1/2009 +0200, Tarek Ziad? wrote:
> >
> >> I think the solution is that Phillip bless someone that have time to
> >> spend on it,
> >> to work on fixing those problems in setuptools.
> >
> > It seems to be the issue here: PJE accepting to delegate 
> maintenance to another
> > person or group of people. This would be a reasonable attitude.

Note that I've previously blessed Jim Fulton, although I suspect that 
he doesn't fall in the "have time to spend on it" category.  To date, 
he's the only person who's shown a really deep understanding of the 
codebase and requirements, although Ian Bicking would probably be a 
close second.

This is not to say that there isn't anyone else who fits those 
criteria, just that I haven't had a chance to see that understanding 
in action.  For example, Philip Jenvey has produced some good 
patches, but they've all been specific to Jython issues so it's hard 
to tell how broad his understanding is.


From ziade.tarek at gmail.com  Mon Jun  1 18:09:20 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 1 Jun 2009 18:09:20 +0200
Subject: [Distutils] Mutual recursion between setuptools and distutils
	with Python 2.7
In-Reply-To: <20090601154655.ABDC03A40D9@sparrow.telecommunity.com>
References: <loom.20090531T180210-628@post.gmane.org>
	<94bdd2610905311508u9edbc2cyf0fee4ae75f50139@mail.gmail.com>
	<loom.20090531T223807-235@post.gmane.org>
	<94bdd2610905311601y32277d6bs906861c78c6470f1@mail.gmail.com>
	<loom.20090601T143301-881@post.gmane.org>
	<94bdd2610906010839v1671386co78dc834c16c0fa8f@mail.gmail.com>
	<20090601154655.ABDC03A40D9@sparrow.telecommunity.com>
Message-ID: <94bdd2610906010909xaa387a3p66ed8d1cc57d4423@mail.gmail.com>

2009/6/1 P.J. Eby <pje at telecommunity.com>:
> Note that I've previously blessed Jim Fulton, although I suspect that he
> doesn't fall in the "have time to spend on it" category. ?To date, he's the
> only person who's shown a really deep understanding of the codebase and
> requirements, although Ian Bicking would probably be a close second.
>
> This is not to say that there isn't anyone else who fits those criteria,
> just that I haven't had a chance to see that understanding in action. ?For
> example, Philip Jenvey has produced some good patches, but they've all been
> specific to Jython issues so it's hard to tell how broad his understanding
> is.

This is an awkward situation.

So if  you, Jim or Ian don't have the time to apply patches and do releases,
and if no one else on earth can do that job, based on your criteria,
what is going to happen exactly ?

Are you planning to work in setuptools code yourself anymore or not ?

When Python 2.7 and Python 3.1 are out, setuptools will not work anymore
unless something happen. It's a software on the top on distutils, it
should evolve with it.

Don't take it personally but I am really angry about the current situation,
I feel like we are all trapped in something here.

From pje at telecommunity.com  Mon Jun  1 18:13:32 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 01 Jun 2009 12:13:32 -0400
Subject: [Distutils] Mutual recursion between setuptools and distutils
 with Python 2.7
In-Reply-To: <94bdd2610905311508u9edbc2cyf0fee4ae75f50139@mail.gmail.com
 >
References: <loom.20090531T180210-628@post.gmane.org>
	<94bdd2610905311508u9edbc2cyf0fee4ae75f50139@mail.gmail.com>
Message-ID: <20090601161044.052493A40D9@sparrow.telecommunity.com>

At 12:08 AM 6/1/2009 +0200, Tarek Ziad? wrote:
>1/ what he wants to do on setuptools side to fix that problem

Just as a reminder, I've already stated I have absolutely no clue 
what to do about this, as I don't even really understand the 
problem.  I also don't see why it wouldn't be easier to fix on the 
distutils side, since it's the distutils that changed to create the 
problem in the first place.


From ziade.tarek at gmail.com  Mon Jun  1 18:14:10 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 1 Jun 2009 18:14:10 +0200
Subject: [Distutils] Mutual recursion between setuptools and distutils
	with Python 2.7
In-Reply-To: <20090601161044.052493A40D9@sparrow.telecommunity.com>
References: <loom.20090531T180210-628@post.gmane.org>
	<94bdd2610905311508u9edbc2cyf0fee4ae75f50139@mail.gmail.com>
	<20090601161044.052493A40D9@sparrow.telecommunity.com>
Message-ID: <94bdd2610906010914m79113aabxcd4d264461057bf8@mail.gmail.com>

2009/6/1 P.J. Eby <pje at telecommunity.com>:
> At 12:08 AM 6/1/2009 +0200, Tarek Ziad? wrote:
>>
>> 1/ what he wants to do on setuptools side to fix that problem
>
> Just as a reminder, I've already stated I have absolutely no clue what to do
> about this, as I don't even really understand the problem. ?I also don't see
> why it wouldn't be easier to fix on the distutils side, since it's the
> distutils that changed to create the problem in the first place.
>

Distutils is not on the top of setuptools. It's the contrary.

How do you expect a software to continue to work when its code base
doesn't evolve anymore
while the rest of the world it relies on evolves ?

The last commit is 7 months old.

Setuptools is also broken under Python 3, and this is not due to that change.
Some people have tried to port it but failed.

From ziade.tarek at gmail.com  Mon Jun  1 19:01:13 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 1 Jun 2009 19:01:13 +0200
Subject: [Distutils] Patch for your setuptools issue
Message-ID: <94bdd2610906011001t2a3174dcvdab34693e097dfec@mail.gmail.com>

Antoine,

here's a patch for your issue. It backports distutils old behaviour
into setuptools so the latter continues to work with Python trunk.

Let me know if you want me to create a distribution for your conveniency

Cheers
Tarek

-- 
Tarek Ziad? | http://ziade.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: setuptools.patch
Type: application/octet-stream
Size: 2408 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090601/5e844c22/attachment.obj>

From solipsis at pitrou.net  Mon Jun  1 19:39:51 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 1 Jun 2009 17:39:51 +0000 (UTC)
Subject: [Distutils] Patch for your setuptools issue
References: <94bdd2610906011001t2a3174dcvdab34693e097dfec@mail.gmail.com>
Message-ID: <loom.20090601T173410-743@post.gmane.org>


Hello Tarek,

> here's a patch for your issue. It backports distutils old behaviour
> into setuptools so the latter continues to work with Python trunk.

Thanks for caring. First I got the following error:

[...]
  File "build/bdist.linux-x86_64/egg/setuptools/command/sdist.py", line 171, in
add_defaults
NameError: global name 'string' is not defined


After the obvious fix, installing pure Python packages is fine. It fails to
compile C extensions though, not being able to find the Python header files:

[...]
Running zope.interface-3.4.1/setup.py -q bdist_egg --dist-dir
/home/antoine/tmp/easy_install-UHUYAk/zope.interface-3.4.1/egg-dist-tmp-1FWUyt
src/zope/interface/_zope_interface_coptimizations.c:15:20: erreur: Python.h :
Aucun fichier ou dossier de ce type
src/zope/interface/_zope_interface_coptimizations.c:16:26: erreur:
structmember.h : Aucun fichier ou dossier de ce type


Regards

Antoine.



From ziade.tarek at gmail.com  Mon Jun  1 19:49:03 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 1 Jun 2009 19:49:03 +0200
Subject: [Distutils] Patch for your setuptools issue
In-Reply-To: <loom.20090601T173410-743@post.gmane.org>
References: <94bdd2610906011001t2a3174dcvdab34693e097dfec@mail.gmail.com>
	<loom.20090601T173410-743@post.gmane.org>
Message-ID: <94bdd2610906011049t7645f640ta603c86d73708e42@mail.gmail.com>

On Mon, Jun 1, 2009 at 7:39 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
 [...]
> Running zope.interface-3.4.1/setup.py -q bdist_egg --dist-dir
> /home/antoine/tmp/easy_install-UHUYAk/zope.interface-3.4.1/egg-dist-tmp-1FWUyt
> src/zope/interface/_zope_interface_coptimizations.c:15:20: erreur: Python.h :
> Aucun fichier ou dossier de ce type
> src/zope/interface/_zope_interface_coptimizations.c:16:26: erreur:
> structmember.h : Aucun fichier ou dossier de ce type

How is your Python installed ? It works for me I have installed python
from a trunk checkout
by running the usual ./configure && make && make install

It looks like you don't have the Python include file,

What is your system ?

Tarek
>
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | http://ziade.org

From solipsis at pitrou.net  Mon Jun  1 20:00:54 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 1 Jun 2009 18:00:54 +0000 (UTC)
Subject: [Distutils] Patch for your setuptools issue
References: <94bdd2610906011001t2a3174dcvdab34693e097dfec@mail.gmail.com>
	<loom.20090601T173410-743@post.gmane.org>
	<94bdd2610906011049t7645f640ta603c86d73708e42@mail.gmail.com>
Message-ID: <loom.20090601T175308-101@post.gmane.org>

Tarek Ziad? <ziade.tarek <at> gmail.com> writes:
> 
> How is your Python installed ? It works for me I have installed python
> from a trunk checkout
> by running the usual ./configure && make && make install

Python is installed in an "usr/" subdirectory of its own checkout. 
Now, the strange thing is:
- if I `cd` into usr and then run `bin/easy_install`, C extensions compile fine
- if I run directly `usr/bin/easy_install`, the headers are not found

Perhaps setuptools/distutils is confused by the fact that there is already a
Makefile (or a python executable) in the current directory?

Regards

Antoine.



From ziade.tarek at gmail.com  Mon Jun  1 20:21:26 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 1 Jun 2009 20:21:26 +0200
Subject: [Distutils] Patch for your setuptools issue
In-Reply-To: <loom.20090601T175308-101@post.gmane.org>
References: <94bdd2610906011001t2a3174dcvdab34693e097dfec@mail.gmail.com>
	<loom.20090601T173410-743@post.gmane.org>
	<94bdd2610906011049t7645f640ta603c86d73708e42@mail.gmail.com>
	<loom.20090601T175308-101@post.gmane.org>
Message-ID: <94bdd2610906011121m103d092n83ff60c08e92ef16@mail.gmail.com>

On Mon, Jun 1, 2009 at 8:00 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Tarek Ziad? <ziade.tarek <at> gmail.com> writes:
>>
>> How is your Python installed ? It works for me I have installed python
>> from a trunk checkout
>> by running the usual ./configure && make && make install
>
> Python is installed in an "usr/" subdirectory of its own checkout.
> Now, the strange thing is:
> - if I `cd` into usr and then run `bin/easy_install`, C extensions compile fine
> - if I run directly `usr/bin/easy_install`, the headers are not found
>
> Perhaps setuptools/distutils is confused by the fact that there is already a
> Makefile (or a python executable) in the current directory?

build_ext calls distutils.sysconfig.get_python_inc() wich looks for
Modules/Setup.dist or  Modules/Setup.local in the current directory
to decide if we are trying to build Python itself, to behave slighlty
differently.

So if you are in a python checkout it's  the case. This is not really
robust, but explains why you have that behavior.

I consider the whole  distutils.sysconfig module quite messy, and I
have proposed in python-ideas its removal in favor
of a standalone sysconfig module in the stdlib that would provide
static information on how Python was build
(eg like a Makefile) + a few helpers

Tarek

>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | http://ziade.org

From pje at telecommunity.com  Mon Jun  1 21:08:26 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 01 Jun 2009 15:08:26 -0400
Subject: [Distutils] Patch for your setuptools issue
In-Reply-To: <94bdd2610906011001t2a3174dcvdab34693e097dfec@mail.gmail.co
 m>
References: <94bdd2610906011001t2a3174dcvdab34693e097dfec@mail.gmail.com>
Message-ID: <20090601190539.445353A40D9@sparrow.telecommunity.com>

At 07:01 PM 6/1/2009 +0200, Tarek Ziad? wrote:
>Antoine,
>
>here's a patch for your issue.

Hm.  Did you send this patch before?  I don't recall seeing it.


From setuptools at bugs.python.org  Mon Jun  1 21:08:15 2009
From: setuptools at bugs.python.org (James)
Date: Mon, 01 Jun 2009 19:08:15 +0000
Subject: [Distutils] [issue75] [PATCH]
In-Reply-To: <1243883295.6.0.728643360371.issue75@psf.upfronthosting.co.za>
Message-ID: <1243883295.6.0.728643360371.issue75@psf.upfronthosting.co.za>


New submission from James <purpleidea at gmail.com>:

Hi, I wrote a patch to improve distutils.
I'm a bit new to the whole distutils vs. setuptools, but in any case i've
decided to submit it here as well. I'm not sure if distutils is accepting it,
but whether they do or not, i'll assume it's source compatible, and perhaps
you'd like to use it too.
Attached is the patch, and the original clean.py file from distutils (that i've
modified)
Link to distutils bug report: http://bugs.python.org/issue6142?

----------
files: distutils-clean.py
messages: 297
nosy: purpleidea
priority: feature
status: unread
title: [PATCH]
Added file: http://bugs.python.org/setuptools/file56/distutils-clean.py

_______________________________________________
Setuptools tracker <setuptools at bugs.python.org>
<http://bugs.python.org/setuptools/issue75>
_______________________________________________
-------------- next part --------------
A non-text attachment was scrubbed...
Name: distutils-clean.py
Type: text/x-python
Size: 3448 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090601/1b7070fb/attachment.py>

From ziade.tarek at gmail.com  Mon Jun  1 21:39:07 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 1 Jun 2009 21:39:07 +0200
Subject: [Distutils] Patch for your setuptools issue
In-Reply-To: <20090601190539.445353A40D9@sparrow.telecommunity.com>
References: <94bdd2610906011001t2a3174dcvdab34693e097dfec@mail.gmail.com>
	<20090601190539.445353A40D9@sparrow.telecommunity.com>
Message-ID: <94bdd2610906011239k49785772ybdbd77852b9361c6@mail.gmail.com>

2009/6/1 P.J. Eby <pje at telecommunity.com>:
> At 07:01 PM 6/1/2009 +0200, Tarek Ziad? wrote:
>>
>> Antoine,
>>
>> here's a patch for your issue.
>
> Hm. ?Did you send this patch before? ?I don't recall seeing it.
>

No, I have just created it. I can add it to setuptools issue tracker
if you think it's useful

From pje at telecommunity.com  Tue Jun  2 00:00:58 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 01 Jun 2009 18:00:58 -0400
Subject: [Distutils] Patch for your setuptools issue
In-Reply-To: <94bdd2610906011239k49785772ybdbd77852b9361c6@mail.gmail.co
 m>
References: <94bdd2610906011001t2a3174dcvdab34693e097dfec@mail.gmail.com>
	<20090601190539.445353A40D9@sparrow.telecommunity.com>
	<94bdd2610906011239k49785772ybdbd77852b9361c6@mail.gmail.com>
Message-ID: <20090601215812.1F1023A40D9@sparrow.telecommunity.com>

At 09:39 PM 6/1/2009 +0200, Tarek Ziad? wrote:
>2009/6/1 P.J. Eby <pje at telecommunity.com>:
> > At 07:01 PM 6/1/2009 +0200, Tarek Ziad? wrote:
> >>
> >> Antoine,
> >>
> >> here's a patch for your issue.
> >
> > Hm.  Did you send this patch before?  I don't recall seeing it.
> >
>
>No, I have just created it. I can add it to setuptools issue tracker
>if you think it's useful

The idea is useful; I think there might be another way to do it now 
that you've given me the idea.  I'd prefer to do it in a way that 
doesn't copy code, i.e., by having __getattr__ return something that 
makes the distutils-level behavior the same as before, but not by 
copying the old distutils code.  That having been said, this might be 
a good patch to go in the meantime.

I am thinking, btw, that it's time to retire the 0.6 stable branch 
and begin only doing new stuff on the trunk, anyway.


From setuptools at bugs.python.org  Tue Jun  2 01:36:08 2009
From: setuptools at bugs.python.org (=?utf-8?q?Tarek_Ziad=C3=A9?=)
Date: Mon, 01 Jun 2009 23:36:08 +0000
Subject: [Distutils] [issue76] make setuptools work with current Python trunk
In-Reply-To: <1243899368.39.0.171491928887.issue76@psf.upfronthosting.co.za>
Message-ID: <1243899368.39.0.171491928887.issue76@psf.upfronthosting.co.za>


New submission from Tarek Ziad?? <ziade.tarek at gmail.com>:

This patch backports old distutils behaviour for add_defaults, to avoid a
recursive loop when setuptools builds the file list.

----------
files: setuptools.patch
messages: 300
nosy: tarek
priority: critical
status: unread
title: make setuptools work with current Python trunk
Added file: http://bugs.python.org/setuptools/file59/setuptools.patch

_______________________________________________
Setuptools tracker <setuptools at bugs.python.org>
<http://bugs.python.org/setuptools/issue76>
_______________________________________________
-------------- next part --------------
A non-text attachment was scrubbed...
Name: setuptools.patch
Type: application/octet-stream
Size: 2401 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090601/f1f42ce6/attachment-0001.obj>

From ziade.tarek at gmail.com  Tue Jun  2 01:52:02 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 2 Jun 2009 01:52:02 +0200
Subject: [Distutils] Patch for your setuptools issue
In-Reply-To: <20090601215812.1F1023A40D9@sparrow.telecommunity.com>
References: <94bdd2610906011001t2a3174dcvdab34693e097dfec@mail.gmail.com>
	<20090601190539.445353A40D9@sparrow.telecommunity.com>
	<94bdd2610906011239k49785772ybdbd77852b9361c6@mail.gmail.com>
	<20090601215812.1F1023A40D9@sparrow.telecommunity.com>
Message-ID: <94bdd2610906011652t6d42fbb4p582130cb96be1cac@mail.gmail.com>

On Tue, Jun 2, 2009 at 12:00 AM, P.J. Eby <pje at telecommunity.com> wrote:
> At 09:39 PM 6/1/2009 +0200, Tarek Ziad? wrote:
>>
>> 2009/6/1 P.J. Eby <pje at telecommunity.com>:
>> > At 07:01 PM 6/1/2009 +0200, Tarek Ziad? wrote:
>> >>
>> >> Antoine,
>> >>
>> >> here's a patch for your issue.
>> >
>> > Hm. ?Did you send this patch before? ?I don't recall seeing it.
>> >
>>
>> No, I have just created it. I can add it to setuptools issue tracker
>> if you think it's useful
>
> The idea is useful; I think there might be another way to do it now that
> you've given me the idea. ?I'd prefer to do it in a way that doesn't copy
> code, i.e., by having __getattr__ return something that makes the
> distutils-level behavior the same as before, but not by copying the old
> distutils code. ?That having been said, this might be a good patch to go in
> the meantime.

I have pushed it here :  http://bugs.python.org/setuptools/issue76

the tests pass using previous python interpreters,

Although it fails with the current python trunk,
it seems that unittest.main behaves differently. I am cc'ing  Michael
for some insights because I think
this has been changed lately.


Traceback (most recent call last):
  File "setup.py", line 95, in <module>
    scripts = [],
  File "/Volumes/MacDev/svn.python.org/python-trunk/Lib/distutils/core.py",
line 149, in setup
    dist.run_commands()
  File "/Volumes/MacDev/svn.python.org/python-trunk/Lib/distutils/dist.py",
line 926, in run_commands
    self.run_command(cmd)
  File "/Volumes/MacDev/svn.python.org/python-trunk/Lib/distutils/dist.py",
line 945, in run_command
    cmd_obj.run()
  File "/Volumes/MacDev/svn.python.org/setuptools/setuptools/command/test.py",
line 121, in run
    self.with_project_on_sys_path(self.run_tests)
  File "/Volumes/MacDev/svn.python.org/setuptools/setuptools/command/test.py",
line 101, in with_project_on_sys_path
    func()
  File "/Volumes/MacDev/svn.python.org/setuptools/setuptools/command/test.py",
line 134, in run_tests
    unittest.main(
  File "/Volumes/MacDev/svn.python.org/python-trunk/Lib/unittest.py",
line 1662, in __init__
    self.runTests()
  File "/Volumes/MacDev/svn.python.org/python-trunk/Lib/unittest.py",
line 1744, in runTests
    self.result = testRunner.run(self.test)
AttributeError: 'NoneType' object has no attribute 'run'



>
> I am thinking, btw, that it's time to retire the 0.6 stable branch and begin
> only doing new stuff on the trunk, anyway.

+1

Maybe a final 0.6 release could be done, including the svn 1.6 patch
(or a patch to make the last 0.6.x release not suffering from svn
changes anymore
e.g. by adding a plugin behaviour)

From ziade.tarek at gmail.com  Tue Jun  2 13:35:18 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 2 Jun 2009 13:35:18 +0200
Subject: [Distutils] Patch for your setuptools issue
In-Reply-To: <4A2505E0.6080703@voidspace.org.uk>
References: <94bdd2610906011001t2a3174dcvdab34693e097dfec@mail.gmail.com>
	<20090601190539.445353A40D9@sparrow.telecommunity.com>
	<94bdd2610906011239k49785772ybdbd77852b9361c6@mail.gmail.com>
	<20090601215812.1F1023A40D9@sparrow.telecommunity.com>
	<94bdd2610906011652t6d42fbb4p582130cb96be1cac@mail.gmail.com>
	<4A2505E0.6080703@voidspace.org.uk>
Message-ID: <94bdd2610906020435r5abb04afgac84257d20cb4d06@mail.gmail.com>

On Tue, Jun 2, 2009 at 12:58 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> This failure happens because in line 136(ish) of setuptools.command.test you
> are calling unittest.main passing in testRunner=None.
>
> I don't think this is caused by changes to unittest itself and would have
> failed with the previous versions. Perhaps changes in distutils or
> setuptools?

The test pass under 2.5, 2.6, so there must be something different in
the environment

>
> Happy to dig further into it if needed.

Thanks for the help, I guess we need to dig into it deeper on our side

Cheers

>
> Michael
>
>
>
> Tarek Ziad? wrote:
>>
>> On Tue, Jun 2, 2009 at 12:00 AM, P.J. Eby <pje at telecommunity.com> wrote:
>>
>>>
>>> At 09:39 PM 6/1/2009 +0200, Tarek Ziad? wrote:
>>>
>>>>
>>>> 2009/6/1 P.J. Eby <pje at telecommunity.com>:
>>>>
>>>>>
>>>>> At 07:01 PM 6/1/2009 +0200, Tarek Ziad? wrote:
>>>>>
>>>>>>
>>>>>> Antoine,
>>>>>>
>>>>>> here's a patch for your issue.
>>>>>>
>>>>>
>>>>> Hm. ?Did you send this patch before? ?I don't recall seeing it.
>>>>>
>>>>>
>>>>
>>>> No, I have just created it. I can add it to setuptools issue tracker
>>>> if you think it's useful
>>>>
>>>
>>> The idea is useful; I think there might be another way to do it now that
>>> you've given me the idea. ?I'd prefer to do it in a way that doesn't copy
>>> code, i.e., by having __getattr__ return something that makes the
>>> distutils-level behavior the same as before, but not by copying the old
>>> distutils code. ?That having been said, this might be a good patch to go
>>> in
>>> the meantime.
>>>
>>
>> I have pushed it here : ?http://bugs.python.org/setuptools/issue76
>>
>> the tests pass using previous python interpreters,
>>
>> Although it fails with the current python trunk,
>> it seems that unittest.main behaves differently. I am cc'ing ?Michael
>> for some insights because I think
>> this has been changed lately.
>>
>>
>> Traceback (most recent call last):
>> ?File "setup.py", line 95, in <module>
>> ? ?scripts = [],
>> ?File "/Volumes/MacDev/svn.python.org/python-trunk/Lib/distutils/core.py",
>> line 149, in setup
>> ? ?dist.run_commands()
>> ?File "/Volumes/MacDev/svn.python.org/python-trunk/Lib/distutils/dist.py",
>> line 926, in run_commands
>> ? ?self.run_command(cmd)
>> ?File "/Volumes/MacDev/svn.python.org/python-trunk/Lib/distutils/dist.py",
>> line 945, in run_command
>> ? ?cmd_obj.run()
>> ?File
>> "/Volumes/MacDev/svn.python.org/setuptools/setuptools/command/test.py",
>> line 121, in run
>> ? ?self.with_project_on_sys_path(self.run_tests)
>> ?File
>> "/Volumes/MacDev/svn.python.org/setuptools/setuptools/command/test.py",
>> line 101, in with_project_on_sys_path
>> ? ?func()
>> ?File
>> "/Volumes/MacDev/svn.python.org/setuptools/setuptools/command/test.py",
>> line 134, in run_tests
>> ? ?unittest.main(
>> ?File "/Volumes/MacDev/svn.python.org/python-trunk/Lib/unittest.py",
>> line 1662, in __init__
>> ? ?self.runTests()
>> ?File "/Volumes/MacDev/svn.python.org/python-trunk/Lib/unittest.py",
>> line 1744, in runTests
>> ? ?self.result = testRunner.run(self.test)
>> AttributeError: 'NoneType' object has no attribute 'run'
>>
>>
>>
>>
>>>
>>> I am thinking, btw, that it's time to retire the 0.6 stable branch and
>>> begin
>>> only doing new stuff on the trunk, anyway.
>>>
>>
>> +1
>>
>> Maybe a final 0.6 release could be done, including the svn 1.6 patch
>> (or a patch to make the last 0.6.x release not suffering from svn
>> changes anymore
>> e.g. by adding a plugin behaviour)
>>
>
>
> --
> http://www.ironpythoninaction.com/
>
>



-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Tue Jun  2 14:11:49 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 2 Jun 2009 14:11:49 +0200
Subject: [Distutils] Patch for your setuptools issue
In-Reply-To: <4A2515A9.9020502@voidspace.org.uk>
References: <94bdd2610906011001t2a3174dcvdab34693e097dfec@mail.gmail.com>
	<20090601190539.445353A40D9@sparrow.telecommunity.com>
	<94bdd2610906011239k49785772ybdbd77852b9361c6@mail.gmail.com>
	<20090601215812.1F1023A40D9@sparrow.telecommunity.com>
	<94bdd2610906011652t6d42fbb4p582130cb96be1cac@mail.gmail.com>
	<4A2505E0.6080703@voidspace.org.uk>
	<94bdd2610906020435r5abb04afgac84257d20cb4d06@mail.gmail.com>
	<4A2515A9.9020502@voidspace.org.uk>
Message-ID: <94bdd2610906020511l4b31022dj34eefc828a8126d8@mail.gmail.com>

On Tue, Jun 2, 2009 at 2:06 PM, Michael Foord <fuzzyman at voidspace.org.uk> wrote:

> Ah - I see the problem. In Python 2.5 testRunner defaulted to None and main
> created its own TextTestRunner. In 2.6 the default was changed to be the
> TextTestRunner class. Unfortunately 2.5 needs either None or an instance of
> a runner.
>
> The easiest thing to do might be to create a dictionary of keyword arguments
> and omit testRunner if it is None. Call main with unittest.main(**args).

Yeah that shall do the trick, thanks

From ziade.tarek at gmail.com  Tue Jun  2 14:26:06 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 2 Jun 2009 14:26:06 +0200
Subject: [Distutils] Patch for your setuptools issue
In-Reply-To: <4A25199E.8050402@voidspace.org.uk>
References: <94bdd2610906011001t2a3174dcvdab34693e097dfec@mail.gmail.com>
	<20090601190539.445353A40D9@sparrow.telecommunity.com>
	<94bdd2610906011239k49785772ybdbd77852b9361c6@mail.gmail.com>
	<20090601215812.1F1023A40D9@sparrow.telecommunity.com>
	<94bdd2610906011652t6d42fbb4p582130cb96be1cac@mail.gmail.com>
	<4A2505E0.6080703@voidspace.org.uk>
	<94bdd2610906020435r5abb04afgac84257d20cb4d06@mail.gmail.com>
	<4A2515A9.9020502@voidspace.org.uk>
	<94bdd2610906020511l4b31022dj34eefc828a8126d8@mail.gmail.com>
	<4A25199E.8050402@voidspace.org.uk>
Message-ID: <94bdd2610906020526h2f44e2e0i4614540d71b83a26@mail.gmail.com>

On Tue, Jun 2, 2009 at 2:22 PM, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> Tarek Ziad? wrote:
>>
>> On Tue, Jun 2, 2009 at 2:06 PM, Michael Foord <fuzzyman at voidspace.org.uk>
>> wrote:
>>
>>
>>>
>>> Ah - I see the problem. In Python 2.5 testRunner defaulted to None and
>>> main
>>> created its own TextTestRunner. In 2.6 the default was changed to be the
>>> TextTestRunner class. Unfortunately 2.5 needs either None or an instance
>>> of
>>> a runner.
>>>
>>> The easiest thing to do might be to create a dictionary of keyword
>>> arguments
>>> and omit testRunner if it is None. Call main with unittest.main(**args).
>>>
>>
>> Yeah that shall do the trick, thanks
>>
>
> It's a bad change in unittest (not me thankfully ;-).
>
> I'll see if I can get it switched back in 2.7 (replacing testRunner with the
> class if it is passed in as None).

In that case, backport it to the next 2.6 maintenance release

here's the patch for setuptools

Cheers


>
> All the best,
>
> Michael
>
> --
> http://www.ironpythoninaction.com/
>
>



-- 
Tarek Ziad? | http://ziade.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fixtest.patch
Type: application/octet-stream
Size: 901 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090602/9373a47f/attachment.obj>

From setuptools at bugs.python.org  Tue Jun  2 14:27:36 2009
From: setuptools at bugs.python.org (=?utf-8?q?Tarek_Ziad=C3=A9?=)
Date: Tue, 02 Jun 2009 12:27:36 +0000
Subject: [Distutils] [issue77] fix tests for python > 2.5
In-Reply-To: <1243945656.06.0.4878092207.issue77@psf.upfronthosting.co.za>
Message-ID: <1243945656.06.0.4878092207.issue77@psf.upfronthosting.co.za>


New submission from Tarek Ziad? <ziade.tarek at gmail.com>:

makes the tests work for python > 2.5

----------
messages: 301
nosy: tarek
priority: bug
status: unread
title: fix tests for python > 2.5

_______________________________________________
Setuptools tracker <setuptools at bugs.python.org>
<http://bugs.python.org/setuptools/issue77>
_______________________________________________

From david.lyon at preisshare.net  Tue Jun  2 14:38:00 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 02 Jun 2009 08:38:00 -0400
Subject: [Distutils] Distutils functionality questions/clarifications...
In-Reply-To: <1243945656.06.0.4878092207.issue77@psf.upfronthosting.co.za>
References: <1243945656.06.0.4878092207.issue77@psf.upfronthosting.co.za>
Message-ID: <0e52434b69b476e786ab27cf8c64adf0@preisshare.net>

Hi All,

After being given the following page...:

 http://docs.python.org/distutils/builtdist.html

I have some comments... distutils looks really good...

from the developer side....

Just some minor questions...

 1) Debian+ubuntu .deb support? working? not-working? Status?

 2) Mac Support? Working? same - how well? documentation is vague

 3) windows installers for packages - can we kill them? or
    how do we move to something better...

I'm wondering what people think about having a gui interface
to drive all this technical wizardry....

When I finish my current workload... I'm wondering what could
be done to make things even better than they currently are
on this front.

David




From ziade.tarek at gmail.com  Tue Jun  2 14:50:35 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 2 Jun 2009 14:50:35 +0200
Subject: [Distutils] Distutils functionality questions/clarifications...
In-Reply-To: <0e52434b69b476e786ab27cf8c64adf0@preisshare.net>
References: <1243945656.06.0.4878092207.issue77@psf.upfronthosting.co.za>
	<0e52434b69b476e786ab27cf8c64adf0@preisshare.net>
Message-ID: <94bdd2610906020550y127ea1cdw7cc2a0954df86ed7@mail.gmail.com>

On Tue, Jun 2, 2009 at 2:38 PM, David Lyon <david.lyon at preisshare.net> wrote:
>
> ?1) Debian+ubuntu .deb support? working? not-working? Status?

The stdeb project provides a layer on the top of distutils for example
to achieve this.

>
> ?2) Mac Support? Working? same - how well? documentation is vague

What kind of mac support you are reffering to ? If you build
something, it will work fine under mac

> ?3) windows installers for packages - can we kill them? or
> ? ?how do we move to something better...

What do you mean by "killing them" ?

>
> I'm wondering what people think about having a gui interface
> to drive all this technical wizardry....

What would your GUI do precisely ?

distutils can probably provide the basis for a third-party application
that wants to provide
some GUI to drive some process.

>
> When I finish my current workload... I'm wondering what could
> be done to make things even better than they currently are
> on this front.

On the packaging side ? tons of things really , welcome aboard ! ;)

>
> David
>
>
>
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | http://ziade.org

From fuzzyman at voidspace.org.uk  Tue Jun  2 12:58:40 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 02 Jun 2009 11:58:40 +0100
Subject: [Distutils] Patch for your setuptools issue
In-Reply-To: <94bdd2610906011652t6d42fbb4p582130cb96be1cac@mail.gmail.com>
References: <94bdd2610906011001t2a3174dcvdab34693e097dfec@mail.gmail.com>	
	<20090601190539.445353A40D9@sparrow.telecommunity.com>	
	<94bdd2610906011239k49785772ybdbd77852b9361c6@mail.gmail.com>	
	<20090601215812.1F1023A40D9@sparrow.telecommunity.com>
	<94bdd2610906011652t6d42fbb4p582130cb96be1cac@mail.gmail.com>
Message-ID: <4A2505E0.6080703@voidspace.org.uk>

This failure happens because in line 136(ish) of setuptools.command.test 
you are calling unittest.main passing in testRunner=None.

I don't think this is caused by changes to unittest itself and would 
have failed with the previous versions. Perhaps changes in distutils or 
setuptools?

Happy to dig further into it if needed.

Michael



Tarek Ziad? wrote:
> On Tue, Jun 2, 2009 at 12:00 AM, P.J. Eby <pje at telecommunity.com> wrote:
>   
>> At 09:39 PM 6/1/2009 +0200, Tarek Ziad? wrote:
>>     
>>> 2009/6/1 P.J. Eby <pje at telecommunity.com>:
>>>       
>>>> At 07:01 PM 6/1/2009 +0200, Tarek Ziad? wrote:
>>>>         
>>>>> Antoine,
>>>>>
>>>>> here's a patch for your issue.
>>>>>           
>>>> Hm.  Did you send this patch before?  I don't recall seeing it.
>>>>
>>>>         
>>> No, I have just created it. I can add it to setuptools issue tracker
>>> if you think it's useful
>>>       
>> The idea is useful; I think there might be another way to do it now that
>> you've given me the idea.  I'd prefer to do it in a way that doesn't copy
>> code, i.e., by having __getattr__ return something that makes the
>> distutils-level behavior the same as before, but not by copying the old
>> distutils code.  That having been said, this might be a good patch to go in
>> the meantime.
>>     
>
> I have pushed it here :  http://bugs.python.org/setuptools/issue76
>
> the tests pass using previous python interpreters,
>
> Although it fails with the current python trunk,
> it seems that unittest.main behaves differently. I am cc'ing  Michael
> for some insights because I think
> this has been changed lately.
>
>
> Traceback (most recent call last):
>   File "setup.py", line 95, in <module>
>     scripts = [],
>   File "/Volumes/MacDev/svn.python.org/python-trunk/Lib/distutils/core.py",
> line 149, in setup
>     dist.run_commands()
>   File "/Volumes/MacDev/svn.python.org/python-trunk/Lib/distutils/dist.py",
> line 926, in run_commands
>     self.run_command(cmd)
>   File "/Volumes/MacDev/svn.python.org/python-trunk/Lib/distutils/dist.py",
> line 945, in run_command
>     cmd_obj.run()
>   File "/Volumes/MacDev/svn.python.org/setuptools/setuptools/command/test.py",
> line 121, in run
>     self.with_project_on_sys_path(self.run_tests)
>   File "/Volumes/MacDev/svn.python.org/setuptools/setuptools/command/test.py",
> line 101, in with_project_on_sys_path
>     func()
>   File "/Volumes/MacDev/svn.python.org/setuptools/setuptools/command/test.py",
> line 134, in run_tests
>     unittest.main(
>   File "/Volumes/MacDev/svn.python.org/python-trunk/Lib/unittest.py",
> line 1662, in __init__
>     self.runTests()
>   File "/Volumes/MacDev/svn.python.org/python-trunk/Lib/unittest.py",
> line 1744, in runTests
>     self.result = testRunner.run(self.test)
> AttributeError: 'NoneType' object has no attribute 'run'
>
>
>
>   
>> I am thinking, btw, that it's time to retire the 0.6 stable branch and begin
>> only doing new stuff on the trunk, anyway.
>>     
>
> +1
>
> Maybe a final 0.6 release could be done, including the svn 1.6 patch
> (or a patch to make the last 0.6.x release not suffering from svn
> changes anymore
> e.g. by adding a plugin behaviour)
>   


-- 
http://www.ironpythoninaction.com/


From fuzzyman at voidspace.org.uk  Tue Jun  2 14:06:01 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 02 Jun 2009 13:06:01 +0100
Subject: [Distutils] Patch for your setuptools issue
In-Reply-To: <94bdd2610906020435r5abb04afgac84257d20cb4d06@mail.gmail.com>
References: <94bdd2610906011001t2a3174dcvdab34693e097dfec@mail.gmail.com>	
	<20090601190539.445353A40D9@sparrow.telecommunity.com>	
	<94bdd2610906011239k49785772ybdbd77852b9361c6@mail.gmail.com>	
	<20090601215812.1F1023A40D9@sparrow.telecommunity.com>	
	<94bdd2610906011652t6d42fbb4p582130cb96be1cac@mail.gmail.com>	
	<4A2505E0.6080703@voidspace.org.uk>
	<94bdd2610906020435r5abb04afgac84257d20cb4d06@mail.gmail.com>
Message-ID: <4A2515A9.9020502@voidspace.org.uk>

Tarek Ziad? wrote:
> On Tue, Jun 2, 2009 at 12:58 PM, Michael Foord
> <fuzzyman at voidspace.org.uk> wrote:
>   
>> This failure happens because in line 136(ish) of setuptools.command.test you
>> are calling unittest.main passing in testRunner=None.
>>
>> I don't think this is caused by changes to unittest itself and would have
>> failed with the previous versions. Perhaps changes in distutils or
>> setuptools?
>>     
>
> The test pass under 2.5, 2.6, so there must be something different in
> the environment
>   

If I run it with Python 2.6 I get the same problem ('python setup.py test').

It works fine under 2.5 for me though and testRunner is still None.

Ah - I see the problem. In Python 2.5 testRunner defaulted to None and 
main created its own TextTestRunner. In 2.6 the default was changed to 
be the TextTestRunner class. Unfortunately 2.5 needs either None or an 
instance of a runner.

The easiest thing to do might be to create a dictionary of keyword 
arguments and omit testRunner if it is None. Call main with 
unittest.main(**args).

Michael



>   
>> Happy to dig further into it if needed.
>>     
>
> Thanks for the help, I guess we need to dig into it deeper on our side
>
> Cheers
>
>   
>> Michael
>>
>>
>>
>> Tarek Ziad? wrote:
>>     
>>> On Tue, Jun 2, 2009 at 12:00 AM, P.J. Eby <pje at telecommunity.com> wrote:
>>>
>>>       
>>>> At 09:39 PM 6/1/2009 +0200, Tarek Ziad? wrote:
>>>>
>>>>         
>>>>> 2009/6/1 P.J. Eby <pje at telecommunity.com>:
>>>>>
>>>>>           
>>>>>> At 07:01 PM 6/1/2009 +0200, Tarek Ziad? wrote:
>>>>>>
>>>>>>             
>>>>>>> Antoine,
>>>>>>>
>>>>>>> here's a patch for your issue.
>>>>>>>
>>>>>>>               
>>>>>> Hm.  Did you send this patch before?  I don't recall seeing it.
>>>>>>
>>>>>>
>>>>>>             
>>>>> No, I have just created it. I can add it to setuptools issue tracker
>>>>> if you think it's useful
>>>>>
>>>>>           
>>>> The idea is useful; I think there might be another way to do it now that
>>>> you've given me the idea.  I'd prefer to do it in a way that doesn't copy
>>>> code, i.e., by having __getattr__ return something that makes the
>>>> distutils-level behavior the same as before, but not by copying the old
>>>> distutils code.  That having been said, this might be a good patch to go
>>>> in
>>>> the meantime.
>>>>
>>>>         
>>> I have pushed it here :  http://bugs.python.org/setuptools/issue76
>>>
>>> the tests pass using previous python interpreters,
>>>
>>> Although it fails with the current python trunk,
>>> it seems that unittest.main behaves differently. I am cc'ing  Michael
>>> for some insights because I think
>>> this has been changed lately.
>>>
>>>
>>> Traceback (most recent call last):
>>>  File "setup.py", line 95, in <module>
>>>    scripts = [],
>>>  File "/Volumes/MacDev/svn.python.org/python-trunk/Lib/distutils/core.py",
>>> line 149, in setup
>>>    dist.run_commands()
>>>  File "/Volumes/MacDev/svn.python.org/python-trunk/Lib/distutils/dist.py",
>>> line 926, in run_commands
>>>    self.run_command(cmd)
>>>  File "/Volumes/MacDev/svn.python.org/python-trunk/Lib/distutils/dist.py",
>>> line 945, in run_command
>>>    cmd_obj.run()
>>>  File
>>> "/Volumes/MacDev/svn.python.org/setuptools/setuptools/command/test.py",
>>> line 121, in run
>>>    self.with_project_on_sys_path(self.run_tests)
>>>  File
>>> "/Volumes/MacDev/svn.python.org/setuptools/setuptools/command/test.py",
>>> line 101, in with_project_on_sys_path
>>>    func()
>>>  File
>>> "/Volumes/MacDev/svn.python.org/setuptools/setuptools/command/test.py",
>>> line 134, in run_tests
>>>    unittest.main(
>>>  File "/Volumes/MacDev/svn.python.org/python-trunk/Lib/unittest.py",
>>> line 1662, in __init__
>>>    self.runTests()
>>>  File "/Volumes/MacDev/svn.python.org/python-trunk/Lib/unittest.py",
>>> line 1744, in runTests
>>>    self.result = testRunner.run(self.test)
>>> AttributeError: 'NoneType' object has no attribute 'run'
>>>
>>>
>>>
>>>
>>>       
>>>> I am thinking, btw, that it's time to retire the 0.6 stable branch and
>>>> begin
>>>> only doing new stuff on the trunk, anyway.
>>>>
>>>>         
>>> +1
>>>
>>> Maybe a final 0.6 release could be done, including the svn 1.6 patch
>>> (or a patch to make the last 0.6.x release not suffering from svn
>>> changes anymore
>>> e.g. by adding a plugin behaviour)
>>>
>>>       
>> --
>> http://www.ironpythoninaction.com/
>>
>>
>>     
>
>
>
>   


-- 
http://www.ironpythoninaction.com/


From fuzzyman at voidspace.org.uk  Tue Jun  2 14:22:54 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 02 Jun 2009 13:22:54 +0100
Subject: [Distutils] Patch for your setuptools issue
In-Reply-To: <94bdd2610906020511l4b31022dj34eefc828a8126d8@mail.gmail.com>
References: <94bdd2610906011001t2a3174dcvdab34693e097dfec@mail.gmail.com>	
	<20090601190539.445353A40D9@sparrow.telecommunity.com>	
	<94bdd2610906011239k49785772ybdbd77852b9361c6@mail.gmail.com>	
	<20090601215812.1F1023A40D9@sparrow.telecommunity.com>	
	<94bdd2610906011652t6d42fbb4p582130cb96be1cac@mail.gmail.com>	
	<4A2505E0.6080703@voidspace.org.uk>	
	<94bdd2610906020435r5abb04afgac84257d20cb4d06@mail.gmail.com>	
	<4A2515A9.9020502@voidspace.org.uk>
	<94bdd2610906020511l4b31022dj34eefc828a8126d8@mail.gmail.com>
Message-ID: <4A25199E.8050402@voidspace.org.uk>

Tarek Ziad? wrote:
> On Tue, Jun 2, 2009 at 2:06 PM, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
>
>   
>> Ah - I see the problem. In Python 2.5 testRunner defaulted to None and main
>> created its own TextTestRunner. In 2.6 the default was changed to be the
>> TextTestRunner class. Unfortunately 2.5 needs either None or an instance of
>> a runner.
>>
>> The easiest thing to do might be to create a dictionary of keyword arguments
>> and omit testRunner if it is None. Call main with unittest.main(**args).
>>     
>
> Yeah that shall do the trick, thanks
>   
It's a bad change in unittest (not me thankfully ;-).

I'll see if I can get it switched back in 2.7 (replacing testRunner with 
the class if it is passed in as None).

All the best,

Michael

-- 
http://www.ironpythoninaction.com/


From david.lyon at preisshare.net  Tue Jun  2 16:32:37 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 02 Jun 2009 10:32:37 -0400
Subject: [Distutils] Distutils functionality questions/clarifications...
In-Reply-To: <94bdd2610906020550y127ea1cdw7cc2a0954df86ed7@mail.gmail.com>
References: <1243945656.06.0.4878092207.issue77@psf.upfronthosting.co.za>
	<0e52434b69b476e786ab27cf8c64adf0@preisshare.net>
	<94bdd2610906020550y127ea1cdw7cc2a0954df86ed7@mail.gmail.com>
Message-ID: <f3b1e9dea91b833c7d9a93474d73dced@preisshare.net>

On Tue, 2 Jun 2009 14:50:35 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
wrote:
>> ?1) Debian+ubuntu .deb support? working? not-working? Status?
> 
> The stdeb project provides a layer on the top of distutils for example
> to achieve this.

Oh ok - well the documentation should really lead people there.. imho

>> ?2) Mac Support? Working? same - how well? documentation is vague
> 
> What kind of mac support you are reffering to ? If you build
> something, it will work fine under mac

Well...

update the documentation.. the table near the top needs to
reflect that.

snip of ... http://docs.python.org/distutils/builtdist.html
-----
The available formats for built distributions are:
Format 	Description 	Notes
gztar 	gzipped tar file (.tar.gz) 	(1),(3)
ztar 	compressed tar file (.tar.Z) 	(3)
tar 	tar file (.tar) 	(3)
zip 	zip file (.zip) 	(4)
rpm 	RPM 	(5)
pkgtool 	Solaris pkgtool 	 
sdux 	HP-UX swinstall 	 
rpm 	RPM 	(5)
wininst 	self-extracting ZIP file for Windows 	(2),(4)
-----

>> ?3) windows installers for packages - can we kill them? or
>> ? ?how do we move to something better...
> 
> What do you mean by "killing them" ?

Remove from circulation.

>> I'm wondering what people think about having a gui interface
>> to drive all this technical wizardry....
> 
> What would your GUI do precisely ?

Allow a novice to take a python "application" or "module" and
run it through all those tools and end up with the desired
built targets....

The kinds of things that need to be managed are:

 - versioning
 - provision for documentation
 - compiling p2exe,py2app
 - windows installers
 
I'm not having too many new ideas because it appears these things
are already implemented...

But in a simple example, a user has a (say) windows pc, wants to build
for mac and linux... has no idea... clicks the appropriate
check boxes... and it all "gets done"...

Perphaps even uploading the lot to their website or pypi...
 
> distutils can probably provide the basis for a third-party application
> that wants to provide
> some GUI to drive some process.

hmmm... it's an interesting thought...

> On the packaging side ? tons of things really

Yes, well I am very close to releasing a Package Manager GUI so
I'm kindof concentrating on that first before I go take on any
new commitments...

David



From p.f.moore at gmail.com  Tue Jun  2 16:58:33 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 2 Jun 2009 15:58:33 +0100
Subject: [Distutils] Distutils functionality questions/clarifications...
In-Reply-To: <f3b1e9dea91b833c7d9a93474d73dced@preisshare.net>
References: <1243945656.06.0.4878092207.issue77@psf.upfronthosting.co.za>
	<0e52434b69b476e786ab27cf8c64adf0@preisshare.net>
	<94bdd2610906020550y127ea1cdw7cc2a0954df86ed7@mail.gmail.com>
	<f3b1e9dea91b833c7d9a93474d73dced@preisshare.net>
Message-ID: <79990c6b0906020758g186923h6028f79579736c1c@mail.gmail.com>

2009/6/2 David Lyon <david.lyon at preisshare.net>:
>>> ?3) windows installers for packages - can we kill them? or
>>> ? ?how do we move to something better...
>>
>> What do you mean by "killing them" ?
>
> Remove from circulation.

Seems a bit drastic, given that very many people rely on them... What
do you propose to replace them with?

Paul.

From nathan at creativecommons.org  Tue Jun  2 17:51:26 2009
From: nathan at creativecommons.org (Nathan Yergler)
Date: Tue, 2 Jun 2009 08:51:26 -0700
Subject: [Distutils] Distutils functionality questions/clarifications...
In-Reply-To: <f3b1e9dea91b833c7d9a93474d73dced@preisshare.net>
References: <1243945656.06.0.4878092207.issue77@psf.upfronthosting.co.za> 
	<0e52434b69b476e786ab27cf8c64adf0@preisshare.net>
	<94bdd2610906020550y127ea1cdw7cc2a0954df86ed7@mail.gmail.com> 
	<f3b1e9dea91b833c7d9a93474d73dced@preisshare.net>
Message-ID: <c1a864630906020851u4046c0c4kb7cd149fe275344a@mail.gmail.com>

On Tue, Jun 2, 2009 at 7:32 AM, David Lyon <david.lyon at preisshare.net> wrote:
> On Tue, 2 Jun 2009 14:50:35 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
> wrote:
>>> ?1) Debian+ubuntu .deb support? working? not-working? Status?
>>
>> The stdeb project provides a layer on the top of distutils for example
>> to achieve this.
>
> Oh ok - well the documentation should really lead people there.. imho
>
>>> ?2) Mac Support? Working? same - how well? documentation is vague
>>
>> What kind of mac support you are reffering to ? If you build
>> something, it will work fine under mac
>
> Well...
>
> update the documentation.. the table near the top needs to
> reflect that.

I think Tarek is just pointing out that you can build a dist and it'll
work on Mac OS X; AFAIK there's no explicit target for it (ala wininst
or RPM).

>
> snip of ... http://docs.python.org/distutils/builtdist.html
> -----
> The available formats for built distributions are:
> Format ?Description ? ? Notes
> gztar ? gzipped tar file (.tar.gz) ? ? ?(1),(3)
> ztar ? ?compressed tar file (.tar.Z) ? ?(3)
> tar ? ? tar file (.tar) ? ? ? ? (3)
> zip ? ? zip file (.zip) ? ? ? ? (4)
> rpm ? ? RPM ? ? (5)
> pkgtool ? ? ? ? Solaris pkgtool
> sdux ? ?HP-UX swinstall
> rpm ? ? RPM ? ? (5)
> wininst ? ? ? ? self-extracting ZIP file for Windows ? ?(2),(4)
> -----
>
>>> ?3) windows installers for packages - can we kill them? or
>>> ? ?how do we move to something better...
>>
>> What do you mean by "killing them" ?
>
> Remove from circulation.
>
>>> I'm wondering what people think about having a gui interface
>>> to drive all this technical wizardry....
>>
>> What would your GUI do precisely ?
>
> Allow a novice to take a python "application" or "module" and
> run it through all those tools and end up with the desired
> built targets....
>
> The kinds of things that need to be managed are:
>
> ?- versioning
> ?- provision for documentation
> ?- compiling p2exe,py2app
> ?- windows installers
>
> I'm not having too many new ideas because it appears these things
> are already implemented...
>
> But in a simple example, a user has a (say) windows pc, wants to build
> for mac and linux... has no idea... clicks the appropriate
> check boxes... and it all "gets done"...
>
> Perphaps even uploading the lot to their website or pypi...
>
>> distutils can probably provide the basis for a third-party application
>> that wants to provide
>> some GUI to drive some process.
>
> hmmm... it's an interesting thought...
>
>> On the packaging side ? tons of things really
>
> Yes, well I am very close to releasing a Package Manager GUI so
> I'm kindof concentrating on that first before I go take on any
> new commitments...
>
> David
>
>
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>

From santagada at gmail.com  Tue Jun  2 20:11:04 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Tue, 2 Jun 2009 15:11:04 -0300
Subject: [Distutils] Distutils functionality questions/clarifications...
In-Reply-To: <c1a864630906020851u4046c0c4kb7cd149fe275344a@mail.gmail.com>
References: <1243945656.06.0.4878092207.issue77@psf.upfronthosting.co.za>
	<0e52434b69b476e786ab27cf8c64adf0@preisshare.net>
	<94bdd2610906020550y127ea1cdw7cc2a0954df86ed7@mail.gmail.com>
	<f3b1e9dea91b833c7d9a93474d73dced@preisshare.net>
	<c1a864630906020851u4046c0c4kb7cd149fe275344a@mail.gmail.com>
Message-ID: <76056F55-B4C8-47CE-A5C2-24E36D40C6C2@gmail.com>


On Jun 2, 2009, at 12:51 PM, Nathan Yergler wrote:

> On Tue, Jun 2, 2009 at 7:32 AM, David Lyon  
> <david.lyon at preisshare.net> wrote:
>> On Tue, 2 Jun 2009 14:50:35 +0200, Tarek Ziad?  
>> <ziade.tarek at gmail.com>
>> wrote:
>>>>  1) Debian+ubuntu .deb support? working? not-working? Status?
>>>
>>> The stdeb project provides a layer on the top of distutils for  
>>> example
>>> to achieve this.
>>
>> Oh ok - well the documentation should really lead people there.. imho
>>
>>>>  2) Mac Support? Working? same - how well? documentation is vague
>>>
>>> What kind of mac support you are reffering to ? If you build
>>> something, it will work fine under mac
>>
>> Well...
>>
>> update the documentation.. the table near the top needs to
>> reflect that.
>
> I think Tarek is just pointing out that you can build a dist and it'll
> work on Mac OS X; AFAIK there's no explicit target for it (ala wininst
> or RPM).

There is makepkg or something, it is just a third party module like  
stdeb no?

--
Leonardo Santagada
santagada at gmail.com




From stef.mientki at gmail.com  Tue Jun  2 20:33:16 2009
From: stef.mientki at gmail.com (Stef Mientki)
Date: Tue, 02 Jun 2009 20:33:16 +0200
Subject: [Distutils] Distutils functionality questions/clarifications...
In-Reply-To: <0e52434b69b476e786ab27cf8c64adf0@preisshare.net>
References: <1243945656.06.0.4878092207.issue77@psf.upfronthosting.co.za>
	<0e52434b69b476e786ab27cf8c64adf0@preisshare.net>
Message-ID: <4A25706C.1070404@gmail.com>

are you familiar with gui2exe:
http://code.google.com/p/gui2exe/

cheers,
Stef

David Lyon wrote:
> Hi All,
>
> After being given the following page...:
>
>  http://docs.python.org/distutils/builtdist.html
>
> I have some comments... distutils looks really good...
>
> from the developer side....
>
> Just some minor questions...
>
>  1) Debian+ubuntu .deb support? working? not-working? Status?
>
>  2) Mac Support? Working? same - how well? documentation is vague
>
>  3) windows installers for packages - can we kill them? or
>     how do we move to something better...
>
> I'm wondering what people think about having a gui interface
> to drive all this technical wizardry....
>
> When I finish my current workload... I'm wondering what could
> be done to make things even better than they currently are
> on this front.
>
> David
>
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>   


From ronaldoussoren at mac.com  Tue Jun  2 21:34:40 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 2 Jun 2009 21:34:40 +0200
Subject: [Distutils] Distutils functionality questions/clarifications...
In-Reply-To: <76056F55-B4C8-47CE-A5C2-24E36D40C6C2@gmail.com>
References: <1243945656.06.0.4878092207.issue77@psf.upfronthosting.co.za>
	<0e52434b69b476e786ab27cf8c64adf0@preisshare.net>
	<94bdd2610906020550y127ea1cdw7cc2a0954df86ed7@mail.gmail.com>
	<f3b1e9dea91b833c7d9a93474d73dced@preisshare.net>
	<c1a864630906020851u4046c0c4kb7cd149fe275344a@mail.gmail.com>
	<76056F55-B4C8-47CE-A5C2-24E36D40C6C2@gmail.com>
Message-ID: <B686E989-B056-4486-A2A7-97ED2F9B1DF5@mac.com>


On 2 Jun, 2009, at 20:11, Leonardo Santagada wrote:

>
> On Jun 2, 2009, at 12:51 PM, Nathan Yergler wrote:
>
>> On Tue, Jun 2, 2009 at 7:32 AM, David Lyon  
>> <david.lyon at preisshare.net> wrote:
>>> On Tue, 2 Jun 2009 14:50:35 +0200, Tarek Ziad? <ziade.tarek at gmail.com 
>>> >
>>> wrote:
>>>>> 1) Debian+ubuntu .deb support? working? not-working? Status?
>>>>
>>>> The stdeb project provides a layer on the top of distutils for  
>>>> example
>>>> to achieve this.
>>>
>>> Oh ok - well the documentation should really lead people there..  
>>> imho
>>>
>>>>> 2) Mac Support? Working? same - how well? documentation is vague
>>>>
>>>> What kind of mac support you are reffering to ? If you build
>>>> something, it will work fine under mac
>>>
>>> Well...
>>>
>>> update the documentation.. the table near the top needs to
>>> reflect that.
>>
>> I think Tarek is just pointing out that you can build a dist and  
>> it'll
>> work on Mac OS X; AFAIK there's no explicit target for it (ala  
>> wininst
>> or RPM).
>
> There is makepkg or something, it is just a third party module like  
> stdeb no?

bdist_mpkg, and it's on pypi.

Ronald

>
> --
> Leonardo Santagada
> santagada at gmail.com
>
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090602/f18f748f/attachment.bin>

From david.lyon at preisshare.net  Wed Jun  3 00:33:11 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 02 Jun 2009 18:33:11 -0400
Subject: [Distutils] Distutils functionality questions/clarifications...
In-Reply-To: <79990c6b0906020758g186923h6028f79579736c1c@mail.gmail.com>
References: <1243945656.06.0.4878092207.issue77@psf.upfronthosting.co.za>
	<0e52434b69b476e786ab27cf8c64adf0@preisshare.net>
	<94bdd2610906020550y127ea1cdw7cc2a0954df86ed7@mail.gmail.com>
	<f3b1e9dea91b833c7d9a93474d73dced@preisshare.net>
	<79990c6b0906020758g186923h6028f79579736c1c@mail.gmail.com>
Message-ID: <399ce1f8c57ad2a748b70faa60a1a4af@preisshare.net>

On Tue, 2 Jun 2009 15:58:33 +0100, Paul Moore <p.f.moore at gmail.com> wrote:
> 2009/6/2 David Lyon <david.lyon at preisshare.net>:
>>>> ?3) windows installers for packages - can we kill them? or
>>>> ? ?how do we move to something better...
>>>
>>> What do you mean by "killing them" ?
>>
>> Remove from circulation.
> 
> Seems a bit drastic, given that very many people rely on them... What
> do you propose to replace them with?

Let me clarify it...

Installers for Apllications imho are just fine.. because users expect
that...

but installers for what are essentially data files .. python packages
is imho not right.

It's best done through pypi....

and a Package Manager haha

David

From p.f.moore at gmail.com  Wed Jun  3 00:44:14 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 2 Jun 2009 23:44:14 +0100
Subject: [Distutils] Distutils functionality questions/clarifications...
In-Reply-To: <399ce1f8c57ad2a748b70faa60a1a4af@preisshare.net>
References: <1243945656.06.0.4878092207.issue77@psf.upfronthosting.co.za>
	<0e52434b69b476e786ab27cf8c64adf0@preisshare.net>
	<94bdd2610906020550y127ea1cdw7cc2a0954df86ed7@mail.gmail.com>
	<f3b1e9dea91b833c7d9a93474d73dced@preisshare.net>
	<79990c6b0906020758g186923h6028f79579736c1c@mail.gmail.com>
	<399ce1f8c57ad2a748b70faa60a1a4af@preisshare.net>
Message-ID: <79990c6b0906021544w3c489672ld4eeca0173779eee@mail.gmail.com>

2009/6/2 David Lyon <david.lyon at preisshare.net>:
> On Tue, 2 Jun 2009 15:58:33 +0100, Paul Moore <p.f.moore at gmail.com> wrote:
>> 2009/6/2 David Lyon <david.lyon at preisshare.net>:
>>>>> ?3) windows installers for packages - can we kill them? or
>>>>> ? ?how do we move to something better...
>>>>
>>>> What do you mean by "killing them" ?
>>>
>>> Remove from circulation.
>>
>> Seems a bit drastic, given that very many people rely on them... What
>> do you propose to replace them with?
>
> Let me clarify it...
>
> Installers for Apllications imho are just fine.. because users expect
> that...
>
> but installers for what are essentially data files .. python packages
> is imho not right.

OK, I accept that is your view. But without a concrete proposal for
what I should use in place of (say) pywin32-213.win32-py2.6.exe to
install the pywin32 package into my Python 2.6 installation on my PC,
I don't see what you are suggesting to replace bdist_wininst
installers.

Eggs? If so, there are well-documented discussions over why some
people (myself included) don't like eggs. There's no point in
rehashing those discussions unless you have something new to add.

> It's best done through pypi....
>
> and a Package Manager haha

Again, that's been talked about a lot. Until someone develops such a
thing, I can't comment (other than to say that I suspect it's harder
than you'd think to please everyone who currently uses bdist_wininst
installers - which you'd have to do if you want to get rid of them).

Paul.

From david.lyon at preisshare.net  Wed Jun  3 00:55:31 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 02 Jun 2009 18:55:31 -0400
Subject: [Distutils] Distutils functionality questions/clarifications...
In-Reply-To: <79990c6b0906021544w3c489672ld4eeca0173779eee@mail.gmail.com>
References: <1243945656.06.0.4878092207.issue77@psf.upfronthosting.co.za>
	<0e52434b69b476e786ab27cf8c64adf0@preisshare.net>
	<94bdd2610906020550y127ea1cdw7cc2a0954df86ed7@mail.gmail.com>
	<f3b1e9dea91b833c7d9a93474d73dced@preisshare.net>
	<79990c6b0906020758g186923h6028f79579736c1c@mail.gmail.com>
	<399ce1f8c57ad2a748b70faa60a1a4af@preisshare.net>
	<79990c6b0906021544w3c489672ld4eeca0173779eee@mail.gmail.com>
Message-ID: <915ca344aa9429ed500534f808f26119@preisshare.net>

On Tue, 2 Jun 2009 23:44:14 +0100, Paul Moore <p.f.moore at gmail.com> wrote:

> OK, I accept that is your view. But without a concrete proposal for
> what I should use in place of (say) pywin32-213.win32-py2.6.exe to
> install the pywin32 package into my Python 2.6 installation on my PC,
> I don't see what you are suggesting to replace bdist_wininst
> installers.

I have no answer for that right now.. 

> Eggs? If so, there are well-documented discussions over why some
> people (myself included) don't like eggs. 

in a nutshell though.. whats wrong with eggs as far as pywin32
is concerned?

> Again, that's been talked about a lot. Until someone develops such a
> thing, I can't comment (other than to say that I suspect it's harder
> than you'd think to please everyone who currently uses bdist_wininst
> installers - which you'd have to do if you want to get rid of them).

For a sys-admin though.. they are a pain in the a.. 

It's not about that anyway... what we need to slowly move to is
having one consistant way of installing packages. I know that isn't
easy... and can't be done in a day....

but for users at the moment... it's just plain too hard...

David

From ronaldoussoren at mac.com  Wed Jun  3 07:55:26 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Wed, 03 Jun 2009 07:55:26 +0200
Subject: [Distutils] Distutils functionality questions/clarifications...
In-Reply-To: <915ca344aa9429ed500534f808f26119@preisshare.net>
References: <1243945656.06.0.4878092207.issue77@psf.upfronthosting.co.za>
	<0e52434b69b476e786ab27cf8c64adf0@preisshare.net>
	<94bdd2610906020550y127ea1cdw7cc2a0954df86ed7@mail.gmail.com>
	<f3b1e9dea91b833c7d9a93474d73dced@preisshare.net>
	<79990c6b0906020758g186923h6028f79579736c1c@mail.gmail.com>
	<399ce1f8c57ad2a748b70faa60a1a4af@preisshare.net>
	<79990c6b0906021544w3c489672ld4eeca0173779eee@mail.gmail.com>
	<915ca344aa9429ed500534f808f26119@preisshare.net>
Message-ID: <42476DD0-9FFE-4EFA-86EB-E5F58C5022FB@mac.com>


On 3 Jun, 2009, at 0:55, David Lyon wrote:
>
>> Eggs? If so, there are well-documented discussions over why some
>> people (myself included) don't like eggs.
>
> in a nutshell though.. whats wrong with eggs as far as pywin32
> is concerned?

The aren't shown in the Add/Remove programs menu.  The same discussion  
can be had about eggs vs. rpms on Linux, and having a package that is  
recognized by the operating system package manager is very useful when  
managing a system.  Having all software managed by the same package  
manager makes live a lot easier when you need to inventory what  
software is installed on a system, and during upgrades.

>
>> Again, that's been talked about a lot. Until someone develops such a
>> thing, I can't comment (other than to say that I suspect it's harder
>> than you'd think to please everyone who currently uses bdist_wininst
>> installers - which you'd have to do if you want to get rid of them).
>
> For a sys-admin though.. they are a pain in the a..

Why is that? Would bdist_msi help improve on that?

Ronald

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090603/f44e853b/attachment.bin>

From david.lyon at preisshare.net  Wed Jun  3 09:40:55 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 03 Jun 2009 03:40:55 -0400
Subject: [Distutils] Distutils functionality questions/clarifications...
In-Reply-To: <42476DD0-9FFE-4EFA-86EB-E5F58C5022FB@mac.com>
References: <1243945656.06.0.4878092207.issue77@psf.upfronthosting.co.za>
	<0e52434b69b476e786ab27cf8c64adf0@preisshare.net>
	<94bdd2610906020550y127ea1cdw7cc2a0954df86ed7@mail.gmail.com>
	<f3b1e9dea91b833c7d9a93474d73dced@preisshare.net>
	<79990c6b0906020758g186923h6028f79579736c1c@mail.gmail.com>
	<399ce1f8c57ad2a748b70faa60a1a4af@preisshare.net>
	<79990c6b0906021544w3c489672ld4eeca0173779eee@mail.gmail.com>
	<915ca344aa9429ed500534f808f26119@preisshare.net>
	<42476DD0-9FFE-4EFA-86EB-E5F58C5022FB@mac.com>
Message-ID: <fb01a74f5b010585df1eca64551121a8@preisshare.net>

On Wed, 03 Jun 2009 07:55:26 +0200, Ronald Oussoren
<ronaldoussoren at mac.com>
wrote:
>> in a nutshell though.. whats wrong with eggs as far as pywin32
>> is concerned?
> 
> The aren't shown in the Add/Remove programs menu.

Oh - you don't have a python package manager... that's all... :-(

Technically though, they shouldn't be displayed in the operating system
application list because they aren't technically "Programs". Rather, they
are just packages of a programming language... therefore.. if you like
subprograms.

I agree - there should be a list where they can be added and removed.

That's exactly what I'm working on now..

> The same discussion  
> can be had about eggs vs. rpms on Linux, and having a package that is  
> recognized by the operating system package manager is very useful when  
> managing a system.  Having all software managed by the same package  
> manager makes live a lot easier when you need to inventory what  
> software is installed on a system, and during upgrades.

Totally agree.

>> ... ..
> Why is that? Would bdist_msi help improve on that?

Nope. For all the above reasons...

David

From ronaldoussoren at mac.com  Wed Jun  3 10:08:01 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Wed, 03 Jun 2009 10:08:01 +0200
Subject: [Distutils] Distutils functionality questions/clarifications...
In-Reply-To: <fb01a74f5b010585df1eca64551121a8@preisshare.net>
References: <1243945656.06.0.4878092207.issue77@psf.upfronthosting.co.za>
	<0e52434b69b476e786ab27cf8c64adf0@preisshare.net>
	<94bdd2610906020550y127ea1cdw7cc2a0954df86ed7@mail.gmail.com>
	<f3b1e9dea91b833c7d9a93474d73dced@preisshare.net>
	<79990c6b0906020758g186923h6028f79579736c1c@mail.gmail.com>
	<399ce1f8c57ad2a748b70faa60a1a4af@preisshare.net>
	<79990c6b0906021544w3c489672ld4eeca0173779eee@mail.gmail.com>
	<915ca344aa9429ed500534f808f26119@preisshare.net>
	<42476DD0-9FFE-4EFA-86EB-E5F58C5022FB@mac.com>
	<fb01a74f5b010585df1eca64551121a8@preisshare.net>
Message-ID: <ED64BC0F-15AC-4024-9AB8-B4D5B8CB3A86@mac.com>


On 3 Jun, 2009, at 9:40, David Lyon wrote:

> On Wed, 03 Jun 2009 07:55:26 +0200, Ronald Oussoren
> <ronaldoussoren at mac.com>
> wrote:
>>> in a nutshell though.. whats wrong with eggs as far as pywin32
>>> is concerned?
>>
>> The aren't shown in the Add/Remove programs menu.
>
> Oh - you don't have a python package manager... that's all... :-(

I already know how to manage python packages, that's not the point.

>
>> The same discussion
>> can be had about eggs vs. rpms on Linux, and having a package that is
>> recognized by the operating system package manager is very useful  
>> when
>> managing a system.  Having all software managed by the same package
>> manager makes live a lot easier when you need to inventory what
>> software is installed on a system, and during upgrades.
>
> Totally agree.

I don't think we actually agree. When I have my system administrator  
hat on I don't want a seperate Python Package Manager, Perl Package  
Manager, Ruby Package Manager, ...., I just want a single tool that  
manages all software.

That said, as a python developer having a tool like easy_install or  
buildout that works the same on all operating systems is also very  
convenient.

Ronald

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090603/082e60b9/attachment-0001.bin>

From p.f.moore at gmail.com  Wed Jun  3 11:27:59 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 3 Jun 2009 10:27:59 +0100
Subject: [Distutils] Distutils functionality questions/clarifications...
In-Reply-To: <42476DD0-9FFE-4EFA-86EB-E5F58C5022FB@mac.com>
References: <1243945656.06.0.4878092207.issue77@psf.upfronthosting.co.za>
	<0e52434b69b476e786ab27cf8c64adf0@preisshare.net>
	<94bdd2610906020550y127ea1cdw7cc2a0954df86ed7@mail.gmail.com>
	<f3b1e9dea91b833c7d9a93474d73dced@preisshare.net>
	<79990c6b0906020758g186923h6028f79579736c1c@mail.gmail.com>
	<399ce1f8c57ad2a748b70faa60a1a4af@preisshare.net>
	<79990c6b0906021544w3c489672ld4eeca0173779eee@mail.gmail.com>
	<915ca344aa9429ed500534f808f26119@preisshare.net>
	<42476DD0-9FFE-4EFA-86EB-E5F58C5022FB@mac.com>
Message-ID: <79990c6b0906030227n25bf1c7dy26f1010b841f21be@mail.gmail.com>

2009/6/3 Ronald Oussoren <ronaldoussoren at mac.com>:
>
> On 3 Jun, 2009, at 0:55, David Lyon wrote:
>>
>>> Eggs? If so, there are well-documented discussions over why some
>>> people (myself included) don't like eggs.
>>
>> in a nutshell though.. whats wrong with eggs as far as pywin32
>> is concerned?
>
> The aren't shown in the Add/Remove programs menu. ?The same discussion can
> be had about eggs vs. rpms on Linux, and having a package that is recognized
> by the operating system package manager is very useful when managing a
> system. ?Having all software managed by the same package manager makes live
> a lot easier when you need to inventory what software is installed on a
> system, and during upgrades.

Plus, the simple fact that there isn't an egg available for pywin32.
(Note: don't take that as a request for one - if one existed I
wouldn't use it for the reason mentioned by Ronald above). You'd have
to ask Mark Hammond why he doesn't produce one. It may be just because
his users don't want one, or there may be other reasons as well.

Paul.

From p.f.moore at gmail.com  Wed Jun  3 11:33:46 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 3 Jun 2009 10:33:46 +0100
Subject: [Distutils] Distutils functionality questions/clarifications...
In-Reply-To: <fb01a74f5b010585df1eca64551121a8@preisshare.net>
References: <1243945656.06.0.4878092207.issue77@psf.upfronthosting.co.za>
	<0e52434b69b476e786ab27cf8c64adf0@preisshare.net>
	<94bdd2610906020550y127ea1cdw7cc2a0954df86ed7@mail.gmail.com>
	<f3b1e9dea91b833c7d9a93474d73dced@preisshare.net>
	<79990c6b0906020758g186923h6028f79579736c1c@mail.gmail.com>
	<399ce1f8c57ad2a748b70faa60a1a4af@preisshare.net>
	<79990c6b0906021544w3c489672ld4eeca0173779eee@mail.gmail.com>
	<915ca344aa9429ed500534f808f26119@preisshare.net>
	<42476DD0-9FFE-4EFA-86EB-E5F58C5022FB@mac.com>
	<fb01a74f5b010585df1eca64551121a8@preisshare.net>
Message-ID: <79990c6b0906030233y1a4a91e3w44f9afbe5b2fd17a@mail.gmail.com>

2009/6/3 David Lyon <david.lyon at preisshare.net>:
> Oh - you don't have a python package manager... that's all... :-(

And until someone produces one, there's no point in comparing what we
have with vapourware.

> Technically though, they shouldn't be displayed in the operating system
> application list because they aren't technically "Programs".

Your opinion. Mine is different. I doubt either of us know Microsoft's
opinion on the matter. (But OS patches aren't "programs" either, and
they are in there, albeit hidden by default)

*If* there was an existing "python package manager", it is possible
that it would provide compelling enough features to convince me to
change my mind. Build it and I'll tell you. Hey, build even some of it
and I'll comment on what you have (as long as I can use it alongside
bdist_wininst installers - I won't change my existing setup just to
test your alternative, sorry).

Paul.

From david.lyon at preisshare.net  Wed Jun  3 17:02:55 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 03 Jun 2009 11:02:55 -0400
Subject: [Distutils] Distutils functionality questions/clarifications...
In-Reply-To: <79990c6b0906030233y1a4a91e3w44f9afbe5b2fd17a@mail.gmail.com>
References: <1243945656.06.0.4878092207.issue77@psf.upfronthosting.co.za>
	<0e52434b69b476e786ab27cf8c64adf0@preisshare.net>
	<94bdd2610906020550y127ea1cdw7cc2a0954df86ed7@mail.gmail.com>
	<f3b1e9dea91b833c7d9a93474d73dced@preisshare.net>
	<79990c6b0906020758g186923h6028f79579736c1c@mail.gmail.com>
	<399ce1f8c57ad2a748b70faa60a1a4af@preisshare.net>
	<79990c6b0906021544w3c489672ld4eeca0173779eee@mail.gmail.com>
	<915ca344aa9429ed500534f808f26119@preisshare.net>
	<42476DD0-9FFE-4EFA-86EB-E5F58C5022FB@mac.com>
	<fb01a74f5b010585df1eca64551121a8@preisshare.net>
	<79990c6b0906030233y1a4a91e3w44f9afbe5b2fd17a@mail.gmail.com>
Message-ID: <19f1130a9454abe849e40ec3f5d40b06@preisshare.net>

On Wed, 3 Jun 2009 10:33:46 +0100, Paul Moore <p.f.moore at gmail.com> wrote:

> *If* there was an existing "python package manager", it is possible
> that it would provide compelling enough features to convince me to
> change my mind. Build it and I'll tell you. Hey, build even some of it
> and I'll comment on what you have ....

ok - that's fair enough... done....

Just follow the download link...

http://sourceforge.net/projects/pythonpkgmgr/

David

From mail at timgolden.me.uk  Wed Jun  3 17:11:45 2009
From: mail at timgolden.me.uk (Tim Golden)
Date: Wed, 03 Jun 2009 16:11:45 +0100
Subject: [Distutils] Distutils functionality questions/clarifications...
In-Reply-To: <19f1130a9454abe849e40ec3f5d40b06@preisshare.net>
References: <1243945656.06.0.4878092207.issue77@psf.upfronthosting.co.za>	<0e52434b69b476e786ab27cf8c64adf0@preisshare.net>	<94bdd2610906020550y127ea1cdw7cc2a0954df86ed7@mail.gmail.com>	<f3b1e9dea91b833c7d9a93474d73dced@preisshare.net>	<79990c6b0906020758g186923h6028f79579736c1c@mail.gmail.com>	<399ce1f8c57ad2a748b70faa60a1a4af@preisshare.net>	<79990c6b0906021544w3c489672ld4eeca0173779eee@mail.gmail.com>	<915ca344aa9429ed500534f808f26119@preisshare.net>	<42476DD0-9FFE-4EFA-86EB-E5F58C5022FB@mac.com>	<fb01a74f5b010585df1eca64551121a8@preisshare.net>	<79990c6b0906030233y1a4a91e3w44f9afbe5b2fd17a@mail.gmail.com>
	<19f1130a9454abe849e40ec3f5d40b06@preisshare.net>
Message-ID: <4A2692B1.2020505@timgolden.me.uk>

David Lyon wrote:
> On Wed, 3 Jun 2009 10:33:46 +0100, Paul Moore <p.f.moore at gmail.com> wrote:
> 
>> *If* there was an existing "python package manager", it is possible
>> that it would provide compelling enough features to convince me to
>> change my mind. Build it and I'll tell you. Hey, build even some of it
>> and I'll comment on what you have ....
> 
> ok - that's fair enough... done....
> 
> Just follow the download link...
> 
> http://sourceforge.net/projects/pythonpkgmgr/

"""
Traceback (most recent call last):
  File "PythonPackageManager.py", line 22, in <module>
  File "PythonPackageManager.py", line 18, in main
  File "wx\_core.pyc", line 7938, in __init__
  File "wx\_core.pyc", line 7512, in _BootstrapApp
  File "PythonPackageManager.py", line 12, in OnInit
  File "MainWindow.pyc", line 29, in create
  File "MainWindow.pyc", line 374, in __init__
  File "Packaging.pyc", line 94, in __init__
  File "Packaging.pyc", line 600, in setworkingversion
  File "Packaging.pyc", line 686, in pythonversonlist
WindowsError: [Error 2] The system cannot find the file specified

"""



From david.lyon at preisshare.net  Wed Jun  3 17:16:25 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 03 Jun 2009 11:16:25 -0400
Subject: [Distutils] Distutils functionality questions/clarifications...
In-Reply-To: <4A2692B1.2020505@timgolden.me.uk>
References: <1243945656.06.0.4878092207.issue77@psf.upfronthosting.co.za>	<0e52434b69b476e786ab27cf8c64adf0@preisshare.net>	<94bdd2610906020550y127ea1cdw7cc2a0954df86ed7@mail.gmail.com>	<f3b1e9dea91b833c7d9a93474d73dced@preisshare.net>	<79990c6b0906020758g186923h6028f79579736c1c@mail.gmail.com>	<399ce1f8c57ad2a748b70faa60a1a4af@preisshare.net>	<79990c6b0906021544w3c489672ld4eeca0173779eee@mail.gmail.com>	<915ca344aa9429ed500534f808f26119@preisshare.net>	<42476DD0-9FFE-4EFA-86EB-E5F58C5022FB@mac.com>	<fb01a74f5b010585df1eca64551121a8@preisshare.net>	<79990c6b0906030233y1a4a91e3w44f9afbe5b2fd17a@mail.gmail.com>
	<19f1130a9454abe849e40ec3f5d40b06@preisshare.net>
	<4A2692B1.2020505@timgolden.me.uk>
Message-ID: <85dcc8ca7bfdf19949a2b7a54fb4eba0@preisshare.net>

On Wed, 03 Jun 2009 16:11:45 +0100, Tim Golden <mail at timgolden.me.uk>
wrote:
> Traceback (most recent call last):
>   File "PythonPackageManager.py", line 22, in <module>
>   File "PythonPackageManager.py", line 18, in main
>   File "wx\_core.pyc", line 7938, in __init__
>   File "wx\_core.pyc", line 7512, in _BootstrapApp
>   File "PythonPackageManager.py", line 12, in OnInit
>   File "MainWindow.pyc", line 29, in create
>   File "MainWindow.pyc", line 374, in __init__
>   File "Packaging.pyc", line 94, in __init__
>   File "Packaging.pyc", line 600, in setworkingversion
>   File "Packaging.pyc", line 686, in pythonversonlist
> WindowsError: [Error 2] The system cannot find the file specified
> 
> """

Thanks for testing it so quickly.... and reporting...

I'll post it as a bug report...

and look into it as soon as I am able...

David



From mail at timgolden.me.uk  Wed Jun  3 17:25:08 2009
From: mail at timgolden.me.uk (Tim Golden)
Date: Wed, 03 Jun 2009 16:25:08 +0100
Subject: [Distutils] Distutils functionality questions/clarifications...
In-Reply-To: <85dcc8ca7bfdf19949a2b7a54fb4eba0@preisshare.net>
References: <1243945656.06.0.4878092207.issue77@psf.upfronthosting.co.za>	<0e52434b69b476e786ab27cf8c64adf0@preisshare.net>	<94bdd2610906020550y127ea1cdw7cc2a0954df86ed7@mail.gmail.com>	<f3b1e9dea91b833c7d9a93474d73dced@preisshare.net>	<79990c6b0906020758g186923h6028f79579736c1c@mail.gmail.com>	<399ce1f8c57ad2a748b70faa60a1a4af@preisshare.net>	<79990c6b0906021544w3c489672ld4eeca0173779eee@mail.gmail.com>	<915ca344aa9429ed500534f808f26119@preisshare.net>	<42476DD0-9FFE-4EFA-86EB-E5F58C5022FB@mac.com>	<fb01a74f5b010585df1eca64551121a8@preisshare.net>	<79990c6b0906030233y1a4a91e3w44f9afbe5b2fd17a@mail.gmail.com>
	<19f1130a9454abe849e40ec3f5d40b06@preisshare.net>
	<4A2692B1.2020505@timgolden.me.uk>
	<85dcc8ca7bfdf19949a2b7a54fb4eba0@preisshare.net>
Message-ID: <4A2695D4.4010706@timgolden.me.uk>

David Lyon wrote:
> On Wed, 03 Jun 2009 16:11:45 +0100, Tim Golden <mail at timgolden.me.uk>
> wrote:
>> Traceback (most recent call last):
>>   File "PythonPackageManager.py", line 22, in <module>
>>   File "PythonPackageManager.py", line 18, in main
>>   File "wx\_core.pyc", line 7938, in __init__
>>   File "wx\_core.pyc", line 7512, in _BootstrapApp
>>   File "PythonPackageManager.py", line 12, in OnInit
>>   File "MainWindow.pyc", line 29, in create
>>   File "MainWindow.pyc", line 374, in __init__
>>   File "Packaging.pyc", line 94, in __init__
>>   File "Packaging.pyc", line 600, in setworkingversion
>>   File "Packaging.pyc", line 686, in pythonversonlist
>> WindowsError: [Error 2] The system cannot find the file specified
>>
>> """
> 
> Thanks for testing it so quickly.... and reporting...
> 
> I'll post it as a bug report...
> 
> and look into it as soon as I am able...
> 
> David
> 
FWIW, the "C:\Program Files\Python Package Manager" directory
it installed into looks very empty:

"""
C:\Program Files\Python Package Manager>dir /b
gdiplus.dll
msvcp71.dll
MSVCR71.dll
pkgmgr
PythonPackageManager.exe
PythonPackageManager.exe.log
uninstall.exe
"""

Maybe something's up with your installer setup?

TJG

From david.lyon at preisshare.net  Wed Jun  3 17:41:03 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 03 Jun 2009 11:41:03 -0400
Subject: [Distutils] Distutils functionality questions/clarifications...
In-Reply-To: <4A2695D4.4010706@timgolden.me.uk>
References: <1243945656.06.0.4878092207.issue77@psf.upfronthosting.co.za>	<0e52434b69b476e786ab27cf8c64adf0@preisshare.net>	<94bdd2610906020550y127ea1cdw7cc2a0954df86ed7@mail.gmail.com>	<f3b1e9dea91b833c7d9a93474d73dced@preisshare.net>	<79990c6b0906020758g186923h6028f79579736c1c@mail.gmail.com>	<399ce1f8c57ad2a748b70faa60a1a4af@preisshare.net>	<79990c6b0906021544w3c489672ld4eeca0173779eee@mail.gmail.com>	<915ca344aa9429ed500534f808f26119@preisshare.net>	<42476DD0-9FFE-4EFA-86EB-E5F58C5022FB@mac.com>	<fb01a74f5b010585df1eca64551121a8@preisshare.net>	<79990c6b0906030233y1a4a91e3w44f9afbe5b2fd17a@mail.gmail.com>
	<19f1130a9454abe849e40ec3f5d40b06@preisshare.net>
	<4A2692B1.2020505@timgolden.me.uk>
	<85dcc8ca7bfdf19949a2b7a54fb4eba0@preisshare.net>
	<4A2695D4.4010706@timgolden.me.uk>
Message-ID: <fc83d7ee71e88f9a42144e710b9c56a3@preisshare.net>

On Wed, 03 Jun 2009 16:25:08 +0100, Tim Golden <mail at timgolden.me.uk>
wrote:
> C:\Program Files\Python Package Manager>dir /b
> gdiplus.dll
> msvcp71.dll
> MSVCR71.dll
> pkgmgr
> PythonPackageManager.exe
> PythonPackageManager.exe.log
> uninstall.exe

> FWIW, the "C:\Program Files\Python Package Manager" directory
> it installed into looks very empty:
> """
> 
> Maybe something's up with your installer setup?

That looks correct.... there aren't many files...

Perphaps you are using vista...

The code is breaking where it is looking in the registry to
see what versions of python are installed on your machine.

I do accept the code is somewhat fragile at this point in
time. I'm only testing on 2.2, 2.5, 2.6 under windows 2000,
xp, and ubuntu at the moment.

The only other thing I could suggest is to download from
source on svn...

svn co https://pythonpkgmgr.svn.sourceforge.net/svnroot/pythonpkgmgr
pythonpkgmgr

I'd welcome any comments.. but I would certainly understand
if you have other more important things to do....

David




 

From mail at timgolden.me.uk  Wed Jun  3 17:47:57 2009
From: mail at timgolden.me.uk (Tim Golden)
Date: Wed, 03 Jun 2009 16:47:57 +0100
Subject: [Distutils] Distutils functionality questions/clarifications...
In-Reply-To: <fc83d7ee71e88f9a42144e710b9c56a3@preisshare.net>
References: <1243945656.06.0.4878092207.issue77@psf.upfronthosting.co.za>	<0e52434b69b476e786ab27cf8c64adf0@preisshare.net>	<94bdd2610906020550y127ea1cdw7cc2a0954df86ed7@mail.gmail.com>	<f3b1e9dea91b833c7d9a93474d73dced@preisshare.net>	<79990c6b0906020758g186923h6028f79579736c1c@mail.gmail.com>	<399ce1f8c57ad2a748b70faa60a1a4af@preisshare.net>	<79990c6b0906021544w3c489672ld4eeca0173779eee@mail.gmail.com>	<915ca344aa9429ed500534f808f26119@preisshare.net>	<42476DD0-9FFE-4EFA-86EB-E5F58C5022FB@mac.com>	<fb01a74f5b010585df1eca64551121a8@preisshare.net>	<79990c6b0906030233y1a4a91e3w44f9afbe5b2fd17a@mail.gmail.com>
	<19f1130a9454abe849e40ec3f5d40b06@preisshare.net>
	<4A2692B1.2020505@timgolden.me.uk>
	<85dcc8ca7bfdf19949a2b7a54fb4eba0@preisshare.net>
	<4A2695D4.4010706@timgolden.me.uk>
	<fc83d7ee71e88f9a42144e710b9c56a3@preisshare.net>
Message-ID: <4A269B2D.7060703@timgolden.me.uk>

David Lyon wrote:
> On Wed, 03 Jun 2009 16:25:08 +0100, Tim Golden <mail at timgolden.me.uk>
> wrote:
>> C:\Program Files\Python Package Manager>dir /b
>> gdiplus.dll
>> msvcp71.dll
>> MSVCR71.dll
>> pkgmgr
>> PythonPackageManager.exe
>> PythonPackageManager.exe.log
>> uninstall.exe
> 
>> FWIW, the "C:\Program Files\Python Package Manager" directory
>> it installed into looks very empty:
>> """
>>
>> Maybe something's up with your installer setup?
> 
> That looks correct.... there aren't many files...

> 
> Perphaps you are using vista...

Nope. XP SP3

> 
> The code is breaking where it is looking in the registry to
> see what versions of python are installed on your machine.

(Which is several; I've got 2.3 to 2.7 in the registry)
> 
> I do accept the code is somewhat fragile at this point in
> time. I'm only testing on 2.2, 2.5, 2.6 under windows 2000,
> xp, and ubuntu at the moment.

> svn co https://pythonpkgmgr.svn.sourceforge.net/svnroot/pythonpkgmgr
> pythonpkgmgr

I'll give that a try.


TJG

From mail at timgolden.me.uk  Wed Jun  3 17:53:54 2009
From: mail at timgolden.me.uk (Tim Golden)
Date: Wed, 03 Jun 2009 16:53:54 +0100
Subject: [Distutils] Distutils functionality questions/clarifications...
In-Reply-To: <4A269B2D.7060703@timgolden.me.uk>
References: <1243945656.06.0.4878092207.issue77@psf.upfronthosting.co.za>	<0e52434b69b476e786ab27cf8c64adf0@preisshare.net>	<94bdd2610906020550y127ea1cdw7cc2a0954df86ed7@mail.gmail.com>	<f3b1e9dea91b833c7d9a93474d73dced@preisshare.net>	<79990c6b0906020758g186923h6028f79579736c1c@mail.gmail.com>	<399ce1f8c57ad2a748b70faa60a1a4af@preisshare.net>	<79990c6b0906021544w3c489672ld4eeca0173779eee@mail.gmail.com>	<915ca344aa9429ed500534f808f26119@preisshare.net>	<42476DD0-9FFE-4EFA-86EB-E5F58C5022FB@mac.com>	<fb01a74f5b010585df1eca64551121a8@preisshare.net>	<79990c6b0906030233y1a4a91e3w44f9afbe5b2fd17a@mail.gmail.com>	<19f1130a9454abe849e40ec3f5d40b06@preisshare.net>	<4A2692B1.2020505@timgolden.me.uk>	<85dcc8ca7bfdf19949a2b7a54fb4eba0@preisshare.net>	<4A2695D4.4010706@timgolden.me.uk>	<fc83d7ee71e88f9a42144e710b9c56a3@preisshare.net>
	<4A269B2D.7060703@timgolden.me.uk>
Message-ID: <4A269C92.3050101@timgolden.me.uk>

Tim Golden wrote:
>> svn co https://pythonpkgmgr.svn.sourceforge.net/svnroot/pythonpkgmgr
>> pythonpkgmgr
> 
> I'll give that a try.


Not that I think it's necessarily the issue, but you've got mixed
tabs and spaces in there, which is never a good recipe. I tried
translating using 4 but didn't seem to work. Probably worth
sorting that out...

TJG

From p.f.moore at gmail.com  Wed Jun  3 19:07:34 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 3 Jun 2009 18:07:34 +0100
Subject: [Distutils] Distutils functionality questions/clarifications...
In-Reply-To: <19f1130a9454abe849e40ec3f5d40b06@preisshare.net>
References: <1243945656.06.0.4878092207.issue77@psf.upfronthosting.co.za>
	<f3b1e9dea91b833c7d9a93474d73dced@preisshare.net>
	<79990c6b0906020758g186923h6028f79579736c1c@mail.gmail.com>
	<399ce1f8c57ad2a748b70faa60a1a4af@preisshare.net>
	<79990c6b0906021544w3c489672ld4eeca0173779eee@mail.gmail.com>
	<915ca344aa9429ed500534f808f26119@preisshare.net>
	<42476DD0-9FFE-4EFA-86EB-E5F58C5022FB@mac.com>
	<fb01a74f5b010585df1eca64551121a8@preisshare.net>
	<79990c6b0906030233y1a4a91e3w44f9afbe5b2fd17a@mail.gmail.com>
	<19f1130a9454abe849e40ec3f5d40b06@preisshare.net>
Message-ID: <79990c6b0906031007n11ad2cddt861bbcc8b65edc82@mail.gmail.com>

2009/6/3 David Lyon <david.lyon at preisshare.net>:
> On Wed, 3 Jun 2009 10:33:46 +0100, Paul Moore <p.f.moore at gmail.com> wrote:
>
>> *If* there was an existing "python package manager", it is possible
>> that it would provide compelling enough features to convince me to
>> change my mind. Build it and I'll tell you. Hey, build even some of it
>> and I'll comment on what you have ....
>
> ok - that's fair enough... done....
>
> Just follow the download link...
>
> http://sourceforge.net/projects/pythonpkgmgr/

Runs OK for me. I got it from Subversion and I'm running from source, FWIW.

It recognises 4 of the packages I have installed, but doesn't see wxPython.

So how do I install something? There's an "Install egg" menu item, so
I guess I have to go & download an egg :-( (I think I already
mentioned I don't like setuptools/eggs...)

No egg for download for pygame. So you'll need to persuade them to
build eggs. Or for nose. Or cx_Oracle. Sigh, this isn't looking good
at the moment.

Fouund an egg for genshi, I'll give it a try. (But it's
version-specific, I have to download the Python 2.6 version, when
genshi appears to be pure-python. That's a particular annoyance of
eggs coming up again.

Never mind.

Tried to install the egg - I got an error "Execution of command
'C:\Apps\Python26\scripts\easy_install.exe
C:\Users\Gustav\Documents\Genshi-0.5.1-py2.6-win32.egg' failed (error
740: the requested operation requires elevation.)" Looks like you need
to support Windows Vista elevation. That's an advantage of normal
(bdist_wininst or bdist_msi) installers - the system seems to do the
elevation prompt for you.

OK, that's as far as I can go without Vista support.

Plus points:
* It doesn't look bad for what I assume is a relatively new project
* It detected (some of) my existing packages

Minus points:
* It uses eggs, and so even pure-python packages have to be
distributed in version-specific binaries. Looks like I'll end up
spending even more time building my own packages with this system
* It doesn't work on Vista - bdist_wininst work unchanged on Vista,
and so does easy_install genshi (by hand) so it's your code that needs
updating (and may well need doing so again for Win7, etc...)
* I can't install so I can't test uninstall, etc.
* I can't tell at this stage if it uses the .pth files that
easy_install does (but I guess so).
* I assume that if I used it to uninstall one of my
bdist_wininst-installed packages, it would fail to uninstall properly
(leave the Add/Remove programs stuff behind). It should either do it
properly or not allow it.

So it's inheriting all the (in my view) bad points of eggs (that I've
documented previously). I misunderstood your intention, I thought you
were proposing something new - rather than simply wrapping
eggs/easy_install to make it more acceptable to people like me. That's
a reasonable goal (and you're not doing badly at it) but it leaves all
of the fundamental non-UI issues with eggs intact.

And I see nothing in this which would persuade a package developer who
wasn't already distributing eggs to switch to doing so. So there's
still the availability issue. (Made worse as your package, unlike
easy_install, doesn't seem to be willing to consume source packages or
bdist_wininst exes).

I hope this helps.
Paul.

From brian at vanguardistas.net  Thu Jun  4 10:54:48 2009
From: brian at vanguardistas.net (Brian Sutherland)
Date: Thu, 4 Jun 2009 10:54:48 +0200
Subject: [Distutils] Low Level API for translating
	distutils/setuptools	metatdata to?Debian metadata
In-Reply-To: <b94538deaf36b06f35b6b78a89cb989e@preisshare.net>
References: <20090529091202.GB22620@Boo.lan> <4A2000B4.90501@astraw.com>
	<b94538deaf36b06f35b6b78a89cb989e@preisshare.net>
Message-ID: <20090604085448.GA9596@Boo.lan>

On Sat, May 30, 2009 at 07:27:06PM -0400, David Lyon wrote:
> On Fri, 29 May 2009 08:35:16 -0700, Andrew Straw <strawman at astraw.com>
> wrote:
> > Brian Sutherland wrote:
> >> I've just published a very small library that does three things so far:
> >> 
> >>     * Provides a mapping between python project names and Debian
> >>       binary/source package names
> >>     * Converts setuptools versions to Debian versions while maintaining
> >>       sort order
> >>     * Can introspect an .egg-info directory to figure out the Debian
> >>       dependencies for use in the debian/control file. It can also
> >>       handle/understand extras (I Hope!)
> >> 
> >> It's basically an attempt to find a common ground between all the
> >> projects doing automated Python->Debian packaging. I have a feeling
> >> everyone is re-implementing this code all the time.
> 
> Maybe not everybody - but I am sure there are a few...
> 
> at least there are many who would like to do it..
> 
> Recently I posted a suggestion to Andrea Gavana about py2exe on the
> wxPython list about his excellent gui interface for py2exe.
> 
> My use case is this... I'm writing a package manager for python..
> I want it to be deployed to every python platform... I dont have
> all those platforms....
> 
> I want p2exe or distutils (Distribution Utils?? is that what it
> stands for?) to take my program and make it for every platform...
> 
>  - windows
>  - mac
>  - linux (ubuntu,debian,suse,redhat)
> 
> It's actually not so many..
> 
> No, I am not talking jibberish or pie in the sky here..
> 
> Here's specifically what needs doing.....
> 
>  1) create a three page "wizard" style app in wxpython
> 
>  2) the app has checkboxes to allow the user to select the 
>     operating systems that they want to deploy to
> 
>  3) p2app for mac, py2exe for windows.. whichever one
>     for linux is driven.. much like gui2exe
> 
>  4) Cheetah templates are used to create the necessary
>     output scripts from the meta information.
> 
>     (building scripts in code is so 1970s.. we don't
>      want to do it that way again)
> 
>  5) The scripts get run and the user ends up with
>     .debs .rpm... windows installer  
>  
> basic structure...
> 
> for plat in user_selected_platforms:
>     build_for_platform(plat)
> 
> def build_for_platform(platform):
>     if platform == "mac":
>         ..
>     elif ....
> 
> I think distutils has been in a "frozen" state for the
> last few years. With no notable improvements apart
> from bug fixes and internal work.
> 
> Given that the general consensus is largely to leave
> package/application management to the operating 
> system, what I describe above is a more modern way to 
> accomplish what people are trying to do with less 
> effort on the part of all...
> 
> If a user makes a python app... why shouldn't they
> be able to run a distutils script to "distribute"
> to a platform they don't have....
> 
> Building ubuntu or debian packages is no longer
> a black art....

To be honest, it still is;)
 
> It's 2009 now nearly 2010...
> 
> When was the last time distutils ever got a
> major upgrade....?
> 
> Is this not what "disutils" should be doing?

I would personally be very happy if distutils kept out of the business
of making packages for anything but python to consume.

Making OS level packages is, in my view, something that should be built
on top of, not inside distutils.

So the current distutils development is going in the right direction for
me: Specifying common metadata formats and making APIs to access that.

> in this day and age?
> 
> David
> 
> 

-- 
Brian Sutherland

From ziade.tarek at gmail.com  Thu Jun  4 11:17:38 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 4 Jun 2009 11:17:38 +0200
Subject: [Distutils] PEP 345, PEP 376, PEP 386
Message-ID: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>

Hello

Here's a status of the current work waiting to be included in
Distutils.  (target: Python 2.7 and Python 3.2)

I have created PEP 386 for the version comparison work, and gathered
in it all the work related to version comparison,
I am not an Fedora, Ubuntu, [put your os here] specialist and the PEP
is in its early stage, so it needs your feedback
of you see false statements, incomplete info or errrors.

STATUS:

- new PEP 386 |?waiting for your feedback

  - http://svn.python.org/projects/peps/trunk/pep-0386.txt


- PEP 376 |?status : waiting for Phillip complementary feedback (and
anyone else of course)

   - up-to-date PEP proposal :
http://bitbucket.org/tarek/pep376/src/tip/docs/pep-0376
   - up-to-date prototype code for pkgutil :
http://bitbucket.org/tarek/pep376/src/tip/pkgutil.py

- PEP 345 | status : waiting for PEP 386 validation


Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org

From brian at vanguardistas.net  Thu Jun  4 11:52:03 2009
From: brian at vanguardistas.net (Brian Sutherland)
Date: Thu, 4 Jun 2009 11:52:03 +0200
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
Message-ID: <20090604095203.GB9596@Boo.lan>

On Thu, Jun 04, 2009 at 11:17:38AM +0200, Tarek Ziad? wrote:
> Hello
> 
> Here's a status of the current work waiting to be included in
> Distutils.  (target: Python 2.7 and Python 3.2)
> 
> I have created PEP 386 for the version comparison work, and gathered
> in it all the work related to version comparison,
> I am not an Fedora, Ubuntu, [put your os here] specialist and the PEP
> is in its early stage, so it needs your feedback
> of you see false statements, incomplete info or errrors.
> 
> STATUS:
> 
> - new PEP 386 |?waiting for your feedback
> 
>   - http://svn.python.org/projects/peps/trunk/pep-0386.txt

    ...  < V('1.0.dev456')
    ...  < V('1.0')
    ...  < V('1.0.dev456post623')

Looks like a typo or very un-intuitive. It doesn't seem right that a
"dev" version sorts after a full release.

> 
> - PEP 376 |?status : waiting for Phillip complementary feedback (and
> anyone else of course)
> 
>    - up-to-date PEP proposal :
> http://bitbucket.org/tarek/pep376/src/tip/docs/pep-0376

Will there be a way to tell distutils that it is not responsible for a
package and should error when trying to uninstall it?

Perhaps not installing the RECORD file at all in this case.

I can imagine distutils uninstalling files previously installed by dpkg
as a shortcut to breaking a machine. Though I'm not sure what will
actually happen in practice. 

>    - up-to-date prototype code for pkgutil :
> http://bitbucket.org/tarek/pep376/src/tip/pkgutil.py
> 
> - PEP 345 | status : waiting for PEP 386 validation
> 
> 
> Regards
> Tarek
> 
> -- 
> Tarek Ziad? | http://ziade.org
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig

-- 
Brian Sutherland

From ziade.tarek at gmail.com  Thu Jun  4 12:37:01 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 4 Jun 2009 12:37:01 +0200
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <20090604095203.GB9596@Boo.lan>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604095203.GB9596@Boo.lan>
Message-ID: <94bdd2610906040337m35a882b0i1c1f7cfa655a38fa@mail.gmail.com>

On Thu, Jun 04, 2009 at 11:17:38AM +0200, Tarek Ziad? wrote:
> Hello
>
> Here's a status of the current work waiting to be included in
> Distutils.  (target: Python 2.7 and Python 3.2)
>
> I have created PEP 386 for the version comparison work, and gathered
> in it all the work related to version comparison,
> I am not an Fedora, Ubuntu, [put your os here] specialist and the PEP
> is in its early stage, so it needs your feedback
> of you see false statements, incomplete info or errrors.
>
> STATUS:
>
> - new PEP 386 | waiting for your feedback
>
>   - http://svn.python.org/projects/peps/trunk/pep-0386.txt

   ...  < V('1.0.dev456')
   ...  < V('1.0')
   ...  < V('1.0.dev456post623')

Looks like a typo or very un-intuitive. It doesn't seem right that a
"dev" version sorts after a full release.

>
> - PEP 376 | status : waiting for Phillip complementary feedback (and
> anyone else of course)
>
>    - up-to-date PEP proposal :
> http://bitbucket.org/tarek/pep376/src/tip/docs/pep-0376

Will there be a way to tell distutils that it is not responsible for a
package and should error when trying to uninstall it?

Perhaps not installing the RECORD file at all in this case.

I can imagine distutils uninstalling files previously installed by dpkg
as a shortcut to breaking a machine. Though I'm not sure what will
actually happen in practice.

>    - up-to-date prototype code for pkgutil :
> http://bitbucket.org/tarek/pep376/src/tip/pkgutil.py
>
> - PEP 345 | status : waiting for PEP 386 validation
>
>
> Regards
> Tarek
>
> --
> Tarek Ziad? | http://ziade.org
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig

--
Brian Sutherland

From ziade.tarek at gmail.com  Thu Jun  4 12:57:12 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 4 Jun 2009 12:57:12 +0200
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <20090604095203.GB9596@Boo.lan>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604095203.GB9596@Boo.lan>
Message-ID: <94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>

Oups messed up sorry -> resending my answer

On Thu, Jun 4, 2009 at 11:52 AM, Brian Sutherland
<brian at vanguardistas.net> wrote:
>>   - http://svn.python.org/projects/peps/trunk/pep-0386.txt
>
>    ...  < V('1.0.dev456')
>    ...  < V('1.0')
>    ...  < V('1.0.dev456post623')
>
> Looks like a typo or very un-intuitive. It doesn't seem right that a
> "dev" version sorts after a full release.

This is a dev version of a post-release version. Which is an edge case
submitted by Phillip.

How would you write it ?

>
>>
>> - PEP 376 | status : waiting for Phillip complementary feedback (and
>> anyone else of course)
>
>
> I can imagine distutils uninstalling files previously installed by dpkg
> as a shortcut to breaking a machine. Though I'm not sure what will
> actually happen in practice.

Distutils defines a standard for an EGG-INFO structure, and provides a
API for the uninstallation that is more likely to be a reference implementation.

Although, It is already providing an install feature.

If uninstalling a package with Distutils, while it was installed by dpkg breaks,

I can imagine that in the very same system, you can also break it
if you *install* packages with Distutils, easy_install, pip because
you shortcut dpkg
as well.

I think this is the responsability of such a system to make sure it handles
all installation and uninstallation.

From p.f.moore at gmail.com  Thu Jun  4 14:23:21 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 4 Jun 2009 13:23:21 +0100
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604095203.GB9596@Boo.lan>
	<94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
Message-ID: <79990c6b0906040523w4cfec90l7617f23940c4d14e@mail.gmail.com>

2009/6/4 Tarek Ziad? <ziade.tarek at gmail.com>:
>>> - PEP 376 | status : waiting for Phillip complementary feedback (and
>>> anyone else of course)
>>
>>
>> I can imagine distutils uninstalling files previously installed by dpkg
>> as a shortcut to breaking a machine. Though I'm not sure what will
>> actually happen in practice.
>
> Distutils defines a standard for an EGG-INFO structure, and provides a
> API for the uninstallation that is more likely to be a reference implementation.
>
> Although, It is already providing an install feature.
>
> If uninstalling a package with Distutils, while it was installed by dpkg breaks,
>
> I can imagine that in the very same system, you can also break it
> if you *install* packages with Distutils, easy_install, pip because
> you shortcut dpkg
> as well.

With Windows, if you install using bdist_wininst and then uninstall
using the (currently nonexistent) distutils uninstall, I'd expect that
it wouldn't remove the Add/Remove programs support items in the
registry, and the Removexxx.exe and xxx-wininst.log files installed by
bdist_wininst. The system would then still think that the package was
installed, even though it isn't. (And the system uninstall may break -
I'm not sure).

Given that Windows doesn't do dependency management, this is a
relatively minor form of breakage, but on a system that does, having
the system metadata wrong could cause incorrect dependencies to be
installed at a later date.

> I think this is the responsability of such a system to make sure it handles
> all installation and uninstallation.

It's surely not the responsibility of the OS to handle you "going
behind its back" - if I did "rm -rf /usr" on a Unix system, I wouldn't
think that it's the responsibility of the system to handle this (other
than by breaking!)

I'd say that it's distutils' responsibility not to offer to uninstall
anything it didn't install.

Paul.

From floris.bruynooghe at gmail.com  Thu Jun  4 14:26:35 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Thu, 4 Jun 2009 13:26:35 +0100
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604095203.GB9596@Boo.lan>
	<94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
Message-ID: <20090604122635.GB9784@laurie.devork>

On Thu, Jun 04, 2009 at 12:57:12PM +0200, Tarek Ziad? wrote:
> 
> On Thu, Jun 4, 2009 at 11:52 AM, Brian Sutherland
> <brian at vanguardistas.net> wrote:
> >>   - http://svn.python.org/projects/peps/trunk/pep-0386.txt
> >
> >    ...  < V('1.0.dev456')
> >    ...  < V('1.0')
> >    ...  < V('1.0.dev456post623')
> >
> > Looks like a typo or very un-intuitive. It doesn't seem right that a
> > "dev" version sorts after a full release.
> 
> This is a dev version of a post-release version. Which is an edge case
> submitted by Phillip.
> 
> How would you write it ?

1.0.post623dev456 is what feels intuitive to me, here's my version of
the last few lines:

...  < V('1.0')
...  < V('1.0.post456dev623')
...  < V('1.0.post456'))

It would even feel more consistent if it was 1.0.port456.dev623


Frankly I don't really understand the post-release requirement (and
the PEP text doesn't help me out there, no explenation of what it is).

If it's like a pre-relase version (alpha, beta, release candidate aka
a, b and c) then I imagine it's a preview release of a bugfix release.
So if you start with 1.0, then discover bugs so need a 1.0.1 but want
to do a preview of that the PEP seems to suggest 1.0.post0, however I
would call it 1.0.1a.  I guess both will sort correctly according to
the PEP so the PEP doesn't care which method you use.  Still find it
confusing tough.


Regards
Floris


-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From brian at vanguardistas.net  Thu Jun  4 14:29:15 2009
From: brian at vanguardistas.net (Brian Sutherland)
Date: Thu, 4 Jun 2009 14:29:15 +0200
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604095203.GB9596@Boo.lan>
	<94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
Message-ID: <a992920f0906040529r7de0ac5cgf2b8d738dbb3be5e@mail.gmail.com>

2009/6/4 Tarek Ziad? <ziade.tarek at gmail.com>:
> Oups messed up sorry -> resending my answer
>
> On Thu, Jun 4, 2009 at 11:52 AM, Brian Sutherland
> <brian at vanguardistas.net> wrote:
>>> ? - http://svn.python.org/projects/peps/trunk/pep-0386.txt
>>
>> ? ?... ?< V('1.0.dev456')
>> ? ?... ?< V('1.0')
>> ? ?... ?< V('1.0.dev456post623')
>>
>> Looks like a typo or very un-intuitive. It doesn't seem right that a
>> "dev" version sorts after a full release.
>
> This is a dev version of a post-release version. Which is an edge case
> submitted by Phillip.
>
> How would you write it ?

    ...  < V('1.0.dev456')
    ...  < V('1.0.dev456post123')  # post-release of a dev version
    ...  < V('1.0')
    ...  < V('1.0.post456dev623')  # dev version of a post-release
    ...  < V('1.0.post456')            # post release

i.e. that a postfix of "dev" means "before" and a postfix of "post" means after.

>>
>>>
>>> - PEP 376 | status : waiting for Phillip complementary feedback (and
>>> anyone else of course)
>>
>>
>> I can imagine distutils uninstalling files previously installed by dpkg
>> as a shortcut to breaking a machine. Though I'm not sure what will
>> actually happen in practice.
>
> Distutils defines a standard for an EGG-INFO structure, and provides a
> API for the uninstallation that is more likely to be a reference implementation.
>
> Although, It is already providing an install feature.
>
> If uninstalling a package with Distutils, while it was installed by dpkg breaks,
>
> I can imagine that in the very same system, you can also break it
> if you *install* packages with Distutils, easy_install, pip because
> you shortcut dpkg
> as well.

Yep. Though I think that nowdays dpkg installs to a different directory than
Distutils' default. So users have to specify extra options to break
their systems.

> I think this is the responsability of such a system to make sure it handles
> all installation and uninstallation.

How should such a system stop distutils (acting as root) from doing an
uninstallation?

From ziade.tarek at gmail.com  Thu Jun  4 14:54:50 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 4 Jun 2009 14:54:50 +0200
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <a992920f0906040529r7de0ac5cgf2b8d738dbb3be5e@mail.gmail.com>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604095203.GB9596@Boo.lan>
	<94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
	<a992920f0906040529r7de0ac5cgf2b8d738dbb3be5e@mail.gmail.com>
Message-ID: <94bdd2610906040554l6ddc4444jc9c5d13cda1d54f9@mail.gmail.com>

Paul
> I'd say that it's distutils' responsibility not to offer to uninstall
> anything it didn't install.

Brian
> Yep. Though I think that nowdays dpkg installs to a different directory than
> Distutils' default. So users have to specify extra options to break
> their systems.

But how does those third-party install the projects ?

Don't they use Distutils' install command under the hood ?

If so, the new PEP 376 APIs are ignoring previous versions of
egg-infos and work only with the ones that complies with the new
standard (it's a quick control right now,  look at the "is_egg_info"
API in http://bitbucket.org/tarek/pep376/src/tip/pkgutil.py)

So, in any case the uninstall command will not work with project that
are filtered out by is_egg_info().

So maybe we could add a "INSTALLER" file with a unique md5 key
provided by the project that installed the package,
and ask for the key when calling this API ? If not provided, it would
use Distutils's md5 key


Tarek

From david at ar.media.kyoto-u.ac.jp  Thu Jun  4 14:38:42 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Thu, 04 Jun 2009 21:38:42 +0900
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <a992920f0906040529r7de0ac5cgf2b8d738dbb3be5e@mail.gmail.com>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>	<20090604095203.GB9596@Boo.lan>	<94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
	<a992920f0906040529r7de0ac5cgf2b8d738dbb3be5e@mail.gmail.com>
Message-ID: <4A27C052.3040602@ar.media.kyoto-u.ac.jp>

Brian Sutherland wrote:
> 2009/6/4 Tarek Ziad? <ziade.tarek at gmail.com>:
>   
>> Oups messed up sorry -> resending my answer
>>
>> On Thu, Jun 4, 2009 at 11:52 AM, Brian Sutherland
>> <brian at vanguardistas.net> wrote:
>>     
>>>>   - http://svn.python.org/projects/peps/trunk/pep-0386.txt
>>>>         
>>>    ...  < V('1.0.dev456')
>>>    ...  < V('1.0')
>>>    ...  < V('1.0.dev456post623')
>>>
>>> Looks like a typo or very un-intuitive. It doesn't seem right that a
>>> "dev" version sorts after a full release.
>>>       
>> This is a dev version of a post-release version. Which is an edge case
>> submitted by Phillip.
>>
>> How would you write it ?
>>     
>
>     ...  < V('1.0.dev456')
>     ...  < V('1.0.dev456post123')  # post-release of a dev version
>     ...  < V('1.0')
>     ...  < V('1.0.post456dev623')  # dev version of a post-release
>     ...  < V('1.0.post456')            # post release
>
> i.e. that a postfix of "dev" means "before" and a postfix of "post" means after.
>
>   
>>>> - PEP 376 | status : waiting for Phillip complementary feedback (and
>>>> anyone else of course)
>>>>         
>>> I can imagine distutils uninstalling files previously installed by dpkg
>>> as a shortcut to breaking a machine. Though I'm not sure what will
>>> actually happen in practice.
>>>       
>> Distutils defines a standard for an EGG-INFO structure, and provides a
>> API for the uninstallation that is more likely to be a reference implementation.
>>
>> Although, It is already providing an install feature.
>>
>> If uninstalling a package with Distutils, while it was installed by dpkg breaks,
>>
>> I can imagine that in the very same system, you can also break it
>> if you *install* packages with Distutils, easy_install, pip because
>> you shortcut dpkg
>> as well.
>>     
>
> Yep. Though I think that nowdays dpkg installs to a different directory than
> Distutils' default. So users have to specify extra options to break
> their systems.
>   

That's unfortunately not true: by default (wo any --prefix option),
distutils install into /usr. So if a user just uses sudo with a
distutils-based package, he is very likely to mess up his system.

I think this should be considered as a distutils bug - by default, most
source distributions install into /usr/local (autoconf-based projects
certainly do).

> How should such a system stop distutils (acting as root) from doing an
> uninstallation?
>   

Ideally, distutils should detect whether it installed the package itself
or not.

IMHO, uninstall is beyond the scope of distutils; it is very difficult
to get right, with so many ways to screw up a system badly.

cheers,

David

From ziade.tarek at gmail.com  Thu Jun  4 15:02:10 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 4 Jun 2009 15:02:10 +0200
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <20090604122635.GB9784@laurie.devork>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604095203.GB9596@Boo.lan>
	<94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
	<20090604122635.GB9784@laurie.devork>
Message-ID: <94bdd2610906040602v7de7c7e6q77eb47842af58a0b@mail.gmail.com>

On Thu, Jun 4, 2009 at 2:26 PM, Floris Bruynooghe
<floris.bruynooghe at gmail.com> wrote:
>>
>> This is a dev version of a post-release version. Which is an edge case
>> submitted by Phillip.
>>
>> How would you write it ?
>
> 1.0.post623dev456 is what feels intuitive to me, here's my version of
> the last few lines:
>
> ... ?< V('1.0')
> ... ?< V('1.0.post456dev623')
> ... ?< V('1.0.post456'))
>
> It would even feel more consistent if it was 1.0.port456.dev623

Sounds reasonable, I'll try something in the prototype, and submit it here

>
>
> Frankly I don't really understand the post-release requirement (and
> the PEP text doesn't help me out there, no explenation of what it is).

Being able to create development releases versions before a post-release
Phillip Eby came with this use case, you should find the original
message in the ML somewhere.

I'll try to make it clearer in the PEP.  Notice that I never had this
use case myself,


Tarek

-- 
Tarek Ziad? | http://ziade.org

From brian at vanguardistas.net  Thu Jun  4 15:23:14 2009
From: brian at vanguardistas.net (Brian Sutherland)
Date: Thu, 4 Jun 2009 15:23:14 +0200
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <94bdd2610906040554l6ddc4444jc9c5d13cda1d54f9@mail.gmail.com>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604095203.GB9596@Boo.lan>
	<94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
	<a992920f0906040529r7de0ac5cgf2b8d738dbb3be5e@mail.gmail.com>
	<94bdd2610906040554l6ddc4444jc9c5d13cda1d54f9@mail.gmail.com>
Message-ID: <a992920f0906040623j6dfd7e85n55166545f96cdbc@mail.gmail.com>

2009/6/4 Tarek Ziad? <ziade.tarek at gmail.com>:
> Paul
>> I'd say that it's distutils' responsibility not to offer to uninstall
>> anything it didn't install.
>
> Brian
>> Yep. Though I think that nowdays dpkg installs to a different directory than
>> Distutils' default. So users have to specify extra options to break
>> their systems.
>
> But how does those third-party install the projects ?
>
> Don't they use Distutils' install command under the hood ?

Yes, most debian packages do, during the build process.

However, the actual install/uninstall on the user's machine is handled by dpkg
(which maintains it's own database of installed files).

> If so, the new PEP 376 APIs are ignoring previous versions of
> egg-infos and work only with the ones that complies with the new
> standard (it's a quick control right now, ?look at the "is_egg_info"
> API in http://bitbucket.org/tarek/pep376/src/tip/pkgutil.py)
>
> So, in any case the uninstall command will not work with project that
> are filtered out by is_egg_info().
>
> So maybe we could add a "INSTALLER" file with a unique md5 key
> provided by the project that installed the package,
> and ask for the key when calling this API ? If not provided, it would
> use Distutils's md5 key

So, with setuptools I was running this while building a package:

    python2.X setup,py install --single-version-externally-managed
--root=debian/$(package) --install-data=usr/lib/$(package)

So, now I would need to run this:

    python2.X setup,py install --root=debian/$(package)
--install-data=usr/lib/$(package) --installer
7e2fccc88b1f74aeee3d37340e8183ed

And that would cause distutils to refuse to uninstall that package by default?

>
>
> Tarek
>

From ziade.tarek at gmail.com  Thu Jun  4 15:23:49 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 4 Jun 2009 15:23:49 +0200
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <4A27C052.3040602@ar.media.kyoto-u.ac.jp>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604095203.GB9596@Boo.lan>
	<94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
	<a992920f0906040529r7de0ac5cgf2b8d738dbb3be5e@mail.gmail.com>
	<4A27C052.3040602@ar.media.kyoto-u.ac.jp>
Message-ID: <94bdd2610906040623s57fb671dn29c8d776ff68cb7c@mail.gmail.com>

2009/6/4 David Cournapeau <david at ar.media.kyoto-u.ac.jp>:
> Ideally, distutils should detect whether it installed the package itself
> or not.

Yes, I think having a marker like I suggested some minutes ago, would help

>
> IMHO, uninstall is beyond the scope of distutils; it is very difficult
> to get right, with so many ways to screw up a system badly.

The idea of the uninstall API is to provide a reference implementation
that can be used
in package managers that will rely on the other APIs, rather than a
complete system.

For instance, Distutils doesn't handle dependencies
installation/uninstallation, while PEP 345 introduces them
in the metadata.

We said earlier this year that it would be better to let projects like
easy_install and al provide advanced features
for installation / uninstallation

From ziade.tarek at gmail.com  Thu Jun  4 15:29:07 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 4 Jun 2009 15:29:07 +0200
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <a992920f0906040623j6dfd7e85n55166545f96cdbc@mail.gmail.com>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604095203.GB9596@Boo.lan>
	<94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
	<a992920f0906040529r7de0ac5cgf2b8d738dbb3be5e@mail.gmail.com>
	<94bdd2610906040554l6ddc4444jc9c5d13cda1d54f9@mail.gmail.com>
	<a992920f0906040623j6dfd7e85n55166545f96cdbc@mail.gmail.com>
Message-ID: <94bdd2610906040629y6316f7b6t6c623b66bca237b8@mail.gmail.com>

On Thu, Jun 4, 2009 at 3:23 PM, Brian Sutherland
<brian at vanguardistas.net> wrote:
>
> So, with setuptools I was running this while building a package:
>
> ? ?python2.X setup,py install --single-version-externally-managed
> --root=debian/$(package) --install-data=usr/lib/$(package)
>
> So, now I would need to run this:
>
> ? ?python2.X setup,py install --root=debian/$(package)
> --install-data=usr/lib/$(package) --installer
> 7e2fccc88b1f74aeee3d37340e8183ed
>
> And that would cause distutils to refuse to uninstall that package by default?

something like that,

but it would be even simpler :  the installer option would be
automatically added by setuptools
when the 'install' command is called, with setuptools md5 marker.
(since setuptools has its own install command)

That said, maybe setuptools would decide to rely on distutils for
uninstallation if its installation turns to be compatible
with distutils'one , and not provide its own marker when you manually
call 'install' like you showed.

And easy_install would have its own marker maybe, if the way it
installs stuff slighlty differs

From ziade.tarek at gmail.com  Thu Jun  4 15:34:01 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 4 Jun 2009 15:34:01 +0200
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <94bdd2610906040629y6316f7b6t6c623b66bca237b8@mail.gmail.com>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604095203.GB9596@Boo.lan>
	<94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
	<a992920f0906040529r7de0ac5cgf2b8d738dbb3be5e@mail.gmail.com>
	<94bdd2610906040554l6ddc4444jc9c5d13cda1d54f9@mail.gmail.com>
	<a992920f0906040623j6dfd7e85n55166545f96cdbc@mail.gmail.com>
	<94bdd2610906040629y6316f7b6t6c623b66bca237b8@mail.gmail.com>
Message-ID: <94bdd2610906040634w7666a60eq496915a3971a0e0a@mail.gmail.com>

On Thu, Jun 4, 2009 at 3:29 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> On Thu, Jun 4, 2009 at 3:23 PM, Brian Sutherland
> <brian at vanguardistas.net> wrote:
>>
>> So, with setuptools I was running this while building a package:
>>
>> ? ?python2.X setup,py install --single-version-externally-managed
>> --root=debian/$(package) --install-data=usr/lib/$(package)
>>
>> So, now I would need to run this:
>>
>> ? ?python2.X setup,py install --root=debian/$(package)
>> --install-data=usr/lib/$(package) --installer
>> 7e2fccc88b1f74aeee3d37340e8183ed

Sorry, maybe I misunderstood your example. (if you see setuptools as a
distutils equivalent)

If you use setuptools to do your own 'manual' installer (by doing
debian-specific stuff), then
yes, you could force the --installer option to avoid
distutils/setuptools to remove it

My previous explanation is still OK though

From brian at vanguardistas.net  Thu Jun  4 15:41:32 2009
From: brian at vanguardistas.net (Brian Sutherland)
Date: Thu, 4 Jun 2009 15:41:32 +0200
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <94bdd2610906040629y6316f7b6t6c623b66bca237b8@mail.gmail.com>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604095203.GB9596@Boo.lan>
	<94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
	<a992920f0906040529r7de0ac5cgf2b8d738dbb3be5e@mail.gmail.com>
	<94bdd2610906040554l6ddc4444jc9c5d13cda1d54f9@mail.gmail.com>
	<a992920f0906040623j6dfd7e85n55166545f96cdbc@mail.gmail.com>
	<94bdd2610906040629y6316f7b6t6c623b66bca237b8@mail.gmail.com>
Message-ID: <a992920f0906040641w37a34263k3ddeb5939e3748d9@mail.gmail.com>

2009/6/4 Tarek Ziad? <ziade.tarek at gmail.com>:
> On Thu, Jun 4, 2009 at 3:23 PM, Brian Sutherland
> <brian at vanguardistas.net> wrote:
>>
>> So, with setuptools I was running this while building a package:
>>
>> ? ?python2.X setup,py install --single-version-externally-managed
>> --root=debian/$(package) --install-data=usr/lib/$(package)
>>
>> So, now I would need to run this:
>>
>> ? ?python2.X setup,py install --root=debian/$(package)
>> --install-data=usr/lib/$(package) --installer
>> 7e2fccc88b1f74aeee3d37340e8183ed
>>
>> And that would cause distutils to refuse to uninstall that package by default?
>
> something like that,
>
> but it would be even simpler : ?the installer option would be
> automatically added by setuptools
> when the 'install' command is called, with setuptools md5 marker.
> (since setuptools has its own install command)
>
> That said, maybe setuptools would decide to rely on distutils for
> uninstallation if its installation turns to be compatible
> with distutils'one , and not provide its own marker when you manually
> call 'install' like you showed.

Ok, so then in this situation the --single-version-externally-managed
setuptools option could tell distutils to use a non-distutils marker
while the default distutils marker is used when this option is not
present.

> And easy_install would have its own marker maybe, if the way it
> installs stuff slighlty differs

ok, seems reasonable:)

Perhaps the --installer option take a string (like "dpkg") and convert
that to an md5 sum?

From ziade.tarek at gmail.com  Thu Jun  4 15:50:24 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 4 Jun 2009 15:50:24 +0200
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <a992920f0906040641w37a34263k3ddeb5939e3748d9@mail.gmail.com>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604095203.GB9596@Boo.lan>
	<94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
	<a992920f0906040529r7de0ac5cgf2b8d738dbb3be5e@mail.gmail.com>
	<94bdd2610906040554l6ddc4444jc9c5d13cda1d54f9@mail.gmail.com>
	<a992920f0906040623j6dfd7e85n55166545f96cdbc@mail.gmail.com>
	<94bdd2610906040629y6316f7b6t6c623b66bca237b8@mail.gmail.com>
	<a992920f0906040641w37a34263k3ddeb5939e3748d9@mail.gmail.com>
Message-ID: <94bdd2610906040650h61747e62qc36257a9142b192d@mail.gmail.com>

On Thu, Jun 4, 2009 at 3:41 PM, Brian Sutherland
<brian at vanguardistas.net> wrote:
>> And easy_install would have its own marker maybe, if the way it
>> installs stuff slighlty differs
>
> ok, seems reasonable:)

I'll work on that. I guess it's time to code the uninstall prototype,

>
> Perhaps the --installer option take a string (like "dpkg") and convert
> that to an md5 sum?
>

Mmm, or maybe drop the md5 completely, and go for a human readable name.

(distutils, dpkg, etc..)

The api would raise an error if the name is not in [A-Za-z0-9_\-]

From floris.bruynooghe at gmail.com  Thu Jun  4 15:51:00 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Thu, 4 Jun 2009 14:51:00 +0100
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <79990c6b0906040523w4cfec90l7617f23940c4d14e@mail.gmail.com>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604095203.GB9596@Boo.lan>
	<94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
	<79990c6b0906040523w4cfec90l7617f23940c4d14e@mail.gmail.com>
Message-ID: <20090604135100.GA10785@laurie.devork>

On Thu, Jun 04, 2009 at 01:23:21PM +0100, Paul Moore wrote:
> 2009/6/4 Tarek Ziad? <ziade.tarek at gmail.com>:
> >>> - PEP 376 | status : waiting for Phillip complementary feedback (and
> >>> anyone else of course)
> >>
> >>
> >> I can imagine distutils uninstalling files previously installed by dpkg
> >> as a shortcut to breaking a machine. Though I'm not sure what will
> >> actually happen in practice.
> >
> > Distutils defines a standard for an EGG-INFO structure, and provides a
> > API for the uninstallation that is more likely to be a reference implementation.
> >
> > Although, It is already providing an install feature.
> >
> > If uninstalling a package with Distutils, while it was installed by dpkg breaks,
> >
> > I can imagine that in the very same system, you can also break it
> > if you *install* packages with Distutils, easy_install, pip because
> > you shortcut dpkg
> > as well.

[...]

> I'd say that it's distutils' responsibility not to offer to uninstall
> anything it didn't install.

So then it's up the the system to stop distutils from removing stuff
it installed, i.e. don't install the RECORD files.  No?


Regards
Floris


-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From brian at vanguardistas.net  Thu Jun  4 15:58:51 2009
From: brian at vanguardistas.net (Brian Sutherland)
Date: Thu, 4 Jun 2009 15:58:51 +0200
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <94bdd2610906040650h61747e62qc36257a9142b192d@mail.gmail.com>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604095203.GB9596@Boo.lan>
	<94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
	<a992920f0906040529r7de0ac5cgf2b8d738dbb3be5e@mail.gmail.com>
	<94bdd2610906040554l6ddc4444jc9c5d13cda1d54f9@mail.gmail.com>
	<a992920f0906040623j6dfd7e85n55166545f96cdbc@mail.gmail.com>
	<94bdd2610906040629y6316f7b6t6c623b66bca237b8@mail.gmail.com>
	<a992920f0906040641w37a34263k3ddeb5939e3748d9@mail.gmail.com>
	<94bdd2610906040650h61747e62qc36257a9142b192d@mail.gmail.com>
Message-ID: <a992920f0906040658h892341ao808196ddc74b5975@mail.gmail.com>

2009/6/4 Tarek Ziad? <ziade.tarek at gmail.com>:
> On Thu, Jun 4, 2009 at 3:41 PM, Brian Sutherland
> <brian at vanguardistas.net> wrote:
>>> And easy_install would have its own marker maybe, if the way it
>>> installs stuff slighlty differs
>>
>> ok, seems reasonable:)
>
> I'll work on that. I guess it's time to code the uninstall prototype,
>
>>
>> Perhaps the --installer option take a string (like "dpkg") and convert
>> that to an md5 sum?
>>
>
> Mmm, or maybe drop the md5 completely, and go for a human readable name.
>
> (distutils, dpkg, etc..)
>
> The api would raise an error if the name is not in [A-Za-z0-9_\-]

I just remembered that setuptools implicitly sets
--single-version-externally-managed when the --root option is set.
Unfortunately my packaging-fu is not good enough to tell if it's an
option in this case as well, i.e. to set --installer to "external".

http://peak.telecommunity.com/DevCenter/setuptools#install-run-easy-install-or-old-style-installation

From p.f.moore at gmail.com  Thu Jun  4 15:59:59 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 4 Jun 2009 14:59:59 +0100
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <94bdd2610906040554l6ddc4444jc9c5d13cda1d54f9@mail.gmail.com>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604095203.GB9596@Boo.lan>
	<94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
	<a992920f0906040529r7de0ac5cgf2b8d738dbb3be5e@mail.gmail.com>
	<94bdd2610906040554l6ddc4444jc9c5d13cda1d54f9@mail.gmail.com>
Message-ID: <79990c6b0906040659l3ff0f394h51d05414d09e2978@mail.gmail.com>

2009/6/4 Tarek Ziad? <ziade.tarek at gmail.com>:
> Paul
>> I'd say that it's distutils' responsibility not to offer to uninstall
>> anything it didn't install.
>
> Brian
>> Yep. Though I think that nowdays dpkg installs to a different directory than
>> Distutils' default. So users have to specify extra options to break
>> their systems.
>
> But how does those third-party install the projects ?
>
> Don't they use Distutils' install command under the hood ?

In the case of Windows, no. Maybe (I don't know) bdist_wininst uses
the install command to build the directory structure (under the build
directory) that is packed into a zip format as part of the final
installer EXE, but there's *other* files and data installed by the
bdist_wininst installer as well as the package data.

The installer generated by bdist_wininst contains an embedded zip of
the installed directory (minus compiled Python files - that's the
mistake eggs make, which cause them to be version-dependent for
pure-python files). The installer, when run, does the following:
- Pops up a GUI asking the user which system-installed Python
directory to install into (from the eligible ones)
- Unzips the embedded directory structure into the appropriate site-packages
- Writes an install log packagename-wininst.log into sys.prefix
recording all of the following actions
- Writes a prebuilt executable, which is *also* embedded in the
installer, as Removepackagename.exe in sys.prefix
- Writes metadata into the registry to add the package to Add/Remove,
which causes Removepackagename.exe to run on uninstall.
- Compile all .py files to .pyc and .pyo

Removepackagename.exe does the following:
- Locate packagename-wininst.log based on the name and location of the
Removepackagename.exe executable
- Remove files and registry entries based on the .log file
- Remove the .log file, and the uninstaller itself

(Note - the above is from memory and how I'd expect things to work.
I'd recommend reading the code if the details matter).

Also, you need to remember that bdist_wininst is not a "third-party"
tool. It's part of distutils (although it's probably written in such a
way that it could easily be extracted as a 3rd party addin - I
honestly don't know).

I'd seriously recommend looking at bdist_wininst as an example - it's
distributed with Python, so the source is easy to find, but it
installs extra system-level metadata above and beyond what setup.py
install does, so you can see what sorts of issues you have to deal
with. (Some of the Unix-level packagers like bdist_rpm may be equally
illuminating, but I've no experience with those).

Note - my understanding is that eggs and easy_install essentially
automate setup.py install (with the only relevant metadata to maintain
being the .pth files) so if your experience is limited to eggs you may
be missing the point of what people are saying here. If my attempts to
explain are unclear, feel free to ask more questions!

Paul.

From ziade.tarek at gmail.com  Thu Jun  4 16:15:11 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 4 Jun 2009 16:15:11 +0200
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <79990c6b0906040659l3ff0f394h51d05414d09e2978@mail.gmail.com>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604095203.GB9596@Boo.lan>
	<94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
	<a992920f0906040529r7de0ac5cgf2b8d738dbb3be5e@mail.gmail.com>
	<94bdd2610906040554l6ddc4444jc9c5d13cda1d54f9@mail.gmail.com>
	<79990c6b0906040659l3ff0f394h51d05414d09e2978@mail.gmail.com>
Message-ID: <94bdd2610906040715n15195ab7o2b34d59dfecde62c@mail.gmail.com>

On Thu, Jun 4, 2009 at 3:59 PM, Paul Moore <p.f.moore at gmail.com> wrote:
>
> Also, you need to remember that bdist_wininst is not a "third-party"
> tool. It's part of distutils (although it's probably written in such a
> way that it could easily be extracted as a 3rd party addin - I
> honestly don't know).

I think that would be great to externalize such command to decouple
its release cycles
from Python (As mentioned during the Langage summit)

The only requirement would be to find someone that would lead its development,

If I find a windows developer that wants to handle this  <hint> <hint>
, we could probably
create a separate project

>
> I'd seriously recommend looking at bdist_wininst as an example - it's
> distributed with Python, so the source is easy to find, but it
> installs extra system-level metadata above and beyond what setup.py
> install does, so you can see what sorts of issues you have to deal
> with. (Some of the Unix-level packagers like bdist_rpm may be equally
> illuminating, but I've no experience with those).

I will, thanks for the tip. I guess bdist_msi should be also our
target for this,

From p.f.moore at gmail.com  Thu Jun  4 16:27:03 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 4 Jun 2009 15:27:03 +0100
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <94bdd2610906040715n15195ab7o2b34d59dfecde62c@mail.gmail.com>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604095203.GB9596@Boo.lan>
	<94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
	<a992920f0906040529r7de0ac5cgf2b8d738dbb3be5e@mail.gmail.com>
	<94bdd2610906040554l6ddc4444jc9c5d13cda1d54f9@mail.gmail.com>
	<79990c6b0906040659l3ff0f394h51d05414d09e2978@mail.gmail.com>
	<94bdd2610906040715n15195ab7o2b34d59dfecde62c@mail.gmail.com>
Message-ID: <79990c6b0906040727t34bcccb8n2c50e0437234844a@mail.gmail.com>

2009/6/4 Tarek Ziad? <ziade.tarek at gmail.com>:
> On Thu, Jun 4, 2009 at 3:59 PM, Paul Moore <p.f.moore at gmail.com> wrote:
>>
>> Also, you need to remember that bdist_wininst is not a "third-party"
>> tool. It's part of distutils (although it's probably written in such a
>> way that it could easily be extracted as a 3rd party addin - I
>> honestly don't know).
>
> I think that would be great to externalize such command to decouple
> its release cycles
> from Python (As mentioned during the Langage summit)
>
> The only requirement would be to find someone that would lead its development,
>
> If I find a windows developer that wants to handle this ?<hint> <hint>
> , we could probably
> create a separate project

Not a chance :-) No way do I have sufficient time, I'm afraid. And
personally, I prefer it built in, so I'm ideologically opposed to the
idea as well :-)

But I'm happy to offer advice and information to anyone who needs to
know more about the way the Windows system works.

>> I'd seriously recommend looking at bdist_wininst as an example - it's
>> distributed with Python, so the source is easy to find, but it
>> installs extra system-level metadata above and beyond what setup.py
>> install does, so you can see what sorts of issues you have to deal
>> with. (Some of the Unix-level packagers like bdist_rpm may be equally
>> illuminating, but I've no experience with those).
>
> I will, thanks for the tip. I guess bdist_msi should be also our
> target for this,

Yes, but bdist_wininst is likely to be a *lot* easier to follow,
because (as far as I understand it) bdist_msi builds a blob of
structured binary data (it's essentially a custom form of database
packed in one file) which is read and parsed by the Windows installer
service. So a lot of the mechanisms are hidden, compared to
bdist_wininst which is a "roll your own" approach to setting up the
required metadata.

Paul.

From brian at vanguardistas.net  Thu Jun  4 16:30:48 2009
From: brian at vanguardistas.net (Brian Sutherland)
Date: Thu, 4 Jun 2009 16:30:48 +0200
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <4A27C052.3040602@ar.media.kyoto-u.ac.jp>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604095203.GB9596@Boo.lan>
	<94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
	<a992920f0906040529r7de0ac5cgf2b8d738dbb3be5e@mail.gmail.com>
	<4A27C052.3040602@ar.media.kyoto-u.ac.jp>
Message-ID: <a992920f0906040730p1d60770fpe1f78c51219c36ec@mail.gmail.com>

2009/6/4 David Cournapeau <david at ar.media.kyoto-u.ac.jp>:
> Brian Sutherland wrote:
>> Yep. Though I think that nowdays dpkg installs to a different directory than
>> Distutils' default. So users have to specify extra options to break
>> their systems.
>>
>
> That's unfortunately not true: by default (wo any --prefix option),
> distutils install into /usr. So if a user just uses sudo with a
> distutils-based package, he is very likely to mess up his system.

I believe that changes from 2.6 onwards:

     https://lists.ubuntu.com/archives/ubuntu-devel/2009-February/027439.html

From pje at telecommunity.com  Thu Jun  4 18:42:59 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 04 Jun 2009 12:42:59 -0400
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.co
 m>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
Message-ID: <20090604164013.F2D093A405F@sparrow.telecommunity.com>

At 11:17 AM 6/4/2009 +0200, Tarek Ziad? wrote:
>- new PEP 386 | waiting for your feedback
>
>   - http://svn.python.org/projects/peps/trunk/pep-0386.txt


 From the PEP:
>Last ".dev456post623" is a development version of a post-release

This appears incorrect to me; it should be a post-release of a 
development version, analagous to ".dev456-r623" in a setuptools version.

I think it would also be helpful to note that one of the new scheme's 
design goals is to be a strict subset of the setuptools version 
algorithm; you sort of implied it in a couple of places (talking 
about the widespread use, and the problem of interpreting such a wide 
variety of schemes), but it might be good to point that out, so it's 
clear that adopting the scheme for your package will not require 
opting out of the setuptools-based infrastructure or depriving anyone 
of their preferred tools.

(In that vein, it also might be helpful to suggest that people try 
using parse_version() comparisons to verify that their new numbers 
are all > their old numbers, if they plan to change schemes.  Despite 
the strict-subset property, there might be problems in the case where 
they are not currently using an equivalent subset.)


>- PEP 376 | status : waiting for Phillip complementary feedback (and
>anyone else of course)
>
>    - up-to-date PEP proposal :
>http://bitbucket.org/tarek/pep376/src/tip/docs/pep-0376
>    - up-to-date prototype code for pkgutil :
>http://bitbucket.org/tarek/pep376/src/tip/pkgutil.py

I'll try to look at this again soon, probably the weekend.


From trentm at gmail.com  Thu Jun  4 18:41:07 2009
From: trentm at gmail.com (Trent Mick)
Date: Thu, 4 Jun 2009 09:41:07 -0700
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <94bdd2610906040602v7de7c7e6q77eb47842af58a0b@mail.gmail.com>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604095203.GB9596@Boo.lan>
	<94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
	<20090604122635.GB9784@laurie.devork>
	<94bdd2610906040602v7de7c7e6q77eb47842af58a0b@mail.gmail.com>
Message-ID: <6db0ea510906040941u738e4a7cj157d940d4da200d2@mail.gmail.com>

On Thu, Jun 4, 2009 at 6:02 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

> On Thu, Jun 4, 2009 at 2:26 PM, Floris Bruynooghe
> <floris.bruynooghe at gmail.com> wrote:
> >>
> >> This is a dev version of a post-release version. Which is an edge case
> >> submitted by Phillip.
> >>
> >> How would you write it ?
> >
> > 1.0.post623dev456 is what feels intuitive to me, here's my version of
> > the last few lines:
> >
> > ...  < V('1.0')
> > ...  < V('1.0.post456dev623')
> > ...  < V('1.0.post456'))
> >
> > It would even feel more consistent if it was 1.0.port456.dev623
>
> Sounds reasonable, I'll try something in the prototype, and submit it here
>

I don't want to whack a dead horse, but when Philip brought up the "dev
release of a post release" I asked for a concrete real-world example. I
didn't get a response to that email. Do other people see a need for this?
My understanding from the PyCon discussions was that the "postNNN" marker
was the equivalent of the "devNNN" marker, but for projects that don't
immediately increment the leading version numbers after a release.

Can people point to some examples of projects using post-release tags, and
that would require the use of a "dev release of a post release"? If there
are real examples, those would also help in explaining the motivation for
this in the versioning scheme in the PEP.

Trent



-- 
Trent Mick
trentm at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090604/a9873dba/attachment.htm>

From pje at telecommunity.com  Thu Jun  4 18:46:03 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 04 Jun 2009 12:46:03 -0400
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.co
 m>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604095203.GB9596@Boo.lan>
	<94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
Message-ID: <20090604164313.9957E3A405F@sparrow.telecommunity.com>

At 12:57 PM 6/4/2009 +0200, Tarek Ziad? wrote:
>On Thu, Jun 4, 2009 at 11:52 AM, Brian Sutherland
><brian at vanguardistas.net> wrote:
> > I can imagine distutils uninstalling files previously installed by dpkg
> > as a shortcut to breaking a machine. Though I'm not sure what will
> > actually happen in practice.
>
>Distutils defines a standard for an EGG-INFO structure, and provides a
>API for the uninstallation that is more likely to be a reference 
>implementation.

There probably needs to be an install option that prevents the file 
record from being written, and this option should be used when 
creating system packages.

3rd-party installation tools also should not overwrite or delete 
files that are not in a RECORD file without prompting or warning.


From trentm at gmail.com  Thu Jun  4 18:44:55 2009
From: trentm at gmail.com (Trent Mick)
Date: Thu, 4 Jun 2009 09:44:55 -0700
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <20090604164013.F2D093A405F@sparrow.telecommunity.com>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604164013.F2D093A405F@sparrow.telecommunity.com>
Message-ID: <6db0ea510906040944m66375278oa680e1f37325a701@mail.gmail.com>

2009/6/4 P.J. Eby <pje at telecommunity.com>

> At 11:17 AM 6/4/2009 +0200, Tarek Ziad? wrote:
>
>> - new PEP 386 | waiting for your feedback
>>
>>  - http://svn.python.org/projects/peps/trunk/pep-0386.txt
>>
>
>
> From the PEP:
>
>> Last ".dev456post623" is a development version of a post-release
>>
>
> This appears incorrect to me; it should be a post-release of a development
> version, analagous to ".dev456-r623" in a setuptools version.
>

In the Pycon discussions the ".devNNN" was meant to *be* the "rNNN" part. A
commonly expected value for NNN would be the source code control revision
number. At least that was my understanding.

Do you know of examples using a ".dev456-r623"-type scheme?

 Trent

-- 
Trent Mick
trentm at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090604/6a922f3e/attachment-0001.htm>

From pje at telecommunity.com  Thu Jun  4 18:47:56 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 04 Jun 2009 12:47:56 -0400
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <79990c6b0906040523w4cfec90l7617f23940c4d14e@mail.gmail.com
 >
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604095203.GB9596@Boo.lan>
	<94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
	<79990c6b0906040523w4cfec90l7617f23940c4d14e@mail.gmail.com>
Message-ID: <20090604164505.CDD9F3A4109@sparrow.telecommunity.com>

At 01:23 PM 6/4/2009 +0100, Paul Moore wrote:
>With Windows, if you install using bdist_wininst and then uninstall
>using the (currently nonexistent) distutils uninstall, I'd expect that
>it wouldn't remove the Add/Remove programs support items in the
>registry, and the Removexxx.exe and xxx-wininst.log files installed by
>bdist_wininst. The system would then still think that the package was
>installed, even though it isn't. (And the system uninstall may break -
>I'm not sure).

A bdist_wininst should skip the RECORD file in that case.  The RECORD 
file should still be *in* the installer zip, though, in case someone 
unpacks it manually.

(Alternatively, perhaps there should be some way to record who/what 
installed the package, and how it should be uninstalled.)



From pje at telecommunity.com  Thu Jun  4 18:53:38 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 04 Jun 2009 12:53:38 -0400
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <94bdd2610906040602v7de7c7e6q77eb47842af58a0b@mail.gmail.co
 m>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604095203.GB9596@Boo.lan>
	<94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
	<20090604122635.GB9784@laurie.devork>
	<94bdd2610906040602v7de7c7e6q77eb47842af58a0b@mail.gmail.com>
Message-ID: <20090604165048.E2CCE3A405F@sparrow.telecommunity.com>

At 03:02 PM 6/4/2009 +0200, Tarek Ziad? wrote:
>On Thu, Jun 4, 2009 at 2:26 PM, Floris Bruynooghe
><floris.bruynooghe at gmail.com> wrote:
> >>
> >> This is a dev version of a post-release version. Which is an edge case
> >> submitted by Phillip.
> >>
> >> How would you write it ?
> >
> > 1.0.post623dev456 is what feels intuitive to me, here's my version of
> > the last few lines:
> >
> > ...  < V('1.0')
> > ...  < V('1.0.post456dev623')
> > ...  < V('1.0.post456'))
> >
> > It would even feel more consistent if it was 1.0.port456.dev623
>
>Sounds reasonable, I'll try something in the prototype, and submit it here
>
> >
> >
> > Frankly I don't really understand the post-release requirement (and
> > the PEP text doesn't help me out there, no explenation of what it is).
>
>Being able to create development releases versions before a post-release
>Phillip Eby came with this use case, you should find the original
>message in the ML somewhere.

Development before postrelease = "1.0post456dev623"
In-progress snapshot of development version = "1.0dev456post623"

The specific use case I was asking about, though, was 
"1.0a1dev-r623", meaning "SVN revision 623 of the development work on 
leading up to 1.0a1" -- and IIRC, in the RationalVersion cheme, this 
should probably just translate to "1.0.a1.dev623".

Hopefully, all these examples I've just given are correct and work 
with the versioning code, as that would still be a strict subset of setuptools.


From pje at telecommunity.com  Thu Jun  4 18:57:30 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 04 Jun 2009 12:57:30 -0400
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <6db0ea510906040941u738e4a7cj157d940d4da200d2@mail.gmail.co
 m>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604095203.GB9596@Boo.lan>
	<94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
	<20090604122635.GB9784@laurie.devork>
	<94bdd2610906040602v7de7c7e6q77eb47842af58a0b@mail.gmail.com>
	<6db0ea510906040941u738e4a7cj157d940d4da200d2@mail.gmail.com>
Message-ID: <20090604165440.BF28F3A405F@sparrow.telecommunity.com>

At 09:41 AM 6/4/2009 -0700, Trent Mick wrote:
>Can people point to some examples of projects using post-release 
>tags, and that would require the use of a "dev release of a post release"?

Any project that uses a post-release tag and has multiple developers 
or multiple commits required to create that post-release version, 
will need the dev tagging to distinguish installed/development 
versions from each other, even if these intermediate versions are 
never "released" to PyPI.  People working from checkouts will need 
them to keep their install/build tools happily humming along, able to 
tell which intermediate checkout version they're workign with and 
whether they are up-to-date.


From trentm at gmail.com  Thu Jun  4 19:18:13 2009
From: trentm at gmail.com (Trent Mick)
Date: Thu, 4 Jun 2009 10:18:13 -0700
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <20090604165048.E2CCE3A405F@sparrow.telecommunity.com>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604095203.GB9596@Boo.lan>
	<94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
	<20090604122635.GB9784@laurie.devork>
	<94bdd2610906040602v7de7c7e6q77eb47842af58a0b@mail.gmail.com>
	<20090604165048.E2CCE3A405F@sparrow.telecommunity.com>
Message-ID: <6db0ea510906041018h594709c6p95b37888c055b4b@mail.gmail.com>

2009/6/4 P.J. Eby <pje at telecommunity.com>

>
> The specific use case I was asking about, though, was "1.0a1dev-r623",
> meaning "SVN revision 623 of the development work on leading up to 1.0a1" --
> and IIRC, in the RationalVersion cheme, this should probably just translate
> to "1.0.a1.dev623".
>

That is correct.

Trent

-- 
Trent Mick
trentm at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090604/f4302d6e/attachment.htm>

From trentm at gmail.com  Thu Jun  4 19:44:27 2009
From: trentm at gmail.com (Trent Mick)
Date: Thu, 4 Jun 2009 10:44:27 -0700
Subject: [Distutils] post-release tags and RationalVersion (Was: PEP 345,
	PEP 376, PEP 	386)
Message-ID: <6db0ea510906041044n3062a809kb80a0283e3e0bbab@mail.gmail.com>

2009/6/4 P.J. Eby <pje at telecommunity.com>
>
> At 09:41 AM 6/4/2009 -0700, Trent Mick wrote:
>>
>> Can people point to some examples of projects using post-release tags, and that would require the use of a "dev release of a post release"?
>
> Any project that uses a post-release tag and has multiple developers or multiple commits required to create that post-release version, will need the dev tagging to distinguish installed/development versions from each other, even if these intermediate versions are never "released" to PyPI. ?People working from checkouts will need them to keep their install/build tools happily humming along, able to tell which intermediate checkout version they're workign with and whether they are up-to-date.
>

Are there any such projects that people can point to?

http://peak.telecommunity.com/DevCenter/setuptools says

> A post-release tag is either a series of letters that are alphabetically greater than or equal to "final", or a dash (-). Post-release tags are generally used to separate patch numbers, port numbers, build numbers, revision numbers, or date stamps from the release number. For example, the version 2.4-r1263 might denote Subversion revision 1263 of a post-release patch of version 2.4. Or you might use 2.4-20051127 to denote a date-stamped post-release.
>


I wonder if there is too much potential for confusion here. The
"2.4-r1263" example: I would interpret that to be a *pre*-release of
version 2.4. However verlib.py's "suggest_rational_version" does
suggest "2.4.post1263" for this. I guess that is fine, because the
RationalVersion spelling (with the explicit "post") is more
self-explanatory.


Back to a "dev version of a post release", given the only examples I've seen:
- "2.4-r1263": given that the post-release is using the Subversion
revision number here, how can a "dev version" if this be meaningful?
- "2.4-20051127": A potential alternative to this would be to just
call it "2.4.20051127" (i.e. not a "post-release of 20051127" but a
"patch-level version of 20051127".

I guess where I'm going is: given that RationalVersion requires
post-release values to be numeric, it seems that a valid solution is
to just have an additional version element. So instead of "2.4-r123"
you use "2.4.123". Instead of "1.0c1-r456" you use "1.0c1.456".

Is there any usage of a post-release that doesn't fit in this scheme?

Is there a potential practical problem with getting users to switch to
that? E.g. Perhaps something with the setup.cfg config variables that
setuptools' versioning supports (I'm not that familiar with them)?

Trent

--
Trent Mick
trentm at gmail.com

From floris.bruynooghe at gmail.com  Thu Jun  4 20:23:36 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Thu, 4 Jun 2009 19:23:36 +0100
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <20090604164313.9957E3A405F@sparrow.telecommunity.com>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604095203.GB9596@Boo.lan>
	<94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
	<20090604164313.9957E3A405F@sparrow.telecommunity.com>
Message-ID: <20090604182336.GA12132@laurie.devork>

On Thu, Jun 04, 2009 at 12:46:03PM -0400, P.J. Eby wrote:
> At 12:57 PM 6/4/2009 +0200, Tarek Ziad? wrote:
>> On Thu, Jun 4, 2009 at 11:52 AM, Brian Sutherland
>> <brian at vanguardistas.net> wrote:
>> > I can imagine distutils uninstalling files previously installed by dpkg
>> > as a shortcut to breaking a machine. Though I'm not sure what will
>> > actually happen in practice.
>>
>> Distutils defines a standard for an EGG-INFO structure, and provides a
>> API for the uninstallation that is more likely to be a reference  
>> implementation.
>
> There probably needs to be an install option that prevents the file  
> record from being written, and this option should be used when creating 
> system packages.
>
> 3rd-party installation tools also should not overwrite or delete files 
> that are not in a RECORD file without prompting or warning.

+1 on both

From pje at telecommunity.com  Thu Jun  4 20:33:36 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 04 Jun 2009 14:33:36 -0400
Subject: [Distutils] post-release tags and RationalVersion (Was: PEP 345,
 PEP  376, PEP  386)
In-Reply-To: <6db0ea510906041044n3062a809kb80a0283e3e0bbab@mail.gmail.co
 m>
References: <6db0ea510906041044n3062a809kb80a0283e3e0bbab@mail.gmail.com>
Message-ID: <20090604183046.C53413A4109@sparrow.telecommunity.com>

At 10:44 AM 6/4/2009 -0700, Trent Mick wrote:
>Back to a "dev version of a post release", given the only examples I've seen:
>- "2.4-r1263": given that the post-release is using the Subversion
>revision number here, how can a "dev version" if this be meaningful?
>- "2.4-20051127": A potential alternative to this would be to just
>call it "2.4.20051127" (i.e. not a "post-release of 20051127" but a
>"patch-level version of 20051127".

In setuptools' case, the original intent was to be compatible with 
projects that do have things like '2.4-1' - e.g., because they're a 
wrapper of a library whose version is 2.4, and the wrapper is the 
first version of that.  If the library then releases a '2.4.1', the 
wrapper for that is then '2.4.1-1'.  However, some projects (see 
Jean-Paul Calderone's recent questions here) have things like '2.4+1' 
as a patch level.

Anyway, the point is that you can have a development of a patch 
level, and patch levels are considered distinct from dotted levels.


>I guess where I'm going is: given that RationalVersion requires
>post-release values to be numeric, it seems that a valid solution is
>to just have an additional version element. So instead of "2.4-r123"
>you use "2.4.123". Instead of "1.0c1-r456" you use "1.0c1.456".
>
>Is there any usage of a post-release that doesn't fit in this scheme?
>
>Is there a potential practical problem with getting users to switch to
>that? E.g. Perhaps something with the setup.cfg config variables that
>setuptools' versioning supports (I'm not that familiar with them)?

I think we're going to have to stage the implementation on the 
setuptools side, first by switching tagging functions to a 
plugin-based operation, and second to provide optional version 
conversion/rationalization.

I personally don't think it will ever make sense to *require* version 
rationalization, since Python is certainly used for private projects 
and companies may have their own numbering schemes.  At most, we can 
warn about versions being potentially not-parseable or being not 
suitable for PyPI distribution if at some point PyPI can reasonably 
become strict about the matter.


From trentm at gmail.com  Thu Jun  4 20:57:59 2009
From: trentm at gmail.com (Trent Mick)
Date: Thu, 4 Jun 2009 11:57:59 -0700
Subject: [Distutils] post-release tags and RationalVersion (Was: PEP 345,
	PEP 376, PEP 	386)
In-Reply-To: <20090604183046.C53413A4109@sparrow.telecommunity.com>
References: <6db0ea510906041044n3062a809kb80a0283e3e0bbab@mail.gmail.com>
	<20090604183046.C53413A4109@sparrow.telecommunity.com>
Message-ID: <6db0ea510906041157i2db44e82x2c39cd1a1a4e2b8b@mail.gmail.com>

2009/6/4 P.J. Eby <pje at telecommunity.com>:
> In setuptools' case, the original intent was to be compatible with projects
> that do have things like '2.4-1' - e.g., because they're a wrapper of a
> library whose version is 2.4, and the wrapper is the first version of that.
> ?If the library then releases a '2.4.1', the wrapper for that is then
> '2.4.1-1'. ?However, some projects (see Jean-Paul Calderone's recent
> questions here) have things like '2.4+1' as a patch level.

I *suppose*. :) Personally I just tack on an external field the main
version number for that. For example:
http://code.google.com/p/python-markdown2/source/browse/trunk/lib/markdown2.py#47
version 1.0.1.12 where the first three numbers are those of the Perl
Markdown.pl on which this is based. Granted that if Markdown.pl
1.0.1.1 came out it would be a little awkward, but that kind of thing
(where the wrapped library adds another version bit) would be rare,
I'd imagine.



Trent

-- 
Trent Mick
trentm at gmail.com

From david at ar.media.kyoto-u.ac.jp  Fri Jun  5 10:21:34 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Fri, 05 Jun 2009 17:21:34 +0900
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <94bdd2610906040623s57fb671dn29c8d776ff68cb7c@mail.gmail.com>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>	
	<20090604095203.GB9596@Boo.lan>	
	<94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>	
	<a992920f0906040529r7de0ac5cgf2b8d738dbb3be5e@mail.gmail.com>	
	<4A27C052.3040602@ar.media.kyoto-u.ac.jp>
	<94bdd2610906040623s57fb671dn29c8d776ff68cb7c@mail.gmail.com>
Message-ID: <4A28D58E.1050802@ar.media.kyoto-u.ac.jp>

Tarek Ziad? wrote:
>
> The idea of the uninstall API is to provide a reference implementation
> that can be used
> in package managers that will rely on the other APIs, rather than a
> complete system.
>   

For package managers which manage everything (pretty much every native
binary installer, be it msi/wininst/rpm/deb), uninstall feature is not
needed anyway. They handle all this by themselves once they know what
they install.

> For instance, Distutils doesn't handle dependencies
> installation/uninstallation, while PEP 345 introduces them
> in the metadata.
>   

I think dependencies is the easy part for uninstallation - it is a
non-issue IMHO (once a decision on how to deal with dependencies is done
:)). The hard problem is how to uninstall one package at the right
location (/usr vs $HOME vs something else on Unix, for example),
reliably. I am generally very much in favor of delegating many things to
3rd party, using metadata instead of tools, but I think
uninstall/registration of packages can only be implemented by python
itself reliably, if only because there can't be several systems at the
same time.

If it is handled by a 3rd party system, I see the following problems:
    - how to know exactly which package to remove (if I have several of
them installed) ?
    - how to get access to the list of installed packages
    - how to deal with doc, generated .pyc/.pyo, configuration files,
rollback, etc...

I don't see how this kind of infrastructure can be provided by a 3rd
party, unless it is guaranteed there will be only one.

David

From ben+python at benfinney.id.au  Fri Jun  5 11:19:39 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 05 Jun 2009 19:19:39 +1000
Subject: [Distutils] Comparison semantics for alphanumeric components of a
	version number (was: Version comparison - round 2)
References: <94bdd2610905290214g495a3ee8oc71710461b202f38@mail.gmail.com>
Message-ID: <87d49jkpg4.fsf@benfinney.id.au>

Tarek Ziad? <ziade.tarek at gmail.com> writes:

>   http://bitbucket.org/tarek/distutilsversion/src/tip/README.txt
> 
> The goal here is to provide a version comparison standard to be
> included in Distutils.

I laud this goal, and thank you for soliciting feedback on this draft.

One thing I haven't seen discussed: Why have such baroque version
comparison semantics been accepted? Surely one of the overarching goals
for such a standard is that it be simple to comprehend, and to behave
unsurprisingly to most users.

Yet the discussion around these non-obvious semantics, trying to have
components interpreted as ?pre-release? and ?post-release? and
?development release? and so on seem to underline the fact that
they're *not* something that there's any consensus on. So why are they
being foisted into a standard for version strings?

Rather than trying to force non-alphanumeric comparison semantics for
alphabetic sequences, why not simply say that alphanumeric comparison
semantics apply for components? That would, at a stroke, end all this
turmoil and IMO futile seeking of some other consensus, when the best
consensus is already what most would expect: alphanumeric comparison.

At the least, I would expect there would need to be a demonstrated,
significantly unified consensus for some specific *other* semantic
before discarding straightforward alphanumeric comparison as the
standard.

-- 
 \           ?Military justice is to justice what military music is to |
  `\                                             music.? ?Groucho Marx |
_o__)                                                                  |
Ben Finney


From p.f.moore at gmail.com  Fri Jun  5 14:43:16 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 5 Jun 2009 13:43:16 +0100
Subject: [Distutils] Comparison semantics for alphanumeric components of
	a version number (was: Version comparison - round 2)
In-Reply-To: <87d49jkpg4.fsf@benfinney.id.au>
References: <94bdd2610905290214g495a3ee8oc71710461b202f38@mail.gmail.com>
	<87d49jkpg4.fsf@benfinney.id.au>
Message-ID: <79990c6b0906050543o1052576eqeced8f1a0be1d2cf@mail.gmail.com>

2009/6/5 Ben Finney <ben+python at benfinney.id.au>:
> Tarek Ziad? <ziade.tarek at gmail.com> writes:
>
>> ? http://bitbucket.org/tarek/distutilsversion/src/tip/README.txt
>>
>> The goal here is to provide a version comparison standard to be
>> included in Distutils.
>
> I laud this goal, and thank you for soliciting feedback on this draft.
>
> One thing I haven't seen discussed: Why have such baroque version
> comparison semantics been accepted? Surely one of the overarching goals
> for such a standard is that it be simple to comprehend, and to behave
> unsurprisingly to most users.
>
> Yet the discussion around these non-obvious semantics, trying to have
> components interpreted as ?pre-release? and ?post-release? and
> ?development release? and so on seem to underline the fact that
> they're *not* something that there's any consensus on. So why are they
> being foisted into a standard for version strings?
>
> Rather than trying to force non-alphanumeric comparison semantics for
> alphabetic sequences, why not simply say that alphanumeric comparison
> semantics apply for components? That would, at a stroke, end all this
> turmoil and IMO futile seeking of some other consensus, when the best
> consensus is already what most would expect: alphanumeric comparison.
>
> At the least, I would expect there would need to be a demonstrated,
> significantly unified consensus for some specific *other* semantic
> before discarding straightforward alphanumeric comparison as the
> standard.

+1. The historical baggage of setuptools (which *had* to cater for
every bizarre convention currently in existence, as its goal was to
"embrace and extend" - something it achieved spectacularly) doesn't
seem to apply when creating a standard Python module.

I'd rather the PEP said "This is how version numbers work (simple,
non-controversial spec here), and this is the standard API for
manipulating them", and then projects that want to conform to the
standard migrate if needed.

But I acknowledge that I have no personal requirement for any of this,
so the only interest I have is an aesthetic one of *not* seeing
overcomplicated, difficult to understand, specifications become part
of the Python stdlib.

Paul.

From ziade.tarek at gmail.com  Fri Jun  5 15:07:31 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 5 Jun 2009 15:07:31 +0200
Subject: [Distutils] Comparison semantics for alphanumeric components of
	a version number (was: Version comparison - round 2)
In-Reply-To: <87d49jkpg4.fsf@benfinney.id.au>
References: <94bdd2610905290214g495a3ee8oc71710461b202f38@mail.gmail.com>
	<87d49jkpg4.fsf@benfinney.id.au>
Message-ID: <94bdd2610906050607x79ac6f7bk7ce898833451f2c4@mail.gmail.com>

On Fri, Jun 5, 2009 at 11:19 AM, Ben Finney<ben+python at benfinney.id.au> wrote:
>
> Yet the discussion around these non-obvious semantics, trying to have
> components interpreted as ?pre-release? and ?post-release? and
> ?development release? and so on seem to underline the fact that
> they're *not* something that there's any consensus on. So why are they
> being foisted into a standard for version strings?

There's a consensus on this in most packaging system out there, and the goal
is to have a rational version system that is understandable by most packagers
so they can work with python projects versions.

For instance, most packaging systems out there will reject your project if
you use "FooBar" as its version number,  then "ZooBar" for its second release.

> Rather than trying to force non-alphanumeric comparison semantics for
> alphabetic sequences, why not simply say that alphanumeric comparison
> semantics apply for components? That would, at a stroke, end all this
> turmoil and IMO futile seeking of some other consensus, when the best
> consensus is already what most would expect: alphanumeric comparison.

I don't think alphanumeric comparison is what most would expect.

For example if you use dates for your version, it'll work perfeclty
with alphanumeric
comparison but Fedora packagers will fail at sorting your versions properly.

Tarek

From ziade.tarek at gmail.com  Fri Jun  5 15:13:32 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 5 Jun 2009 15:13:32 +0200
Subject: [Distutils] Comparison semantics for alphanumeric components of
	a version number (was: Version comparison - round 2)
In-Reply-To: <79990c6b0906050543o1052576eqeced8f1a0be1d2cf@mail.gmail.com>
References: <94bdd2610905290214g495a3ee8oc71710461b202f38@mail.gmail.com>
	<87d49jkpg4.fsf@benfinney.id.au>
	<79990c6b0906050543o1052576eqeced8f1a0be1d2cf@mail.gmail.com>
Message-ID: <94bdd2610906050613t2f1535b4yf30fe95c3084ba16@mail.gmail.com>

On Fri, Jun 5, 2009 at 2:43 PM, Paul Moore<p.f.moore at gmail.com> wrote:
> I'd rather the PEP said "This is how version numbers work (simple,
> non-controversial spec here), and this is the standard API for
> manipulating them", and then projects that want to conform to the
> standard migrate if needed.

That's exactly the goal.

Maybe there's a missing introduction on how version numbers works
Not a standard at first, but the principles of releasing a dev
release, an alpha, and so on..


> But I acknowledge that I have no personal requirement for any of this,
> so the only interest I have is an aesthetic one of *not* seeing
> overcomplicated, difficult to understand, specifications become part
> of the Python stdlib.

Well if the specification is difficult to understand or overcomplicated
it'll fail for sure.

But so far, besides that very specific case for the post-release dev
tag, I don't find
it complicated at all. Another win I can see is that it will help developers use
better versions numbers for their projects imho.

Regards
Tarek

From p.f.moore at gmail.com  Fri Jun  5 15:37:50 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 5 Jun 2009 14:37:50 +0100
Subject: [Distutils] Comparison semantics for alphanumeric components of
	a version number (was: Version comparison - round 2)
In-Reply-To: <94bdd2610906050613t2f1535b4yf30fe95c3084ba16@mail.gmail.com>
References: <94bdd2610905290214g495a3ee8oc71710461b202f38@mail.gmail.com>
	<87d49jkpg4.fsf@benfinney.id.au>
	<79990c6b0906050543o1052576eqeced8f1a0be1d2cf@mail.gmail.com>
	<94bdd2610906050613t2f1535b4yf30fe95c3084ba16@mail.gmail.com>
Message-ID: <79990c6b0906050637n13384cebme27c753725260aa7@mail.gmail.com>

2009/6/5 Tarek Ziad? <ziade.tarek at gmail.com>:
>> But I acknowledge that I have no personal requirement for any of this,
>> so the only interest I have is an aesthetic one of *not* seeing
>> overcomplicated, difficult to understand, specifications become part
>> of the Python stdlib.
>
> Well if the specification is difficult to understand or overcomplicated
> it'll fail for sure.
>
> But so far, besides that very specific case for the post-release dev
> tag, I don't find
> it complicated at all. Another win I can see is that it will help developers use
> better versions numbers for their projects imho.

And yet, in spite of repeated requests for specific examples of
projects using this post-dev stuff, which haven't been forthcoming,
why is nobody saying "it's an unnecessary complication, we'll drop
it"?

OK, I'll say it: Having post and dev specifications is unnecessary.
Please remove it from the spec.

As an alternative, allow a trailing '-' followed by arbitrary
alphanumeric (a-zA-Z0-9) data. If this is all numeric, sort as
numbers, otherwise sort textually. Numeric vs non-numeric is sorted as
text. That covers

  -vendorid (as used by such things as cygwin, RPMs, etc - 1.2.1-3)
  -revision (e.g. subversion revision numbers 1.2-32471)
  -changeset (e.g. DVCS changeset id 1.3-0c30df5c527b)
  ... and anything else people want to invent (dates, developer's age, ...)

I repeat - I don't have any experience in this area. But surely the
people who do should be able to explain to me *why* they need
something more complex than this?

(Note: I'm assuming that the point of setting a standard is that
people move to it - arguments like "Project XXX uses version numbers
like so" don't count unless there's a justification why switching
version numbering format isn't an option when they are planning to
switch version parsing code in the first place!)

Or I'm completely confused. (But I don't seem to be the only one).

Paul.

From ben+python at benfinney.id.au  Fri Jun  5 15:41:14 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 05 Jun 2009 23:41:14 +1000
Subject: [Distutils] Comparison semantics for alphanumeric components of
	a version number
References: <94bdd2610905290214g495a3ee8oc71710461b202f38@mail.gmail.com>
	<87d49jkpg4.fsf@benfinney.id.au>
	<79990c6b0906050543o1052576eqeced8f1a0be1d2cf@mail.gmail.com>
	<94bdd2610906050613t2f1535b4yf30fe95c3084ba16@mail.gmail.com>
Message-ID: <87oct2kdc5.fsf@benfinney.id.au>

Tarek Ziad? <ziade.tarek at gmail.com> writes:

> On Fri, Jun 5, 2009 at 2:43 PM, Paul Moore<p.f.moore at gmail.com> wrote:
> > I'd rather the PEP said "This is how version numbers work (simple,
> > non-controversial spec here), and this is the standard API for
> > manipulating them", and then projects that want to conform to the
> > standard migrate if needed.
> 
> That's exactly the goal.

If so, that goal is being missed with all the attempts to wedge in
complicated semantics for pre-, post-, ?devpost?, and so on. Without
all that special treatment requiring long explanation, the specification
becomes much more simple, and the comparisons obvious:

    N.N[.X]+

where ?N? is the set [0-9], and ?X? is the set [0-9A-Za-z].

    >>> from verlib import RationalVersion as V
    >>> (V('1.0')
    ...  < V('1.0.a1')
    ...  < V('1.0.a2')
    ...  < V('1.0.a2.1')
    ...  < V('1.0.b2')
    ...  < V('1.0.c1')
    ...  < V('1.0.dev456post623')
    ...  < V('1.0.post456'))
    True

Are there many packages out there that don't follow this scheme? Of
course; that's trivially true whatever scheme is picked, which is
precisely the problem being addressed by choosing a simple, obvious
standard.

Choose anything *but* simple alphanumeric comparison for each component,
and you've expressly chosen something less obvious and more complicated
to describe.

To depart from simplicity of specification and obvious semantics, there
has to be something pretty big to gain ? big enough to convince the
mass of packages out there that it's worth switching from a home-brew
version scheme to something still complicated and non-obvious. What is
that? I haven't seen it demonstrated, hence this thread.

> Well if the specification is difficult to understand or
> overcomplicated it'll fail for sure.

Not only that, but I think the level of acceptance will be inversely
proportional to how non-obvious and complicated the specification is.

> But so far, besides that very specific case for the post-release dev
> tag, I don't find it complicated at all.

Then why not drop that part and stick to simple alphanumeric comparison
of each component?

-- 
 \        ?Ubi dubium, ibi libertas.? (?Where there is doubt, there is |
  `\                                                        freedom.?) |
_o__)                                                                  |
Ben Finney


From ben+python at benfinney.id.au  Fri Jun  5 15:55:48 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 05 Jun 2009 23:55:48 +1000
Subject: [Distutils] Comparison semantics for alphanumeric components of
	a version number
References: <94bdd2610905290214g495a3ee8oc71710461b202f38@mail.gmail.com>
	<87d49jkpg4.fsf@benfinney.id.au>
	<94bdd2610906050607x79ac6f7bk7ce898833451f2c4@mail.gmail.com>
Message-ID: <87k53qkcnv.fsf@benfinney.id.au>

Tarek Ziad? <ziade.tarek at gmail.com> writes:

> There's a consensus on this in most packaging system out there, and
> the goal is to have a rational version system that is understandable
> by most packagers so they can work with python projects versions.

Have you actually read the version comparison specifications of other
packaging systems? How do you think they compare to the current draft
specification here?

Those that do specify are *much* more obvious and simple in their
comparison of individual components than what is currently in the
?distutilsversion? specification. Usually it's a simple case of ?once
we've got the individual components of a version string, they compare
alphanumerically?.

> I don't think alphanumeric comparison is what most would expect.

I think it's far more likely to be what someone would expect than any
other *specific* arbitrary ordering of words that is chosen. You make
this point yourself in the current draft:

    ? it is much preferable if the versioning spec is such that a human
    can make a reasonable attempt at that sorting without having to run
    it against some code.

What evidence is there that some arbitrary ordering of specific sets of
words is going to be correctly guessed by such a person than a simple
?alphanumeric ordering? specification?

> For example if you use dates for your version, it'll work perfeclty
> with alphanumeric comparison but Fedora packagers will fail at sorting
> your versions properly.

I don't see how this is relevant. I'm not talking about removing the
need for dividing a version string into separately-comparable
components. That's clearly useful, and anyway is pretty uncontroversial
and widespread.

I'm arguing that, having got those components, and needing to compare
them *within an individual component position* for ordering, the best
candidate for ?simple to describe and remember, and obvious to guess?
is that the components are compared alphanumerically with no special
exceptions.

-- 
 \       ?An idea isn't responsible for the people who believe in it.? |
  `\                                      ?Donald Robert Perry Marquis |
_o__)                                                                  |
Ben Finney


From brandon at rhodesmill.org  Fri Jun  5 16:03:09 2009
From: brandon at rhodesmill.org (Brandon Craig Rhodes)
Date: Fri, 05 Jun 2009 10:03:09 -0400
Subject: [Distutils] Comparison semantics for alphanumeric components of
	a version number
In-Reply-To: <87oct2kdc5.fsf@benfinney.id.au> (Ben Finney's message of "Fri,
	05 Jun 2009 23:41:14 +1000")
References: <94bdd2610905290214g495a3ee8oc71710461b202f38@mail.gmail.com>
	<87d49jkpg4.fsf@benfinney.id.au>
	<79990c6b0906050543o1052576eqeced8f1a0be1d2cf@mail.gmail.com>
	<94bdd2610906050613t2f1535b4yf30fe95c3084ba16@mail.gmail.com>
	<87oct2kdc5.fsf@benfinney.id.au>
Message-ID: <87tz2u7p7m.fsf@rhodesmill.org>

Ben Finney <ben+python at benfinney.id.au> writes:

>     >>> from verlib import RationalVersion as V
>     >>> (V('1.0')
>     ...  < V('1.0.a1')
>     ...  < V('1.0.a2')
>     ...  < V('1.0.a2.1')
>     ...  < V('1.0.b2')
>     ...  < V('1.0.c1')
>     ...  < V('1.0.dev456post623')
>     ...  < V('1.0.post456'))
>     True

+1

With a caveat (or maybe this is the way it already works?) that every
series of adjacent digits in the version be compared as an integer
occupying a single character's worth of string, regardless of how many
digits compose the integer - which, I personally think, is how strings
should always be compared, and is the first thing I'd fix if I could go
back in time and improve something about Unix on the PDP in 1970 so that
it would have followed as the default string comparison mechanism
through the rest of time.  The fact that "log.7" sorts after "log.10" is
simply stupid, and trying to fix the problem with leading zeros is a
silly hack that breaks as soon as you get more log files that you
thought you'd need.

In other words:

  - Take the strings you are comparing:
         "1.0.post45pre12"
         "1.0.post9pre12"

  - Consider adjacent series of digits to be integers, not strings:
         (1, '.', 0, '.post', 45, 'pre', 12)
         (1, '.', 0, '.post',  9, 'pre', 12)

  - Compare the results, making (in this case) the number 9 come before
    the number 45, correcting the normal problem that the string "9" is
    lexicographically after the string "45".

-- 
Brandon Craig Rhodes   brandon at rhodesmill.org   http://rhodesmill.org/brandon

From ziade.tarek at gmail.com  Fri Jun  5 16:11:49 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 5 Jun 2009 16:11:49 +0200
Subject: [Distutils] Comparison semantics for alphanumeric components of
	a version number
In-Reply-To: <87k53qkcnv.fsf@benfinney.id.au>
References: <94bdd2610905290214g495a3ee8oc71710461b202f38@mail.gmail.com>
	<87d49jkpg4.fsf@benfinney.id.au>
	<94bdd2610906050607x79ac6f7bk7ce898833451f2c4@mail.gmail.com>
	<87k53qkcnv.fsf@benfinney.id.au>
Message-ID: <94bdd2610906050711n79ee9a82ne5b4b2a0b0ff950d@mail.gmail.com>

On Fri, Jun 5, 2009 at 3:55 PM, Ben Finney<ben+python at benfinney.id.au> wrote:
> Tarek Ziad? <ziade.tarek at gmail.com> writes:
>
>> There's a consensus on this in most packaging system out there, and
>> the goal is to have a rational version system that is understandable
>> by most packagers so they can work with python projects versions.
>
> Have you actually read the version comparison specifications of other
> packaging systems? How do you think they compare to the current draft
> specification here?

This is very upsetting !   This specification is not something I just
pulled out of my hat.

We worked during two evenings during Pycon with people from Fedora and
Ubuntu on
that (Toshio and Matthias Klose). Those people are packaging Python projects
for their systems and have problems because of the lack of proper
versioning sometimes.

We were like +10 people the first night brainstorming on that topic.

The proposal is something that everyone agreed was "good enough" there for
packagers to work with.

The only complexity we (I in fact) added later is the use case Philip
brought up for the post-releases,
and this something we are currently discussing in details in the ML/

But with your "N.N[.X]+" proposal,  you are missing some things this
PEP tries to adress,
such as dealing with alpha, beta and candidate, or development versions.

>
> I don't see how this is relevant. I'm not talking about removing the
> need for dividing a version string into separately-comparable
> components. That's clearly useful, and anyway is pretty uncontroversial
> and widespread.
> I'm arguing that, having got those components, and needing to compare
> them *within an individual component position* for ordering, the best
> candidate for ?simple to describe and remember, and obvious to guess?
> is that the components are compared alphanumerically with no special
> exceptions.

The RationalVersion class takes your string and split it in several components,
does a bit of processing, and then compare the components alphanumerically.

But there are extra rules for some of those components, (like
alpha/beta/candidate restriction)

From ben+python at benfinney.id.au  Fri Jun  5 16:12:13 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 06 Jun 2009 00:12:13 +1000
Subject: [Distutils] Comparison semantics for alphanumeric components of
	a version number
References: <94bdd2610905290214g495a3ee8oc71710461b202f38@mail.gmail.com>
	<87d49jkpg4.fsf@benfinney.id.au>
	<94bdd2610906050607x79ac6f7bk7ce898833451f2c4@mail.gmail.com>
	<87k53qkcnv.fsf@benfinney.id.au>
Message-ID: <87bpp2kbwi.fsf@benfinney.id.au>

Ben Finney <ben+python at benfinney.id.au> writes:

> Those that do specify are *much* more obvious and simple in their
> comparison of individual components than what is currently in the
> ?distutilsversion? specification. Usually it's a simple case of ?once
> we've got the individual components of a version string, they compare
> alphanumerically?.
[?]

> I'm arguing that, having got those components, and needing to compare
> them *within an individual component position* for ordering, the best
> candidate for ?simple to describe and remember, and obvious to guess?
> is that the components are compared alphanumerically with no special
> exceptions.

As pointed out, I'm oversimplifying: the consensus I've seen includes
the semantic that sequences of digits in any component are to be treated
as integers and compared accordingly. I think that should be part of the
specification (and it was so obvious in my mind that I forgot to express
it :-)

-- 
 \       ?Truth is stranger than fiction, but it is because fiction is |
  `\     obliged to stick to possibilities, truth isn't.? ?Mark Twain, |
_o__)                                          _Following the Equator_ |
Ben Finney


From ben+python at benfinney.id.au  Fri Jun  5 16:20:47 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 06 Jun 2009 00:20:47 +1000
Subject: [Distutils] Comparison semantics for alphanumeric components of
	a version number
References: <94bdd2610905290214g495a3ee8oc71710461b202f38@mail.gmail.com>
	<87d49jkpg4.fsf@benfinney.id.au>
	<94bdd2610906050607x79ac6f7bk7ce898833451f2c4@mail.gmail.com>
	<87k53qkcnv.fsf@benfinney.id.au>
	<94bdd2610906050711n79ee9a82ne5b4b2a0b0ff950d@mail.gmail.com>
Message-ID: <877hzqkbi8.fsf@benfinney.id.au>

Tarek Ziad? <ziade.tarek at gmail.com> writes:

> On Fri, Jun 5, 2009 at 3:55 PM, Ben Finney<ben+python at benfinney.id.au> wrote:
> > Tarek Ziad? <ziade.tarek at gmail.com> writes:
> >
> >> There's a consensus on this in most packaging system out there, and
> >> the goal is to have a rational version system that is understandable
> >> by most packagers so they can work with python projects versions.
> >
> > Have you actually read the version comparison specifications of other
> > packaging systems? How do you think they compare to the current draft
> > specification here?
> 
> This is very upsetting !   This specification is not something I just
> pulled out of my hat.

I didn't intend to imply anything like that.

> We worked during two evenings during Pycon with people from Fedora and
> Ubuntu on that (Toshio and Matthias Klose). Those people are packaging
> Python projects for their systems and have problems because of the
> lack of proper versioning sometimes.

This is the burden of conferences, and the danger of thinking that a
room full of ten people can somehow determine the consensus of the whole
community. Ideas that seem great when everyone is face to face still
need to be justified at length to the larger community.

> But with your "N.N[.X]+" proposal, you are missing some things this
> PEP tries to adress, such as dealing with alpha, beta and candidate,
> or development versions.

Then please, point to (or marshal people to write convincingly) the
justification for why these complicating factors are needed, and why
what they buy is worth the price.

The specification and discussion here acknowledges that accommodating
the weight of traditional version numbering schemes isn't a primary
goal: specifying something useful, obvious, and simple to explain and
remember is more important. I'm attempting to cast these special
complicating exceptions in that harsh light and see why they should
survive.

> The RationalVersion class takes your string and split it in several
> components, does a bit of processing, and then compare the components
> alphanumerically.
> 
> But there are extra rules for some of those components, (like
> alpha/beta/candidate restriction)

And it is exactly that extra baggage that is in question here.

-- 
 \        ?Most people don't realize that large pieces of coral, which |
  `\       have been painted brown and attached to the skull by common |
_o__)    wood screws, can make a child look like a deer.? ?Jack Handey |
Ben Finney


From ben+python at benfinney.id.au  Fri Jun  5 16:22:41 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 06 Jun 2009 00:22:41 +1000
Subject: [Distutils] Comparison semantics for alphanumeric components of
	a version number
References: <94bdd2610905290214g495a3ee8oc71710461b202f38@mail.gmail.com>
	<87d49jkpg4.fsf@benfinney.id.au>
	<79990c6b0906050543o1052576eqeced8f1a0be1d2cf@mail.gmail.com>
	<94bdd2610906050613t2f1535b4yf30fe95c3084ba16@mail.gmail.com>
	<87oct2kdc5.fsf@benfinney.id.au> <87tz2u7p7m.fsf@rhodesmill.org>
Message-ID: <873aaekbf2.fsf@benfinney.id.au>

Brandon Craig Rhodes <brandon at rhodesmill.org> writes:

> With a caveat (or maybe this is the way it already works?) that every
> series of adjacent digits in the version be compared as an integer
> occupying a single character's worth of string, regardless of how many
> digits compose the integer

Right. This is what I meant, but failed to express in my drive to
simplicity. Thanks for picking up on it.

-- 
 \       ?My classmates would copulate with anything that moved, but I |
  `\               never saw any reason to limit myself.? ?Emo Philips |
_o__)                                                                  |
Ben Finney


From ziade.tarek at gmail.com  Fri Jun  5 16:51:22 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 5 Jun 2009 16:51:22 +0200
Subject: [Distutils] Comparison semantics for alphanumeric components of
	a version number
In-Reply-To: <877hzqkbi8.fsf@benfinney.id.au>
References: <94bdd2610905290214g495a3ee8oc71710461b202f38@mail.gmail.com>
	<87d49jkpg4.fsf@benfinney.id.au>
	<94bdd2610906050607x79ac6f7bk7ce898833451f2c4@mail.gmail.com>
	<87k53qkcnv.fsf@benfinney.id.au>
	<94bdd2610906050711n79ee9a82ne5b4b2a0b0ff950d@mail.gmail.com>
	<877hzqkbi8.fsf@benfinney.id.au>
Message-ID: <94bdd2610906050751s7e5c8d7erbd98c6e5fbf55a05@mail.gmail.com>

On Fri, Jun 5, 2009 at 4:20 PM, Ben Finney<ben+python at benfinney.id.au> wrote:
>> We worked during two evenings during Pycon with people from Fedora and
>> Ubuntu on that (Toshio and Matthias Klose). Those people are packaging
>> Python projects for their systems and have problems because of the
>> lack of proper versioning sometimes.
>
> This is the burden of conferences, and the danger of thinking that a
> room full of ten people can somehow determine the consensus of the whole
> community. Ideas that seem great when everyone is face to face still
> need to be justified at length to the larger community.

I know I won't reach a consensus for the whole community. And there's no
need at all to reach it. It's impossible anyway.

As a matter of fact, there are already version comparison systems in Distutils,
you can read the PEP to have their descriptions. I have extracted their doc.

I think the StrictVersion in Distutils is quite similar to what you
are describing.

No one will force people to use the one we are defining, like no one
forced people
to use StrictVersion or LooseVersion.

But at some point, if you package  is being distributed to a wide
audience you need an
undertsandable versioning scheme, this new version can  help.

Not because it provides a better version scheme, but just because
your consumers will say : "Hey, I know how to read this version number"
without having to decrypt it everytime, depending on the projects and the teams.

And the problem is that the one that currently exists in Distutils and
Setuptools, or the one
you have described, doesn't fill that requirement because they are not
strict enough
from an os-packager point of view (some reasons are mentioned in the PEP)

Let's state it differently : this PEP is a proposal for a *strict*
versioning comparison tool
that what we already have, for an easier understanding of packagers.

Maybe we need to write that "how to use version numbers"  document at first.
Not to say yours is bad or mine is better, but just to write down how
release cycles work
and how versions increments occurs. Then how each proposal would fit in there.
The need of development versions will eventually become obvious for instance.

Regards
Tarek

From p.f.moore at gmail.com  Fri Jun  5 16:59:41 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 5 Jun 2009 15:59:41 +0100
Subject: [Distutils] Comparison semantics for alphanumeric components of
	a version number
In-Reply-To: <94bdd2610906050751s7e5c8d7erbd98c6e5fbf55a05@mail.gmail.com>
References: <94bdd2610905290214g495a3ee8oc71710461b202f38@mail.gmail.com>
	<87d49jkpg4.fsf@benfinney.id.au>
	<94bdd2610906050607x79ac6f7bk7ce898833451f2c4@mail.gmail.com>
	<87k53qkcnv.fsf@benfinney.id.au>
	<94bdd2610906050711n79ee9a82ne5b4b2a0b0ff950d@mail.gmail.com>
	<877hzqkbi8.fsf@benfinney.id.au>
	<94bdd2610906050751s7e5c8d7erbd98c6e5fbf55a05@mail.gmail.com>
Message-ID: <79990c6b0906050759r2e7d2de7kfca07c0d59318a63@mail.gmail.com>

2009/6/5 Tarek Ziad? <ziade.tarek at gmail.com>:
> No one will force people to use the one we are defining, like no one
> forced people
> to use StrictVersion or LooseVersion.

But it's being defined via a PEP (rather than hidden in the code, as
with Strict/LooseVsersion) so it has a higher level of visibility and
authoritativeness. So it should be held to higher standards. Maybe I
won't be forced to use it, but I suspect I will be *expected* to. And
quite possibly disadvantaged if I don't.

Paul.

From pje at telecommunity.com  Fri Jun  5 17:54:55 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 05 Jun 2009 11:54:55 -0400
Subject: [Distutils] Comparison semantics for alphanumeric components of
 a version number (was: Version comparison - round 2)
In-Reply-To: <94bdd2610906050613t2f1535b4yf30fe95c3084ba16@mail.gmail.co
 m>
References: <94bdd2610905290214g495a3ee8oc71710461b202f38@mail.gmail.com>
	<87d49jkpg4.fsf@benfinney.id.au>
	<79990c6b0906050543o1052576eqeced8f1a0be1d2cf@mail.gmail.com>
	<94bdd2610906050613t2f1535b4yf30fe95c3084ba16@mail.gmail.com>
Message-ID: <20090605155207.8E27A3A4109@sparrow.telecommunity.com>

At 03:13 PM 6/5/2009 +0200, Tarek Ziad? wrote:
>But so far, besides that very specific case for the post-release dev
>tag, I don't find
>it complicated at all.

...and I'm a little bit concerned that perhaps there was some 
confusion about that use case and maybe something got more 
complicated than it actually needs to be:

>The specific use case I was asking about, though, was 
>"1.0a1dev-r623", meaning "SVN revision 623 of the development work 
>on leading up to 1.0a1" -- and IIRC, in the RationalVersion cheme, 
>this should probably just translate to "1.0.a1.dev623".

The other use case I might've been concerned about would be 
"2.4-1dev-r623", which IIUC would be "2.4.post1.dev623" under RationalVersion.

IOW, I'm not sure that a RationalVersion needs more than one each of 
"post" and "dev", nor that they need to be in any order other than "post, dev".

I think there was some confusion on my part at first, because under 
setuptools' versioning scheme, "-r" is technically a "post-release" 
tag, but when used in a scheme like '1.0a1dev-r623', its only effect 
is to compare greater than '1.0a1dev623' - a version that wouldn't be 
used in that case, anyway!

So, that has some effect on what suggest_version should do, but not 
so much on the core versioning scheme.

Does that make anything clearer?  I want to make sure we don't end up 
with any extra complication due to my terminology confusion early on.


From ziade.tarek at gmail.com  Fri Jun  5 18:00:56 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 5 Jun 2009 18:00:56 +0200
Subject: [Distutils] Comparison semantics for alphanumeric components of
	a version number
In-Reply-To: <79990c6b0906050759r2e7d2de7kfca07c0d59318a63@mail.gmail.com>
References: <94bdd2610905290214g495a3ee8oc71710461b202f38@mail.gmail.com>
	<87d49jkpg4.fsf@benfinney.id.au>
	<94bdd2610906050607x79ac6f7bk7ce898833451f2c4@mail.gmail.com>
	<87k53qkcnv.fsf@benfinney.id.au>
	<94bdd2610906050711n79ee9a82ne5b4b2a0b0ff950d@mail.gmail.com>
	<877hzqkbi8.fsf@benfinney.id.au>
	<94bdd2610906050751s7e5c8d7erbd98c6e5fbf55a05@mail.gmail.com>
	<79990c6b0906050759r2e7d2de7kfca07c0d59318a63@mail.gmail.com>
Message-ID: <94bdd2610906050900p5938254dv9e020a7aa0e93ec2@mail.gmail.com>

On Fri, Jun 5, 2009 at 4:59 PM, Paul Moore<p.f.moore at gmail.com> wrote:
> 2009/6/5 Tarek Ziad? <ziade.tarek at gmail.com>:
>> No one will force people to use the one we are defining, like no one
>> forced people
>> to use StrictVersion or LooseVersion.
>
> But it's being defined via a PEP (rather than hidden in the code, as
> with Strict/LooseVsersion) so it has a higher level of visibility and
> authoritativeness. So it should be held to higher standards. Maybe I
> won't be forced to use it, but I suspect I will be *expected* to. And
> quite possibly disadvantaged if I don't.

Probably so yes, if you use the install_requires field PEP 345 introduces,
where you will define dependencies with their versions.

And if you don't use dev. flags for example, you will have to deal with them
if they are present in other projects you depend on. It's a real need.

We could use the setuptools standard here, because it's the de-facto
standard for this metadata today, (and that's what I have proposed
first during the Pycon sessions), but some use cases were raised
and the work+proposal you see in PEP 386 followed.

I see only advantages on having a strict, well-documented standard
for versions numbers., that we know is good enough for non-python
packagers.

Let's see what is going to come out of the other threads (with Phillip
and Trent on the edge case). I do believe we can have something that'll reach
consensus at some point when these edge cases are resolved,
because the whole thing will probably work with much simpler cases
like you have shown;

At least I hope we all agree that : 2009.05.12 is not a good version number,
and that we all want a major/minor scheme.

Regards
Tarek

From p.f.moore at gmail.com  Fri Jun  5 18:13:22 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 5 Jun 2009 17:13:22 +0100
Subject: [Distutils] Comparison semantics for alphanumeric components of
	a version number
In-Reply-To: <94bdd2610906050900p5938254dv9e020a7aa0e93ec2@mail.gmail.com>
References: <94bdd2610905290214g495a3ee8oc71710461b202f38@mail.gmail.com>
	<87d49jkpg4.fsf@benfinney.id.au>
	<94bdd2610906050607x79ac6f7bk7ce898833451f2c4@mail.gmail.com>
	<87k53qkcnv.fsf@benfinney.id.au>
	<94bdd2610906050711n79ee9a82ne5b4b2a0b0ff950d@mail.gmail.com>
	<877hzqkbi8.fsf@benfinney.id.au>
	<94bdd2610906050751s7e5c8d7erbd98c6e5fbf55a05@mail.gmail.com>
	<79990c6b0906050759r2e7d2de7kfca07c0d59318a63@mail.gmail.com>
	<94bdd2610906050900p5938254dv9e020a7aa0e93ec2@mail.gmail.com>
Message-ID: <79990c6b0906050913k1e94bdffhb3ff23cc29ec2dd4@mail.gmail.com>

2009/6/5 Tarek Ziad? <ziade.tarek at gmail.com>:
> At least I hope we all agree that : 2009.05.12 is not a good version number,
> and that we all want a major/minor scheme.

Well, Twisted and Ubuntu might both disagree :-) But it's no more of a
problem than 1.7.5 - the numbers are just a bit bigger, and there are
gaps.

Frankly, I prefer that to 1.2post5.dev6-r1234. Or is that
1.2dev6-post5_r1234??? :-)

Paul.

From greno at verizon.net  Fri Jun  5 20:22:21 2009
From: greno at verizon.net (Gerry Reno)
Date: Fri, 05 Jun 2009 14:22:21 -0400
Subject: [Distutils] Packaging:  bdist_deb patches
In-Reply-To: <49B7B62F.40306@verizon.net>
References: <49B7B62F.40306@verizon.net>
Message-ID: <4A29625D.2020005@verizon.net>

Gerry Reno wrote:
> Tarek,
> Speaking of packaging patches. Would there be any chance that you 
> could get the 'bdist_deb' patches integrated in Distutils prior to PyCon?
>
Hi Tarek,
Has there been any progress on getting the 'bdist_deb' target (akin to 
'bdist_rpm') implemented in Distutils? We are looking for behavior that 
would be similar to bdist_rpm driven by setup.cfg with fully automated 
generation of packages. There is in existence a set of patches that 
supposedly does this. Has any progress been made to integrate those patches?

Regards,
Gerry


From david.lyon at preisshare.net  Sat Jun  6 04:26:06 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Fri, 05 Jun 2009 22:26:06 -0400
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <94bdd2610906040715n15195ab7o2b34d59dfecde62c@mail.gmail.com>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604095203.GB9596@Boo.lan>
	<94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
	<a992920f0906040529r7de0ac5cgf2b8d738dbb3be5e@mail.gmail.com>
	<94bdd2610906040554l6ddc4444jc9c5d13cda1d54f9@mail.gmail.com>
	<79990c6b0906040659l3ff0f394h51d05414d09e2978@mail.gmail.com>
	<94bdd2610906040715n15195ab7o2b34d59dfecde62c@mail.gmail.com>
Message-ID: <860dd9a0c6f99ed7b4cb6f3967ec2eaa@preisshare.net>

On Thu, 4 Jun 2009 16:15:11 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
wrote:
> I think that would be great to externalize such command to decouple
> its release cycles from Python (As mentioned during the Langage summit)
> 
> The only requirement would be to find someone that would lead its
> development,
> 
> If I find a windows developer that wants to handle this  <hint> <hint>
> , we could probably create a separate project

It's a lot of work for anybody doing any seperate project.

What are you expectations for this?

David

From barthelemy at crans.org  Sat Jun  6 08:44:16 2009
From: barthelemy at crans.org (=?ISO-8859-1?Q?S=E9bastien_Barth=E9lemy?=)
Date: Sat, 6 Jun 2009 08:44:16 +0200
Subject: [Distutils] a --user option for develop?
Message-ID: <78f7ab620906052344g3f4da10epa6f8a5f4fc4e16b1@mail.gmail.com>

Hello all,

I discovered the joys of the --user option: no more need for
PYTHONPATH in many cases, simply run

python setup.py install --user

And I wonder if it would be possible to add this functionality/option
to the develop command too?


Thanks !

From g.brandl at gmx.net  Sun Jun  7 00:43:34 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 07 Jun 2009 00:43:34 +0200
Subject: [Distutils] Comparison semantics for alphanumeric components of
 a version number
In-Reply-To: <79990c6b0906050637n13384cebme27c753725260aa7@mail.gmail.com>
References: <94bdd2610905290214g495a3ee8oc71710461b202f38@mail.gmail.com>	<87d49jkpg4.fsf@benfinney.id.au>	<79990c6b0906050543o1052576eqeced8f1a0be1d2cf@mail.gmail.com>	<94bdd2610906050613t2f1535b4yf30fe95c3084ba16@mail.gmail.com>
	<79990c6b0906050637n13384cebme27c753725260aa7@mail.gmail.com>
Message-ID: <h0ergf$j8s$1@ger.gmane.org>

Paul Moore schrieb:
> 2009/6/5 Tarek Ziad? <ziade.tarek at gmail.com>:
>>> But I acknowledge that I have no personal requirement for any of this,
>>> so the only interest I have is an aesthetic one of *not* seeing
>>> overcomplicated, difficult to understand, specifications become part
>>> of the Python stdlib.
>>
>> Well if the specification is difficult to understand or overcomplicated
>> it'll fail for sure.
>>
>> But so far, besides that very specific case for the post-release dev
>> tag, I don't find
>> it complicated at all. Another win I can see is that it will help developers use
>> better versions numbers for their projects imho.
> 
> And yet, in spite of repeated requests for specific examples of
> projects using this post-dev stuff, which haven't been forthcoming,
> why is nobody saying "it's an unnecessary complication, we'll drop
> it"?

Not everybody is on this list.  At the discussion at PyCon, one participant
(IIRC it was Zooko) specifically asked for this option to be included.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From greno at verizon.net  Sun Jun  7 00:20:00 2009
From: greno at verizon.net (Gerry Reno)
Date: Sat, 06 Jun 2009 18:20:00 -0400
Subject: [Distutils] Packaging:  bdist_deb patches
In-Reply-To: <49B7B62F.40306@verizon.net>
References: <49B7B62F.40306@verizon.net>
Message-ID: <4A2AEB90.4060607@verizon.net>

Gerry Reno wrote:
> Tarek,
> Speaking of packaging patches. Would there be any chance that you 
> could get the 'bdist_deb' patches integrated in Distutils prior to PyCon?
>
Hi Tarek,
Has there been any progress on getting the 'bdist_deb' target (akin to
'bdist_rpm') implemented in Distutils? We are looking for behavior that
would be similar to bdist_rpm driven by setup.cfg with fully automated
generation of packages. There is in existence a set of patches that
supposedly does this. Has any progress been made to integrate those patches?

Regards,
Gerry



From ronaldoussoren at mac.com  Sun Jun  7 16:28:49 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Sun, 07 Jun 2009 07:28:49 -0700
Subject: [Distutils] PEP 376 comments
Message-ID: <FBC8A392-2135-41D1-9129-4688C5771C7D@mac.com>

I have some feedback on PEP376, both the pep itself and the pkgutil  
code. I'll start with comments on the PEP.

* I don't like the definition of a project: this seems to define what  
distutils calls a distribution, and that is not necessarily and  
application.

* The description of the status quo is not entirely acurate w.r.t.  
setuptools, you describe only one of the possible ways setuptools can  
install a project (pip seems to use another one of setuptools' ways of  
installing a project, and setuptools can also install projects inside  
of zipfiles in multiple ways).

* Regarding the RECORD file: it is not a "CSV-like" file, it is a real  
CSV file. I'd specify the exact options for the 'csv' module that will  
be used, rather than writing that the default options are used and  
then explaining what those are.

* Should the PEP specify the encoding of text-files? PEP314 doesn't  
seem to specify the encoding of PKG-INFO files, which can cause  
problems when a field contains data that isn't ASCII.


All of these are minor issues. The following are imho more serious.

* The PEP doesn't describe how this PEP interacts with PEP302. That  
is, how should the "egg-info" machinery work when a project is not  
installed on the filesystem but in a zipfile. I'm not primarily  
interested in the ".egg" zipfiles that setuptools uses, but I am  
worried about how this will affect tools like py2exe and py2app that  
bundle the python code used by an application into a zipfile.

* Why is there a "paths" argument for the global functions (such as  
"get_file_users")? The description claims the functions will use  
sys.path and hence it is not necessary to have an argument to specify  
the path.

* There is no API to list the files in the egg-info directory, you  
could build one yourself on top of the get_installed_files method but  
should IMO be part of the public API.


And now on to the implementation:

* EggInfo.get_file: the implementation seems to want to load a file  
from the .egg-info directory, the PEP itself is unclear about what  
this method is intended to do.

If get_file should open a file in the egg-info directory I'd raise an  
exception if the path argument specifies an absolute path.

I wonder if it wouldn't be better to have a function that returns the  
contents of the file instead of one that returns a file-like object.  
Especially when thinking of PEP302 integration.

Wouldn't it be better to have two separate methods instead of "binary"  
argument.  In 3.x file-like objects behave slightly different w.r.t  
binary and text stream ("bytes" vs. "str" as the result of read).

* EggInfo.get_installed_files: if 'local' is True the yielded paths  
are made absolute w.r.t. the egg-info directory rather than the  
directory containing the egg-info directory.

* The global functions seem to maintain and modify global state,  
wouldn't this cause problems if I specify different values of the path  
arguments in different pieces of code?


Ronald

From ronaldoussoren at mac.com  Sun Jun  7 16:36:16 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Sun, 07 Jun 2009 07:36:16 -0700
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <94bdd2610906040554l6ddc4444jc9c5d13cda1d54f9@mail.gmail.com>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604095203.GB9596@Boo.lan>
	<94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
	<a992920f0906040529r7de0ac5cgf2b8d738dbb3be5e@mail.gmail.com>
	<94bdd2610906040554l6ddc4444jc9c5d13cda1d54f9@mail.gmail.com>
Message-ID: <79CB1097-E1B0-4938-AADE-31A7DC34BA91@mac.com>


On 4 Jun, 2009, at 5:54, Tarek Ziad? wrote:

> Paul
>> I'd say that it's distutils' responsibility not to offer to uninstall
>> anything it didn't install.
>
> Brian
>> Yep. Though I think that nowdays dpkg installs to a different  
>> directory than
>> Distutils' default. So users have to specify extra options to break
>> their systems.
>
> But how does those third-party install the projects ?
>
> Don't they use Distutils' install command under the hood ?
>
> If so, the new PEP 376 APIs are ignoring previous versions of
> egg-infos and work only with the ones that complies with the new
> standard (it's a quick control right now,  look at the "is_egg_info"
> API in http://bitbucket.org/tarek/pep376/src/tip/pkgutil.py)
>
> So, in any case the uninstall command will not work with project that
> are filtered out by is_egg_info().
>
> So maybe we could add a "INSTALLER" file with a unique md5 key
> provided by the project that installed the package,
> and ask for the key when calling this API ? If not provided, it would
> use Distutils's md5 key

Why should the the INSTALLER file contain a md5 value? I'd use either  
the name of the packager (dpkg,rpm,...), or even that plus the name of  
the external package:

	rpm,py26-zlib

This would make it possible for a PEP376-based uninstaller to give  
more meaningful error messages when you try to uninstall a project  
that was installed by a system package. That is, it could say "this  
project was installed using the 'py26-zlib' system-package, use 'rpm'  
to uninstall" rather than "go away I didn't install this".

BTW. I'd say that the INSTALLER should default to "distutils", that  
is: don't install the INSTALLER file unless someone explicitly asks  
for it by specifying a different value for the installer. A tool like  
bdist_rpm could of course automaticly set that different value.

Ronald
>
>
> Tarek
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig


From ronaldoussoren at mac.com  Sun Jun  7 16:46:10 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Sun, 07 Jun 2009 07:46:10 -0700
Subject: [Distutils] setuptools and py3k
Message-ID: <EE65BCC4-F1AE-40C1-83CE-6A5D60D97BFD@mac.com>

Hi,

I want to port pyobjc to python 3.1 and need a py3k port of setuptools  
for that. I'm slowy moving forward with the latter, partly based on  
earlier discussions on this list.  I don't have anything useful to  
share at the moment, beyond a setup3.py file (see below).

But first a question:  some of the tests use docutils and I'm having a  
hard time getting those to run with both python3 and python2. The bit  
that's particularly annoying is this bit of api_tests.txt:

    Note that asking for a conflicting version of a distribution  
already in a
    working set triggers a ``pkg_resources.VersionConflict`` error:

        >>> ws.find(Requirement.parse("Bar==1.0")) # doctest:  
+NORMALIZE_WHITESPACE
        Traceback (most recent call last):
          ...
        VersionConflict: (Bar 0.9 (http://example.com/something),
                       Requirement.parse('Bar==1.0'))

This passes with 2.6, but fails in 3.1 because it expects  
'pkg_resources.VersionConflict' rather than just 'VersionConflict'.  
What's the best way to work around this issue?


The attached file "setup3.py" is a script that incrementally copies  
the setuptools source-tree to a temporary directory, runs 2to3 on that  
copy and exec-s the contents of the translated 'setup.py' file. This  
seems to be good enough to be able to install setuptools for python2  
and python3 using the same code-base.   That said, this is just the  
first step of a python3 port there need to be changes to the  
setuptools sources to be able to actually use the translated sources.

Ronald

-------------- next part --------------
A non-text attachment was scrubbed...
Name: setup3.py
Type: text/x-python-script
Size: 3282 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090607/ef407691/attachment.bin>
-------------- next part --------------


From p.f.moore at gmail.com  Sun Jun  7 18:48:04 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 7 Jun 2009 17:48:04 +0100
Subject: [Distutils] Comparison semantics for alphanumeric components of
	a version number
In-Reply-To: <h0ergf$j8s$1@ger.gmane.org>
References: <94bdd2610905290214g495a3ee8oc71710461b202f38@mail.gmail.com>
	<87d49jkpg4.fsf@benfinney.id.au>
	<79990c6b0906050543o1052576eqeced8f1a0be1d2cf@mail.gmail.com>
	<94bdd2610906050613t2f1535b4yf30fe95c3084ba16@mail.gmail.com>
	<79990c6b0906050637n13384cebme27c753725260aa7@mail.gmail.com>
	<h0ergf$j8s$1@ger.gmane.org>
Message-ID: <79990c6b0906070948i1a50380aw59b06b73eee72dd7@mail.gmail.com>

2009/6/6 Georg Brandl <g.brandl at gmx.net>:
> Paul Moore schrieb:
>> And yet, in spite of repeated requests for specific examples of
>> projects using this post-dev stuff, which haven't been forthcoming,
>> why is nobody saying "it's an unnecessary complication, we'll drop
>> it"?
>
> Not everybody is on this list. ?At the discussion at PyCon, one participant
> (IIRC it was Zooko) specifically asked for this option to be included.

FWIW, Zooko is on the list. If it wasn't him, someone who knows who it
was needs to ask them to supply their justification for the list -
given that there are a number of objections being raised here which
need to be addressed.

I don't think "someone said they need it" is sufficient justification
by itself. I'm not trying to criticise the work done at PyCon, but I
think the discussions (and not just the conclusions) need to be
publicised better.

Paul.

From ben+python at benfinney.id.au  Mon Jun  8 02:50:51 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 08 Jun 2009 10:50:51 +1000
Subject: [Distutils] Comparison semantics for alphanumeric components of
	a version number
References: <94bdd2610905290214g495a3ee8oc71710461b202f38@mail.gmail.com>
	<87d49jkpg4.fsf@benfinney.id.au>
	<94bdd2610906050607x79ac6f7bk7ce898833451f2c4@mail.gmail.com>
	<87k53qkcnv.fsf@benfinney.id.au>
	<94bdd2610906050711n79ee9a82ne5b4b2a0b0ff950d@mail.gmail.com>
	<877hzqkbi8.fsf@benfinney.id.au>
	<94bdd2610906050751s7e5c8d7erbd98c6e5fbf55a05@mail.gmail.com>
Message-ID: <877hznft04.fsf@benfinney.id.au>

Tarek Ziad? <ziade.tarek at gmail.com> writes:

> On Fri, Jun 5, 2009 at 4:20 PM, Ben Finney<ben+python at benfinney.id.au> wrote:
> >> We worked during two evenings during Pycon with people from Fedora and
> >> Ubuntu on that (Toshio and Matthias Klose). Those people are packaging
> >> Python projects for their systems and have problems because of the
> >> lack of proper versioning sometimes.
> >
> > This is the burden of conferences, and the danger of thinking that a
> > room full of ten people can somehow determine the consensus of the
> > whole community. Ideas that seem great when everyone is face to face
> > still need to be justified at length to the larger community.
> 
> I know I won't reach a consensus for the whole community. And there's
> no need at all to reach it. It's impossible anyway.

You seem to think ?consensus? means ?unanimous agreement?; that's
not what it means. For consensus, it's only necessary that the group as
a whole acts on the belief that agreement has been reached; it doesn't
need to extend to every individual member.

That doesn't make getting the consensus of the Python community *easy*;
but it's certainly not impossible.

> And the problem is that the [version comparison algorithm] that
> currently exists in Distutils and Setuptools, or the one you have
> described, doesn't fill that requirement because they are not strict
> enough from an os-packager point of view (some reasons are mentioned
> in the PEP)

The one I've described is exactly as strict as the one in the current
draft (though I haven't formally described it). The difference is not in
strictness, but simplicity.

> Let's state it differently : this PEP is a proposal for a *strict*
> versioning comparison tool that what we already have, for an easier
> understanding of packagers.

Yes, I agree that strictness is essential to being able to apply such an
algorithm automatically. That's not in dispute.

-- 
 \      ?God forbid that any book should be banned. The practice is as |
  `\                  indefensible as infanticide.? ?Dame Rebecca West |
_o__)                                                                  |
Ben Finney


From ben+python at benfinney.id.au  Mon Jun  8 02:53:46 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 08 Jun 2009 10:53:46 +1000
Subject: [Distutils] Comparison semantics for alphanumeric components of
	a version number
References: <94bdd2610905290214g495a3ee8oc71710461b202f38@mail.gmail.com>
	<87d49jkpg4.fsf@benfinney.id.au>
	<79990c6b0906050543o1052576eqeced8f1a0be1d2cf@mail.gmail.com>
	<94bdd2610906050613t2f1535b4yf30fe95c3084ba16@mail.gmail.com>
	<79990c6b0906050637n13384cebme27c753725260aa7@mail.gmail.com>
	<h0ergf$j8s$1@ger.gmane.org>
	<79990c6b0906070948i1a50380aw59b06b73eee72dd7@mail.gmail.com>
Message-ID: <873aabfsv9.fsf@benfinney.id.au>

Paul Moore <p.f.moore at gmail.com> writes:

> I don't think "someone said they need it" is sufficient justification
> by itself. I'm not trying to criticise the work done at PyCon, but I
> think the discussions (and not just the conclusions) need to be
> publicised better.

Yes, that's exactly what I'm asking for. Before making it more complex
than it apparently needs to be, the justifications need to be subjected
to public scrutiny and, if they don't survive, discarded.

That doesn't pre-judge the outcome of such scrutiny: precisely the
opposite, I *don't* want such additional complexity pre-judged as
acceptable.

-- 
 \      ?I am too firm in my consciousness of the marvelous to be ever |
  `\       fascinated by the mere supernatural ?? ?Joseph Conrad, _The |
_o__)                                                     Shadow-Line_ |
Ben Finney


From ben+python at benfinney.id.au  Mon Jun  8 02:58:38 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 08 Jun 2009 10:58:38 +1000
Subject: [Distutils] Comparison semantics for alphanumeric components of
	a version number
References: <94bdd2610905290214g495a3ee8oc71710461b202f38@mail.gmail.com>
	<87d49jkpg4.fsf@benfinney.id.au>
	<79990c6b0906050543o1052576eqeced8f1a0be1d2cf@mail.gmail.com>
	<94bdd2610906050613t2f1535b4yf30fe95c3084ba16@mail.gmail.com>
	<20090605155207.8E27A3A4109@sparrow.telecommunity.com>
Message-ID: <87y6s3ee2p.fsf@benfinney.id.au>

"P.J. Eby" <pje at telecommunity.com> writes:

> IOW, I'm not sure that a RationalVersion needs more than one each of
> "post" and "dev", nor that they need to be in any order other than
> "post, dev".

Why are those special cases needed *at all*? Why is it not enough to say
?within a component, compare sequences of digits as integers, and other
characters ASCII-ordered??

I don't see there is sufficient consistency in the usage of these
special words to justify enshrining *one particular* interpretation of a
selected set of them. Why not cast them all out and keep the
specification thereby simpler?

-- 
 \                   ?The best ad-libs are rehearsed.? ?Graham Kennedy |
  `\                                                                   |
_o__)                                                                  |
Ben Finney


From ziade.tarek at gmail.com  Mon Jun  8 11:58:18 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 8 Jun 2009 11:58:18 +0200
Subject: [Distutils] PEP 376 comments
In-Reply-To: <FBC8A392-2135-41D1-9129-4688C5771C7D@mac.com>
References: <FBC8A392-2135-41D1-9129-4688C5771C7D@mac.com>
Message-ID: <94bdd2610906080258m2ce4398bg1f072176ff26a023@mail.gmail.com>

2009/6/7 Ronald Oussoren <ronaldoussoren at mac.com>:
> I have some feedback on PEP376, both the pep itself and the pkgutil code. I'll start with comments on the PEP.

Thanks for this detailed feedback.

>
> * I don't like the definition of a project: this seems to define what distutils calls a distribution, and that is not necessarily and application.

Fixed.

>
> * The description of the status quo is not entirely acurate w.r.t. setuptools, you describe only one of the possible ways setuptools can install a project (pip seems to use another one of setuptools' ways of installing a project, and setuptools can also install projects inside of zipfiles in multiple ways).

The goal is to keep only one way to install projects. I've added more
details in the PEP

>
> * Regarding the RECORD file: it is not a "CSV-like" file, it is a real CSV file. I'd specify the exact options for the 'csv' module that will be used, rather than writing that the default options are used and then explaining what those are.

Changed.

>
> * Should the PEP specify the encoding of text-files? PEP314 doesn't seem to specify the encoding of PKG-INFO files, which can cause problems when a field contains data that isn't ASCII.

The encoding used is utf-8 since 2.6.  I think we should rather update
PEP 314, and mention it in the upcoming PEP 345 as well,

>
>
> All of these are minor issues. The following are imho more serious.
>
> * The PEP doesn't describe how this PEP interacts with PEP302. That is, how should the "egg-info" machinery work when a project is not installed on the filesystem but in a zipfile. I'm not primarily interested in the ".egg" zipfiles that setuptools uses, but I am worried about how this will affect tools like py2exe and py2app that bundle the python code used by an application into a zipfile.

I need to work on this.

>
> * Why is there a "paths" argument for the global functions (such as "get_file_users")? The description claims the functions will use sys.path and hence it is not necessary to have an argument to specify the path.

That's because you can use these apis to work on an arbitrary list of
paths in a packaging tool.  I have changed the
description accordingly.

>
> * There is no API to list the files in the egg-info directory, you could build one yourself on top of the get_installed_files method but should IMO be part of the public API.

Added.

>
>
> And now on to the implementation:
>
> * EggInfo.get_file: the implementation seems to want to load a file from the .egg-info directory, the PEP itself is unclear about what this method is intended to do.

Added more details. I've also renamed some APIs to make things clearer
(get_file -> get_egg_info_file)
(I am also thinking about renaming EggInfo to a better name)

>
> If get_file should open a file in the egg-info directory I'd raise an exception if the path argument specifies an absolute path.

Added.

>
> I wonder if it wouldn't be better to have a function that returns the contents of the file instead of one that returns a file-like object. Especially when thinking of PEP302 integration.

Why ?

>
> Wouldn't it be better to have two separate methods instead of "binary" argument. ?In 3.x file-like objects behave slightly different w.r.t binary and text stream ("bytes" vs. "str" as the result of read).

Will work on that.

>
> * EggInfo.get_installed_files: if 'local' is True the yielded paths are made absolute w.r.t. the egg-info directory rather than the directory containing the egg-info directory.


Yes that was a bug. Fixed in the prototype.

>
> * The global functions seem to maintain and modify global state, wouldn't this cause problems if I specify different values of the path arguments in different pieces of code?

The cache just prevents re-reading a directory content.  Do you have a
scenario in mind of a possible problem ?
The only problem I can see is when a project is uninstalled by a
program while these APIs are used by another program.

By the way, I'm thinking about adding something else for that part: A
singleton-like class for EggInfoDirectory. That
would make sure there's only one instance for each path.

Tarek

From ziade.tarek at gmail.com  Mon Jun  8 12:10:08 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 8 Jun 2009 12:10:08 +0200
Subject: [Distutils] PEP 345, PEP 376, PEP 386
In-Reply-To: <79CB1097-E1B0-4938-AADE-31A7DC34BA91@mac.com>
References: <94bdd2610906040217v2b9e05adxfb17e6a502ef335f@mail.gmail.com>
	<20090604095203.GB9596@Boo.lan>
	<94bdd2610906040357l45adb700n9b668a0858d84703@mail.gmail.com>
	<a992920f0906040529r7de0ac5cgf2b8d738dbb3be5e@mail.gmail.com>
	<94bdd2610906040554l6ddc4444jc9c5d13cda1d54f9@mail.gmail.com>
	<79CB1097-E1B0-4938-AADE-31A7DC34BA91@mac.com>
Message-ID: <94bdd2610906080310o27a1f43evb253e0cbca5a2b6d@mail.gmail.com>

On Sun, Jun 7, 2009 at 4:36 PM, Ronald Oussoren<ronaldoussoren at mac.com> wrote:
>> So maybe we could add a "INSTALLER" file with a unique md5 key
>> provided by the project that installed the package,
>> and ask for the key when calling this API ? If not provided, it would
>> use Distutils's md5 key
>
> Why should the the INSTALLER file contain a md5 value? I'd use either the
> name of the packager (dpkg,rpm,...), or even that plus the name of the
> external package:
>
> ? ? ? ?rpm,py26-zlib
>
> This would make it possible for a PEP376-based uninstaller to give more
> meaningful error messages when you try to uninstall a project that was
> installed by a system package. That is, it could say "this project was
> installed using the 'py26-zlib' system-package, use 'rpm' to uninstall"
> rather than "go away I didn't install this".

Yes that's what I said later, and I changed the PEP accordingly,


>
> BTW. I'd say that the INSTALLER should default to "distutils", that is:
> don't install the INSTALLER file unless someone explicitly asks for it by
> specifying a different value for the installer. A tool like bdist_rpm could
> of course automaticly set that different value.

That's the case besides the fact that the INSTALLER file is created
even with the
default distutils value. I don't see any good reason not to create it
nevertheless.

From ziade.tarek at gmail.com  Mon Jun  8 14:28:52 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 8 Jun 2009 14:28:52 +0200
Subject: [Distutils] Packaging: bdist_deb patches
In-Reply-To: <4A2AEB90.4060607@verizon.net>
References: <49B7B62F.40306@verizon.net> <4A2AEB90.4060607@verizon.net>
Message-ID: <94bdd2610906080528x4724d980oc54af2b3e7464385@mail.gmail.com>

Hi Gerry,

During the summit at Pycon, we have said that it would be a better
strategy not to include
within Distutils os-specific tools for various reasons (and also to
remove existing ones) :

- it's better for them to have their own release cycles
- it's hard for me, as distutils maintainer, to maintain and make
evolve os-specific tools. People that are specialists on those OS
  will do a better job.

Now the problems to reach that goal are:

- some people are reluctant not to have everything included in the
stdlib (I am not). I don't think we can all agree on this,
  and since Guido have encouraged this strategy during the summit, I
guess I'm more inclined to follow it nevertheless.

- we need to detect for each existing command (rpm, etc) a project
that can take over, prior to remove it from Distutils
when appliable .

Now, for debian, since it's not included in distutils, the question is
: what is the most used/advanced project ? sdteb ?

++
Tarek

From tseaver at palladion.com  Mon Jun  8 14:53:45 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Mon, 08 Jun 2009 08:53:45 -0400
Subject: [Distutils] setuptools and py3k
In-Reply-To: <EE65BCC4-F1AE-40C1-83CE-6A5D60D97BFD@mac.com>
References: <EE65BCC4-F1AE-40C1-83CE-6A5D60D97BFD@mac.com>
Message-ID: <h0j1n5$u9k$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ronald Oussoren wrote:

> But first a question:  some of the tests use docutils and I'm having a  
> hard time getting those to run with both python3 and python2. The bit  
> that's particularly annoying is this bit of api_tests.txt:
> 
>     Note that asking for a conflicting version of a distribution  
> already in a
>     working set triggers a ``pkg_resources.VersionConflict`` error:
> 
>         >>> ws.find(Requirement.parse("Bar==1.0")) # doctest:  
> +NORMALIZE_WHITESPACE*
>         Traceback (most recent call last):
>           ...
>         VersionConflict: (Bar 0.9 (http://example.com/something),
>                        Requirement.parse('Bar==1.0'))
> 
> This passes with 2.6, but fails in 3.1 because it expects  
> 'pkg_resources.VersionConflict' rather than just 'VersionConflict'.  
> What's the best way to work around this issue?

Don't rely on the repr of an exception.  This kind of problem is why I
hate using doctest to test edge cases.  If you must, then something like:

   >>> from pkg_resources import VersionConflict
   >>> try:
   ...     ws.find(Requirement.parse("Bar==1.0"))
   ... except VersionConflict:
   ...     pass
   ... else:
   ...     print 'VersionConflict not raised.'

with no output expected.  Note that this is actually *less* verbose and
easier to understand than the repr bit.  Of course, a traditional unit
test would be even shorter and easier to understand.

> The attached file "setup3.py" is a script that incrementally copies  
> the setuptools source-tree to a temporary directory, runs 2to3 on that  
> copy and exec-s the contents of the translated 'setup.py' file. This  
> seems to be good enough to be able to install setuptools for python2  
> and python3 using the same code-base.   That said, this is just the  
> first step of a python3 port there need to be changes to the  
> setuptools sources to be able to actually use the translated sources.

Great work.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFKLQnZ+gerLs4ltQ4RAh3cAJ4wHai8q+svdYKjDLB3sk8LwshIMgCg1PDL
xFCDzoM52IwA1n0rUlBsWpU=
=zl9H
-----END PGP SIGNATURE-----


From ronaldoussoren at mac.com  Mon Jun  8 15:04:42 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Mon, 08 Jun 2009 06:04:42 -0700
Subject: [Distutils] PEP 376 comments
In-Reply-To: <94bdd2610906080258m2ce4398bg1f072176ff26a023@mail.gmail.com>
References: <FBC8A392-2135-41D1-9129-4688C5771C7D@mac.com>
	<94bdd2610906080258m2ce4398bg1f072176ff26a023@mail.gmail.com>
Message-ID: <5D2DA27E-1EC7-4032-86C1-F535A3D7DCB3@mac.com>


On 8 Jun, 2009, at 2:58, Tarek Ziad? wrote:

> 2009/6/7 Ronald Oussoren <ronaldoussoren at mac.com>:
>
>>
>> * The global functions seem to maintain and modify global state,  
>> wouldn't this cause problems if I specify different values of the  
>> path arguments in different pieces of code?
>
> The cache just prevents re-reading a directory content.  Do you have a
> scenario in mind of a possible problem ?
> The only problem I can see is when a project is uninstalled by a
> program while these APIs are used by another program.

I guess the problem only occurs if "egg_infos" is part of the public  
interface. What if module A calls get_egg_infos(), then module B calls  
get_egg_infos(somePath), then module A uses "egg_infos" assuming that  
it still refers to the same set of paths.

There's also the issue of API complexity. The global functions don't  
add a lot to using an EggInfoDirectories object when you are working  
with a different set of paths than sys.path.

Ronald

From j.schauer at email.de  Mon Jun  8 16:42:22 2009
From: j.schauer at email.de (Johannes Schauer)
Date: Mon, 08 Jun 2009 16:42:22 +0200
Subject: [Distutils] generating and cleaning documentation
Message-ID: <4A2D234E.8060009@email.de>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

hello!

our project uses epydoc to create it's documentation.
is there a way to let build do the documentation generation?
and how to get clean into the mix to remove the generated documentation?
is there a way to build the doc without introducing a new command?
cheers
josch
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkotI04ACgkQXYV+f49DE/UzcgCeNlPPyO4uUX7x1FYnaL/5lGA0
6vUAmwfWqPmB0ZL7067gSGYIgrzNfJz5
=lGS0
-----END PGP SIGNATURE-----

From p.f.moore at gmail.com  Mon Jun  8 17:11:58 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 8 Jun 2009 16:11:58 +0100
Subject: [Distutils] PEP 376 comments
In-Reply-To: <5D2DA27E-1EC7-4032-86C1-F535A3D7DCB3@mac.com>
References: <FBC8A392-2135-41D1-9129-4688C5771C7D@mac.com>
	<94bdd2610906080258m2ce4398bg1f072176ff26a023@mail.gmail.com>
	<5D2DA27E-1EC7-4032-86C1-F535A3D7DCB3@mac.com>
Message-ID: <79990c6b0906080811t7b893f91r3507f523042db59d@mail.gmail.com>

2009/6/8 Ronald Oussoren <ronaldoussoren at mac.com>:
> I guess the problem only occurs if "egg_infos" is part of the public
> interface. What if module A calls get_egg_infos(), then module B calls
> get_egg_infos(somePath), then module A uses "egg_infos" assuming that it
> still refers to the same set of paths.

Vaguely off-topic, but is it only me that hates the abbreviation
"infos"? It can only realistically be an abbreviation for the word
"informations" - and there's no such word :-(

Paul.

From ziade.tarek at gmail.com  Mon Jun  8 17:16:33 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 8 Jun 2009 17:16:33 +0200
Subject: [Distutils] PEP 376 comments
In-Reply-To: <79990c6b0906080811t7b893f91r3507f523042db59d@mail.gmail.com>
References: <FBC8A392-2135-41D1-9129-4688C5771C7D@mac.com>
	<94bdd2610906080258m2ce4398bg1f072176ff26a023@mail.gmail.com>
	<5D2DA27E-1EC7-4032-86C1-F535A3D7DCB3@mac.com>
	<79990c6b0906080811t7b893f91r3507f523042db59d@mail.gmail.com>
Message-ID: <94bdd2610906080816o4c35aab0yf84a2b1be43d1a39@mail.gmail.com>

On Mon, Jun 8, 2009 at 5:11 PM, Paul Moore<p.f.moore at gmail.com> wrote:
> 2009/6/8 Ronald Oussoren <ronaldoussoren at mac.com>:
>> I guess the problem only occurs if "egg_infos" is part of the public
>> interface. What if module A calls get_egg_infos(), then module B calls
>> get_egg_infos(somePath), then module A uses "egg_infos" assuming that it
>> still refers to the same set of paths.
>
> Vaguely off-topic, but is it only me that hates the abbreviation
> "infos"? It can only realistically be an abbreviation for the word
> "informations" - and there's no such word :-(

Mmm, that's my frenglish, Sir  ;)  (If you see other weird stuff
please let me know)

So I guess "get_egg_info" is the shortest, proper form ?


>
> Paul.
>



-- 
Tarek Ziad? | http://ziade.org

From ronaldoussoren at mac.com  Mon Jun  8 17:23:46 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Mon, 08 Jun 2009 08:23:46 -0700
Subject: [Distutils] PEP 376 comments
In-Reply-To: <94bdd2610906080816o4c35aab0yf84a2b1be43d1a39@mail.gmail.com>
References: <FBC8A392-2135-41D1-9129-4688C5771C7D@mac.com>
	<94bdd2610906080258m2ce4398bg1f072176ff26a023@mail.gmail.com>
	<5D2DA27E-1EC7-4032-86C1-F535A3D7DCB3@mac.com>
	<79990c6b0906080811t7b893f91r3507f523042db59d@mail.gmail.com>
	<94bdd2610906080816o4c35aab0yf84a2b1be43d1a39@mail.gmail.com>
Message-ID: <75A9B731-72C4-477D-B660-DAD382A8D780@mac.com>


On 8 Jun, 2009, at 8:16, Tarek Ziad? wrote:

> On Mon, Jun 8, 2009 at 5:11 PM, Paul Moore<p.f.moore at gmail.com> wrote:
>> 2009/6/8 Ronald Oussoren <ronaldoussoren at mac.com>:
>>> I guess the problem only occurs if "egg_infos" is part of the public
>>> interface. What if module A calls get_egg_infos(), then module B  
>>> calls
>>> get_egg_infos(somePath), then module A uses "egg_infos" assuming  
>>> that it
>>> still refers to the same set of paths.
>>
>> Vaguely off-topic, but is it only me that hates the abbreviation
>> "infos"? It can only realistically be an abbreviation for the word
>> "informations" - and there's no such word :-(
>
> Mmm, that's my frenglish, Sir  ;)  (If you see other weird stuff
> please let me know)
>
> So I guess "get_egg_info" is the shortest, proper form ?

The thingy we're getting is called an "EggInfo", which would IMHO mean  
that "get_egg_infos" is technically the correct name for something  
that returns a list of them. I'd interpret the singular form as a  
function that returns one EggInfo object.

Disclaimer: I'm not a native english speaker

Ronald
>
>
>>
>> Paul.
>>
>
>
>
> -- 
> Tarek Ziad? | http://ziade.org


From ronaldoussoren at mac.com  Mon Jun  8 17:27:32 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Mon, 08 Jun 2009 08:27:32 -0700
Subject: [Distutils] setuptools and py3k
In-Reply-To: <h0j1n5$u9k$1@ger.gmane.org>
References: <EE65BCC4-F1AE-40C1-83CE-6A5D60D97BFD@mac.com>
	<h0j1n5$u9k$1@ger.gmane.org>
Message-ID: <4DBAED55-17CF-4CD7-BDBB-EE05A5C8643B@mac.com>


On 8 Jun, 2009, at 5:53, Tres Seaver wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Ronald Oussoren wrote:
>
>> But first a question:  some of the tests use docutils and I'm  
>> having a
>> hard time getting those to run with both python3 and python2. The bit
>> that's particularly annoying is this bit of api_tests.txt:
>>
>>    Note that asking for a conflicting version of a distribution
>> already in a
>>    working set triggers a ``pkg_resources.VersionConflict`` error:
>>
>>>>> ws.find(Requirement.parse("Bar==1.0")) # doctest:
>> +NORMALIZE_WHITESPACE*
>>        Traceback (most recent call last):
>>          ...
>>        VersionConflict: (Bar 0.9 (http://example.com/something),
>>                       Requirement.parse('Bar==1.0'))
>>
>> This passes with 2.6, but fails in 3.1 because it expects
>> 'pkg_resources.VersionConflict' rather than just 'VersionConflict'.
>> What's the best way to work around this issue?
>
> Don't rely on the repr of an exception.  This kind of problem is why I
> hate using doctest to test edge cases.  If you must, then something  
> like:
>
>>>> from pkg_resources import VersionConflict
>>>> try:
>   ...     ws.find(Requirement.parse("Bar==1.0"))
>   ... except VersionConflict:
>   ...     pass
>   ... else:
>   ...     print 'VersionConflict not raised.'
>
> with no output expected.  Note that this is actually *less* verbose  
> and
> easier to understand than the repr bit.  Of course, a traditional unit
> test would be even shorter and easier to understand.

I like your workaround. A tradition unittest would be better, but I'd  
prefer to keep the patches for py3k support as small as possible.
>
>> The attached file "setup3.py" is a script that incrementally copies
>> the setuptools source-tree to a temporary directory, runs 2to3 on  
>> that
>> copy and exec-s the contents of the translated 'setup.py' file. This
>> seems to be good enough to be able to install setuptools for python2
>> and python3 using the same code-base.   That said, this is just the
>> first step of a python3 port there need to be changes to the
>> setuptools sources to be able to actually use the translated sources.
>
> Great work.

This is only a first step, I'm found a number of annoying issues that  
might require interface changes to get right (such a number of  
functions in pkg_resources that claim to return a string, but actually  
return bytes (by opening a file in binary mode and returning the file  
contents).  Hopefully I'll have a set of patches soonish, that would  
make discussion easier.

Ronald
>
>
> Tres.
> - --
> ===================================================================
> Tres Seaver          +1 540-429-0999          tseaver at palladion.com
> Palladion Software   "Excellence by Design"    http://palladion.com
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iD8DBQFKLQnZ+gerLs4ltQ4RAh3cAJ4wHai8q+svdYKjDLB3sk8LwshIMgCg1PDL
> xFCDzoM52IwA1n0rUlBsWpU=
> =zl9H
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig


From ziade.tarek at gmail.com  Mon Jun  8 17:33:06 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 8 Jun 2009 17:33:06 +0200
Subject: [Distutils] PEP 376 comments
In-Reply-To: <5D2DA27E-1EC7-4032-86C1-F535A3D7DCB3@mac.com>
References: <FBC8A392-2135-41D1-9129-4688C5771C7D@mac.com>
	<94bdd2610906080258m2ce4398bg1f072176ff26a023@mail.gmail.com>
	<5D2DA27E-1EC7-4032-86C1-F535A3D7DCB3@mac.com>
Message-ID: <94bdd2610906080833t3458da9ct5b3c6f37933cfacf@mail.gmail.com>

On Mon, Jun 8, 2009 at 3:04 PM, Ronald Oussoren<ronaldoussoren at mac.com> wrote:
>
> On 8 Jun, 2009, at 2:58, Tarek Ziad? wrote:
>
>> 2009/6/7 Ronald Oussoren <ronaldoussoren at mac.com>:
>>
>>>
>>> * The global functions seem to maintain and modify global state, wouldn't
>>> this cause problems if I specify different values of the path arguments in
>>> different pieces of code?
>>
>> The cache just prevents re-reading a directory content. ?Do you have a
>> scenario in mind of a possible problem ?
>> The only problem I can see is when a project is uninstalled by a
>> program while these APIs are used by another program.
>
> I guess the problem only occurs if "egg_infos" is part of the public
> interface. What if module A calls get_egg_infos(), then module B calls
> get_egg_infos(somePath), then module A uses "egg_infos" assuming that it
> still refers to the same set of paths.

I'll work on that.

>
> There's also the issue of API complexity. The global functions don't add a
> lot to using an EggInfoDirectories object when you are working with a
> different set of paths than sys.path.

We have created collections to make it easy to use from a third party
application
that wants to work with the directories, and make it easy to bypass the cache.

These global functions are just shortcuts for the most common usage
and not really intended
to do custom work. Maybe the paths argument could be completely
removed si they are specialized for sys.path.

>
> Ronald



-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Mon Jun  8 17:35:13 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 8 Jun 2009 17:35:13 +0200
Subject: [Distutils] PEP 376 comments
In-Reply-To: <75A9B731-72C4-477D-B660-DAD382A8D780@mac.com>
References: <FBC8A392-2135-41D1-9129-4688C5771C7D@mac.com>
	<94bdd2610906080258m2ce4398bg1f072176ff26a023@mail.gmail.com>
	<5D2DA27E-1EC7-4032-86C1-F535A3D7DCB3@mac.com>
	<79990c6b0906080811t7b893f91r3507f523042db59d@mail.gmail.com>
	<94bdd2610906080816o4c35aab0yf84a2b1be43d1a39@mail.gmail.com>
	<75A9B731-72C4-477D-B660-DAD382A8D780@mac.com>
Message-ID: <94bdd2610906080835i14f3a699vb8a3e6d6d82b0162@mail.gmail.com>

On Mon, Jun 8, 2009 at 5:23 PM, Ronald Oussoren<ronaldoussoren at mac.com> wrote:
>
> On 8 Jun, 2009, at 8:16, Tarek Ziad? wrote:
>
>> On Mon, Jun 8, 2009 at 5:11 PM, Paul Moore<p.f.moore at gmail.com> wrote:
>>>
>>> 2009/6/8 Ronald Oussoren <ronaldoussoren at mac.com>:
>>>>
>>>> I guess the problem only occurs if "egg_infos" is part of the public
>>>> interface. What if module A calls get_egg_infos(), then module B calls
>>>> get_egg_infos(somePath), then module A uses "egg_infos" assuming that it
>>>> still refers to the same set of paths.
>>>
>>> Vaguely off-topic, but is it only me that hates the abbreviation
>>> "infos"? It can only realistically be an abbreviation for the word
>>> "informations" - and there's no such word :-(
>>
>> Mmm, that's my frenglish, Sir ?;) ?(If you see other weird stuff
>> please let me know)
>>
>> So I guess "get_egg_info" is the shortest, proper form ?
>
> The thingy we're getting is called an "EggInfo", which would IMHO mean that
> "get_egg_infos" is technically the correct name for something that returns a
> list of them. I'd interpret the singular form as a function that returns one
> EggInfo object.

I have a problem now with the EggInfo name, because of the repetition:

EggInfo.get_egg_info_file

I think EggInfo could be renamed to something better, like DistInfo maybe,
(not PkgInfo because it handles more that PKG-INFO)

Tarek

From p.f.moore at gmail.com  Mon Jun  8 17:35:22 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 8 Jun 2009 16:35:22 +0100
Subject: [Distutils] PEP 376 comments
In-Reply-To: <94bdd2610906080816o4c35aab0yf84a2b1be43d1a39@mail.gmail.com>
References: <FBC8A392-2135-41D1-9129-4688C5771C7D@mac.com>
	<94bdd2610906080258m2ce4398bg1f072176ff26a023@mail.gmail.com>
	<5D2DA27E-1EC7-4032-86C1-F535A3D7DCB3@mac.com>
	<79990c6b0906080811t7b893f91r3507f523042db59d@mail.gmail.com>
	<94bdd2610906080816o4c35aab0yf84a2b1be43d1a39@mail.gmail.com>
Message-ID: <79990c6b0906080835y736ad7bbu7e198ac6bc2c71b0@mail.gmail.com>

2009/6/8 Tarek Ziad? <ziade.tarek at gmail.com>:

> Mmm, that's my frenglish, Sir ?;) ?(If you see other weird stuff
> please let me know)

Well, it's better than my French, so who am I to complain? (As
everyone knows, the English speak French by USING ENGLISH WORDS, BUT
SHOUTING... :-))

> So I guess "get_egg_info" is the shortest, proper form ?

That would be fine, yes. (Though I have a sneaking suspicion that I
saw one place which had both _info and _infos, with the _infos form
being the way of getting multiple copies of the things that _info gets
one of. That's harder to deal with - I'll have a look and see if I can
find the place I mean.)

[...]

2009/6/8 Ronald Oussoren <ronaldoussoren at mac.com>:
> The thingy we're getting is called an "EggInfo", which would IMHO mean that
> "get_egg_infos" is technically the correct name for something that returns a
> list of them. I'd interpret the singular form as a function that returns one
> EggInfo object.

That's the place I mean. But I'm not sure I like the idea of calling
it an "EggInfo". I'll see if I can think of a better name (but not
being familiar with the domain, I'm not sure I'll be able to). If it
has to be an "EggInfo", then you shouldn't break it with underscores -
so you have

    get_egginfo - get an EggInfo object from wherever
    get_egginfos - get a list of EggInfo objects

(and make sure that the invented term EggInfo is defined somewhere
before it gets used here).

Paul.

From sridharr at activestate.com  Mon Jun  8 19:36:23 2009
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Mon, 08 Jun 2009 10:36:23 -0700
Subject: [Distutils] PEP 376 comments
In-Reply-To: <94bdd2610906080258m2ce4398bg1f072176ff26a023@mail.gmail.com>
References: <FBC8A392-2135-41D1-9129-4688C5771C7D@mac.com>
	<94bdd2610906080258m2ce4398bg1f072176ff26a023@mail.gmail.com>
Message-ID: <4A2D4C17.3010105@activestate.com>

On 09-06-08 02:58 AM, Tarek Ziad? wrote:
>> >
>> >  * Should the PEP specify the encoding of text-files? PEP314 doesn't seem to specify the encoding of PKG-INFO files, which can cause problems when a field contains data that isn't ASCII.
>
> The encoding used is utf-8 since 2.6.  I think we should rather update
> PEP 314, and mention it in the upcoming PEP 345 as well,

For the python-wifi-0.3.1 package, I noticed that PKG-INFO uses 'latin1' 
encoding.

 > grep Author python_wifi.egg-info/PKG-INFO | head -n 1
Author: R?man Joost

 > grep Author python_wifi.egg-info/PKG-INFO | hexdump -C | head -n 1
00000000  41 75 74 68 6f 72 3a 20  52 f3 6d 61 6e 20 4a 6f  |Author: 
R.man Jo|

Note: latin1 uses "f3", while utf-8 uses "c3 b3".


The reason for not using utf-8 in PKG-INFO is perhaps due to the 
presence of "# -*- coding: latin1 -*-" in setup.py. Cf. PEP 0263

Indeed, even 'python setup.py --author' sends latin1 encoded bytes (not 
utf-8).

From ronaldoussoren at mac.com  Mon Jun  8 22:48:09 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Mon, 08 Jun 2009 13:48:09 -0700
Subject: [Distutils] PEP 376 comments
In-Reply-To: <4A2D4C17.3010105@activestate.com>
References: <FBC8A392-2135-41D1-9129-4688C5771C7D@mac.com>
	<94bdd2610906080258m2ce4398bg1f072176ff26a023@mail.gmail.com>
	<4A2D4C17.3010105@activestate.com>
Message-ID: <EA36B0D3-A9B7-4A06-9532-5E0D362297CD@mac.com>


On 8 Jun, 2009, at 10:36, Sridhar Ratnakumar wrote:

> On 09-06-08 02:58 AM, Tarek Ziad? wrote:
>>> >
>>> >  * Should the PEP specify the encoding of text-files? PEP314  
>>> doesn't seem to specify the encoding of PKG-INFO files, which can  
>>> cause problems when a field contains data that isn't ASCII.
>>
>> The encoding used is utf-8 since 2.6.  I think we should rather  
>> update
>> PEP 314, and mention it in the upcoming PEP 345 as well,
>
> For the python-wifi-0.3.1 package, I noticed that PKG-INFO uses  
> 'latin1' encoding.

Not quite. My best guess is that it doesn't use any encoding, it just  
dumps the bytes in the string you specified in the PKG-INFO file (at  
least for python2, I haven't checked what distutils does in python3).

Ronald

From p.f.moore at gmail.com  Mon Jun  8 23:05:07 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 8 Jun 2009 22:05:07 +0100
Subject: [Distutils] PEP 376 comments
In-Reply-To: <79990c6b0906080835y736ad7bbu7e198ac6bc2c71b0@mail.gmail.com>
References: <FBC8A392-2135-41D1-9129-4688C5771C7D@mac.com>
	<94bdd2610906080258m2ce4398bg1f072176ff26a023@mail.gmail.com>
	<5D2DA27E-1EC7-4032-86C1-F535A3D7DCB3@mac.com>
	<79990c6b0906080811t7b893f91r3507f523042db59d@mail.gmail.com>
	<94bdd2610906080816o4c35aab0yf84a2b1be43d1a39@mail.gmail.com>
	<79990c6b0906080835y736ad7bbu7e198ac6bc2c71b0@mail.gmail.com>
Message-ID: <79990c6b0906081405i2026afcevaa858aaa87361273@mail.gmail.com>

2009/6/8 Paul Moore <p.f.moore at gmail.com>:
> That's the place I mean. But I'm not sure I like the idea of calling
> it an "EggInfo". I'll see if I can think of a better name (but not
> being familiar with the domain, I'm not sure I'll be able to).

Hmm.

Reading the PEP, the EggInfo class represents the contents of the
.egg-info directory. OK, but .egg-info is a directory containing
information about the egg. The class name implies that directory
contains an "egg info" (ie, an "egg information"). But that's not
grammatically correct - you don't have "an information".

So the question is, what is a good (singular noun) word for the
collection of data in the .egg-info directory?

I don't know enough of the concepts/terminology for this stuff to be
able to suggest anything, though.

Paul.

From pje at telecommunity.com  Mon Jun  8 23:14:57 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 08 Jun 2009 17:14:57 -0400
Subject: [Distutils] PEP 376 comments
In-Reply-To: <79990c6b0906080835y736ad7bbu7e198ac6bc2c71b0@mail.gmail.co
 m>
References: <FBC8A392-2135-41D1-9129-4688C5771C7D@mac.com>
	<94bdd2610906080258m2ce4398bg1f072176ff26a023@mail.gmail.com>
	<5D2DA27E-1EC7-4032-86C1-F535A3D7DCB3@mac.com>
	<79990c6b0906080811t7b893f91r3507f523042db59d@mail.gmail.com>
	<94bdd2610906080816o4c35aab0yf84a2b1be43d1a39@mail.gmail.com>
	<79990c6b0906080835y736ad7bbu7e198ac6bc2c71b0@mail.gmail.com>
Message-ID: <20090608211206.D977E3A402D@sparrow.telecommunity.com>

At 04:35 PM 6/8/2009 +0100, Paul Moore wrote:
>2009/6/8 Ronald Oussoren <ronaldoussoren at mac.com>:
> > The thingy we're getting is called an "EggInfo", which would IMHO mean that
> > "get_egg_infos" is technically the correct name for something 
> that returns a
> > list of them. I'd interpret the singular form as a function that 
> returns one
> > EggInfo object.
>
>That's the place I mean. But I'm not sure I like the idea of calling
>it an "EggInfo". I'll see if I can think of a better name (but not
>being familiar with the domain, I'm not sure I'll be able to).

FWIW, pkg_resources uses "Distribution", and it has functions such as 
find_distributions().


From kevin at bud.ca  Tue Jun  9 00:07:33 2009
From: kevin at bud.ca (Kevin Teague)
Date: Mon, 8 Jun 2009 15:07:33 -0700 (PDT)
Subject: [Distutils] PEP 376 comments
In-Reply-To: <20090608211206.D977E3A402D@sparrow.telecommunity.com>
References: <FBC8A392-2135-41D1-9129-4688C5771C7D@mac.com> 
	<94bdd2610906080258m2ce4398bg1f072176ff26a023@mail.gmail.com> 
	<5D2DA27E-1EC7-4032-86C1-F535A3D7DCB3@mac.com>
	<79990c6b0906080811t7b893f91r3507f523042db59d@mail.gmail.com> 
	<94bdd2610906080816o4c35aab0yf84a2b1be43d1a39@mail.gmail.com> 
	<79990c6b0906080835y736ad7bbu7e198ac6bc2c71b0@mail.gmail.com> 
	<20090608211206.D977E3A402D@sparrow.telecommunity.com>
Message-ID: <efaba99f-1878-44c8-b738-9fdec150cab7@z7g2000vbh.googlegroups.com>

Glancing over the EggInfo, EggInfoDirectories, and EggInfoDirectory
APIs, it might be interesting if these were able to match with the PEP
3119 interfaces (http://www.python.org/dev/peps/pep-3119/).

For example, the EggInfoDirectories has __iter__, append, clear and
remove methods, but the source code comments list them as 'container
APIs'. Technically though, that's an Iterable interface and a partial
implementation of the MutableSequence interface.

The EggInfoDirectories.get_egg_infos(project_name) is also suggestive
of the Mapping interface - if this method was shortened from
get_egg_infos() to just get().

Maybe it would be confusing if an EggInfoDirectories was a
MutableSequence which could only have paths added to it, but returned
an iterable of EggInfoDirectory, but could also be accessed as a
Mapping where the keys are project names and those values are
EggInfos ... but maybe something along those lines would work?

From sridharr at activestate.com  Tue Jun  9 00:18:37 2009
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Mon, 08 Jun 2009 15:18:37 -0700
Subject: [Distutils] PEP 376 comments
In-Reply-To: <EA36B0D3-A9B7-4A06-9532-5E0D362297CD@mac.com>
References: <FBC8A392-2135-41D1-9129-4688C5771C7D@mac.com>
	<94bdd2610906080258m2ce4398bg1f072176ff26a023@mail.gmail.com>
	<4A2D4C17.3010105@activestate.com>
	<EA36B0D3-A9B7-4A06-9532-5E0D362297CD@mac.com>
Message-ID: <4A2D8E3D.6050004@activestate.com>

On 09-06-08 01:48 PM, Ronald Oussoren wrote:
>
> On 8 Jun, 2009, at 10:36, Sridhar Ratnakumar wrote:
>
>> On 09-06-08 02:58 AM, Tarek Ziad? wrote:
>>>> >
>>>> >  * Should the PEP specify the encoding of text-files? PEP314
>>>> doesn't seem to specify the encoding of PKG-INFO files, which can
>>>> cause problems when a field contains data that isn't ASCII.
>>>
>>> The encoding used is utf-8 since 2.6.  I think we should rather update
>>> PEP 314, and mention it in the upcoming PEP 345 as well,
>>
>> For the python-wifi-0.3.1 package, I noticed that PKG-INFO uses
>> 'latin1' encoding.
>
> Not quite.

In this particular package, yes.. but not generally of course.

> My best guess is that it doesn't use any encoding, it just
> dumps the bytes in the string you specified in the PKG-INFO file (at
> least for python2, I haven't checked what distutils does in python3).

Correct; that is what I thought. Tarek said "The encoding used is utf-8 
since 2.6" .. for which I provided this counter-example (python-wifi-0.3.1).

From david.lyon at preisshare.net  Tue Jun  9 01:50:11 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Mon, 08 Jun 2009 19:50:11 -0400
Subject: [Distutils]
 =?utf-8?q?Packaging=3A_bdist=5Fdeb_patches_/_platform?=
 =?utf-8?q?_specific_code?=
In-Reply-To: <94bdd2610906080528x4724d980oc54af2b3e7464385@mail.gmail.com>
References: <49B7B62F.40306@verizon.net> <4A2AEB90.4060607@verizon.net>
	<94bdd2610906080528x4724d980oc54af2b3e7464385@mail.gmail.com>
Message-ID: <34c7b7f6afca67f0f32fe364c32edbe2@preisshare.net>

On Mon, 8 Jun 2009 14:28:52 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
wrote:
> During the summit at Pycon, we have said that it would be a better
> strategy not to include
> within Distutils os-specific tools for various reasons (and also to
> remove existing ones) :
> 
> - it's better for them to have their own release cycles
> - it's hard for me, as distutils maintainer, to maintain and make
> evolve os-specific tools. People that are specialists on those OS
>   will do a better job.

Let's think about this.... and what it means.....

It seems that this direction comes down to "we don't want to maintain it"
rather than anything else.

The whole point of distutils is to be able to make up a package in
such a way that it can be distributed to every platform.... right?

So removing the code that is specific to specific platforms is just
collapsing the whole project.

Rather than "removing" platform specific code... and rendering
distutils crippled... why not work on developing the code so that
it operates more like a "cross-compiler"...

That is...

runs on any platform...

builds for any platform....

David 







From ben+python at benfinney.id.au  Tue Jun  9 02:13:38 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Tue, 09 Jun 2009 10:13:38 +1000
Subject: [Distutils] PEP 376 comments
References: <FBC8A392-2135-41D1-9129-4688C5771C7D@mac.com>
	<94bdd2610906080258m2ce4398bg1f072176ff26a023@mail.gmail.com>
	<5D2DA27E-1EC7-4032-86C1-F535A3D7DCB3@mac.com>
	<79990c6b0906080811t7b893f91r3507f523042db59d@mail.gmail.com>
	<94bdd2610906080816o4c35aab0yf84a2b1be43d1a39@mail.gmail.com>
	<75A9B731-72C4-477D-B660-DAD382A8D780@mac.com>
Message-ID: <87r5xuclhp.fsf@benfinney.id.au>

Ronald Oussoren <ronaldoussoren at mac.com> writes:

> On 8 Jun, 2009, at 8:16, Tarek Ziad? wrote:
> 
> > On Mon, Jun 8, 2009 at 5:11 PM, Paul Moore<p.f.moore at gmail.com> wrote:
> >> Vaguely off-topic, but is it only me that hates the abbreviation
> >> "infos"? It can only realistically be an abbreviation for the word
> >> "informations" - and there's no such word :-(

It's not only you; it's bad English, which harms communication
perceptibly.

> > Mmm, that's my frenglish, Sir  ;)  (If you see other weird stuff
> > please let me know)
> >
> > So I guess "get_egg_info" is the shortest, proper form ?
> 
> The thingy we're getting is called an "EggInfo", which would IMHO mean
> that "get_egg_infos" is technically the correct name for something
> that returns a list of them. I'd interpret the singular form as a
> function that returns one EggInfo object.
> 
> Disclaimer: I'm not a native english speaker

The singular form is correct for uncountable nouns, such as substances
or concepts. Nouns like ?equipment? or ?information? are
uncountable, so do not get pluralised: we don't specify the *number*,
but rather the *amount* of that noun.
<URL:http://esl.about.com/od/grammarforbeginners/a/g_cucount.htm>

Now, in this case, the term is being used to name the type of a Python
object, and Python objects certainly *are* countable :-) I would want to
name the container or collection: perhaps ?get_egg_info_list?, but
someone who knows the full purpose of that function can probably improve
that.

-- 
 \         ?Broken promises don't upset me. I just think, why did they |
  `\                                         believe me?? ?Jack Handey |
_o__)                                                                  |
Ben Finney


From david.lyon at preisshare.net  Tue Jun  9 04:04:20 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Mon, 08 Jun 2009 22:04:20 -0400
Subject: [Distutils] =?utf-8?q?What=27s_the_Distutils/setuptools_api_to_in?=
	=?utf-8?q?stall_an_=2Eegg=3F?=
In-Reply-To: <e1ee33321aebdd2c78c81343fe4d0882@preisshare.net>
References: <94bdd2610905250329n21fc744bu52327e2ed0c15440@mail.gmail.com>
	<f924b071972615dceaa73580f53402fa@preisshare.net>
	<94bdd2610905280321s5d8811eat26e6aae3bb3b7c1a@mail.gmail.com>
	<e1ee33321aebdd2c78c81343fe4d0882@preisshare.net>
Message-ID: <95a82d74f4a8d6e41e560a6b226e712a@preisshare.net>


Hi All,

What's the distutils api call that I need to make to install
an .egg.

At the moment I'm using easy_install and the command line
but under vista it requires escalated privilages.

There must be a "batteries-included" way within python
either in setuptools or distutils to install an .EGG
from pypi? That is without telling the users that they
must download either easy_install or pip before they
can do the .egg install.

Thanks in advance

David

From santagada at gmail.com  Tue Jun  9 04:37:42 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Mon, 8 Jun 2009 23:37:42 -0300
Subject: [Distutils] Packaging: bdist_deb patches / platform specific
	code
In-Reply-To: <34c7b7f6afca67f0f32fe364c32edbe2@preisshare.net>
References: <49B7B62F.40306@verizon.net> <4A2AEB90.4060607@verizon.net>
	<94bdd2610906080528x4724d980oc54af2b3e7464385@mail.gmail.com>
	<34c7b7f6afca67f0f32fe364c32edbe2@preisshare.net>
Message-ID: <E3D9C8B5-9650-4323-8991-CBEBF9802E19@gmail.com>


On Jun 8, 2009, at 8:50 PM, David Lyon wrote:

> On Mon, 8 Jun 2009 14:28:52 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
> wrote:
>> During the summit at Pycon, we have said that it would be a better
>> strategy not to include
>> within Distutils os-specific tools for various reasons (and also to
>> remove existing ones) :
>>
>> - it's better for them to have their own release cycles
>> - it's hard for me, as distutils maintainer, to maintain and make
>> evolve os-specific tools. People that are specialists on those OS
>>  will do a better job.
>
> Let's think about this.... and what it means.....
>
> It seems that this direction comes down to "we don't want to  
> maintain it"
> rather than anything else.
>
> The whole point of distutils is to be able to make up a package in
> such a way that it can be distributed to every platform.... right?
>
> So removing the code that is specific to specific platforms is just
> collapsing the whole project.
>
> Rather than "removing" platform specific code... and rendering
> distutils crippled... why not work on developing the code so that
> it operates more like a "cross-compiler"...
>
> That is...
>
> runs on any platform...
>
> builds for any platform....


Because this is almost impossible... for example building a win_inst  
package on linux (let alone other unixes, etc) is very very hard  
(specially if it involves extensions), the same for rpm, deb and mpkg.

Distutils should provide all tools to make this happen, but the  
package creation itself should be let to each system. This is the same  
problem with svn and setuptools where you end up with non working code  
for long periods of time. Now that python and probably a ton of other  
projects are going to use mercurial this can even become worse.

--
Leonardo Santagada
santagada at gmail.com




From david.lyon at preisshare.net  Tue Jun  9 05:15:06 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Mon, 08 Jun 2009 23:15:06 -0400
Subject: [Distutils]
 =?utf-8?q?Packaging=3A_bdist=5Fdeb_patches_/_platform?=
 =?utf-8?q?_specific_code?=
In-Reply-To: <E3D9C8B5-9650-4323-8991-CBEBF9802E19@gmail.com>
References: <49B7B62F.40306@verizon.net> <4A2AEB90.4060607@verizon.net>
	<94bdd2610906080528x4724d980oc54af2b3e7464385@mail.gmail.com>
	<34c7b7f6afca67f0f32fe364c32edbe2@preisshare.net>
	<E3D9C8B5-9650-4323-8991-CBEBF9802E19@gmail.com>
Message-ID: <9508b01923109d4f42c2bb6cde766b87@preisshare.net>

On Mon, 8 Jun 2009 23:37:42 -0300, Leonardo Santagada <santagada at gmail.com>
wrote:
>> builds for any platform....
> 
> Because this is almost impossible... 

Not true...

Don't forget, cygwin runs under windows, and wine runs under linux

So it's not so difficult to build windows targets under linux and
linux targets under windows.

Cross compiling has been common practice for the last 30 years.

> for example building a win_inst  
> package on linux (let alone other unixes, etc) is very very hard  
> (specially if it involves extensions).

It should work under wine, or should be made to work under wine.

This is an example of something that can be adapted to work
under linux with not too many problems.

> the same for rpm, deb and mpkg.

Granted. They're are a bit more tricky. But essentially they
are all just zip files with some entry and exit scripts.

> Distutils should provide all tools to make this happen, but the  
> package creation itself should be let to each system. 

At a minimum, distutils should be able to create an .egg that will
work on every system.

I still believe in cross-compilers... even python itself is a
sort of cross compiler.... killing the cross compilation side of
distutils seems way worse than enhancing it....

David


From david at ar.media.kyoto-u.ac.jp  Tue Jun  9 05:20:17 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Tue, 09 Jun 2009 12:20:17 +0900
Subject: [Distutils] Packaging: bdist_deb patches / platform specific
 code
In-Reply-To: <9508b01923109d4f42c2bb6cde766b87@preisshare.net>
References: <49B7B62F.40306@verizon.net>
	<4A2AEB90.4060607@verizon.net>	<94bdd2610906080528x4724d980oc54af2b3e7464385@mail.gmail.com>	<34c7b7f6afca67f0f32fe364c32edbe2@preisshare.net>	<E3D9C8B5-9650-4323-8991-CBEBF9802E19@gmail.com>
	<9508b01923109d4f42c2bb6cde766b87@preisshare.net>
Message-ID: <4A2DD4F1.7010503@ar.media.kyoto-u.ac.jp>

David Lyon wrote:
>
> Not true...
>
> Don't forget, cygwin runs under windows

And how does that help to build a linux binary ?

> , and wine runs under linux
>   

Yes, but Visual Studio does not. So you are out of luck if you need full
compatibility with pre-built python binaries on python.org. It works
almost ok with mingw, but not always - and it is far from working well
on windows 64 bits BTW.

Anyway, as long as distutils works under windows, distutils will work
under wine - the problem is the toolchain, not distutils.

> Cross compiling has been common practice for the last 30 years.
>   

And still hard to get right. That should certainly not be built in
distutils - a sane build tool needs very little knowledge about cross
compilation, really. Python itself cannot be cross-compiled out of the
box anyway.

> At a minimum, distutils should be able to create an .egg that will
> work on every system.
>   

Before implementing things that no other system has ever done, we should
focus on basic things that distutils cannot do today and are expected
from a build/distribution tool.

Those are hard problems to get right, saying that it is not hard because
it is just moving files and scripts does not really help. That's like
saying programming is easy because it is just inputting characters in a
text file.

cheers,

David

From david.lyon at preisshare.net  Tue Jun  9 05:36:25 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Mon, 08 Jun 2009 23:36:25 -0400
Subject: [Distutils]
 =?utf-8?q?Packaging=3A_bdist=5Fdeb_patches_/_platform?=
 =?utf-8?q?_specific_code?=
In-Reply-To: <E3D9C8B5-9650-4323-8991-CBEBF9802E19@gmail.com>
References: <49B7B62F.40306@verizon.net> <4A2AEB90.4060607@verizon.net>
	<94bdd2610906080528x4724d980oc54af2b3e7464385@mail.gmail.com>
	<34c7b7f6afca67f0f32fe364c32edbe2@preisshare.net>
	<E3D9C8B5-9650-4323-8991-CBEBF9802E19@gmail.com>
Message-ID: <7498ab0b5527891c87f02fa5525f62f6@preisshare.net>

On Mon, 8 Jun 2009 23:37:42 -0300, Leonardo Santagada <santagada at gmail.com>
wrote:
>> That is...
>>
>> runs on any platform...
>>
>> builds for any platform....
> 
> 
> Because this is almost impossible... for example building a win_inst  
> package on linux (let alone other unixes, etc) is very very hard  
> (specially if it involves extensions), the same for rpm, deb and mpkg.

quoting...

http://docs.python.org/distutils/builtdist.html#cross-compiling-on-windows

"A built distribution is how you make life as easy as possible for
installers of your module distribution: for users of RPM-based Linux
systems, it?s a binary RPM; for Windows users, it?s an executable
installer; for Debian-based Linux users, it?s a Debian package; and so
forth. Obviously, no one person will be able to create built distributions
for every platform under the sun, so the Distutils are designed to enable
module developers to concentrate on their specialty?writing code and
creating source distributions"



From david.lyon at preisshare.net  Tue Jun  9 05:52:49 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Mon, 08 Jun 2009 23:52:49 -0400
Subject: [Distutils]
 =?utf-8?q?Packaging=3A_bdist=5Fdeb_patches_/_platform?=
 =?utf-8?q?_specific_code?=
In-Reply-To: <4A2DD4F1.7010503@ar.media.kyoto-u.ac.jp>
References: <49B7B62F.40306@verizon.net>
	<4A2AEB90.4060607@verizon.net>	<94bdd2610906080528x4724d980oc54af2b3e7464385@mail.gmail.com>	<34c7b7f6afca67f0f32fe364c32edbe2@preisshare.net>	<E3D9C8B5-9650-4323-8991-CBEBF9802E19@gmail.com>
	<9508b01923109d4f42c2bb6cde766b87@preisshare.net>
	<4A2DD4F1.7010503@ar.media.kyoto-u.ac.jp>
Message-ID: <853735a66e33d13dab5dec6ac3f51421@preisshare.net>

On Tue, 09 Jun 2009 12:20:17 +0900, David Cournapeau
>> Don't forget, cygwin runs under windows
> 
> And how does that help to build a linux binary ?

I wasn't thinking of it for building binaries.. rather
using various packaging tools that could be adapted or
compiled from source to run. 

Not all distributions need to be binary executables.
> 
>> , and wine runs under linux
>>   
> 
> Yes, but Visual Studio does not. So you are out of luck if you need full
> compatibility with pre-built python binaries on python.org. It works
> almost ok with mingw, but not always - and it is far from working well
> on windows 64 bits BTW.

Yes. Point taken.

But not everybody is building C extensions for python.

A lot of python libraries are built with absolutely no C code within
them at all.

> Anyway, as long as distutils works under windows, distutils will work
> under wine - the problem is the toolchain, not distutils.

Yes. That's correct.

>> Cross compiling has been common practice for the last 30 years.
> 
> And still hard to get right. That should certainly not be built in
> distutils - 

Well distutils already has partial support for it.

And the alternative being disccused here is removing the platform 
specific code.

quoting
http://docs.python.org/distutils/builtdist.html#cross-compiling-on-windows

"A built distribution is how you make life as easy as possible for
installers of your module distribution: for users of RPM-based Linux
systems, it?s a binary RPM; for Windows users, it?s an executable
installer; for Debian-based Linux users, it?s a Debian package; and so
forth. Obviously, no one person will be able to create built distributions
for every platform under the sun, so the Distutils are designed to enable
module developers to concentrate on their specialty?writing code and
creating source distributions"

>> At a minimum, distutils should be able to create an .egg that will
>> work on every system.
> 
> Before implementing things that no other system has ever done, we should
> focus on basic things that distutils cannot do today and are expected
> from a build/distribution tool.

I'm just reading the distutils documentation.....

The documentation is setting an expectation...

Yes - you can be expected to work towards the goals that are documented
within your project.

Change the documentation and don't promise it if it is not going to
be delivered.....

> Those are hard problems to get right, saying that it is not hard because
> it is just moving files and scripts does not really help. That's like
> saying programming is easy because it is just inputting characters in a
> text file.

yeah - but there are clever people working on this stuff....

They should be encouraged to work towards the solution as documented.

David


From david.lyon at preisshare.net  Tue Jun  9 06:04:17 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 09 Jun 2009 00:04:17 -0400
Subject: [Distutils]
 =?utf-8?q?Packaging=3A_bdist=5Fdeb_patches_/_platform?=
 =?utf-8?q?_specific_code?=
In-Reply-To: <4A2DD4F1.7010503@ar.media.kyoto-u.ac.jp>
References: <49B7B62F.40306@verizon.net>
	<4A2AEB90.4060607@verizon.net>	<94bdd2610906080528x4724d980oc54af2b3e7464385@mail.gmail.com>	<34c7b7f6afca67f0f32fe364c32edbe2@preisshare.net>	<E3D9C8B5-9650-4323-8991-CBEBF9802E19@gmail.com>
	<9508b01923109d4f42c2bb6cde766b87@preisshare.net>
	<4A2DD4F1.7010503@ar.media.kyoto-u.ac.jp>
Message-ID: <5013e21e9d6f6e8c79bb594ba9c450fc@preisshare.net>

On Tue, 09 Jun 2009 12:20:17 +0900, David Cournapeau
> Before implementing things that no other system has ever done, we should
> focus on basic things that distutils cannot do today and are expected
> from a build/distribution tool.

"If you have a pure module distribution (only containing pure Python
modules and packages), the resulting installer will be version independent
and have a name like foo-1.0.win32.exe. These installers can even be
created on Unix platforms or Mac OS X."

http://docs.python.org/distutils/builtdist.html#creating-windows-installers

David

From santagada at gmail.com  Tue Jun  9 07:22:39 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Tue, 9 Jun 2009 02:22:39 -0300
Subject: [Distutils] Packaging: bdist_deb patches / platform specific
	code
In-Reply-To: <853735a66e33d13dab5dec6ac3f51421@preisshare.net>
References: <49B7B62F.40306@verizon.net>
	<4A2AEB90.4060607@verizon.net>	<94bdd2610906080528x4724d980oc54af2b3e7464385@mail.gmail.com>	<34c7b7f6afca67f0f32fe364c32edbe2@preisshare.net>	<E3D9C8B5-9650-4323-8991-CBEBF9802E19@gmail.com>
	<9508b01923109d4f42c2bb6cde766b87@preisshare.net>
	<4A2DD4F1.7010503@ar.media.kyoto-u.ac.jp>
	<853735a66e33d13dab5dec6ac3f51421@preisshare.net>
Message-ID: <7B5409E9-DAB8-4F70-8F3F-47E9DFE990DE@gmail.com>


On Jun 9, 2009, at 12:52 AM, David Lyon wrote:

>>
>> Anyway, as long as distutils works under windows, distutils will work
>> under wine - the problem is the toolchain, not distutils.
>
> Yes. That's correct.
>
>>> Cross compiling has been common practice for the last 30 years.
>>
>> And still hard to get right. That should certainly not be built in
>> distutils -
>
> Well distutils already has partial support for it.
>
> And the alternative being disccused here is removing the platform
> specific code.
>
> quoting
> http://docs.python.org/distutils/builtdist.html#cross-compiling-on-windows
>
> "A built distribution is how you make life as easy as possible for
> installers of your module distribution: for users of RPM-based Linux
> systems, it?s a binary RPM; for Windows users, it?s an executable
> installer; for Debian-based Linux users, it?s a Debian package; and so
> forth. Obviously, no one person will be able to create built  
> distributions
> for every platform under the sun, so the Distutils are designed to  
> enable
> module developers to concentrate on their specialty?writing code and
> creating source distributions"
>
>>> At a minimum, distutils should be able to create an .egg that will
>>> work on every system.
>>
>> Before implementing things that no other system has ever done, we  
>> should
>> focus on basic things that distutils cannot do today and are expected
>> from a build/distribution tool.
>
> I'm just reading the distutils documentation.....
>
> The documentation is setting an expectation...
>
> Yes - you can be expected to work towards the goals that are  
> documented
> within your project.
>
> Change the documentation and don't promise it if it is not going to
> be delivered.....


The point is, getting past the trivial stuff, building packages for  
different systems is hard, maintaining the code to build packages to  
other systems is way way harder.

For example, Tarek may be using OS/2 warp (Its just an example people)  
and for him to maintain the rpm/deb/mpkg/win32_inst stuff is actually  
impossible, so he wants that people on those platforms to take  
responsability for a little part of distutils so when rpm-ng 6.0 comes  
either red hat people would say that building python package is not  
supported or they are going to update the rpm code which is external  
to disutils.

The point is not exactly cross compilation, but cross maintaining of  
distribution specific code. The only way this actually works now, I  
would guess, is because rpm format hasn't changed much, and win32_inst  
has a huge user base.

--
Leonardo Santagada
santagada at gmail.com




From trentm at gmail.com  Tue Jun  9 09:50:39 2009
From: trentm at gmail.com (Trent Mick)
Date: Tue, 9 Jun 2009 00:50:39 -0700
Subject: [Distutils] Comparison semantics for alphanumeric components of
	a version number
In-Reply-To: <79990c6b0906070948i1a50380aw59b06b73eee72dd7@mail.gmail.com>
References: <94bdd2610905290214g495a3ee8oc71710461b202f38@mail.gmail.com>
	<87d49jkpg4.fsf@benfinney.id.au>
	<79990c6b0906050543o1052576eqeced8f1a0be1d2cf@mail.gmail.com>
	<94bdd2610906050613t2f1535b4yf30fe95c3084ba16@mail.gmail.com>
	<79990c6b0906050637n13384cebme27c753725260aa7@mail.gmail.com>
	<h0ergf$j8s$1@ger.gmane.org>
	<79990c6b0906070948i1a50380aw59b06b73eee72dd7@mail.gmail.com>
Message-ID: <6db0ea510906090050s4b489e05j343bacc3d4588bef@mail.gmail.com>

On Sun, Jun 7, 2009 at 9:48 AM, Paul Moore<p.f.moore at gmail.com> wrote:
> 2009/6/6 Georg Brandl <g.brandl at gmx.net>:
>> Paul Moore schrieb:
>>> And yet, in spite of repeated requests for specific examples of
>>> projects using this post-dev stuff, which haven't been forthcoming,
>>> why is nobody saying "it's an unnecessary complication, we'll drop
>>> it"?
>>
>> Not everybody is on this list. ?At the discussion at PyCon, one participant
>> (IIRC it was Zooko) specifically asked for this option to be included.
>
> FWIW, Zooko is on the list. If it wasn't him, someone who knows who it
> was needs to ask them to supply their justification for the list -
> given that there are a number of objections being raised here which
> need to be addressed.

Yes, my memory as well is that it was Zooko. IIRC, at the time I asked
for an example of a project that used "post-releases" [^1] and the
example was Twisted.

I tried to look at the details of how Twisted's versioning works a
little bit (I'm not a Twisted developer), but before I discuss that a
little bit I think I should cover my understanding of why ".dev" and
".post" were deemed useful at the time.

(/me writes for too long). I'll send a separate email mostly
attempting to give my recollections of the Pycon discussions in answer
to:

[Paul Moore]
> I'm not trying to criticise the work done at PyCon, but I
> think the discussions (and not just the conclusions) need to be
> publicised better.


My email only covers up (but excluding) the ".dev" and ".post" parts
of the version scheme. I'll try to get to the those tomorrow night.

Trent

-- 
Trent Mick
trentm at gmail.com

From trentm at gmail.com  Tue Jun  9 09:58:48 2009
From: trentm at gmail.com (Trent Mick)
Date: Tue, 9 Jun 2009 00:58:48 -0700
Subject: [Distutils] recollections of Pycon distutils versioning discussion
	(part 1)
Message-ID: <6db0ea510906090058x76c8cadfj22e7d0dc619b56aa@mail.gmail.com>

Here-in my recollections of the Pycon distutils versioning
discussions... but only
up to the part where reasoning for the ".dev" and ".post" parts are added to the
scheme.

Hopefully the following will be helpful for reference in the current
version discussions.

# Goals

- as self-explanatory and clear versioning as reasonable. A whole dog's
  breakfast of versions is just a pain.
- deterministic translation btwn version string and version tuple. Bonus
  points if two reasonable humans would sort them the same way.
- capable dependency specs and reasonably readable

# Preliminaries

When I say things like "this isn't common on PyPI" below, this is from
analysing a dump of all the version strings currently in use on PyPI. This
list was produced by Martin von Loewis during Pycon.


# Super simple

A very simple versioning scheme for released packages would be:

    major.minor.patch

where those are all numeric fields. Update the "patch"-level when making
a small change without compatibility issues. Update the "minor" field
when adding a feature. Update the "major" field when making backward
incompat changes. Easy.

Incidentally this is what Ruby suggests for Gem authors ("Gems" are Ruby
packages), though they call the last field "build":

[http://rubygems.org/read/chapter/7]
> Any "public" release of a gem should have a different version. Normally
> that means incrementing the build number. This means a developer can
> generate builds all day long for himself, but as soon as he/she makes a
> public release, the version must be updated.


# "But I want to do an alpha release!"

I don't think I'm overstating in saying that most of us (those that care to
help in defining Python packaging tools) would want to allow alpha/beta
releases. Certainly this was true at the Pycon discussions. This gives us:

    N.N.NaN    # e.g. "1.0.0a2"
    N.N.NbN    # e.g. "2.6.0b1"

Alternatives like the following were discarded:

- "1.0.0.a2" The '.' before the 'a' separator, while nice for parsing
  is not common practice at all.
- "1.0.0alpha2" While this *does* appear on PyPI, it is less common than
  just using the single character. As well, Python itself uses 'a'.
- "1.0.0a" No alpha version. The concensus was to not support this. Reasonable
  people disagreed on whether this would imply "a0" or "a1". It was felt
  that explicit was better than implicit here. As well, Python itself always
  has a version on the alpha/beta.


# To "c", or not to "rc"?

Doing a release candidate is reasonable too, it was felt. That gives us:

    N.N.NaN    # e.g. "1.0.0a2"
    N.N.NbN    # e.g. "2.6.0b1"
    N.N.NcN    # e.g. "2.6.0c1"

Why "c" instead of "rc"? All of "c", "rc", and "candidate" are in use on
PyPI (I don't have percentages right now). "c" won because:

- that's what Python itself uses
- the one-character symmetry with 'a' and 'b' is nice
- it was felt that 'c' clearly enough indicated "release candidate"


So far I don't expect anything to be too controversial (but I'm probably
wrong :).


# Just three N's?

The current PyPI versions include quite a few versions with just two
"N's" -- e.g. "0.1", "3.5a2" -- as well as a some, though fewer, with
four N's -- e.g. "1.5.4.3". This gives us (this is just a pseudo-pattern):

  N.N[.N]*[(abc)N]

It was felt that just a single N -- e.g. "1" -- should be disallowed.
However, the upper limit was left unbounded, i.e. this is allowed:

  1.2.3.4.5.6.7.8.9a3

An example of where more N's is useful is for a Python module that wraps
a third-party library. Say that library ("libfoo") version is 2.5.2, a
reasonable version for "python-libfoo" might be "2.5.2.1.0" where the
first three bits track the "libfoo" version.


# Multiple N's after the "abc".

"1.2.0a3.4" or in the pseudo-pattern I've been using:

  N.N[.N]*[(abc)N[.N]*]

This was discussed and added. I don't recall who supported this.

Personally I've not had a need for this. 29 out of 4975 PyPI versions (in
MvL's list generated during Pycon) use this -- and in 7 of those that last
".N" is a date stamp (e.g. "1.0a2.20070215") where I think the datestamp is
meant as a sort of ".dev" or "pre-release" tag or build number.


# ... the rest

I'll try to post tomorrow night with my recollections and current
understanding of the rational for the ".dev" and ".post" parts of the
proposed version scheme.

Cheers,
Trent

-- 
Trent Mick
trentm at gmail.com

From ziade.tarek at gmail.com  Tue Jun  9 10:29:09 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 9 Jun 2009 10:29:09 +0200
Subject: [Distutils] recollections of Pycon distutils versioning
	discussion (part 1)
In-Reply-To: <6db0ea510906090058x76c8cadfj22e7d0dc619b56aa@mail.gmail.com>
References: <6db0ea510906090058x76c8cadfj22e7d0dc619b56aa@mail.gmail.com>
Message-ID: <94bdd2610906090129y424c21b8ib81ecf1d56ee461f@mail.gmail.com>

Great post, thanks a lot Trent !

Maybe you could push the summary in
http://wiki.python.org/moin/Distutils/VersionComparison
to decouple it from the Mailing List

Regards
Tarek

On Tue, Jun 9, 2009 at 9:58 AM, Trent Mick<trentm at gmail.com> wrote:
> Here-in my recollections of the Pycon distutils versioning
> discussions... but only
> up to the part where reasoning for the ".dev" and ".post" parts are added to the
> scheme.
>
> Hopefully the following will be helpful for reference in the current
> version discussions.
>
> # Goals
>
> - as self-explanatory and clear versioning as reasonable. A whole dog's
> ?breakfast of versions is just a pain.
> - deterministic translation btwn version string and version tuple. Bonus
> ?points if two reasonable humans would sort them the same way.
> - capable dependency specs and reasonably readable
>
> # Preliminaries
>
> When I say things like "this isn't common on PyPI" below, this is from
> analysing a dump of all the version strings currently in use on PyPI. This
> list was produced by Martin von Loewis during Pycon.
>
>
> # Super simple
>
> A very simple versioning scheme for released packages would be:
>
> ? ?major.minor.patch
>
> where those are all numeric fields. Update the "patch"-level when making
> a small change without compatibility issues. Update the "minor" field
> when adding a feature. Update the "major" field when making backward
> incompat changes. Easy.
>
> Incidentally this is what Ruby suggests for Gem authors ("Gems" are Ruby
> packages), though they call the last field "build":
>
> [http://rubygems.org/read/chapter/7]
>> Any "public" release of a gem should have a different version. Normally
>> that means incrementing the build number. This means a developer can
>> generate builds all day long for himself, but as soon as he/she makes a
>> public release, the version must be updated.
>
>
> # "But I want to do an alpha release!"
>
> I don't think I'm overstating in saying that most of us (those that care to
> help in defining Python packaging tools) would want to allow alpha/beta
> releases. Certainly this was true at the Pycon discussions. This gives us:
>
> ? ?N.N.NaN ? ?# e.g. "1.0.0a2"
> ? ?N.N.NbN ? ?# e.g. "2.6.0b1"
>
> Alternatives like the following were discarded:
>
> - "1.0.0.a2" The '.' before the 'a' separator, while nice for parsing
> ?is not common practice at all.
> - "1.0.0alpha2" While this *does* appear on PyPI, it is less common than
> ?just using the single character. As well, Python itself uses 'a'.
> - "1.0.0a" No alpha version. The concensus was to not support this. Reasonable
> ?people disagreed on whether this would imply "a0" or "a1". It was felt
> ?that explicit was better than implicit here. As well, Python itself always
> ?has a version on the alpha/beta.
>
>
> # To "c", or not to "rc"?
>
> Doing a release candidate is reasonable too, it was felt. That gives us:
>
> ? ?N.N.NaN ? ?# e.g. "1.0.0a2"
> ? ?N.N.NbN ? ?# e.g. "2.6.0b1"
> ? ?N.N.NcN ? ?# e.g. "2.6.0c1"
>
> Why "c" instead of "rc"? All of "c", "rc", and "candidate" are in use on
> PyPI (I don't have percentages right now). "c" won because:
>
> - that's what Python itself uses
> - the one-character symmetry with 'a' and 'b' is nice
> - it was felt that 'c' clearly enough indicated "release candidate"
>
>
> So far I don't expect anything to be too controversial (but I'm probably
> wrong :).
>
>
> # Just three N's?
>
> The current PyPI versions include quite a few versions with just two
> "N's" -- e.g. "0.1", "3.5a2" -- as well as a some, though fewer, with
> four N's -- e.g. "1.5.4.3". This gives us (this is just a pseudo-pattern):
>
> ?N.N[.N]*[(abc)N]
>
> It was felt that just a single N -- e.g. "1" -- should be disallowed.
> However, the upper limit was left unbounded, i.e. this is allowed:
>
> ?1.2.3.4.5.6.7.8.9a3
>
> An example of where more N's is useful is for a Python module that wraps
> a third-party library. Say that library ("libfoo") version is 2.5.2, a
> reasonable version for "python-libfoo" might be "2.5.2.1.0" where the
> first three bits track the "libfoo" version.
>
>
> # Multiple N's after the "abc".
>
> "1.2.0a3.4" or in the pseudo-pattern I've been using:
>
> ?N.N[.N]*[(abc)N[.N]*]
>
> This was discussed and added. I don't recall who supported this.
>
> Personally I've not had a need for this. 29 out of 4975 PyPI versions (in
> MvL's list generated during Pycon) use this -- and in 7 of those that last
> ".N" is a date stamp (e.g. "1.0a2.20070215") where I think the datestamp is
> meant as a sort of ".dev" or "pre-release" tag or build number.
>
>
> # ... the rest
>
> I'll try to post tomorrow night with my recollections and current
> understanding of the rational for the ".dev" and ".post" parts of the
> proposed version scheme.
>
> Cheers,
> Trent
>
> --
> Trent Mick
> trentm at gmail.com
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | http://ziade.org

From p.f.moore at gmail.com  Tue Jun  9 12:10:48 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 9 Jun 2009 11:10:48 +0100
Subject: [Distutils] recollections of Pycon distutils versioning
	discussion (part 1)
In-Reply-To: <6db0ea510906090058x76c8cadfj22e7d0dc619b56aa@mail.gmail.com>
References: <6db0ea510906090058x76c8cadfj22e7d0dc619b56aa@mail.gmail.com>
Message-ID: <79990c6b0906090310g40f21a2ela5d1674fd3cbaa27@mail.gmail.com>

2009/6/9 Trent Mick <trentm at gmail.com>:
> Here-in my recollections of the Pycon distutils versioning
> discussions... but only
> up to the part where reasoning for the ".dev" and ".post" parts are added to the
> scheme.
>
> Hopefully the following will be helpful for reference in the current
> version discussions.
>
> # Goals

 +1 to all this

> # Preliminaries
>
> When I say things like "this isn't common on PyPI" below, this is from
> analysing a dump of all the version strings currently in use on PyPI. This
> list was produced by Martin von Loewis during Pycon.

OK

> # Super simple
>
> A very simple versioning scheme for released packages would be:
>
> ? ?major.minor.patch
[...]
> # "But I want to do an alpha release!"
>
> I don't think I'm overstating in saying that most of us (those that care to
> help in defining Python packaging tools) would want to allow alpha/beta
> releases. Certainly this was true at the Pycon discussions. This gives us:
>
> ? ?N.N.NaN ? ?# e.g. "1.0.0a2"
> ? ?N.N.NbN ? ?# e.g. "2.6.0b1"
[...]
> # To "c", or not to "rc"?
>
> Doing a release candidate is reasonable too, it was felt. That gives us:
>
> ? ?N.N.NaN ? ?# e.g. "1.0.0a2"
> ? ?N.N.NbN ? ?# e.g. "2.6.0b1"
> ? ?N.N.NcN ? ?# e.g. "2.6.0c1"
[...]
> So far I don't expect anything to be too controversial (but I'm probably
> wrong :).

All +1 from me.

> # Just three N's?
>
> The current PyPI versions include quite a few versions with just two
> "N's" -- e.g. "0.1", "3.5a2" -- as well as a some, though fewer, with
> four N's -- e.g. "1.5.4.3". This gives us (this is just a pseudo-pattern):
>
> ?N.N[.N]*[(abc)N]
>
> It was felt that just a single N -- e.g. "1" -- should be disallowed.
> However, the upper limit was left unbounded, i.e. this is allowed:
>
> ?1.2.3.4.5.6.7.8.9a3

I see no issue with this - I'd expect it to be a relatively rarely
used generalisation, but conceptually it costs nothing, so I'm fine
with it. Actually, I do know of (non-Python) cases where a single N is
used - less is probably the best known - but I certainly don't care
enough to insist one way or the other.

> An example of where more N's is useful is for a Python module that wraps
> a third-party library. Say that library ("libfoo") version is 2.5.2, a
> reasonable version for "python-libfoo" might be "2.5.2.1.0" where the
> first three bits track the "libfoo" version.

Purely theoretical example, I assume? I doubt I'd do this, but again,
who cares? No conceptual cost.

> # Multiple N's after the "abc".
>
> "1.2.0a3.4" or in the pseudo-pattern I've been using:
>
> ?N.N[.N]*[(abc)N[.N]*]
>
> This was discussed and added. I don't recall who supported this.
>
> Personally I've not had a need for this. 29 out of 4975 PyPI versions (in
> MvL's list generated during Pycon) use this -- and in 7 of those that last
> ".N" is a date stamp (e.g. "1.0a2.20070215") where I think the datestamp is
> meant as a sort of ".dev" or "pre-release" tag or build number.

-1

29/4975 is hardly anything, and I don't believe this should be defined
on a "someone uses it so we should allow it" basis.

If someone supports this, they should be presenting a good use case,
with an explanation of why it is of value *to the end user* (ie, not
just to the project developers).

For something to consider - my view is that if you've done 2.2a1 and
you want a new release, it should be 2.2a2 - not 2.2a1.1 or any such
thing. If it's not good enough to be an a2, then why are you releasing
it? Note: there's an assumption implicit in this that a "version" is
something attached to a release - I have little sympathy with the idea
that every single Subversion revision (or Mercurial changeset, or
whatever) should have a unique "version" number. Unreleased versions
should be identified differently (and nobody should be specifying
dependencies on unreleased versions, before anybody suggests that!)

> # ... the rest
>
> I'll try to post tomorrow night with my recollections and current
> understanding of the rational for the ".dev" and ".post" parts of the
> proposed version scheme.

Thanks for posting this. So far, it's relatively uncontroversial, but
it still makes a great summary of the arguments and conclusions.

Paul.

From floris.bruynooghe at gmail.com  Tue Jun  9 13:47:34 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Tue, 9 Jun 2009 12:47:34 +0100
Subject: [Distutils] recollections of Pycon distutils
	versioning	discussion (part 1)
In-Reply-To: <79990c6b0906090310g40f21a2ela5d1674fd3cbaa27@mail.gmail.com>
References: <6db0ea510906090058x76c8cadfj22e7d0dc619b56aa@mail.gmail.com>
	<79990c6b0906090310g40f21a2ela5d1674fd3cbaa27@mail.gmail.com>
Message-ID: <20090609114734.GA30798@laurie.devork>

On Tue, Jun 09, 2009 at 11:10:48AM +0100, Paul Moore wrote:
> 2009/6/9 Trent Mick <trentm at gmail.com>:
> > # Preliminaries
> >
> > When I say things like "this isn't common on PyPI" below, this is from
> > analysing a dump of all the version strings currently in use on PyPI. This
> > list was produced by Martin von Loewis during Pycon.
> 
> OK

FYI, the mail is here:

http://mail.python.org/pipermail/distutils-sig/2009-March/011194.html

And the list here:

http://www.dcl.hpi.uni-potsdam.de/home/loewis/versions


Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From greno at verizon.net  Tue Jun  9 18:45:50 2009
From: greno at verizon.net (Gerry Reno)
Date: Tue, 09 Jun 2009 12:45:50 -0400
Subject: [Distutils] Packaging: bdist_deb patches
In-Reply-To: <94bdd2610906080528x4724d980oc54af2b3e7464385@mail.gmail.com>
References: <49B7B62F.40306@verizon.net> <4A2AEB90.4060607@verizon.net>
	<94bdd2610906080528x4724d980oc54af2b3e7464385@mail.gmail.com>
Message-ID: <4A2E91BE.8040302@verizon.net>

Tarek Ziad? wrote:
> Hi Gerry,
>
> During the summit at Pycon, we have said that it would be a better
> strategy not to include
> within Distutils os-specific tools for various reasons (and also to
> remove existing ones) :
>
> - it's better for them to have their own release cycles
> - it's hard for me, as distutils maintainer, to maintain and make
> evolve os-specific tools. People that are specialists on those OS
>   will do a better job.
>
> Now the problems to reach that goal are:
>
> - some people are reluctant not to have everything included in the
> stdlib (I am not). I don't think we can all agree on this,
>   and since Guido have encouraged this strategy during the summit, I
> guess I'm more inclined to follow it nevertheless.
>
> - we need to detect for each existing command (rpm, etc) a project
> that can take over, prior to remove it from Distutils
> when appliable .
>
> Now, for debian, since it's not included in distutils, the question is
> : what is the most used/advanced project ? sdteb ?
>
> ++
> Tarek
>
>   
Tarek,
  I understand your position on this.  However, here is what needs 
considered if you want to split out this functionality.  'bdist_rpm' 
operates rather well and you can automate the generation of packages 
using this target.  What I would hope not to see is that we end up with 
an entire fragmentation of different approaches to developing similar 
targets for other platforms such as targets like 'bdist_deb', 
'bdist_mac', etc.  Whatever group would be selected to take over this 
functionality should take over responsibility for all these targets and 
they be developed in a coordinated manner so that it would be easily 
understood by the user how to generated packages for multiple platforms 
using the targets and similar settings in setup.cfg and any auxiliary 
files similar to rpm_install_sh.txt.  If a group can be selected to 
handle these all these targets with the above goals in mind then I think 
it would be possible to split this functionality out of the core.  But, 
if the cannot be done, then I think this functionality should remain 
with the core in order to avoid fragmentation to the approaches in 
developing these 'bdist_xxx' targets.

Regards,
Gerry


From p.f.moore at gmail.com  Tue Jun  9 21:04:15 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 9 Jun 2009 20:04:15 +0100
Subject: [Distutils] Packaging: bdist_deb patches
In-Reply-To: <94bdd2610906080528x4724d980oc54af2b3e7464385@mail.gmail.com>
References: <49B7B62F.40306@verizon.net> <4A2AEB90.4060607@verizon.net>
	<94bdd2610906080528x4724d980oc54af2b3e7464385@mail.gmail.com>
Message-ID: <79990c6b0906091204p15b259c0k5af363ba14afbaa3@mail.gmail.com>

2009/6/8 Tarek Ziad? <ziade.tarek at gmail.com>:
> During the summit at Pycon, we have said that it would be a better
> strategy not to include
> within Distutils os-specific tools for various reasons (and also to
> remove existing ones) :
>
> - it's better for them to have their own release cycles

I don't see why. The one I'm interested in (bdist_wininst) doesn't
change often, and doesn't need a different release cycle (IMHO).

> - it's hard for me, as distutils maintainer, to maintain and make
> evolve os-specific tools. People that are specialists on those OS
> ?will do a better job.

Sorry, but while I appreciate your difficulty, that's not a good
enough reason. The rest of Python core is maintained by people who
don't have access to all OSes. Packages like subprocess and threading
manage. Just take patches from interested community members - you act
as a manager rather than a developer in those areas, but that's OK.

> Now the problems to reach that goal are:
>
> - some people are reluctant not to have everything included in the
> stdlib (I am not). I don't think we can all agree on this,
> ?and since Guido have encouraged this strategy during the summit, I
> guess I'm more inclined to follow it nevertheless.

While "Guido says so" beats any other argument, I'd like to hear some
good reasons - at least for bdist_wininst. For the Linux package
formats, where there's a lot of interest in packaging and strong views
on what's "right", I can see the point. For Windows, where nobody gets
excited about it, moving bdist_wininst out of the core risks nobody
caring enough to keep it going, and thus no packaging solution at all
on Windows.

> - we need to detect for each existing command (rpm, etc) a project
> that can take over, prior to remove it from Distutils
> when appliable .

And a policy on what to do if no such project can be found.

Paul.

From trentm at gmail.com  Tue Jun  9 21:52:38 2009
From: trentm at gmail.com (Trent Mick)
Date: Tue, 9 Jun 2009 12:52:38 -0700
Subject: [Distutils] recollections of Pycon distutils versioning
	discussion (part 1)
In-Reply-To: <79990c6b0906090310g40f21a2ela5d1674fd3cbaa27@mail.gmail.com>
References: <6db0ea510906090058x76c8cadfj22e7d0dc619b56aa@mail.gmail.com>
	<79990c6b0906090310g40f21a2ela5d1674fd3cbaa27@mail.gmail.com>
Message-ID: <6db0ea510906091252t90070e1wc184b02d3bf38a8e@mail.gmail.com>

>> An example of where more N's is useful is for a Python module that wraps
>> a third-party library. Say that library ("libfoo") version is 2.5.2, a
>> reasonable version for "python-libfoo" might be "2.5.2.1.0" where the
>> first three bits track the "libfoo" version.
>
> Purely theoretical example, I assume? I doubt I'd do this, but again,
> who cares? No conceptual cost.
>

Something similar I do for my "python-markdown2" module:
http://code.google.com/p/python-markdown2/source/browse/trunk/lib/markdown2.py#47



>> ?N.N[.N]*[(abc)N[.N]*]
>>
>> ...
>
> -1
>
> 29/4975 is hardly anything, and I don't believe this should be defined
> on a "someone uses it so we should allow it" basis.
>
> If someone supports this, they should be presenting a good use case,
> with an explanation of why it is of value *to the end user* (ie, not
> just to the project developers).

Yes, I'd like to send a separate email to this list (perhaps to
python-list) that asks if anyone can pipe in with a good use
case/justification for this.

Anyway, in this email I'm just trying to put down my current
understanding of the proposal and the debate.

> Note: there's an assumption implicit in this that a "version" is
> something attached to a release - I have little sympathy with the idea
> that every single Subversion revision (or Mercurial changeset, or
> whatever) should have a unique "version" number. Unreleased versions
> should be identified differently (and nobody should be specifying
> dependencies on unreleased versions, before anybody suggests that!)

Ah... I'll get into that with the ".dev" stuff. :) Hopefully tonight.


Trent

-- 
Trent Mick
trentm at gmail.com

From santagada at gmail.com  Tue Jun  9 22:42:15 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Tue, 9 Jun 2009 17:42:15 -0300
Subject: [Distutils] Packaging: bdist_deb patches
In-Reply-To: <4A2E91BE.8040302@verizon.net>
References: <49B7B62F.40306@verizon.net> <4A2AEB90.4060607@verizon.net>
	<94bdd2610906080528x4724d980oc54af2b3e7464385@mail.gmail.com>
	<4A2E91BE.8040302@verizon.net>
Message-ID: <74BC6461-7CE1-4173-A8B9-F9EEEB2146A8@gmail.com>


On Jun 9, 2009, at 1:45 PM, Gerry Reno wrote:
>>
> Tarek,
> I understand your position on this.  However, here is what needs  
> considered if you want to split out this functionality.  'bdist_rpm'  
> operates rather well and you can automate the generation of packages  
> using this target.  What I would hope not to see is that we end up  
> with an entire fragmentation of different approaches to developing  
> similar targets for other platforms such as targets like  
> 'bdist_deb', 'bdist_mac', etc.  Whatever group would be selected to  
> take over this functionality should take over responsibility for all  
> these targets and they be developed in a coordinated manner so that  
> it would be easily understood by the user how to generated packages  
> for multiple platforms using the targets and similar settings in  
> setup.cfg and any auxiliary files similar to rpm_install_sh.txt.  If  
> a group can be selected to handle these all these targets with the  
> above goals in mind then I think it would be possible to split this  
> functionality out of the core.  But, if the cannot be done, then I  
> think this functionality should remain with the core in order to  
> avoid fragmentation to the approaches in developing these  
> 'bdist_xxx' targets.


What exactly you mean by all these? If I'm not mistaken only rpm,  
windows and a generic unixy binary packages exists in distutils today,  
and rpm packages are hardly close to a windows installer package (in  
terms of install scripts etc) so I don't see what you mean by they  
being the same now and your fear of they growing different.

Also I don't see how having a bit of code inside the stdlib is easier  
to maintain than code outside it. Seeing how long a patch takes to be  
applied in the stdlib and how no one can receive commit rights to work  
on just a submodule of it I think that working as an outside project  
is actually beneficial to all those projects in case something change  
on their package tool, and if nothing changes they can keep up pretty  
easily with distutils.

ps: Tarek is one of the exceptions right? He works only on distutils.

--
Leonardo Santagada
santagada at gmail.com




From ronaldoussoren at mac.com  Tue Jun  9 22:53:32 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 09 Jun 2009 13:53:32 -0700
Subject: [Distutils] Packaging: bdist_deb patches / platform specific
	code
In-Reply-To: <9508b01923109d4f42c2bb6cde766b87@preisshare.net>
References: <49B7B62F.40306@verizon.net> <4A2AEB90.4060607@verizon.net>
	<94bdd2610906080528x4724d980oc54af2b3e7464385@mail.gmail.com>
	<34c7b7f6afca67f0f32fe364c32edbe2@preisshare.net>
	<E3D9C8B5-9650-4323-8991-CBEBF9802E19@gmail.com>
	<9508b01923109d4f42c2bb6cde766b87@preisshare.net>
Message-ID: <CAE55AEF-84FA-4DCF-B6C8-129A2CEA7359@mac.com>


On 8 Jun, 2009, at 20:15, David Lyon wrote:

>
>> the same for rpm, deb and mpkg.
>
> Granted. They're are a bit more tricky. But essentially they
> are all just zip files with some entry and exit scripts.

To pick some nits: the mpkg one cannot be created on any other system  
than an OSX one because one of the files that's included in an OSX  
package has an undocumented format and is created using a tool that's  
only available on OSX.

Ronald

From ronaldoussoren at mac.com  Tue Jun  9 23:01:35 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 09 Jun 2009 14:01:35 -0700
Subject: [Distutils] Packaging: bdist_deb patches / platform
	specific	code
In-Reply-To: <7B5409E9-DAB8-4F70-8F3F-47E9DFE990DE@gmail.com>
References: <49B7B62F.40306@verizon.net> <4A2AEB90.4060607@verizon.net>
	<94bdd2610906080528x4724d980oc54af2b3e7464385@mail.gmail.com>
	<34c7b7f6afca67f0f32fe364c32edbe2@preisshare.net>
	<E3D9C8B5-9650-4323-8991-CBEBF9802E19@gmail.com>
	<9508b01923109d4f42c2bb6cde766b87@preisshare.net>
	<4A2DD4F1.7010503@ar.media.kyoto-u.ac.jp>
	<853735a66e33d13dab5dec6ac3f51421@preisshare.net>
	<7B5409E9-DAB8-4F70-8F3F-47E9DFE990DE@gmail.com>
Message-ID: <AD501BE0-3FF5-449E-9AE7-5C1BC0C597EE@mac.com>


On 8 Jun, 2009, at 22:22, Leonardo Santagada wrote:

>
> The point is not exactly cross compilation, but cross maintaining of  
> distribution specific code. The only way this actually works now, I  
> would guess, is because rpm format hasn't changed much, and  
> win32_inst has a huge user base.

Even for bdist_rpm it is questionable if the implementation is correct  
w.r.t. the policies of various distributions.  Given the discussions  
about this in the past we wouldn't need a "bdist_rpm" or "bdist_deb"  
but "bdist_redhat", "bdist_suse", "bdist_debian", ...

Bdist_wininst and bdist_msi are somewhat special because the OS vendor  
does not have a specific policy on how to install python software.

Ronald



From eric at trueblade.com  Tue Jun  9 23:58:16 2009
From: eric at trueblade.com (Eric Smith)
Date: Tue, 9 Jun 2009 17:58:16 -0400 (EDT)
Subject: [Distutils] Packaging: bdist_deb patches
In-Reply-To: <79990c6b0906091204p15b259c0k5af363ba14afbaa3@mail.gmail.com>
References: <49B7B62F.40306@verizon.net> <4A2AEB90.4060607@verizon.net>
	<94bdd2610906080528x4724d980oc54af2b3e7464385@mail.gmail.com>
	<79990c6b0906091204p15b259c0k5af363ba14afbaa3@mail.gmail.com>
Message-ID: <1609.96.231.30.173.1244584696.squirrel@mail.trueblade.com>

> 2009/6/8 Tarek Ziad? <ziade.tarek at gmail.com>:

< plans and reasons to drop some commands from distutils, which I agree
with >

>> - we need to detect for each existing command (rpm, etc) a project
>> that can take over, prior to remove it from Distutils
>> when appliable .
>
> And a policy on what to do if no such project can be found.

I don't think we need a policy at all. If no one wants to support it, it
gets dropped. You can't tell volunteers what they have to do.

Eric.


From p.f.moore at gmail.com  Wed Jun 10 00:34:12 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 9 Jun 2009 23:34:12 +0100
Subject: [Distutils] What precisely is meant by removing bdist modules from
	the stdlib? (Was: Packaging: bdist_deb patches)
Message-ID: <79990c6b0906091534h225fdbc7ob0883670b3b74fd0@mail.gmail.com>

2009/6/9 Eric Smith <eric at trueblade.com>:
>> 2009/6/8 Tarek Ziad? <ziade.tarek at gmail.com>:
>
> < plans and reasons to drop some commands from distutils, which I agree
> with >
>
>>> - we need to detect for each existing command (rpm, etc) a project
>>> that can take over, prior to remove it from Distutils
>>> when appliable .
>>
>> And a policy on what to do if no such project can be found.
>
> I don't think we need a policy at all. If no one wants to support it, it
> gets dropped. You can't tell volunteers what they have to do.

No, it could be kept in the stdlib as is. If no-one wants to do the
work to handle the proposed change (from in the stdlib to externally
supported) then why not keep the status quo?

You seem to be saying that the correct approach should be to spend
effort removing bdist_wininst and others from the stdlib regardless of
whether someone is willing to commit to maintaining them externally.
I'm saying that unless someone wants to take on the job of being an
external maintainer, we should do nothing.

If bdist_wininst gets a special exception and is kept in the stdlib,
I'll shut up - I have no vested interest in any of the other bdist
forms. Otherwise, I fail to see the point of this purge -
bdist_wininst is important to Windows users, and I think it's
important that it stays in the stdlib.

To provide some concrete facts, here's the top of hg log for bdist_wininst.py:

changeset:   39428:a8950f06efc1
branch:      trunk
user:        mark.hammond
date:        Wed May 28 03:54:55 2008 +0200
summary:     [svn r63755] bdist_wininst now works correctly when both
--skip-build and --plat-name are specified.

changeset:   39004:f56b6234cf8d
branch:      trunk
user:        mark.hammond
date:        Fri May 02 14:48:15 2008 +0200
summary:     [svn r62636] #2581: Vista UAC/elevation support for bdist_wininst

changeset:   38807:15196e08f04a
branch:      trunk
user:        mark.hammond
date:        Mon Apr 07 03:53:39 2008 +0200
summary:     [svn r62197] Issue #2513: enable 64bit cross compilation
on windows.

changeset:   37535:1cd6136356e0
branch:      trunk
user:        christian.heimes
date:        Mon Dec 31 15:47:07 2007 +0100
summary:     [svn r59620] Added wininst-9.0.exe executable for VS 2008

changeset:   31774:201bf25423b8
branch:      trunk
user:        loewis
date:        Wed Mar 23 19:54:36 2005 +0100
summary:     [svn r38697] Make dist_files a triple, with the Python
target version included,

changeset:   31770:924c8e8c201a
branch:      trunk
user:        loewis
date:        Tue Mar 22 23:23:29 2005 +0100
summary:     [svn r38692] Fix registration of output file.

changeset:   31762:b47b4a1ac59c
branch:      trunk
user:        loewis
date:        Mon Mar 21 21:56:35 2005 +0100
summary:     [svn r38684] Add the upload command. Make all dist
commands register their

Doesn't look like a huge burden - since 2005, there have been 7
changes - 3 contributed by Mark Hammond, 3 by Martin von Loewis, and
one by Christian Heimes. All of whom I would assume would continue to
provide such changes if bdist_wininst stays in the stdlib, but who may
well not if it becomes externally maintained.

Note: If I seem to be making a lot of fuss about bdist_wininst, that's
because the only bdist modules in the stdlib right now are bdist_msi,
bdist_wininst, bdist_rpm and bdist_dumb. Either this policy is aimed
to a substantial extent at Windows, or it should be stated as "remove
bdist_rpm from the stdlib".

Can someone clearly articulate what is proposed here? I can see 4 main options:

1. Remove all 4 bdist modules (wininst, msi, rpm, dumb) from the
stdlib, and let them be picked up by external maintainers, or die.
2. Remove bdist_rpm from the stdlib, so that RPMs don't have special
status out of the Linux packaging formats, but retain bdist_msi,
bdist_wininst and bdist_dumb (effectively, the "windows is a special
exception" option).
3. Solicit offers from the community to maintain the 4 bdist_xxx
modules as external packages on PyPI. If no such offers are
forthcoming, continue as at present.
4. Simply make a formal policy that no new bdist_ commands will be
implemented in the stdlib, and leave open the possibility that someone
comes along and offers to pull one or more of the existing commands
out into an external module on PyPI (hardly any different from the
current status quo, except in being articulated a bit more
explicitly).

The only one which seriously concerns me is (1) out of the above. If
someone (Tarek?) comes out and says that (1) is not what is being
proposed, I'm happy to let the rest of the discussions take their
course.

Paul.

PS I have a paranoid feeling that there's a subtext somewhere here to
the effect that participants in the discussion don't see why there's
such a big deal, as everyone should be using
setuptools/easy_install/eggs anyway. Please let's avoid getting into
that here. Can we assume for the purposes of this discussion that
easy_install and eggs are not relevant, as the target audience for the
bdist_ commands is *precisely* those people who don't want to use
setuptools?

From david.lyon at preisshare.net  Wed Jun 10 00:36:15 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 09 Jun 2009 18:36:15 -0400
Subject: [Distutils]
 =?utf-8?q?Packaging=3A_bdist=5Fdeb_patches_/_platform?=
 =?utf-8?q?_specific=09code?=
In-Reply-To: <AD501BE0-3FF5-449E-9AE7-5C1BC0C597EE@mac.com>
References: <49B7B62F.40306@verizon.net> <4A2AEB90.4060607@verizon.net>
	<94bdd2610906080528x4724d980oc54af2b3e7464385@mail.gmail.com>
	<34c7b7f6afca67f0f32fe364c32edbe2@preisshare.net>
	<E3D9C8B5-9650-4323-8991-CBEBF9802E19@gmail.com>
	<9508b01923109d4f42c2bb6cde766b87@preisshare.net>
	<4A2DD4F1.7010503@ar.media.kyoto-u.ac.jp>
	<853735a66e33d13dab5dec6ac3f51421@preisshare.net>
	<7B5409E9-DAB8-4F70-8F3F-47E9DFE990DE@gmail.com>
	<AD501BE0-3FF5-449E-9AE7-5C1BC0C597EE@mac.com>
Message-ID: <406373460d8f1692ddfa8496939577c3@preisshare.net>

On Tue, 09 Jun 2009 14:01:35 -0700, Ronald Oussoren
<ronaldoussoren at mac.com>
wrote:
> Even for bdist_rpm it is questionable if the implementation is correct  
> w.r.t. the policies of various distributions.  Given the discussions  
> about this in the past we wouldn't need a "bdist_rpm" or "bdist_deb"  
> but "bdist_redhat", "bdist_suse", "bdist_debian", ...

Makes perfect sense to me... I'd include ubuntu in the list of primaries..

> Bdist_wininst and bdist_msi are somewhat special because the OS vendor  
> does not have a specific policy on how to install python software.

I'd suggest that they merge into bdist_win. Then be designated with 
"bdist_win /wininst" and "bdist_win /msi" respectively. One could
be made the default option but I don't have a preference.

David



From eric at trueblade.com  Wed Jun 10 00:40:50 2009
From: eric at trueblade.com (Eric Smith)
Date: Tue, 09 Jun 2009 18:40:50 -0400
Subject: [Distutils] What precisely is meant by removing bdist modules
 from the stdlib? (Was: Packaging: bdist_deb patches)
In-Reply-To: <79990c6b0906091534h225fdbc7ob0883670b3b74fd0@mail.gmail.com>
References: <79990c6b0906091534h225fdbc7ob0883670b3b74fd0@mail.gmail.com>
Message-ID: <4A2EE4F2.60308@trueblade.com>

I'll share my thoughts on the other points when I have some time, but 
I'd like to address this one now:

Paul Moore wrote:
> PS I have a paranoid feeling that there's a subtext somewhere here to
> the effect that participants in the discussion don't see why there's
> such a big deal, as everyone should be using
> setuptools/easy_install/eggs anyway. Please let's avoid getting into
> that here. Can we assume for the purposes of this discussion that
> easy_install and eggs are not relevant, as the target audience for the
> bdist_ commands is *precisely* those people who don't want to use
> setuptools?
> 

I'm a huge user of bdist_rpm. I definitely agree that easy_install and 
eggs are not relevant to this discussion. This is only about distutils 
and what it includes.

And while I'm for removing the bdist_* commands from distutils, I'm 
neutral on removing their functionality from the standard library. I 
think they'll all ultimately require a lot of work as distutils evolves.

Eric.


From zooko at zooko.com  Wed Jun 10 16:30:26 2009
From: zooko at zooko.com (Zooko Wilcox-O'Hearn)
Date: Wed, 10 Jun 2009 08:30:26 -0600
Subject: [Distutils] Packaging:  bdist_deb patches
In-Reply-To: <4A29625D.2020005@verizon.net>
References: <49B7B62F.40306@verizon.net> <4A29625D.2020005@verizon.net>
Message-ID: <EB4F9EF7-042B-4CF8-A9CC-F551B944D0E8@zooko.com>

On Jun 5, 2009, at 12:22 PM, Gerry Reno wrote:

> Has there been any progress on getting the 'bdist_deb' target (akin  
> to 'bdist_rpm') implemented in Distutils? We are looking for  
> behavior that would be similar to bdist_rpm driven by setup.cfg  
> with fully automated generation of packages. There is in existence  
> a set of patches that supposedly does this. Has any progress been  
> made to integrate those patches?


I use stdeb for this purpose:

http://pypi.python.org/pypi/stdeb

For example, here is a buildbot which automatically builds a new .deb  
from the Python source whenever a new patch is committed to the  
revision control repository:

http://allmydata.org/buildbot-pycryptopp/builders/linux-amd64-ubuntu- 
hardy-yukyuk/builds/82/steps/stdeb/logs/stdio

Regards,

Zooko

From setuptools at bugs.python.org  Wed Jun 10 19:54:13 2009
From: setuptools at bugs.python.org (Zooko O'Whielacronx)
Date: Wed, 10 Jun 2009 17:54:13 +0000
Subject: [Distutils] [issue78] egg platform names don't reflect unicode
	variant (UCS2, UCS4)
In-Reply-To: <1244656453.49.0.250715417793.issue78@psf.upfronthosting.co.za>
Message-ID: <1244656453.49.0.250715417793.issue78@psf.upfronthosting.co.za>


New submission from Zooko O'Whielacronx <zooko at zooko.com>:

A user of Tahoe-LAFS encountered an error in which pyOpenSSL emitted:

exceptions.UnicodeDecodeError: 'utf8' codec can't decode bytes in position 0-5:
unsupported Unicode code range

It took some effort on the part of the user and the Tahoe-LAFS devs to delve
into the code and figure out how an invalid string got into a certificate inside
pyOpenSSL.  Eventually the user (David Abrahams) figured out that the issue was
this one:

http://markmail.org/message/bla5vrwlv3kn3n7e

He had installed a pyOpenSSL egg which had been built with UCS4, but his Python
interpreter was UCS2.  According to the thread linked above, the best way to fix
this is for distutils get_platform() to include the unicode variant in its
output, and then for setuptools to test the compatibility of that field when
choosing an egg.  Is that right?

What's the next step?

----------
messages: 302
nosy: zooko
priority: bug
status: unread
title: egg platform names don't reflect unicode variant (UCS2, UCS4)

_______________________________________________
Setuptools tracker <setuptools at bugs.python.org>
<http://bugs.python.org/setuptools/issue78>
_______________________________________________

From david.lyon at preisshare.net  Thu Jun 11 00:55:37 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 10 Jun 2009 18:55:37 -0400
Subject: [Distutils] =?utf-8?q?Packaging=3A__bdist=5Fdeb_patches?=
In-Reply-To: <EB4F9EF7-042B-4CF8-A9CC-F551B944D0E8@zooko.com>
References: <49B7B62F.40306@verizon.net> <4A29625D.2020005@verizon.net>
	<EB4F9EF7-042B-4CF8-A9CC-F551B944D0E8@zooko.com>
Message-ID: <0d4e82918dd59d81c8cf21f061acddb8@preisshare.net>

On Wed, 10 Jun 2009 08:30:26 -0600, Zooko Wilcox-O'Hearn <zooko at zooko.com>
wrote:
> I use stdeb for this purpose:
> 
> http://pypi.python.org/pypi/stdeb
> 
> For example, here is a buildbot which automatically builds a new .deb  
> from the Python source whenever a new patch is committed to the  
> revision control repository:
> 
> http://allmydata.org/buildbot-pycryptopp/builders/linux-amd64-ubuntu- 
> hardy-yukyuk/builds/82/steps/stdeb/logs/stdio

That sounds excellent....

To my mind, that's what bdist_* and pypi need....

Not talking about anything contriversial or new or big....

But just better integration so that we can get a build of our
projects done more smoothly and uploaded to pypi on several
different python versions and platforms...

with the built in qa facilities that might already exist
to us.....

Maybe it is logical to discuss splitting bdist_* from
distutils. Given that there is so much platform specific
stuff in bdist_*. But then what does it leave distutils
as? "packageutils" ? 

I think there is some logical division between platform 
independant package definition issues (which I know 
certain people like working on - and that's good) and 
the package delivery side. (bdist)

The two seem fairly seperate to me. Yet they both make
up distutils.

Just following on and trying to clarify what other
people have put in the discussion pot...

David



From thomas at thomas-lotze.de  Thu Jun 11 08:55:08 2009
From: thomas at thomas-lotze.de (Thomas Lotze)
Date: Thu, 11 Jun 2009 08:55:08 +0200
Subject: [Distutils] [Buildout] Download API and extends caching - please
	review
Message-ID: <pan.2009.06.11.06.55.07.449140@ID-174572.user.uni-berlin.de>

As discussed before on this list, I've been working on an internal
download API for zc.buildout that takes care of handling the download
cache, offline mode, MD5 checks etc and is used for fetching configuration
files (as specified by the buildout:extends option). I consider the
feature finished and I'd be grateful if someone could review it before it
gets merged to the trunk. The branch I've been developing this on is
<svn://svn.zope.org/repos/main/zc.buildout/branches/tlotze-download-api>.
Thank you.

-- 
Thomas




From trentm at gmail.com  Thu Jun 11 09:42:29 2009
From: trentm at gmail.com (Trent Mick)
Date: Thu, 11 Jun 2009 00:42:29 -0700
Subject: [Distutils] recollections of Pycon distutils versioning discussion
	(part 2)
Message-ID: <6db0ea510906110042x4d31ab1dr35a7865315638daf@mail.gmail.com>

Here-in the second part of my recollections of the Pycon distutils versioning
discussions. This first part is here:
http://mail.python.org/pipermail/distutils-sig/2009-June/012143.html

The first part left us with this pseudo-pattern:

  N.N[.N]*[(abc)N[.N]*]

Some examples to help illustrate:

  1.0
  1.4.2.45.2.2.5
  1.2.3a4
  1.34.5.6c1
  3.0b2.1

That leaves the ".devN" and ".postN" business of the current version
proposal (PEP 386).

# A brief setuptools lesson

(Mainly this is because I needed it -- I'm not intimately familiar with
setuptools. Skip this section if you know this already.)

Setuptools defines
(http://peak.telecommunity.com/DevCenter/setuptools#specifying-your-project-s-version)
"pre-release tags" and "post-release tags" on a version number. In setuptools
one part of the "pre-release tags" are the "a", "b", "c" (for alpha, beta,
candidate) parts of the proposed pattern that we've already discussed. The
remaining part of "pre-release tags" is the use of "dev" or "pre", e.g.:

    1.0dev
    2.5.2pre
    3.1dev459259

(Strictly speaking also "preview" and any string that sorts less than
"final", but "dev" and "pre" are the common ones.)

"Post-release tags" are any non-numeric in the version string that sorts
greater than "final" or the special case "-". Practically speaking it is
almost always the latter because setuptools provides command-line
options (http://peak.telecommunity.com/DevCenter/setuptools#egg-info) to
automatically append an svn rev ("-rNNNNN") or a date ("-YYYYMMDD").
Examples:

    1.0-r2345
    5.4.3-20090107


# ".devNNNN" in the new versioning proposal

Reminder: I'm just trying to describe my recollections of the Pycon
discussions here. I'll post my current opinions in a separate email. I.e.,
don't shoot the messenger. :)

In general, people tended to agree that development builds are a reasonable
thing. Your project is in the run-up to 1.0 and you'll be doing regular
builds at incremental revisions of your source code control system.
Setuptools encourages (by offering certain command-line options) spelling
this kind of thing this way (though, of course there are other spellings of
the equivalent):

    1.0dev-r342
    1.0dev-r343
    1.0dev-r350
    1.0dev-r355

At this point in the Pycon discussions I vaguely recall that there was some
opposition to having support for spelling development builds in the
versioning scheme at all. However, I don't fully recall. If so, they were won
over (or drowned out) by (a) others giving examples of doing daily/nightly
builds of product X -- including me describing nightlies of the Komodo
product (not a
Python project) I work on -- and (b) the start of the debate on how to spell
"1.0dev-r342", those presuming that its inclusion was a forgone conclusion.
I'll touch more on this in a followup email.

Ensuing debate, pleading, fisticuffs, cajoling ... leading to

    N.N[.N]*[(abc)N[.N]*][.devN]
    1.0.dev456
    2.5.0b2.dev523

Why "dev" at all and not "-", "_", or "~" or just another "."? None of the
alternatives were explicit that this is a release that should sort *earlier*
than the version without it. Also, '-' is used to setuptools to mean a
*post*-release; '_' causes problems for Debian packaging; '~' is used in
Debian package versioning (IIRC, best to avoid using it); just '.' (e.g.
'1.0a1.456') can be ambiguous.

Why "dev"? Because it is by far the most common string used for this in
current PyPI versions. "pre" a far second.

Why ".devN" (1.0a1.dev456) instead of just "devN" (1.0a1dev456)? Partly a
feeling that the '.' helped separate from the leading part -- if "1.0.a1"
didn't look ridiculous we probably would have chosen that too.  :)
Also partly because there is justifiably little point in quibbling for
hours over a single '.' here.


# ".postNNNN" in the new versioning proposal

Amid fading sounds of hands being dusted off, backs being patted and
rising discussions of what's for dinner ("Salad bar or steak?" "What's
the special tonight?") the question:

    "What about post-releases?" -- Zooko

Groans. "Who uses that?" "Twisted, for example." "Really? Hrm, okay,
let's tack on a '.postN' (like the '.devN') and call it a night."

The discussion *was* longer than that, but not a whole lot longer. Mostly the
justification for ".postN" in the proposal rested on the example of at least
a couple significant Python projects using this (e.g. Twisted). An example
of the resulting sorting order:

    1.0a1
    1.0a2.dev456
    1.0a2
    1.0a2.1.dev456
    1.0a2.1
    1.0b1.dev456
    1.0b2
    1.0c1.dev456
    1.0c1
    1.0.dev456
    1.0
    1.0.post456

and the final pseudo-pattern:

    N.N[.N]*[(a|b|c)N[.N]*][(.devN|.postN)]

Those are my recollections of the Pycon versioning discussions. I want to
following up (in a separate email) with my current opinions.



Cheers,
Trent

-- 
Trent Mick
trentm at gmail.com

From p.f.moore at gmail.com  Thu Jun 11 10:37:03 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 11 Jun 2009 09:37:03 +0100
Subject: [Distutils] recollections of Pycon distutils versioning
	discussion (part 2)
In-Reply-To: <6db0ea510906110042x4d31ab1dr35a7865315638daf@mail.gmail.com>
References: <6db0ea510906110042x4d31ab1dr35a7865315638daf@mail.gmail.com>
Message-ID: <79990c6b0906110137h26e32765w55f164169da86154@mail.gmail.com>

2009/6/11 Trent Mick <trentm at gmail.com>:
> Those are my recollections of the Pycon versioning discussions. I want to
> following up (in a separate email) with my current opinions.

Thank you for posting this. It helps *a lot*.

Some questions that come immediately to mind:

1. From your description of events, and from discussion here, the
"post" tag is clearly the least thought through aspect of all this. So
a proponent of this should step up to justify why 1.0.5.post3 is
required, when 1.0.5.3 is available already.

2. It sounds like all of the discussions were focused on formalising a
*definition* of version string syntax, with no mention of what use
would be made of the definition (i.e., what software benefits from a
well-defined version-parsing API). Can someone please enumerate the
expected use cases for the new version parsing API? The obvious one is
dependency handling - I'll add my thoughts on this separately. I can't
think of any other compelling use cases.

Paul.

From ziade.tarek at gmail.com  Thu Jun 11 10:53:42 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 11 Jun 2009 10:53:42 +0200
Subject: [Distutils] setuptools and py3k
In-Reply-To: <4DBAED55-17CF-4CD7-BDBB-EE05A5C8643B@mac.com>
References: <EE65BCC4-F1AE-40C1-83CE-6A5D60D97BFD@mac.com>
	<h0j1n5$u9k$1@ger.gmane.org>
	<4DBAED55-17CF-4CD7-BDBB-EE05A5C8643B@mac.com>
Message-ID: <94bdd2610906110153v4df6bf18re36c6588bea13149@mail.gmail.com>

On Mon, Jun 8, 2009 at 5:27 PM, Ronald Oussoren<ronaldoussoren at mac.com> wrote:
>>
>> Great work.

+1

>
> This is only a first step, I'm found a number of annoying issues that might
> require interface changes to get right (such a number of functions in
> pkg_resources that claim to return a string, but actually return bytes (by
> opening a file in binary mode and returning the file contents). ?Hopefully
> I'll have a set of patches soonish, that would make discussion easier.

There are also a lot of important pending patches for setuptools 'trunk'

Phillip, can we move forward and retire the 0.6.x branch with a final release,
then start to work on the python 3 port  in the 0.7 branch ?

Do you need help to do this / how may I help to speed up this final release ?


Regards
Tarek

From trentm at gmail.com  Thu Jun 11 11:29:31 2009
From: trentm at gmail.com (Trent Mick)
Date: Thu, 11 Jun 2009 02:29:31 -0700
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
	".postNNN"
Message-ID: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>

PEP 386 is proposing a new versioning scheme for distutils. One that
currently includes ways to spell development releases (with a ".devNNN"
suffix) and "post-releases" (with a ".postNNN" suffix).

This is a long email. You can jump to the bottom for conclusions.

First a survey of what "the other guys" do.

# Perl

CPAN. A free for all. A couple quotes from a Perl core developer here at
ActiveState:

> There is no standardized version numbering scheme.  This module is supposed
> to be it:
>
>    http://search.cpan.org/~jpeacock/version-0.76/lib/version.pod
>
> But I think it probably created as many problems as it solved.


# Tcl

http://www.tcl.tk/man/tcl8.5/TclCmd/package.htm#M22

Strict version scheme, and because this is part of a core language feature
it is effectively enforced. The scheme is:

    N[.N]+[(a|b)N]

For examples of versions, ActiveState has a repo of binary builds of Tcl
extensions called "teapot" (the "TEA" stands for Tcl Extension Architecture)
here: http://teapot.activestate.com/index


# PHP

PEAR, http://pear.php.net/packages.php
535 packages: relatively small

Strict version scheme (http://pear.php.net/group/docs/20040226-vn.php)
"which will be mandatory for all packages from now on". They *do* support
a "dev" state -- "0.9.5 (dev)" -- that is less than an "alpha" state,
but NOT with an attached revision/date numeric value.

Note that I did some poking around on the PEAR site and I couldn't find
any packages with "dev"-state release.


# Others?

Anybody?


# Some current PyPI numbers

Conclusions are **bolded** for skimming.

Using the PyPI XML-RPC API I got a dump of all current project names
and the list of versions of uploaded packages for that project. A line
like this for each project:

    Markdown: 1.7, 1.7, 1.7, 2.0, 2.0, 2.0, 2.0, 2.0.1, 2.0.1, 2.0.1

This shows the order in which packages were uploaded... duplicates because
separate package types for the same version (sdist, bdist_egg, etc.).

    script: http://dl.getdropbox.com/u/1301040/distutilsver/gatherpypi.py
    data: http://dl.getdropbox.com/u/1301040/distutilsver/file_versions_from_name.txt

The current verlib.py has a "suggest_rational_version" that has pretty
good coverage on getting the equivalent rational version number from
a pypi version (16133 out of 17089 in my count). I counted uses of ".devNNN"
and ".postNNN" in all the currently used PyPI versions:

**Out of a total 17089 versions released to PyPI there are 281 (in 90
of about 6726 projects, 1.3%) that use a post-release tag.** These are:
http://dl.getdropbox.com/u/1301040/distutilsver/file_versions_from_name_using_post.txt

**Out of a total 17089 versions released to PyPI there are 738 (in 292
of about 6726 projects, 4.3%) that use a "dev" tag.** These are:
http://dl.getdropbox.com/u/1301040/distutilsver/file_versions_from_name_using_dev.txt

Looking at those that do there is a much lower percentage that *frequently*
use the post- and pre-release tags. For example, two of SQLObject's
*many* releases using a pre-release "dev-rNNNN" tag:

    SQLObject: 0.9.5, 0.7.8, 0.8.5, 0.9.1, 0.8.6b1, 0.9.2b1, 0.7.9b1,
        0.8.6b2, 0.7.9b2, 0.9.2b2, 0.7.9b3, 0.8.6b3, 0.9.2b3, 0.7.9, 0.8.6,
        0.9.2, 0.7.10, 0.9.3, 0.8.7, 0.10.0b1, 0.10.0b2, 0.9.4b1, 0.10.0b3,
        0.10.0b4, 0.9.4, 0.10.0, 0.10.1, 0.9.6, 0.10.2b1, 0.9.7b1,
        0.10.2b1dev-r3466, 0.9.7, 0.10dev-r3481, 0.10.2, 0.10.3, 0.9.8, 0.9.9,
        0.10.4, 0.9.10, 0.10.5, 0.10.6, 0.9.11

**I.e. some might have just been a one off trying out the related setuptools
command-line options for this. For others it might have just been a
mistake.** For example, notice that the SQLObject "0.10.2b1dev-r3466"
release happened *after* the "0.10.2b1". That "0.10.2b1dev-r3466" SQLObject
release has subsequently been deleted from PyPI (indicating a mistake),
though older releases are still there (e.g.
http://pypi.python.org/pypi/SQLObject/0.9.3).

There are indicates that the setuptools "-rNNNN" *post-release* tag is
being accidentally mis-interpreted. According to setuptools:

    >>> from pkg_resources import parse_version as V
    >>> V("0.7") < V("0.7-r15094")
    True

However, looking at the "Chandler-EVDBPlugin" project: "0.7-r15094" was
uploaded 2007-07-24, and "0.7" was uploaded 2007-09-11 (later). Basically
I suspect the maintainer of "Chandler-EVDBPlugin" misinterpreted how to
use the setuptools "--tag-svn-revision" facility. **I.e. I suspect there
are significantly fewer than 1% of projects uploading packages to PyPI
that need/want a "post-release" tag facility**.

It would be interesting to get numbers of whether this setuptools
dependency pattern is at all frequently used:

    install_requires = ["OtherProject>=0.2a1.dev-r143"]

I.e. a dependency on an uploaded-to-PyPI "dev" release.


# The "Twisted" example

The Twisted example was a justifier for including ".postNNNN" in the versioning
scheme. See my previous email:
http://mail.python.org/pipermail/distutils-sig/2009-June/012161.html

However, while it appears to be true that Twisted doesn't
appear to increment the version number in the source code until very
close to release [1] (opposite to what Python itself, for example, does),
I'm not sure this constitutes "using post-releases".

- Twisted doesn't release packages to PyPI
(http://pypi.python.org/pypi/Twisted/)
- The archive of Twisted releases doesn't include post-releases.
  http://tmrc.mit.edu/mirror/twisted/Twisted/
- Perhaps not relevant, but Twisted's "Version" class seems to indicate a
  leaning towards "*pre*releases"
  http://twistedmatrix.com/trac/browser/trunk/twisted/python/versions.py#L38

[1] When correlating changes to Twisted's version file
    http://twistedmatrix.com/trac/browser/trunk/twisted/_version.py
    to Twisted release dates.


# My conclusion/opinion

In so far as the proposed new distutils versioning scheme is to be used
for enforcing/encouraging versions for packages uploaded to PyPI, I don't
believe there is a strong case for including ".devNNN" and certainly not
".postNNN".

- It just isn't used often enough to justify the complexity in the version
  scheme for packages on PyPI.
- Setuptools' facility to do `install_requires = ["OtherProject==dev"]`
  is a good thing, but this doesn't require the ".devNNNN" and post-release
  tags on sdist/bdist packages uploaded to PyPI.
- The other language repo systems don't demonstrate a need for this.
- Pressure on project authors to do normal releases (rather than having a
quick "dev sorta-release") when user's want it is a *good thing*.
- Dropping this complexity in the version scheme documentation for users
would allow us to spend more time educating Python module authors on
more important best practices like incrementing the major version number
for backward incompatible changes.


Cheers,
Trent

-- 
Trent Mick
trentm at gmail.com

From p.f.moore at gmail.com  Thu Jun 11 11:51:36 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 11 Jun 2009 10:51:36 +0100
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
	".postNNN"
In-Reply-To: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>
Message-ID: <79990c6b0906110251qff9ab31y62b3c0062f6839b3@mail.gmail.com>

2009/6/11 Trent Mick <trentm at gmail.com>:
> # My conclusion/opinion
>
> In so far as the proposed new distutils versioning scheme is to be used
> for enforcing/encouraging versions for packages uploaded to PyPI, I don't
> believe there is a strong case for including ".devNNN" and certainly not
> ".postNNN".
>
> - It just isn't used often enough to justify the complexity in the version
> ?scheme for packages on PyPI.
> - Setuptools' facility to do `install_requires = ["OtherProject==dev"]`
> ?is a good thing, but this doesn't require the ".devNNNN" and post-release
> ?tags on sdist/bdist packages uploaded to PyPI.
> - The other language repo systems don't demonstrate a need for this.
> - Pressure on project authors to do normal releases (rather than having a
> quick "dev sorta-release") when user's want it is a *good thing*.
> - Dropping this complexity in the version scheme documentation for users
> would allow us to spend more time educating Python module authors on
> more important best practices like incrementing the major version number
> for backward incompatible changes.

+1. That saved me from having to write an email which made this
argument, but far less effectively :-)

Paul.

From ben+python at benfinney.id.au  Thu Jun 11 13:49:48 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 11 Jun 2009 21:49:48 +1000
Subject: [Distutils] recollections of Pycon distutils versioning
	discussion (part 2)
References: <6db0ea510906110042x4d31ab1dr35a7865315638daf@mail.gmail.com>
Message-ID: <87bpovat2b.fsf@benfinney.id.au>

Trent Mick <trentm at gmail.com> writes:

> Here-in the second part of my recollections of the Pycon distutils
> versioning discussions. This first part is here:
> http://mail.python.org/pipermail/distutils-sig/2009-June/012143.html

Thank you for this series, it is good to have these summaries as a focus
for discussion.

> The first part left us with this pseudo-pattern:
> 
>   N.N[.N]*[(abc)N[.N]*]
> 
> Some examples to help illustrate:
> 
>   1.0
>   1.4.2.45.2.2.5
>   1.2.3a4
>   1.34.5.6c1
>   3.0b2.1

An important part of the specification here is how versions compare
sequentially.

My understanding is the above version strings should be compared by the
following rules:

  * A version string is interpreted as a tuple of components. Each
    component is a sequence of characters from the set [0-9A-Za-z]. Each
    component is separated from others by a single full-stop (?.?)
    character.

  * Two version strings are compared for sequence by comparing their
    component tuples.

    * Each pair of components (from the two version tuples) is compared
      from left to right.

    * Contiguous sequences of digits are interpreted as integers and
      compared numerically; other characters compare as per the ASCII
      character set sequence.

  * The first component in turn that differs between the two tuples
    thereby determines the sequencing of the two version strings; if all
    corresponding components compare equal then the two strings
    represent the same version.

My experience with version string interpretations suggests the above
rules formally describe a fairly solid consensus about how version
strings should compare.

> That leaves the ".devN" and ".postN" business of the current version
> proposal (PEP 386).

Right, thank you for treating this as an addendum.

> # A brief setuptools lesson
> 
> (Mainly this is because I needed it -- I'm not intimately familiar
> with setuptools. Skip this section if you know this already.)

I think this description of setuptools's specifics is interesting only
in that it describes one particular implementation; I do *not* think any
authority should be vested in this implementation merely because it's
setuptools.

> # ".devNNNN" in the new versioning proposal
> 
> In general, people tended to agree that development builds are a
> reasonable thing.

No disagreement here.

> Your project is in the run-up to 1.0 and you'll be doing regular
> builds at incremental revisions of your source code control system.
> Setuptools encourages (by offering certain command-line options)
> spelling this kind of thing this way (though, of course there are
> other spellings of the equivalent):
> 
>     1.0dev-r342
>     1.0dev-r343
>     1.0dev-r350
>     1.0dev-r355

My beef comes in at this point. Why on earth should we have a
specification that enshrines the above, which is only one of many
incompatible special-case extensions to an otherwise simple comparison
algorithm?

It's especially puzzling why we would choose something more complex when
?the run-up to 1.0? is easily versioned as version numbers that will
compare as previous to version 1.0.

    0.0.5
    0.18.3
    0.23
    0.999.dev-r342
    0.999.dev-r343
    0.999.dev-r350
    0.999.dev-r355
    1.0

> At this point in the Pycon discussions I vaguely recall that there was
> some opposition to having support for spelling development builds in
> the versioning scheme at all. However, I don't fully recall.

It's precisely the fact that it's (AFAICT) far more controversial than
the solid consensus on the version comparisons without special cases
that makes me think it has no place in a specification that we hope to
be broadly accepted.

> If so, they were won over (or drowned out) by (a) others giving
> examples of doing daily/nightly builds of product X -- including me
> describing nightlies of the Komodo product (not a Python project) I
> work on -- and (b) the start of the debate on how to spell
> "1.0dev-r342", those presuming that its inclusion was a forgone
> conclusion. I'll touch more on this in a followup email.

If that's an accurate representation of the discussion, I want to point
out the huge non sequitur that has occurred. I've demonstrated above
that one can easily have nightly builds of product X *and* have a simple
version comparison algorithm that has no special-cased words.

So it's a false dilemma to present it as ?simple version comparison
versus versions for nightly builds?. We *can* have both, so anyone who
wants to take one of those away still has all their persuasion work
ahead of them.

> Why "dev" at all and not "-", "_", or "~" or just another "."?
[?]

> Why "dev"?
[?]

> Why ".devN" (1.0a1.dev456) instead of just "devN" (1.0a1dev456)?
[?]

All of these are secondary to ?Why special-case any word at all,
instead of just using versions that *already* compare in the right
sequence by a simple uncontroversial algorithm with no special cases??

> # ".postNNNN" in the new versioning proposal
> 
> Amid fading sounds of hands being dusted off, backs being patted and
> rising discussions of what's for dinner ("Salad bar or steak?" "What's
> the special tonight?") the question:
> 
>     "What about post-releases?" -- Zooko

What's wrong with appending more components so that these post-releases
will compare in the right sequence anyway?

Another demonstration:

    1.0      # release of 1.0

    1.0.0.1  # first post-1.0 release
    1.0.0.2  # second post-1.0 release

    1.0.1    # working on the next release
    1.0.2
    1.0.3

The ?work leading up to version 1.1? has already been tagged version
1.0.1, 1.0.2 etc. Then the need for an additional release of the version
1.0 code is needed; that's tagged version 1.0.0.1, and the next such
release can be 1.0.0.2, etc. while 1.0.3 is developed.

This *works*, and is easily understood and IME obvious to know where the
versions compare. So again, anyone who wants to take away one of
?simple version comparison? and ?ability to tag post-release versions
in correct sequence? has all their persuasive work ahead of them.

So I think both of ?.devXXX needs to be special-cased? and ?.postXXX
needs to be special-cased? are demonstrably false, and as a result the
inclusion of these into the specification is not sufficiently justified.

> Those are my recollections of the Pycon versioning discussions. I want
> to following up (in a separate email) with my current opinions.

Thanks again.

-- 
 \         ?Truth would quickly cease to become stranger than fiction, |
  `\                     once we got as used to it.? ?Henry L. Mencken |
_o__)                                                                  |
Ben Finney


From ben+python at benfinney.id.au  Thu Jun 11 13:53:37 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 11 Jun 2009 21:53:37 +1000
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
	".postNNN"
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>
	<79990c6b0906110251qff9ab31y62b3c0062f6839b3@mail.gmail.com>
Message-ID: <877hzjasvy.fsf@benfinney.id.au>

Paul Moore <p.f.moore at gmail.com> writes:

> 2009/6/11 Trent Mick <trentm at gmail.com>:
> > # My conclusion/opinion
> >
> > In so far as the proposed new distutils versioning scheme is to be
> > used for enforcing/encouraging versions for packages uploaded to
> > PyPI, I don't believe there is a strong case for including ".devNNN"
> > and certainly not ".postNNN".
> 
> +1. That saved me from having to write an email which made this
> argument, but far less effectively :-)

+1, this is also my position. I didn't see Trent's above message in
time, but hopefully my different expression will be useful.

-- 
 \     ?Wrinkles should merely indicate where smiles have been.? ?Mark |
  `\                                    Twain, _Following the Equator_ |
_o__)                                                                  |
Ben Finney


From ziade.tarek at gmail.com  Thu Jun 11 14:11:03 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 11 Jun 2009 14:11:03 +0200
Subject: [Distutils] PEP 376 comments
In-Reply-To: <4A2D8E3D.6050004@activestate.com>
References: <FBC8A392-2135-41D1-9129-4688C5771C7D@mac.com>
	<94bdd2610906080258m2ce4398bg1f072176ff26a023@mail.gmail.com>
	<4A2D4C17.3010105@activestate.com>
	<EA36B0D3-A9B7-4A06-9532-5E0D362297CD@mac.com>
	<4A2D8E3D.6050004@activestate.com>
Message-ID: <94bdd2610906110511q6e6d5efo90770f80fb9f36f2@mail.gmail.com>

On Tue, Jun 9, 2009 at 12:18 AM, Sridhar
Ratnakumar<sridharr at activestate.com> wrote:
>
>> My best guess is that it doesn't use any encoding, it just
>> dumps the bytes in the string you specified in the PKG-INFO file (at
>> least for python2, I haven't checked what distutils does in python3).
>
> Correct; that is what I thought. Tarek said "The encoding used is utf-8
> since 2.6" .. for which I provided this counter-example (python-wifi-0.3.1).
>

In Python 2 >= 2.6 distutils uses dist.PKG_INFO_ENCODING (which is  'utf-8')

then, uses this encoding when it writes the PKG-INFO file.

That is:

<<<<
        if isinstance(value, unicode):
            value = value.encode(PKG_INFO_ENCODING)
        else:
            value = str(value)
        file.write('%s: %s\n' % (name, value))
>>>>

In Python 3 it just writes the fields like they are provided.



-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Thu Jun 11 14:58:34 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 11 Jun 2009 14:58:34 +0200
Subject: [Distutils] PEP 376 APIs reworked (was: Re:  PEP 376 comments)
Message-ID: <94bdd2610906110558n5b3cf17ciec43a5987b4b75de@mail.gmail.com>

I've reworked the APIs using all the feedback:

- now using basic types (dict + set) to make the code simpler
- renamed the classes using the "Distribution" root name
- the global functions are also now sys.path-based and there's no more
global variable
- egg_info -> egginfo
- ...

Please comment !


On Tue, Jun 9, 2009 at 12:07 AM, Kevin Teague<kevin at bud.ca> wrote:
> Glancing over the EggInfo, EggInfoDirectories, and EggInfoDirectory
> APIs, it might be interesting if these were able to match with the PEP
> 3119 interfaces (http://www.python.org/dev/peps/pep-3119/).
>
> For example, the EggInfoDirectories has __iter__, append, clear and
> remove methods, but the source code comments list them as 'container
> APIs'. Technically though, that's an Iterable interface and a partial
> implementation of the MutableSequence interface.
>
> The EggInfoDirectories.get_egg_infos(project_name) is also suggestive
> of the Mapping interface - if this method was shortened from
> get_egg_infos() to just get().
>
> Maybe it would be confusing if an EggInfoDirectories was a
> MutableSequence which could only have paths added to it, but returned
> an iterable of EggInfoDirectory, but could also be accessed as a
> Mapping where the keys are project names and those values are
> EggInfos ... but maybe something along those lines would work?
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Thu Jun 11 14:59:45 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 11 Jun 2009 14:59:45 +0200
Subject: [Distutils] PEP 376 APIs reworked (was: Re: PEP 376 comments)
In-Reply-To: <94bdd2610906110558n5b3cf17ciec43a5987b4b75de@mail.gmail.com>
References: <94bdd2610906110558n5b3cf17ciec43a5987b4b75de@mail.gmail.com>
Message-ID: <94bdd2610906110559n7d5a9c8ak5b75a0ebdbe21540@mail.gmail.com>

notice that I didn't change the PEP yet, all the changes are located in :

http://bitbucket.org/tarek/pep376/src/tip/pkgutil.py

I'll change the PEP text when the prototype is stabilized

On Thu, Jun 11, 2009 at 2:58 PM, Tarek Ziad?<ziade.tarek at gmail.com> wrote:
> I've reworked the APIs using all the feedback:
>
> - now using basic types (dict + set) to make the code simpler
> - renamed the classes using the "Distribution" root name
> - the global functions are also now sys.path-based and there's no more
> global variable
> - egg_info -> egginfo
> - ...
>
> Please comment !
>
>
> On Tue, Jun 9, 2009 at 12:07 AM, Kevin Teague<kevin at bud.ca> wrote:
>> Glancing over the EggInfo, EggInfoDirectories, and EggInfoDirectory
>> APIs, it might be interesting if these were able to match with the PEP
>> 3119 interfaces (http://www.python.org/dev/peps/pep-3119/).
>>
>> For example, the EggInfoDirectories has __iter__, append, clear and
>> remove methods, but the source code comments list them as 'container
>> APIs'. Technically though, that's an Iterable interface and a partial
>> implementation of the MutableSequence interface.
>>
>> The EggInfoDirectories.get_egg_infos(project_name) is also suggestive
>> of the Mapping interface - if this method was shortened from
>> get_egg_infos() to just get().
>>
>> Maybe it would be confusing if an EggInfoDirectories was a
>> MutableSequence which could only have paths added to it, but returned
>> an iterable of EggInfoDirectory, but could also be accessed as a
>> Mapping where the keys are project names and those values are
>> EggInfos ... but maybe something along those lines would work?
>> _______________________________________________
>> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
>> http://mail.python.org/mailman/listinfo/distutils-sig
>>
>
>
>
> --
> Tarek Ziad? | http://ziade.org
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | http://ziade.org

From exarkun at divmod.com  Thu Jun 11 15:31:58 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Thu, 11 Jun 2009 09:31:58 -0400
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
 ".postNNN"
In-Reply-To: <877hzjasvy.fsf@benfinney.id.au>
Message-ID: <20090611133159.22176.785529549.divmod.quotient.4403@henry.divmod.com>

On Thu, 11 Jun 2009 21:53:37 +1000, Ben Finney <ben+python at benfinney.id.au> wrote:
>Paul Moore <p.f.moore at gmail.com> writes:
>
>> 2009/6/11 Trent Mick <trentm at gmail.com>:
>> > # My conclusion/opinion
>> >
>> > In so far as the proposed new distutils versioning scheme is to be
>> > used for enforcing/encouraging versions for packages uploaded to
>> > PyPI, I don't believe there is a strong case for including ".devNNN"
>> > and certainly not ".postNNN".
>>
>> +1. That saved me from having to write an email which made this
>> argument, but far less effectively :-)
>
>+1, this is also my position. I didn't see Trent's above message in
>time, but hopefully my different expression will be useful.
>

What .devNNN and .postNNN provide that doesn't seem to be provided for
elsewhere in this scheme is the inclusion of a revision control identifier
(at least for vcs which have small integer revision identifiers) in the
version number.  This information makes it more easily possible to determine
which revision (ie, non-release) of software is being used.

This is why Twisted uses "post releases" (thanks for the summaries, Trent,
I had no idea what a "post release" version was before reading them).

The suggestion seems to be that projects using post releases should instead
just add another component to their version.  This would make the current
version of Twisted 8.2.0.27002.

Jean-Paul

From pje at telecommunity.com  Thu Jun 11 15:55:41 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 11 Jun 2009 09:55:41 -0400
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
 ".postNNN"
In-Reply-To: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.co
 m>
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>
Message-ID: <20090611135255.EFA2F3A402D@sparrow.telecommunity.com>

At 02:29 AM 6/11/2009 -0700, Trent Mick wrote:
>It would be interesting to get numbers of whether this setuptools
>dependency pattern is at all frequently used:
>
>     install_requires = ["OtherProject>=0.2a1.dev-r143"]
>
>I.e. a dependency on an uploaded-to-PyPI "dev" release.

PyPI uploads aren't a suitable basis for analyzing "dev" use cases, 
since the whole point of having a "dev" tag is for *non-released* 
versions.  (E.g., in-progress development via SVN.)  Dev tags are so 
that while you're doing development, your locally-installed versions 
can be distinguished from one another.


From zooko at zooko.com  Thu Jun 11 15:51:29 2009
From: zooko at zooko.com (Zooko Wilcox-O'Hearn)
Date: Thu, 11 Jun 2009 07:51:29 -0600
Subject: [Distutils] post-release revisions (was: recollections of Pycon
	distutils versioning discussion (part 1))
In-Reply-To: <6db0ea510906090058x76c8cadfj22e7d0dc619b56aa@mail.gmail.com>
References: <6db0ea510906090058x76c8cadfj22e7d0dc619b56aa@mail.gmail.com>
Message-ID: <829FF863-51DC-4214-B2A7-9FDEE82A08D5@zooko.com>

Trent Mick:

Thank you very much for the summary emails.

On Jun 9, 2009, at 1:58 AM, Trent Mick wrote:

> # Super simple
...
>     major.minor.patch
...
> # "But I want to do an alpha release!"
>
> I don't think I'm overstating in saying that most of us (those that  
> care to help in defining Python packaging tools) would want to  
> allow alpha/beta releases. Certainly this was true at the Pycon  
> discussions. This gives us:
>
>     N.N.NaN    # e.g. "1.0.0a2"
>     N.N.NbN    # e.g. "2.6.0b1"

This is where we branch between the two ways that people do it.  Some  
people count up to a future release, other people count away from  
past releases.  (Of course probably some people do both.)

In Tahoe (and I think in Twisted, and Nevow, and Foolscap), we  
typically don't count up to a release so that $REL-$SOMETHING is a  
predecessor to the $REL release.  Instead we count away from $REL, so  
that $REL-$SOMETHING is a successor to the $REL release.

Here's an example:

http://allmydata.org/source/tahoe/tarballs/

There are files in that directory named allmydata-tahoe-1.4.1- 
r3904.tar.gz, allmydata-tahoe-1.4.1-r3905.tar.gz, allmydata- 
tahoe-1.4.1-r3908.tar.gz, etc.  Each of these is newer than the  
previous one, and all of them are newer than the v1.4.1 release.

The -r$NUMBER is a count of patches in our revision control  
repository (so it is pretty much like the SVN revision numbers that  
Twisted and Nevow use in their version numbers, and it is like the  
SVN revision numbers that setuptools can append to a version string  
for you).

When the time comes for the Tahoe v1.5 release (Real Soon Now!), we  
will eventually have a release numbered 1.4.1-r3948 (assuming that it  
takes us 40 more patches from now to get ready for the next stable  
release), and then the next tarball after that will be named  
allmydata-tahoe-1.5.0.tar.gz.  (Technically, we could also name it  
allmydata-tahoe-1.5.0-r3949.tar.gz and everything would work, but we  
leave the -r$COUNT component off when this is the first release of a  
new version number.)

So, we don't use the "a/b/c" indicators, but we do use what you are  
calling "post-release" indicators.  Currently that is spelled "-r",  
which is how Tahoe, Twisted, and setuptools do it.  At PyCon I agreed  
that it wouldn't hurt to change the spelling to "-post" for clarity  
and for parallelism with "-pre".  (I don't have the authority to  
agree to anything on behalf of the Twisted or setuptools projects --  
I was just agreeing to stop arguing about it.  :-))

Regards,

Zooko

From p.f.moore at gmail.com  Thu Jun 11 16:07:35 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 11 Jun 2009 15:07:35 +0100
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
	".postNNN"
In-Reply-To: <20090611133159.22176.785529549.divmod.quotient.4403@henry.divmod.com>
References: <877hzjasvy.fsf@benfinney.id.au>
	<20090611133159.22176.785529549.divmod.quotient.4403@henry.divmod.com>
Message-ID: <79990c6b0906110707i6c681c3bt6945f8745024979e@mail.gmail.com>

2009/6/11 Jean-Paul Calderone <exarkun at divmod.com>:
> The suggestion seems to be that projects using post releases should instead
> just add another component to their version. ?This would make the current
> version of Twisted 8.2.0.27002.

That seems fine to me. DVCS users (where changeset IDs are SHA ids and
not comparable) might have an issue, but that's going to be true of
any scheme.

Paul.

From p.f.moore at gmail.com  Thu Jun 11 16:13:05 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 11 Jun 2009 15:13:05 +0100
Subject: [Distutils] post-release revisions (was: recollections of Pycon
	distutils versioning discussion (part 1))
In-Reply-To: <829FF863-51DC-4214-B2A7-9FDEE82A08D5@zooko.com>
References: <6db0ea510906090058x76c8cadfj22e7d0dc619b56aa@mail.gmail.com>
	<829FF863-51DC-4214-B2A7-9FDEE82A08D5@zooko.com>
Message-ID: <79990c6b0906110713hc0aac66j469a34cab64ae69c@mail.gmail.com>

2009/6/11 Zooko Wilcox-O'Hearn <zooko at zooko.com>:
>
> This is where we branch between the two ways that people do it. ?Some people
> count up to a future release, other people count away from past releases.
> ?(Of course probably some people do both.)
>
> In Tahoe (and I think in Twisted, and Nevow, and Foolscap), we typically
> don't count up to a release so that $REL-$SOMETHING is a predecessor to the
> $REL release. ?Instead we count away from $REL, so that $REL-$SOMETHING is a
> successor to the $REL release.
>
> Here's an example:
>
> http://allmydata.org/source/tahoe/tarballs/
>
> There are files in that directory named allmydata-tahoe-1.4.1-r3904.tar.gz,
> allmydata-tahoe-1.4.1-r3905.tar.gz, allmydata-tahoe-1.4.1-r3908.tar.gz, etc.
> ?Each of these is newer than the previous one, and all of them are newer
> than the v1.4.1 release.

So just go with 1.4.1.3908

> So, we don't use the "a/b/c" indicators, but we do use what you are calling
> "post-release" indicators. ?Currently that is spelled "-r", which is how
> Tahoe, Twisted, and setuptools do it. ?At PyCon I agreed that it wouldn't
> hurt to change the spelling to "-post" for clarity and for parallelism with
> "-pre". ?(I don't have the authority to agree to anything on behalf of the
> Twisted or setuptools projects -- I was just agreeing to stop arguing about
> it. ?:-))

If it's just arguing about which spelling to use, there should be
nothing wrong with "." rather than "-r", and it has the huge
advantages of simplicity and consistency.

Actually, using Ben's (slightly) extended definition, you could even
use ".r" if you're wedded to the idea of having something other than a
plain number.

Paul.

From p.f.moore at gmail.com  Thu Jun 11 16:16:46 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 11 Jun 2009 15:16:46 +0100
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
	".postNNN"
In-Reply-To: <20090611135255.EFA2F3A402D@sparrow.telecommunity.com>
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>
	<20090611135255.EFA2F3A402D@sparrow.telecommunity.com>
Message-ID: <79990c6b0906110716q11b3a280l3065107d4ef95e76@mail.gmail.com>

2009/6/11 P.J. Eby <pje at telecommunity.com>:
> PyPI uploads aren't a suitable basis for analyzing "dev" use cases, since
> the whole point of having a "dev" tag is for *non-released* versions.
> ?(E.g., in-progress development via SVN.)

If it's non-released, I've yet to see a clear explanation of why the
PEP is relevant. Who is going to use an API from the PEP to parse your
"version number", and why?

>?Dev tags are so that while you're
> doing development, your locally-installed versions can be distinguished from
> one another.

Distinguished by what? What code (that you didn't write yourself,
purely for internal use) needs to parse your dev tag?

Paul.

From zooko at zooko.com  Thu Jun 11 16:15:30 2009
From: zooko at zooko.com (Zooko Wilcox-O'Hearn)
Date: Thu, 11 Jun 2009 08:15:30 -0600
Subject: [Distutils] recollections of Pycon distutils versioning
	discussion (part 2)
In-Reply-To: <79990c6b0906110137h26e32765w55f164169da86154@mail.gmail.com>
References: <6db0ea510906110042x4d31ab1dr35a7865315638daf@mail.gmail.com>
	<79990c6b0906110137h26e32765w55f164169da86154@mail.gmail.com>
Message-ID: <2CE34B1E-CC24-4F5F-B673-E18B65F74058@zooko.com>

On Jun 11, 2009, at 2:37 AM, Paul Moore wrote:

> 1. From your description of events, and from discussion here, the  
> "post" tag is clearly the least thought through aspect of all this.  
> So a proponent of this should step up to justify why 1.0.5.post3 is  
> required, when 1.0.5.3 is available already.

My motivation is that the leading sequence of numbers is chosen by a  
human to communicate some information such as major rewrite (major),  
feature addition (minor), or bugfix (micro), while the numbers after  
the "-r" or the ".post" are chosen by the version control system to  
simply count patches or give a secure hash of the current tree state  
or whatever.  Another bit of information that we thus encode into the  
version number is whether it is a stable release or a snapshot --  
stable releases don't have a -r$COUNT in their version number.

The current stable release of Tahoe is v1.4.1, as visible on PyPI:  
http://pypi.python.org/pypi/allmydata-tahoe .  The current snapshot  
is v1.4.1-r3908, as visible on our web server: http://allmydata.org/ 
source/tahoe/tarballs/?C=M;O=D .

If the new "rational version number" definition excludes ".post", and  
if I choose to make Tahoe snapshot version numbers be rational  
version numbers, then I could make snapshots be named e.g.  
v1.4.1.3908.  Then I would have v1.4.1.3909, etc. until one day I  
would have v1.4.1.3948 and then v1.5.0.  The next snapshot would be  
numbered v1.5.0.3949.  I would hope that people who are looking for  
stable releases don't find the v1.5.0.3949 tarball (since it isn't on  
PyPI), or if they do find it that they realize from the extra long  
version number that it is a snapshot instead of a stable release.

I'm willing to change my build system to produce $MAJ.$MIN.$MIC.post 
$COUNT instead of $MAJ.$MIN.$MIC-r$COUNT, in order to achieve  
rationality (i.e., in order to make my versions look more like other  
people's versions and in order to be compatible with some  
hypothetical far-future tool which is picky and refuses to use  
software with irrational version numbers).  I'm not yet sure whether  
I'm willing to change it to $MAJ.$MIN.$MIC.$COUNT.

Regards,

Zooko

From exarkun at divmod.com  Thu Jun 11 16:23:20 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Thu, 11 Jun 2009 10:23:20 -0400
Subject: [Distutils] recollections of Pycon distutils versioning
 discussion (part 2)
In-Reply-To: <2CE34B1E-CC24-4F5F-B673-E18B65F74058@zooko.com>
Message-ID: <20090611142320.22176.1480928315.divmod.quotient.4424@henry.divmod.com>

On Thu, 11 Jun 2009 08:15:30 -0600, Zooko Wilcox-O'Hearn <zooko at zooko.com> wrote:
>On Jun 11, 2009, at 2:37 AM, Paul Moore wrote:
>>1. From your description of events, and from discussion here, the  "post" 
>>tag is clearly the least thought through aspect of all this.  So a 
>>proponent of this should step up to justify why 1.0.5.post3 is  required, 
>>when 1.0.5.3 is available already.
>
>My motivation is that the leading sequence of numbers is chosen by a  human 
>to communicate some information such as major rewrite (major),  feature 
>addition (minor), or bugfix (micro), while the numbers after  the "-r" or 
>the ".post" are chosen by the version control system to  simply count 
>patches or give a secure hash of the current tree state  or whatever. 
>Another bit of information that we thus encode into the  version number is 
>whether it is a stable release or a snapshot --  stable releases don't have 
>a -r$COUNT in their version number.
>
>The current stable release of Tahoe is v1.4.1, as visible on PyPI: 
>http://pypi.python.org/pypi/allmydata-tahoe .  The current snapshot  is 
>v1.4.1-r3908, as visible on our web server: http://allmydata.org/ 
>source/tahoe/tarballs/?C=M;O=D .
>
>If the new "rational version number" definition excludes ".post", and  if I 
>choose to make Tahoe snapshot version numbers be rational  version numbers, 
>then I could make snapshots be named e.g.  v1.4.1.3908.  Then I would have 
>v1.4.1.3909, etc. until one day I  would have v1.4.1.3948 and then v1.5.0. 
>The next snapshot would be  numbered v1.5.0.3949.  I would hope that people 
>who are looking for  stable releases don't find the v1.5.0.3949 tarball 
>(since it isn't on  PyPI), or if they do find it that they realize from the 
>extra long  version number that it is a snapshot instead of a stable 
>release.
>
>I'm willing to change my build system to produce $MAJ.$MIN.$MIC.post $COUNT 
>instead of $MAJ.$MIN.$MIC-r$COUNT, in order to achieve  rationality (i.e., 
>in order to make my versions look more like other  people's versions and in 
>order to be compatible with some  hypothetical far-future tool which is 
>picky and refuses to use  software with irrational version numbers).  I'm 
>not yet sure whether  I'm willing to change it to $MAJ.$MIN.$MIC.$COUNT.

This is basically how I feel about Twisted (and Nevow, Axiom, Mantissa,
Epsilon, Sine, Quotient, etc) right noow.

Jean-Paul

From eric at trueblade.com  Thu Jun 11 16:29:14 2009
From: eric at trueblade.com (Eric Smith)
Date: Thu, 11 Jun 2009 10:29:14 -0400
Subject: [Distutils] post-release revisions
In-Reply-To: <829FF863-51DC-4214-B2A7-9FDEE82A08D5@zooko.com>
References: <6db0ea510906090058x76c8cadfj22e7d0dc619b56aa@mail.gmail.com>
	<829FF863-51DC-4214-B2A7-9FDEE82A08D5@zooko.com>
Message-ID: <4A3114BA.1040009@trueblade.com>

Zooko Wilcox-O'Hearn wrote:
> Trent Mick:
> 
> Thank you very much for the summary emails.

Indeed!

>> This gives us:
>>
>>     N.N.NaN    # e.g. "1.0.0a2"
>>     N.N.NbN    # e.g. "2.6.0b1"
> 
> This is where we branch between the two ways that people do it.  Some 
> people count up to a future release, other people count away from past 
> releases.  (Of course probably some people do both.)

I think this is an excellent point, and I think that both ways of doing 
it need to be supported. The "counting up" method is especially useful 
if you're moving toward 2 different versions, simultaneously. Think 
about Python itself moving to 3.0. What should development versions of 
Python 3.0 have been called, when both 2.6 and 3.0 were being developed? 
If only "counting away" were supported, would all 3.0 build have been 
called "alpha"? Certainly calling 2.6 and 3.0 "2.5 plus something" would 
have been wrong, and a nightmare.

[From another thread]

P.J. Eby wrote:
 > PyPI uploads aren't a suitable basis for analyzing "dev" use cases,
 > since the whole point of having a "dev" tag is for *non-released*
 > versions.  (E.g., in-progress development via SVN.)  Dev tags are so
 > that while you're doing development, your locally-installed versions can
 > be distinguished from one another.

True. When I'm "sneaking up" on an alpha release, I need to have version 
comparisons work in my test environment, and I'd prefer to use "-dev". 
These releases would never be published to PyPI or anywhere else, other 
than my internal servers.

Eric.


From p.f.moore at gmail.com  Thu Jun 11 16:30:55 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 11 Jun 2009 15:30:55 +0100
Subject: [Distutils] recollections of Pycon distutils versioning
	discussion (part 2)
In-Reply-To: <2CE34B1E-CC24-4F5F-B673-E18B65F74058@zooko.com>
References: <6db0ea510906110042x4d31ab1dr35a7865315638daf@mail.gmail.com>
	<79990c6b0906110137h26e32765w55f164169da86154@mail.gmail.com>
	<2CE34B1E-CC24-4F5F-B673-E18B65F74058@zooko.com>
Message-ID: <79990c6b0906110730y15505166y3f7c44cc56fee69b@mail.gmail.com>

2009/6/11 Zooko Wilcox-O'Hearn <zooko at zooko.com>:
> If the new "rational version number" definition excludes ".post", and if I
> choose to make Tahoe snapshot version numbers be rational version numbers,
> then I could make snapshots be named e.g. v1.4.1.3908. ?Then I would have
> v1.4.1.3909, etc. until one day I would have v1.4.1.3948 and then v1.5.0.
> ?The next snapshot would be numbered v1.5.0.3949. ?I would hope that people
> who are looking for stable releases don't find the v1.5.0.3949 tarball
> (since it isn't on PyPI), or if they do find it that they realize from the
> extra long version number that it is a snapshot instead of a stable release.
>
> I'm willing to change my build system to produce $MAJ.$MIN.$MIC.post$COUNT
> instead of $MAJ.$MIN.$MIC-r$COUNT, in order to achieve rationality (i.e., in
> order to make my versions look more like other people's versions and in
> order to be compatible with some hypothetical far-future tool which is picky
> and refuses to use software with irrational version numbers). ?I'm not yet
> sure whether I'm willing to change it to $MAJ.$MIN.$MIC.$COUNT.

I really don't fully understand your motivation here (how "stable" a
release is, is more about where you get it from and how it is
advertised, than about the version number) but if it's going to stop
the arguments, let's go with Ben's definition, specified in a previous
email (dot-separated components, each component sorted
alphanumerically, with sequences of digits treated as numbers). Then
you can use .r12345, or .post12345, or even
.helloguysIcanabusethesystem12345. But please don't enshrine specific
fixed strings in the spec.

Paul

From eric at trueblade.com  Thu Jun 11 16:33:32 2009
From: eric at trueblade.com (Eric Smith)
Date: Thu, 11 Jun 2009 10:33:32 -0400
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
 ".postNNN"
In-Reply-To: <79990c6b0906110716q11b3a280l3065107d4ef95e76@mail.gmail.com>
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>	<20090611135255.EFA2F3A402D@sparrow.telecommunity.com>
	<79990c6b0906110716q11b3a280l3065107d4ef95e76@mail.gmail.com>
Message-ID: <4A3115BC.6040306@trueblade.com>

Paul Moore wrote:
> 2009/6/11 P.J. Eby <pje at telecommunity.com>:
>> PyPI uploads aren't a suitable basis for analyzing "dev" use cases, since
>> the whole point of having a "dev" tag is for *non-released* versions.
>>  (E.g., in-progress development via SVN.)
> 
> If it's non-released, I've yet to see a clear explanation of why the
> PEP is relevant. Who is going to use an API from the PEP to parse your
> "version number", and why?

I would, if my test environment used buildout to grab the latest 
versions of my own packages from my own servers. And that's in fact how 
I operate internally.

> Distinguished by what? What code (that you didn't write yourself,
> purely for internal use) needs to parse your dev tag?

Because it's my own code this use case doesn't need to be supported?

Eric.


From zooko at zooko.com  Thu Jun 11 16:41:14 2009
From: zooko at zooko.com (Zooko Wilcox-O'Hearn)
Date: Thu, 11 Jun 2009 08:41:14 -0600
Subject: [Distutils] recollections of Pycon distutils versioning
	discussion (part 2)
In-Reply-To: <79990c6b0906110730y15505166y3f7c44cc56fee69b@mail.gmail.com>
References: <6db0ea510906110042x4d31ab1dr35a7865315638daf@mail.gmail.com>
	<79990c6b0906110137h26e32765w55f164169da86154@mail.gmail.com>
	<2CE34B1E-CC24-4F5F-B673-E18B65F74058@zooko.com>
	<79990c6b0906110730y15505166y3f7c44cc56fee69b@mail.gmail.com>
Message-ID: <BCF91A19-65C1-44A3-BE5B-B9BBF09E8D3C@zooko.com>

If the "rational version number" spec allows arbitrary alphanumeric  
components, then I can write something like $MAJ.$MIC.$MIN.snapshot 
$COUNT.  Any tool which wants to compare my version numbers to each  
other will do the right thing -- newer snapshots are higher than  
older snapshots, snapshots are higher than the release (i.e. the same  
version string with the ".snapshot$COUNT" part truncated off), and  
newer releases are higher than any snapshot of an older release.

So far so good.  Now, should the "rational version number spec"  
*also* encourage those of us who use this technique to use the same  
spelling for ".snapshot" / "-r" / ".post"?  This would not effect any  
version comparison, but it would be nice for us humans if everyone  
chose the same word when they mean the same thing.

Regards,

Zooko

From p.f.moore at gmail.com  Thu Jun 11 16:48:20 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 11 Jun 2009 15:48:20 +0100
Subject: [Distutils] post-release revisions
In-Reply-To: <4A3114BA.1040009@trueblade.com>
References: <6db0ea510906090058x76c8cadfj22e7d0dc619b56aa@mail.gmail.com>
	<829FF863-51DC-4214-B2A7-9FDEE82A08D5@zooko.com>
	<4A3114BA.1040009@trueblade.com>
Message-ID: <79990c6b0906110748h7cdab88k723b96d8797ba4cf@mail.gmail.com>

2009/6/11 Eric Smith <eric at trueblade.com>:
> I think this is an excellent point, and I think that both ways of doing it
> need to be supported. The "counting up" method is especially useful if
> you're moving toward 2 different versions, simultaneously. Think about
> Python itself moving to 3.0. What should development versions of Python 3.0
> have been called, when both 2.6 and 3.0 were being developed? If only
> "counting away" were supported, would all 3.0 build have been called
> "alpha"? Certainly calling 2.6 and 3.0 "2.5 plus something" would have been
> wrong, and a nightmare.

They weren't called anything (OK, internally all such releases have
version number 2.6a0 and 3.0a0 respectively, but individual Subversion
commits had no distinct version). And that's precisely right, as they
are internal commits, and were not released.

Why is there a need for every internal commit to have a distinct
"version number"? I'm not saying there's no need to be able to
identify the code base, but why must everything be lumped into the
version? In my experience (and in the practice used by Python core)
version numbers are for released versions. Snapshots, nightly builds,
personal developer builds, etc, don't have version numbers.

> True. When I'm "sneaking up" on an alpha release, I need to have version
> comparisons work in my test environment, and I'd prefer to use "-dev". These
> releases would never be published to PyPI or anywhere else, other than my
> internal servers.

I'm sorry, but why? How does your test environment work so that you
are testing 2 versions at once, and need to compare "version numbers"
programmatically? And why isn't something *additional* to the version
number, like the core's sys.subversion, suitable?

Please note, in case I seem excessively aggressive - my assumption is
that the standard defined by the PEP doesn't have to be used by
everyone, in all cases. You presumably have code that works at the
moment, and you are perfectly entitled to continue using it. If you
switch to the APIs proposed in the PEP, then presumably you see some
benefits in following the standard. There's clearly a cost in doing so
- one part of the cost is changing your code, and I don't think it's
unreasonable for another part of that cost to be changing your version
numbering scheme (at least for your released versions). Whether that
cost is too high is a decision you have to make yourself. Designing
the PEP is a compromise between all the various aspects involved.

Paul.

From p.f.moore at gmail.com  Thu Jun 11 16:50:16 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 11 Jun 2009 15:50:16 +0100
Subject: [Distutils] recollections of Pycon distutils versioning
	discussion (part 2)
In-Reply-To: <BCF91A19-65C1-44A3-BE5B-B9BBF09E8D3C@zooko.com>
References: <6db0ea510906110042x4d31ab1dr35a7865315638daf@mail.gmail.com>
	<79990c6b0906110137h26e32765w55f164169da86154@mail.gmail.com>
	<2CE34B1E-CC24-4F5F-B673-E18B65F74058@zooko.com>
	<79990c6b0906110730y15505166y3f7c44cc56fee69b@mail.gmail.com>
	<BCF91A19-65C1-44A3-BE5B-B9BBF09E8D3C@zooko.com>
Message-ID: <79990c6b0906110750w1cd42621wb458daff30954b3e@mail.gmail.com>

2009/6/11 Zooko Wilcox-O'Hearn <zooko at zooko.com>:
> So far so good. ?Now, should the "rational version number spec" *also*
> encourage those of us who use this technique to use the same spelling for
> ".snapshot" / "-r" / ".post"? ?This would not effect any version comparison,
> but it would be nice for us humans if everyone chose the same word when they
> mean the same thing.

Not if it perpetuates this endless discussion, IMHO :-)

Paul.

From p.f.moore at gmail.com  Thu Jun 11 17:10:20 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 11 Jun 2009 16:10:20 +0100
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
	".postNNN"
In-Reply-To: <4A3115BC.6040306@trueblade.com>
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>
	<20090611135255.EFA2F3A402D@sparrow.telecommunity.com>
	<79990c6b0906110716q11b3a280l3065107d4ef95e76@mail.gmail.com>
	<4A3115BC.6040306@trueblade.com>
Message-ID: <79990c6b0906110810k6e37a778tebc2f6627623ad91@mail.gmail.com>

2009/6/11 Eric Smith <eric at trueblade.com>:
> Paul Moore wrote:
>>
>> 2009/6/11 P.J. Eby <pje at telecommunity.com>:
>>>
>>> PyPI uploads aren't a suitable basis for analyzing "dev" use cases, since
>>> the whole point of having a "dev" tag is for *non-released* versions.
>>> ?(E.g., in-progress development via SVN.)
>>
>> If it's non-released, I've yet to see a clear explanation of why the
>> PEP is relevant. Who is going to use an API from the PEP to parse your
>> "version number", and why?
>
> I would, if my test environment used buildout to grab the latest versions of
> my own packages from my own servers. And that's in fact how I operate
> internally.
>
>> Distinguished by what? What code (that you didn't write yourself,
>> purely for internal use) needs to parse your dev tag?
>
> Because it's my own code this use case doesn't need to be supported?

That's not what I meant, but I suspect that if I try to clarify, I'll
make things worse, as I'm losing track of things at this point :-(

Does Ben's proposed solution satisfy your needs? If not, let's try to
understand why, and we can start the discussion again with a clean
slate.

From pje at telecommunity.com  Thu Jun 11 17:53:15 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 11 Jun 2009 11:53:15 -0400
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
 ".postNNN"
In-Reply-To: <79990c6b0906110716q11b3a280l3065107d4ef95e76@mail.gmail.co
 m>
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>
	<20090611135255.EFA2F3A402D@sparrow.telecommunity.com>
	<79990c6b0906110716q11b3a280l3065107d4ef95e76@mail.gmail.com>
Message-ID: <20090611155023.368A83A402D@sparrow.telecommunity.com>

At 03:16 PM 6/11/2009 +0100, Paul Moore wrote:
>2009/6/11 P.J. Eby <pje at telecommunity.com>:
> > PyPI uploads aren't a suitable basis for analyzing "dev" use cases, since
> > the whole point of having a "dev" tag is for *non-released* versions.
> >  (E.g., in-progress development via SVN.)
>
>If it's non-released, I've yet to see a clear explanation of why the
>PEP is relevant. Who is going to use an API from the PEP to parse your
>"version number", and why?
>
> > Dev tags are so that while you're
> > doing development, your locally-installed versions can be 
> distinguished from
> > one another.
>
>Distinguished by what? What code (that you didn't write yourself,
>purely for internal use) needs to parse your dev tag?

Distinguished by setuptools for processing version requirements of 
scripts, or require() statements in code, and installation 
requirements of newly-installed code.

For example, if I'm working on two projects that are distributed via 
SVN and one depends on the other, if I update one, it may require an 
update of the other; the failure of the .dev#### version requirement 
in the first one will inform me of the need to "svn up" the second 
project and rerun "setup.py develop" on it.

This is a routine circumstance in at least my development cycle; I 
would expect that it's the case in other open source development 
workflows as well as proprietary ones.


From pje at telecommunity.com  Thu Jun 11 18:04:08 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 11 Jun 2009 12:04:08 -0400
Subject: [Distutils] post-release revisions
In-Reply-To: <79990c6b0906110748h7cdab88k723b96d8797ba4cf@mail.gmail.com
 >
References: <6db0ea510906090058x76c8cadfj22e7d0dc619b56aa@mail.gmail.com>
	<829FF863-51DC-4214-B2A7-9FDEE82A08D5@zooko.com>
	<4A3114BA.1040009@trueblade.com>
	<79990c6b0906110748h7cdab88k723b96d8797ba4cf@mail.gmail.com>
Message-ID: <20090611160116.9D0813A402D@sparrow.telecommunity.com>

At 03:48 PM 6/11/2009 +0100, Paul Moore wrote:
>Please note, in case I seem excessively aggressive - my assumption is
>that the standard defined by the PEP doesn't have to be used by
>everyone, in all cases. You presumably have code that works at the
>moment, and you are perfectly entitled to continue using it.

That depends on whether this PEP's version scheme is also going to be 
embedded in the code implied by PEP 376.  If so, then nobody will 
have any reason to switch from using pkg_resources to the PEP 376 
pkgutil code, because they won't be able to manage their test, dev, 
and collaboration environments that way.

>  If you
>switch to the APIs proposed in the PEP, then presumably you see some
>benefits in following the standard.

That's precisely it - there aren't any visible benefits to doing so, 
but there *are* visible problems.

We need to have a way to translate existing version schemes to the 
new one, if we're to be able to have a reasonable upgrade path -- 
considering that these PEPs' only benefit to adopters (at least among 
those using setuptools) is that the code will be in some future 
stdlib version, and support simple uninstalls.


From ianb at colorstudy.com  Thu Jun 11 18:03:19 2009
From: ianb at colorstudy.com (Ian Bicking)
Date: Thu, 11 Jun 2009 11:03:19 -0500
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
	".postNNN"
In-Reply-To: <20090611155023.368A83A402D@sparrow.telecommunity.com>
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com> 
	<20090611135255.EFA2F3A402D@sparrow.telecommunity.com>
	<79990c6b0906110716q11b3a280l3065107d4ef95e76@mail.gmail.com> 
	<20090611155023.368A83A402D@sparrow.telecommunity.com>
Message-ID: <b654cd2e0906110903p6ef3c5c2la560216b7246c9c7@mail.gmail.com>

On Thu, Jun 11, 2009 at 10:53 AM, P.J. Eby<pje at telecommunity.com> wrote:
> At 03:16 PM 6/11/2009 +0100, Paul Moore wrote:
>>
>> 2009/6/11 P.J. Eby <pje at telecommunity.com>:
>> > PyPI uploads aren't a suitable basis for analyzing "dev" use cases,
>> > since
>> > the whole point of having a "dev" tag is for *non-released* versions.
>> > ?(E.g., in-progress development via SVN.)
>>
>> If it's non-released, I've yet to see a clear explanation of why the
>> PEP is relevant. Who is going to use an API from the PEP to parse your
>> "version number", and why?
>>
>> > Dev tags are so that while you're
>> > doing development, your locally-installed versions can be distinguished
>> > from
>> > one another.
>>
>> Distinguished by what? What code (that you didn't write yourself,
>> purely for internal use) needs to parse your dev tag?
>
> Distinguished by setuptools for processing version requirements of scripts,
> or require() statements in code, and installation requirements of
> newly-installed code.
>
> For example, if I'm working on two projects that are distributed via SVN and
> one depends on the other, if I update one, it may require an update of the
> other; the failure of the .dev#### version requirement in the first one will
> inform me of the need to "svn up" the second project and rerun "setup.py
> develop" on it.
>
> This is a routine circumstance in at least my development cycle; I would
> expect that it's the case in other open source development workflows as well
> as proprietary ones.

Agreed, I do this all the time.  Pylons dev versions also regularly
rely on other packages with a dev version, and people regularly use
these non-released versions, with dependencies detected and installed
via dependency_links.


-- 
Ian Bicking  |  http://blog.ianbicking.org  |  http://topplabs.org/civichacker

From sklein at cpcug.org  Thu Jun 11 18:26:17 2009
From: sklein at cpcug.org (Stanley A. Klein)
Date: Thu, 11 Jun 2009 12:26:17 -0400 (EDT)
Subject: [Distutils] Problem using bdist_rpm with Fedora 10
Message-ID: <34043.71.163.211.210.1244737577.squirrel@www.cpcug.org>

I tried to build java2python using bdist_rpm on Fedora 10 and got the
following errors:

RPM build errors:
    Installed (but unpackaged) file(s) found:
   /usr/lib/python2.5/site-packages/java2python/__init__.pyo
   /usr/lib/python2.5/site-packages/java2python/lib/__init__.pyo
   /usr/lib/python2.5/site-packages/java2python/lib/defaultconfig.pyo
   /usr/lib/python2.5/site-packages/java2python/lib/lexer.pyo
   /usr/lib/python2.5/site-packages/java2python/lib/parser.pyo
   /usr/lib/python2.5/site-packages/java2python/lib/sourcetypes.pyo
   /usr/lib/python2.5/site-packages/java2python/lib/walker.pyo
error: command 'rpmbuild' failed with exit status 1

This had been an issue in the past, and the recommended solution was to
prepare a setup.cfg containing

[install]
optimize = 1

This had worked in the past, but it had no effect this time.

I found some discussion of the issue and some aspect of it is supposedly
addressed in a change to Fedora.  I upgraded to the latest rpm packages
(version -4.6.1-1.fc10), but got the same behavior.

How can I proceed to do the build?

Thanks.


Stan Klein








From p.f.moore at gmail.com  Thu Jun 11 19:01:00 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 11 Jun 2009 18:01:00 +0100
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
	".postNNN"
In-Reply-To: <20090611155023.368A83A402D@sparrow.telecommunity.com>
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>
	<20090611135255.EFA2F3A402D@sparrow.telecommunity.com>
	<79990c6b0906110716q11b3a280l3065107d4ef95e76@mail.gmail.com>
	<20090611155023.368A83A402D@sparrow.telecommunity.com>
Message-ID: <79990c6b0906111001r4aeba710hde3e5e0eaa528608@mail.gmail.com>

2009/6/11 P.J. Eby <pje at telecommunity.com>:
>> > Dev tags are so that while you're
>> > doing development, your locally-installed versions can be distinguished
>> > from one another.
>>
>> Distinguished by what? What code (that you didn't write yourself,
>> purely for internal use) needs to parse your dev tag?
>
> Distinguished by setuptools for processing version requirements of scripts,
> or require() statements in code, and installation requirements of
> newly-installed code.

So will setuptools be modified to use the new code? If not, the PEP is
directly competing with setuptools. And the new PEP will probably not
be adopted by anyone currently using setuptools (after all, switching
is all cost and no benefit).

If setuptools *will* be changing, then setuptools users have to choose
whether to conform to the new version of setuptools (and hence the
spec) or to remain on an old/forked version of setuptools. That's a
genuine choice - although maybe not one that people will like being
given.

My understanding was that setuptools was a success at least in part
due to its policy of catering for as many variations on existing
policy as possible, effectively refusing to enforce policy, but rather
to adapt to existing use. If the new PEP isn't designed precisely to
limit variation, and enforce policy even if that does mean excluding
certain current usages, then I don't see why the PEP doesn't just say
"adopt setuptools version code" [1].

So I guess I'm missing the point of the PEP, given the existence of setuptools.

> For example, if I'm working on two projects that are distributed via SVN and
> one depends on the other, if I update one, it may require an update of the
> other; the failure of the .dev#### version requirement in the first one will
> inform me of the need to "svn up" the second project and rerun "setup.py
> develop" on it.
>
> This is a routine circumstance in at least my development cycle; I would
> expect that it's the case in other open source development workflows as well
> as proprietary ones.

But setup.py develop is a setuptools extension. The PEP doesn't say
anything about setuptools. Are you saying that setuptools *will* be
modified to use the PEP version rules? And hence, what are you saying
about the relationship between the principle I stated above (assuming
I didn't misinterpret things) and the (implicit) goals of the PEP
process?

Paul.

[1] Actually, if I was being cynical, I could say that the reason is
to avoid alienating people who don't like / use setuptools. Either
way, it may be that the reason I'm making such a fuss is that I'm
(unconsciously) reacting against the same aspects of the PEP that I
dislike in setuptools.

From python at venix.com  Thu Jun 11 19:02:18 2009
From: python at venix.com (Lloyd Kvam)
Date: Thu, 11 Jun 2009 13:02:18 -0400
Subject: [Distutils] Problem using bdist_rpm with Fedora 10
In-Reply-To: <34043.71.163.211.210.1244737577.squirrel@www.cpcug.org>
References: <34043.71.163.211.210.1244737577.squirrel@www.cpcug.org>
Message-ID: <1244739738.17125.35.camel@venix-laptop>

On Thu, 2009-06-11 at 12:26 -0400, Stanley A. Klein wrote:
> I tried to build java2python using bdist_rpm on Fedora 10 and got the
> following errors:
> 
> RPM build errors:
>     Installed (but unpackaged) file(s) found:
>    /usr/lib/python2.5/site-packages/java2python/__init__.pyo
>    /usr/lib/python2.5/site-packages/java2python/lib/__init__.pyo
>    /usr/lib/python2.5/site-packages/java2python/lib/defaultconfig.pyo
>    /usr/lib/python2.5/site-packages/java2python/lib/lexer.pyo
>    /usr/lib/python2.5/site-packages/java2python/lib/parser.pyo
>    /usr/lib/python2.5/site-packages/java2python/lib/sourcetypes.pyo
>    /usr/lib/python2.5/site-packages/java2python/lib/walker.pyo
> error: command 'rpmbuild' failed with exit status 1
> 
> This had been an issue in the past, and the recommended solution was to
> prepare a setup.cfg containing
> 
> [install]
> optimize = 1
> 
> This had worked in the past, but it had no effect this time.
> 
> I found some discussion of the issue and some aspect of it is supposedly
> addressed in a change to Fedora.  I upgraded to the latest rpm packages
> (version -4.6.1-1.fc10), but got the same behavior.
> 
> How can I proceed to do the build?

https://bugzilla.redhat.com/show_bug.cgi?id=236535#c20
I believe this patch is still appropriate.  Presumably it is not in the
latest rpms.

> 
> Thanks.
> 
> 
> Stan Klein
> 
> 
> 
> 
> 
> 
> 
> _______________________________________________
> Fedora-python-devel-list mailing list
> Fedora-python-devel-list at redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-python-devel-list
-- 
Lloyd Kvam
Venix Corp
DLSLUG/GNHLUG library
http://dlslug.org/library.html
http://www.librarything.com/catalog/dlslug
http://www.librarything.com/rsshtml/recent/dlslug
http://www.librarything.com/rss/recent/dlslug


From trentm at gmail.com  Thu Jun 11 20:05:36 2009
From: trentm at gmail.com (Trent Mick)
Date: Thu, 11 Jun 2009 11:05:36 -0700
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
	".postNNN"
In-Reply-To: <b654cd2e0906110903p6ef3c5c2la560216b7246c9c7@mail.gmail.com>
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>
	<20090611135255.EFA2F3A402D@sparrow.telecommunity.com>
	<79990c6b0906110716q11b3a280l3065107d4ef95e76@mail.gmail.com>
	<20090611155023.368A83A402D@sparrow.telecommunity.com>
	<b654cd2e0906110903p6ef3c5c2la560216b7246c9c7@mail.gmail.com>
Message-ID: <6db0ea510906111105y79c346e3jf60e282172de16ad@mail.gmail.com>

On Thu, Jun 11, 2009 at 9:03 AM, Ian Bicking<ianb at colorstudy.com> wrote:
> On Thu, Jun 11, 2009 at 10:53 AM, P.J. Eby<pje at telecommunity.com> wrote:
>> For example, if I'm working on two projects that are distributed via SVN and
>> one depends on the other, if I update one, it may require an update of the
>> other; the failure of the .dev#### version requirement in the first one will
>> inform me of the need to "svn up" the second project and rerun "setup.py
>> develop" on it.
>>
>> This is a routine circumstance in at least my development cycle; I would
>> expect that it's the case in other open source development workflows as well
>> as proprietary ones.
>
> Agreed, I do this all the time. ?Pylons dev versions also regularly
> rely on other packages with a dev version, and people regularly use
> these non-released versions, with dependencies detected and installed
> via dependency_links.

If there were a setup.py metadata field (called "dev_revision" or
"build_number" or something), separate from the "version" field, that
was used to hold the value for sorting/distinguishing unreleased
versions... could that work?

I.e. Have the shorter "N.N.N[(a|b|c)N]" scheme for "version" to be
used for "released" packages. And have a separate field (or fields)
for use in dependency handling of unreleased versions? Putting the two
together is resulting in package uploads to PyPI
(foo-1.2.3.dev-r456.tar.gz) that I think were never intended in the
design

A weird thing about defining a sort order for:

  1.2.3.dev-r450
  1.2.3          # this is the released version (svn rev was r454)
  1.2.3-r456

is that it is comparing apples and oranges. The "1.2.3" released
version *had* a VCS revision number (or release date, or "number of
patches").


Trent

-- 
Trent Mick
trentm at gmail.com

From ziade.tarek at gmail.com  Thu Jun 11 20:59:16 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 11 Jun 2009 20:59:16 +0200
Subject: [Distutils] Problem using bdist_rpm with Fedora 10
In-Reply-To: <1244739738.17125.35.camel@venix-laptop>
References: <34043.71.163.211.210.1244737577.squirrel@www.cpcug.org>
	<1244739738.17125.35.camel@venix-laptop>
Message-ID: <94bdd2610906111159k37c6be5fp5b8f4cd8d4630a30@mail.gmail.com>

The change mentioned in the red-hat patch has been included in Python
the upcoming Python releases.

If you still have an eeror with -O1, it's something else. You'll have
to provide more information



On Thu, Jun 11, 2009 at 7:02 PM, Lloyd Kvam<python at venix.com> wrote:
> On Thu, 2009-06-11 at 12:26 -0400, Stanley A. Klein wrote:
>> I tried to build java2python using bdist_rpm on Fedora 10 and got the
>> following errors:
>>
>> RPM build errors:
>> ? ? Installed (but unpackaged) file(s) found:
>> ? ?/usr/lib/python2.5/site-packages/java2python/__init__.pyo
>> ? ?/usr/lib/python2.5/site-packages/java2python/lib/__init__.pyo
>> ? ?/usr/lib/python2.5/site-packages/java2python/lib/defaultconfig.pyo
>> ? ?/usr/lib/python2.5/site-packages/java2python/lib/lexer.pyo
>> ? ?/usr/lib/python2.5/site-packages/java2python/lib/parser.pyo
>> ? ?/usr/lib/python2.5/site-packages/java2python/lib/sourcetypes.pyo
>> ? ?/usr/lib/python2.5/site-packages/java2python/lib/walker.pyo
>> error: command 'rpmbuild' failed with exit status 1
>>
>> This had been an issue in the past, and the recommended solution was to
>> prepare a setup.cfg containing
>>
>> [install]
>> optimize = 1
>>
>> This had worked in the past, but it had no effect this time.
>>
>> I found some discussion of the issue and some aspect of it is supposedly
>> addressed in a change to Fedora. ?I upgraded to the latest rpm packages
>> (version -4.6.1-1.fc10), but got the same behavior.
>>
>> How can I proceed to do the build?
>
> https://bugzilla.redhat.com/show_bug.cgi?id=236535#c20
> I believe this patch is still appropriate. ?Presumably it is not in the
> latest rpms.
>
>>
>> Thanks.
>>
>>
>> Stan Klein
>>
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> Fedora-python-devel-list mailing list
>> Fedora-python-devel-list at redhat.com
>> https://www.redhat.com/mailman/listinfo/fedora-python-devel-list
> --
> Lloyd Kvam
> Venix Corp
> DLSLUG/GNHLUG library
> http://dlslug.org/library.html
> http://www.librarything.com/catalog/dlslug
> http://www.librarything.com/rsshtml/recent/dlslug
> http://www.librarything.com/rss/recent/dlslug
>
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Thu Jun 11 21:52:19 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 11 Jun 2009 21:52:19 +0200
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
	".postNNN"
In-Reply-To: <6db0ea510906111105y79c346e3jf60e282172de16ad@mail.gmail.com>
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>
	<20090611135255.EFA2F3A402D@sparrow.telecommunity.com>
	<79990c6b0906110716q11b3a280l3065107d4ef95e76@mail.gmail.com>
	<20090611155023.368A83A402D@sparrow.telecommunity.com>
	<b654cd2e0906110903p6ef3c5c2la560216b7246c9c7@mail.gmail.com>
	<6db0ea510906111105y79c346e3jf60e282172de16ad@mail.gmail.com>
Message-ID: <94bdd2610906111252m39e6c20bsee5df434f8d7823e@mail.gmail.com>

On Thu, Jun 11, 2009 at 8:05 PM, Trent Mick<trentm at gmail.com> wrote:
> On Thu, Jun 11, 2009 at 9:03 AM, Ian Bicking<ianb at colorstudy.com> wrote:
>> On Thu, Jun 11, 2009 at 10:53 AM, P.J. Eby<pje at telecommunity.com> wrote:
>>> For example, if I'm working on two projects that are distributed via SVN and
>>> one depends on the other, if I update one, it may require an update of the
>>> other; the failure of the .dev#### version requirement in the first one will
>>> inform me of the need to "svn up" the second project and rerun "setup.py
>>> develop" on it.
>>>
>>> This is a routine circumstance in at least my development cycle; I would
>>> expect that it's the case in other open source development workflows as well
>>> as proprietary ones.
>>
>> Agreed, I do this all the time. ?Pylons dev versions also regularly
>> rely on other packages with a dev version, and people regularly use
>> these non-released versions, with dependencies detected and installed
>> via dependency_links.

Me too, dev version have to be included in the comparison tool.

>
> If there were a setup.py metadata field (called "dev_revision" or
> "build_number" or something), separate from the "version" field, that
> was used to hold the value for sorting/distinguishing unreleased
> versions... could that work?
>
> I.e. Have the shorter "N.N.N[(a|b|c)N]" scheme for "version" to be
> used for "released" packages. And have a separate field (or fields)
> for use in dependency handling of unreleased versions? Putting the two
> together is resulting in package uploads to PyPI

What would be the difference then with the initial proposal ? You
would end up merging the "short" version
with the dev field to be able to sort different versions of the same
distribution.

If we have dev versions, we have to include them in the scheme

Tarek

From trentm at gmail.com  Thu Jun 11 22:38:33 2009
From: trentm at gmail.com (Trent Mick)
Date: Thu, 11 Jun 2009 13:38:33 -0700
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
	".postNNN"
In-Reply-To: <94bdd2610906111252m39e6c20bsee5df434f8d7823e@mail.gmail.com>
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>
	<20090611135255.EFA2F3A402D@sparrow.telecommunity.com>
	<79990c6b0906110716q11b3a280l3065107d4ef95e76@mail.gmail.com>
	<20090611155023.368A83A402D@sparrow.telecommunity.com>
	<b654cd2e0906110903p6ef3c5c2la560216b7246c9c7@mail.gmail.com>
	<6db0ea510906111105y79c346e3jf60e282172de16ad@mail.gmail.com>
	<94bdd2610906111252m39e6c20bsee5df434f8d7823e@mail.gmail.com>
Message-ID: <6db0ea510906111338n3b7b32a1w430f3a592e722388@mail.gmail.com>

>> I.e. Have the shorter "N.N.N[(a|b|c)N]" scheme for "version" to be
>> used for "released" packages. And have a separate field (or fields)
>> for use in dependency handling of unreleased versions? Putting the two
>> together is resulting in package uploads to PyPI
>
> What would be the difference then with the initial proposal ? You
> would end up merging the "short" version
> with the dev field to be able to sort different versions of the same
> distribution.
>
> If we have dev versions, we have to include them in the scheme

I've been thinking from the p.o.v. of what releases get up on PyPI....
and I gather that those releases are the ones that lead to potential
packaging in RPM and .deb repositories.

Say "version" and "build_number" (or whatever name for the latter) are
separate fields. Only "version" is used for putting in package names
(sdist, bdist_*). However the setup() fields for dependency info can
specify checks against both version and build_number.

The difference with the initial proposal (if I'm not missing something) is that:

- packages looking like "foo-1.2.3.dev-r456.tar.gz" don't get uploaded
to PyPI (yeah!)
- the meta data in my released version can still state what SCC
revision (in the build_number field) it was built against
- when I specify a dependency against a particular build_number of a
package, I don't care if that build_number happened to be a released
version or a dev version


Trent

-- 
Trent Mick
trentm at gmail.com

From pje at telecommunity.com  Thu Jun 11 23:12:49 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 11 Jun 2009 17:12:49 -0400
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
 ".postNNN"
In-Reply-To: <79990c6b0906111001r4aeba710hde3e5e0eaa528608@mail.gmail.co
 m>
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>
	<20090611135255.EFA2F3A402D@sparrow.telecommunity.com>
	<79990c6b0906110716q11b3a280l3065107d4ef95e76@mail.gmail.com>
	<20090611155023.368A83A402D@sparrow.telecommunity.com>
	<79990c6b0906111001r4aeba710hde3e5e0eaa528608@mail.gmail.com>
Message-ID: <20090611211326.D35A73A40D9@sparrow.telecommunity.com>

At 06:01 PM 6/11/2009 +0100, Paul Moore wrote:
>2009/6/11 P.J. Eby <pje at telecommunity.com>:
> >> > Dev tags are so that while you're
> >> > doing development, your locally-installed versions can be distinguished
> >> > from one another.
> >>
> >> Distinguished by what? What code (that you didn't write yourself,
> >> purely for internal use) needs to parse your dev tag?
> >
> > Distinguished by setuptools for processing version requirements of scripts,
> > or require() statements in code, and installation requirements of
> > newly-installed code.
>
>So will setuptools be modified to use the new code?

It doesn't have to be; the RationalVersion scheme is both a strict 
subset of setuptools' scheme, and also has the property that most 
in-field uses of setuptools-based versioning schemes can be 
*mechanically* translated to RationalVersion instances, using an 
appropriate implementation of suggest_version().

This means that if somebody chooses to adopt RationalVersion (as 
currently defined in the PEP), setuptools will read and process it as 
correctly as it does more exotic numbering schemes.

IOW, setuptools doesn't need to be changed, unless it was to say, add 
warnings for non-convertible versions...  and that could actually be 
done with a plugin.


From pje at telecommunity.com  Thu Jun 11 23:16:18 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 11 Jun 2009 17:16:18 -0400
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
 ".postNNN"
In-Reply-To: <6db0ea510906111338n3b7b32a1w430f3a592e722388@mail.gmail.co
 m>
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>
	<20090611135255.EFA2F3A402D@sparrow.telecommunity.com>
	<79990c6b0906110716q11b3a280l3065107d4ef95e76@mail.gmail.com>
	<20090611155023.368A83A402D@sparrow.telecommunity.com>
	<b654cd2e0906110903p6ef3c5c2la560216b7246c9c7@mail.gmail.com>
	<6db0ea510906111105y79c346e3jf60e282172de16ad@mail.gmail.com>
	<94bdd2610906111252m39e6c20bsee5df434f8d7823e@mail.gmail.com>
	<6db0ea510906111338n3b7b32a1w430f3a592e722388@mail.gmail.com>
Message-ID: <20090611211327.8E0EF3A411E@sparrow.telecommunity.com>

At 01:38 PM 6/11/2009 -0700, Trent Mick wrote:
>- when I specify a dependency against a particular build_number of a
>package, I don't care if that build_number happened to be a released
>version or a dev version

However, to specify that dependency you're going to need a way to 
represent the build number as part of a requirement string, at which 
point we're right back where we started.


From trentm at gmail.com  Thu Jun 11 23:44:23 2009
From: trentm at gmail.com (Trent Mick)
Date: Thu, 11 Jun 2009 14:44:23 -0700
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
	".postNNN"
In-Reply-To: <20090611211327.8E0EF3A411E@sparrow.telecommunity.com>
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>
	<20090611135255.EFA2F3A402D@sparrow.telecommunity.com>
	<79990c6b0906110716q11b3a280l3065107d4ef95e76@mail.gmail.com>
	<20090611155023.368A83A402D@sparrow.telecommunity.com>
	<b654cd2e0906110903p6ef3c5c2la560216b7246c9c7@mail.gmail.com>
	<6db0ea510906111105y79c346e3jf60e282172de16ad@mail.gmail.com>
	<94bdd2610906111252m39e6c20bsee5df434f8d7823e@mail.gmail.com>
	<6db0ea510906111338n3b7b32a1w430f3a592e722388@mail.gmail.com>
	<20090611211327.8E0EF3A411E@sparrow.telecommunity.com>
Message-ID: <6db0ea510906111444y5a4d91b6p7bf770fb436f443d@mail.gmail.com>

>> - when I specify a dependency against a particular build_number of a
>> package, I don't care if that build_number happened to be a released
>> version or a dev version
>
> However, to specify that dependency you're going to need a way to represent
> the build number as part of a requirement string, at which point we're right
> back where we started.

Perhaps. I'm wondering if the separation of "version" (does not
include the ".dev") and "build_number" helps clear up some of the
cases.

If "1.2.3.dev456"-type version strings don't appear in packages
released to PyPI, then the job of the downstream RPM/.deb packagers is
easier (they then don't need to care about the spelling of the version
with the build number). Have a "RationalReleaseVersion()" that is just
the non-dev part of the proposal.

Yes, as you say, the requirement/dependency fields (presumably they
will be strings) need a way to spell the "build_number" part. However,
the large set of setup.py authors that don't need to understand or use
dependency strings don't need to see that.

Trent

-- 
Trent Mick
trentm at gmail.com

From stephen.c.waterbury at nasa.gov  Thu Jun 11 22:51:24 2009
From: stephen.c.waterbury at nasa.gov (Stephen Waterbury)
Date: Thu, 11 Jun 2009 16:51:24 -0400
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
 ".postNNN"
In-Reply-To: <6db0ea510906111338n3b7b32a1w430f3a592e722388@mail.gmail.com>
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>	<20090611135255.EFA2F3A402D@sparrow.telecommunity.com>	<79990c6b0906110716q11b3a280l3065107d4ef95e76@mail.gmail.com>	<20090611155023.368A83A402D@sparrow.telecommunity.com>	<b654cd2e0906110903p6ef3c5c2la560216b7246c9c7@mail.gmail.com>	<6db0ea510906111105y79c346e3jf60e282172de16ad@mail.gmail.com>	<94bdd2610906111252m39e6c20bsee5df434f8d7823e@mail.gmail.com>
	<6db0ea510906111338n3b7b32a1w430f3a592e722388@mail.gmail.com>
Message-ID: <4A316E4C.30408@nasa.gov>

Trent Mick wrote:
> I've been thinking from the p.o.v. of what releases get up on PyPI....
> and I gather that those releases are the ones that lead to potential
> packaging in RPM and .deb repositories.

I think that is not necessarily true -- I seem to recall some discussion
in the PyCon sessions about OS distro folks sometimes having to
package dev versions ... hopefully there are some distro packagers
here who can chime in on this ...

Steve


From jason at mail.cornsyrup.org  Thu Jun 11 22:42:23 2009
From: jason at mail.cornsyrup.org (jason)
Date: Thu, 11 Jun 2009 13:42:23 -0700
Subject: [Distutils] =?utf-8?q?easy=5Finstall_self_upgrade/version_check_b?=
	=?utf-8?q?ug?=
Message-ID: <b77cc058255c5d3a518d29d02e4d1fcb@mail.cornsyrup.org>


I've run into a problem where easy_install/setuptools continues to
recommend upgrading itself even after the upgrade has been installed. Steps
to reproduce on CentOS 5.2:

include ~/python in PYTHONPATH
include ~/bin in PATH

create ~/.pydistutils.cfg 
[install]
install_lib = ~/python
install_scripts = ~/bin

$ easy_install flup
...
Running flup-1.0.3.dev-20090608/setup.py -q bdist_egg --dist-dir
/tmp/easy_install-rPyZ6a/flup-1.0.3.dev-20090608/egg-dist-tmp-pm0o4x
The required version of setuptools (>=0.6c9) is not available, and
can't be installed while this script is running. Please install
 a more recent version first, using 'easy_install -U setuptools'.

(Currently using setuptools 0.6c5 (/usr/lib/python2.4/site-packages))
error: Setup script exited with 2
$ easy_install -U setuptools
...
Processing dependencies for setuptools
$ which easy_install
~/bin/easy_install
$ python -c 'import setuptools; print setuptools.__version__'
0.6c9
$ easy_install flup
...
Running flup-1.0.3.dev-20090608/setup.py -q bdist_egg --dist-dir
/tmp/easy_install-WoxTch/flup-1.0.3.dev-20090608/egg-dist-tmp-oFotRt
The required version of setuptools (>=0.6c9) is not available, and
can't be installed while this script is running. Please install
 a more recent version first, using 'easy_install -U setuptools'.

(Currently using setuptools 0.6c5 (/usr/lib/python2.4/site-packages))
error: Setup script exited with 2


At this point I thought it might be a problem with flup's setup.py or
ez_setup.py, but manually running python setup.py bdist_egg outside of
easy_install works fine.

From ben+python at benfinney.id.au  Fri Jun 12 00:18:11 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 12 Jun 2009 08:18:11 +1000
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
	".postNNN"
References: <877hzjasvy.fsf@benfinney.id.au>
	<20090611133159.22176.785529549.divmod.quotient.4403@henry.divmod.com>
Message-ID: <87ab4e9zz0.fsf@benfinney.id.au>

Jean-Paul Calderone <exarkun at divmod.com> writes:

> The suggestion seems to be that projects using post releases should
> instead just add another component to their version.

Yes. That makes it immediately obvious that the ?post release number?
component sorts within version numbers like any other component.

> This would make the current version of Twisted 8.2.0.27002.

Note that the specification I propose allows any alphanumeric character
[0-9A-Za-z], so you could also make the version string ?8.2.0.r27002? if
you want the ?revision number? component to be visibly different while
still having an obvious sequencing semantic.

-- 
 \     ?Here is a test to see if your mission on earth is finished. If |
  `\                          you are alive, it isn't.? ?Francis Bacon |
_o__)                                                                  |
Ben Finney


From ben+python at benfinney.id.au  Fri Jun 12 00:24:25 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 12 Jun 2009 08:24:25 +1000
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
	".postNNN"
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>
	<20090611135255.EFA2F3A402D@sparrow.telecommunity.com>
	<79990c6b0906110716q11b3a280l3065107d4ef95e76@mail.gmail.com>
	<20090611155023.368A83A402D@sparrow.telecommunity.com>
	<b654cd2e0906110903p6ef3c5c2la560216b7246c9c7@mail.gmail.com>
Message-ID: <87y6ry8l46.fsf@benfinney.id.au>

Ian Bicking <ianb at colorstudy.com> writes:

> Agreed, I do this all the time. Pylons dev versions also regularly
> rely on other packages with a dev version, and people regularly use
> these non-released versions, with dependencies detected and installed
> via dependency_links.

I don't see how any of this argues for special-cased tokens in a version
string. Isn't this all possible anyway with a version comparison
algorithm with no special-cased tokens?

-- 
 \     ?It is not enough to have a good mind. The main thing is to use |
  `\                                         it well.? ?Rene Descartes |
_o__)                                                                  |
Ben Finney


From ben+python at benfinney.id.au  Fri Jun 12 00:22:48 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 12 Jun 2009 08:22:48 +1000
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
	".postNNN"
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>
	<20090611135255.EFA2F3A402D@sparrow.telecommunity.com>
Message-ID: <873aa69zrb.fsf@benfinney.id.au>

"P.J. Eby" <pje at telecommunity.com> writes:

> Dev tags are so that while you're doing development, your
> locally-installed versions can be distinguished from one another.

How is ?distinguished from one another? different from ?compare
sequentially previous or subsequent to one another??

What is wrong with using the version comparison semantics without
special-cased words for this purpose?

-- 
 \           ?Never express yourself more clearly than you are able to |
  `\                                               think.? ?Niels Bohr |
_o__)                                                                  |
Ben Finney


From ben+python at benfinney.id.au  Fri Jun 12 00:33:27 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 12 Jun 2009 08:33:27 +1000
Subject: [Distutils] recollections of Pycon distutils versioning
	discussion (part 2)
References: <6db0ea510906110042x4d31ab1dr35a7865315638daf@mail.gmail.com>
	<79990c6b0906110137h26e32765w55f164169da86154@mail.gmail.com>
	<2CE34B1E-CC24-4F5F-B673-E18B65F74058@zooko.com>
	<79990c6b0906110730y15505166y3f7c44cc56fee69b@mail.gmail.com>
	<BCF91A19-65C1-44A3-BE5B-B9BBF09E8D3C@zooko.com>
Message-ID: <87ski68kp4.fsf@benfinney.id.au>

Zooko Wilcox-O'Hearn <zooko at zooko.com> writes:

> So far so good.  Now, should the "rational version number spec"

I would like to avoid this term, since version strings do not represent
rational numbers (nor, indeed, should they be interpreted as any kind of
single number).

If we must have a name for it, I propose ?consistent version comparison
spec? as less confusing than the above term.

> *also* encourage those of us who use this technique to use the same
> spelling for ".snapshot" / "-r" / ".post"?

This is where I don't think the specification should express an opinion.
Keep it simple and declarative.

> This would not effect any version comparison, but it would be nice for
> us humans if everyone chose the same word when they mean the same
> thing.

It might be nice, but it's not necessary to the specification since
version comparison only needs to be consistent within versions of *the
same thing*, so for the sake of specification it doesn't matter if
different projects choose different tokens.

The specification should provide *examples*, and make those examples
sensible; but don't favour anything that's not meant to be normative.

-- 
 \     ?I cannot conceive that anybody will require multiplications at |
  `\   the rate of 40,000 or even 4,000 per hour ?? ?F. H. Wales, 1936 |
_o__)                                                                  |
Ben Finney


From ben+python at benfinney.id.au  Fri Jun 12 00:39:08 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 12 Jun 2009 08:39:08 +1000
Subject: [Distutils] post-release revisions
References: <6db0ea510906090058x76c8cadfj22e7d0dc619b56aa@mail.gmail.com>
	<829FF863-51DC-4214-B2A7-9FDEE82A08D5@zooko.com>
	<4A3114BA.1040009@trueblade.com>
	<79990c6b0906110748h7cdab88k723b96d8797ba4cf@mail.gmail.com>
	<20090611160116.9D0813A402D@sparrow.telecommunity.com>
Message-ID: <87ocsu8kfn.fsf@benfinney.id.au>

"P.J. Eby" <pje at telecommunity.com> writes:

> That depends on whether this PEP's version scheme is also going to be
> embedded in the code implied by PEP 376. If so, then nobody will have
> any reason to switch from using pkg_resources to the PEP 376 pkgutil
> code, because they won't be able to manage their test, dev, and
> collaboration environments that way.

What prevents them from doing so merely by choosing version strings that
conform to the specification?

> That's precisely it - there aren't any visible benefits to doing so,
> but there *are* visible problems.

I think a significant benefit will be that the version comparison
semantics will be implemented *in Python*, and I know there are very
many people who would like to drop a dependency on third-party tools.

Alternatively, if this version string scheme is to be implemented in PEP
365, that has the same effect: it will be ?part of Python?, which is a
huge benefit compared to an implementation outside Python.

> We need to have a way to translate existing version schemes to the new
> one, if we're to be able to have a reasonable upgrade path --

This certainly deserves consideration. However, I remind readers that
part of the problem we're trying to solve here is that version string
schemes are *already* non-standard and inconsistent; we would have to
choose only a sub-set of existing version schemes to be supported for
upgrade.

-- 
 \              ?In the long run, the utility of all non-Free software |
  `\      approaches zero. All non-Free software is a dead end.? ?Mark |
_o__)                                                    Pilgrim, 2006 |
Ben Finney


From ben+python at benfinney.id.au  Fri Jun 12 01:01:18 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 12 Jun 2009 09:01:18 +1000
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
	".postNNN"
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>
	<20090611135255.EFA2F3A402D@sparrow.telecommunity.com>
	<79990c6b0906110716q11b3a280l3065107d4ef95e76@mail.gmail.com>
	<20090611155023.368A83A402D@sparrow.telecommunity.com>
	<b654cd2e0906110903p6ef3c5c2la560216b7246c9c7@mail.gmail.com>
	<6db0ea510906111105y79c346e3jf60e282172de16ad@mail.gmail.com>
	<94bdd2610906111252m39e6c20bsee5df434f8d7823e@mail.gmail.com>
	<6db0ea510906111338n3b7b32a1w430f3a592e722388@mail.gmail.com>
	<20090611211327.8E0EF3A411E@sparrow.telecommunity.com>
	<6db0ea510906111444y5a4d91b6p7bf770fb436f443d@mail.gmail.com>
Message-ID: <87hbym8jep.fsf@benfinney.id.au>

Trent Mick <trentm at gmail.com> writes:

> Perhaps. I'm wondering if the separation of "version" (does not
> include the ".dev") and "build_number" helps clear up some of the
> cases.

If they are separate components (where each component from left to right
is entirely subordinate to the preceding component), then that
separation is implied. Version ?1.2.3? can have as many ?.devXXXX? as
desired, all of which will compare previous to version ?1.2.4?.

> If "1.2.3.dev456"-type version strings don't appear in packages
> released to PyPI, then the job of the downstream RPM/.deb packagers is
> easier (they then don't need to care about the spelling of the version
> with the build number).

If the version comparison semantics are such that simple ?compare
non-digits per ASCII, compare digit sequences as integers? works within
a component, I'm not aware of any distribution downstream that can't
just use them as-is. What specific problems can you see with that?

> Have a "RationalReleaseVersion()" that is just the non-dev part of the
> proposal.

Let's choose a name that doesn't add to the confusion between the
non-number ?version string? and the mathematical concept of ?rational
number?. Perhaps ?SimpleVersion? is a better name.

-- 
 \       "Success is going from one failure to the next without a loss |
  `\                             of enthusiasm."  -- Winston Churchill |
_o__)                                                                  |
Ben Finney


From ben+python at benfinney.id.au  Fri Jun 12 01:50:59 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 12 Jun 2009 09:50:59 +1000
Subject: [Distutils] recollections of Pycon distutils versioning
	discussion (part 2)
References: <6db0ea510906110042x4d31ab1dr35a7865315638daf@mail.gmail.com>
	<87bpovat2b.fsf@benfinney.id.au>
Message-ID: <87d49a8h3w.fsf@benfinney.id.au>

Ben Finney <ben+python at benfinney.id.au> writes:

> Note that the specification I propose allows any alphanumeric
> character [0-9A-Za-z], so you could also make the version string
> ?8.2.0.r27002? if you want the ?revision number? component to be
> visibly different while still having an obvious sequencing semantic.

Ben Finney <ben+python at benfinney.id.au> writes:

> My understanding is the above version strings should be compared by the
> following rules:
> 
>   * A version string is interpreted as a tuple of components. Each
>     component is a sequence of characters from the set [0-9A-Za-z]. Each
>     component is separated from others by a single full-stop (?.?)
>     character.
[?]
> 
>     * Contiguous sequences of digits are interpreted as integers and
>       compared numerically; other characters compare as per the ASCII
>       character set sequence.

I realise now that this has an unintended effect: that version strings
which have letters in differing case will compare ASCIIbetically, which
may be non-obvious:

    1.2.C1
    1.2.D1
    1.2.REV876
    1.2.a1
    1.2.b1
    1.2.rev543

I hereby simplify the above specification and its semantics, by
declaring upper-case letters outside the scope of a version string. A
component can have characters from the set [0-9a-z], removing the above
cases of non-obvious comparison.

    1.2.a1
    1.2.b1
    1.2.c1
    1.2.d1
    1.2.rev543
    1.2.rev876

-- 
 \      ?The process by which banks create money is so simple that the |
  `\     mind is repelled.? ?John Kenneth Galbraith, _Money: Whence It |
_o__)                                       Came, Where It Went_, 1975 |
Ben Finney


From eric at trueblade.com  Fri Jun 12 02:28:35 2009
From: eric at trueblade.com (Eric Smith)
Date: Thu, 11 Jun 2009 20:28:35 -0400
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
 ".postNNN"
In-Reply-To: <87hbym8jep.fsf@benfinney.id.au>
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>	<20090611135255.EFA2F3A402D@sparrow.telecommunity.com>	<79990c6b0906110716q11b3a280l3065107d4ef95e76@mail.gmail.com>	<20090611155023.368A83A402D@sparrow.telecommunity.com>	<b654cd2e0906110903p6ef3c5c2la560216b7246c9c7@mail.gmail.com>	<6db0ea510906111105y79c346e3jf60e282172de16ad@mail.gmail.com>	<94bdd2610906111252m39e6c20bsee5df434f8d7823e@mail.gmail.com>	<6db0ea510906111338n3b7b32a1w430f3a592e722388@mail.gmail.com>	<20090611211327.8E0EF3A411E@sparrow.telecommunity.com>	<6db0ea510906111444y5a4d91b6p7bf770fb436f443d@mail.gmail.com>
	<87hbym8jep.fsf@benfinney.id.au>
Message-ID: <4A31A133.9030100@trueblade.com>

Ben Finney wrote:
> If they are separate components (where each component from left to right
> is entirely subordinate to the preceding component), then that
> separation is implied. Version ?1.2.3? can have as many ?.devXXXX? as
> desired, all of which will compare previous to version ?1.2.4?.

This demonstrates the "count up from" versus "count up to" problem. If 
I'm getting ready to release 1.2.4, I think I'm working on 
1.2.4-dev20090611 (or similar), not that I'm working on 1.2.3.9999.3.

> If the version comparison semantics are such that simple ?compare
> non-digits per ASCII, compare digit sequences as integers? works within
> a component, I'm not aware of any distribution downstream that can't
> just use them as-is. What specific problems can you see with that?

I think the "count up to" a release version is the problem, and the only 
one I can think of. Before Twisted releases 8.2.2, do they really need 
to call all of the versions 8.2.1.99.1, 8.2.1.99.2, etc., to avoid 
confusion with a possible 8.2.1 release? Don't you think some 
information is lost by not conveying "this will become 8.2.2 when it is 
the release version"? I grant you that your comparison scheme with 
version components works for the computer, I just think it loses 
information that's important to humans.

And as others have said, I too use these development version numbers in 
internal build tools which automatically update dependencies, so I am 
comparing version numbers.

>> Have a "RationalReleaseVersion()" that is just the non-dev part of the
>> proposal.
> 
> Let's choose a name that doesn't add to the confusion between the
> non-number ?version string? and the mathematical concept of ?rational
> number?. Perhaps ?SimpleVersion? is a better name.

I agree on "Rational" being confusing. "SimpleVersion" sounds reasonable 
to me.

Eric.

From pje at telecommunity.com  Fri Jun 12 02:45:04 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 11 Jun 2009 20:45:04 -0400
Subject: [Distutils] easy_install self upgrade/version check b ug
In-Reply-To: <b77cc058255c5d3a518d29d02e4d1fcb@mail.cornsyrup.org>
References: <b77cc058255c5d3a518d29d02e4d1fcb@mail.cornsyrup.org>
Message-ID: <20090612004211.4AF563A402D@sparrow.telecommunity.com>

At 01:42 PM 6/11/2009 -0700, jason wrote:
>(Currently using setuptools 0.6c5 (/usr/lib/python2.4/site-packages))
>error: Setup script exited with 2
>$ easy_install -U setuptools
>...
>Processing dependencies for setuptools

You omitted the one part of the output that would tell us what the 
problem is; i.e. where the "easy_install -U setuptools" installed 
itself to.  (Apparently, it's not /usr/lib/python2.4/site-packages!)


From pje at telecommunity.com  Fri Jun 12 02:48:30 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 11 Jun 2009 20:48:30 -0400
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
 ".postNNN"
In-Reply-To: <873aa69zrb.fsf@benfinney.id.au>
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>
	<20090611135255.EFA2F3A402D@sparrow.telecommunity.com>
	<873aa69zrb.fsf@benfinney.id.au>
Message-ID: <20090612004538.9873A3A402D@sparrow.telecommunity.com>

At 08:22 AM 6/12/2009 +1000, Ben Finney wrote:
>"P.J. Eby" <pje at telecommunity.com> writes: > Dev tags are so that 
>while you're doing development, your > locally-installed versions 
>can be distinguished from one another. How is ???distinguished from 
>one another??? different from ???compare sequentially previous or 
>subsequent to one another????

First, more than one version can co-exist in .egg form, and there may 
be a need to depend on an exact version.  Second, as already shown in 
other examples, there is a need for the installed development version 
to be compared against a desired development version, and these 
requirements are expressed by version comparison strings.


>  What is wrong with using the version comparison semantics without 
> special-cased words for this purpose?

Please explain how you intend to designate "development" versions 
that compare less than the equivalent non-development versions in your scheme.


From pje at telecommunity.com  Fri Jun 12 02:54:02 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 11 Jun 2009 20:54:02 -0400
Subject: [Distutils] post-release revisions
In-Reply-To: <87ocsu8kfn.fsf@benfinney.id.au>
References: <6db0ea510906090058x76c8cadfj22e7d0dc619b56aa@mail.gmail.com>
	<829FF863-51DC-4214-B2A7-9FDEE82A08D5@zooko.com>
	<4A3114BA.1040009@trueblade.com>
	<79990c6b0906110748h7cdab88k723b96d8797ba4cf@mail.gmail.com>
	<20090611160116.9D0813A402D@sparrow.telecommunity.com>
	<87ocsu8kfn.fsf@benfinney.id.au>
Message-ID: <20090612005109.0E6E63A402D@sparrow.telecommunity.com>

At 08:39 AM 6/12/2009 +1000, Ben Finney wrote:
>However, I remind readers that part of the problem we're trying to 
>solve here is that version string schemes are *already* non-standard 
>and inconsistent; we would have to choose only a sub-set of existing 
>version schemes to be supported for upgrade.

Indeed, and such a subset has already been proposed.  Apparently, 
some people don't like "dev" and "post" being a part of it.

I can *almost* see a case for dropping "post" - I personally have not 
used it, and Trent has shown evidence that its usage is relatively 
rare.  And I could *almost* see tacking on extra zeros to signify a 
post tag, at the cost of readability, and the need for any tracked 
packages (where the Python version of the project is versioned as 
patchlevels of the foreign project) to maintain a consistent numbering scheme.

However, I do not see an equivalent case for dropping "dev", since 
it's an important part of development workflows, and I don't see any 
way to simulate it in an an otherwise-linear scheme.


From tseaver at palladion.com  Fri Jun 12 03:11:38 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 11 Jun 2009 21:11:38 -0400
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
	".postNNN"
In-Reply-To: <6db0ea510906111444y5a4d91b6p7bf770fb436f443d@mail.gmail.com>
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>	<20090611135255.EFA2F3A402D@sparrow.telecommunity.com>	<79990c6b0906110716q11b3a280l3065107d4ef95e76@mail.gmail.com>	<20090611155023.368A83A402D@sparrow.telecommunity.com>	<b654cd2e0906110903p6ef3c5c2la560216b7246c9c7@mail.gmail.com>	<6db0ea510906111105y79c346e3jf60e282172de16ad@mail.gmail.com>	<94bdd2610906111252m39e6c20bsee5df434f8d7823e@mail.gmail.com>	<6db0ea510906111338n3b7b32a1w430f3a592e722388@mail.gmail.com>	<20090611211327.8E0EF3A411E@sparrow.telecommunity.com>
	<6db0ea510906111444y5a4d91b6p7bf770fb436f443d@mail.gmail.com>
Message-ID: <h0sa0a$g7b$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Trent Mick wrote:
>>> - when I specify a dependency against a particular build_number of a
>>> package, I don't care if that build_number happened to be a released
>>> version or a dev version
>> However, to specify that dependency you're going to need a way to represent
>> the build number as part of a requirement string, at which point we're right
>> back where we started.
> 
> Perhaps. I'm wondering if the separation of "version" (does not
> include the ".dev") and "build_number" helps clear up some of the
> cases.
> 
> If "1.2.3.dev456"-type version strings don't appear in packages
> released to PyPI, then the job of the downstream RPM/.deb packagers is
> easier (they then don't need to care about the spelling of the version
> with the build number). Have a "RationalReleaseVersion()" that is just
> the non-dev part of the proposal.
> 
> Yes, as you say, the requirement/dependency fields (presumably they
> will be strings) need a way to spell the "build_number" part. However,
> the large set of setup.py authors that don't need to understand or use
> dependency strings don't need to see that.

Assuming that we add the requisite 'build_number' field to PKG_INFO,
could we allow spelling a dependency on a combined version + build
number using an "odd" spelling, such as: '1.2.3#4567' or '1.2.3 at 4567'?
This spelling would be *disallowed* for "released" packages, but could
still satisfy the folks who use such dependencies in internal-only
development mode.

Effectively, my proposal here is to move the whole "build_number" topic
out of the PEP 386 (which is about versioning), and into PEP 345
(whichadds dependencies on distributions to PKG_INFO).


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFKMatK+gerLs4ltQ4RAj4YAKDH4BU+KKnl4uRaegyPqeXE54Q4SgCgiHeG
tQDCm6McvuckIH8s38kDmSg=
=zdWb
-----END PGP SIGNATURE-----


From pje at telecommunity.com  Fri Jun 12 03:48:09 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 11 Jun 2009 21:48:09 -0400
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
 ".postNNN"
In-Reply-To: <h0sa0a$g7b$1@ger.gmane.org>
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>
	<20090611135255.EFA2F3A402D@sparrow.telecommunity.com>
	<79990c6b0906110716q11b3a280l3065107d4ef95e76@mail.gmail.com>
	<20090611155023.368A83A402D@sparrow.telecommunity.com>
	<b654cd2e0906110903p6ef3c5c2la560216b7246c9c7@mail.gmail.com>
	<6db0ea510906111105y79c346e3jf60e282172de16ad@mail.gmail.com>
	<94bdd2610906111252m39e6c20bsee5df434f8d7823e@mail.gmail.com>
	<6db0ea510906111338n3b7b32a1w430f3a592e722388@mail.gmail.com>
	<20090611211327.8E0EF3A411E@sparrow.telecommunity.com>
	<6db0ea510906111444y5a4d91b6p7bf770fb436f443d@mail.gmail.com>
	<h0sa0a$g7b$1@ger.gmane.org>
Message-ID: <20090612014516.089413A402D@sparrow.telecommunity.com>

At 09:11 PM 6/11/2009 -0400, Tres Seaver wrote:
>Assuming that we add the requisite 'build_number' field to PKG_INFO,
>could we allow spelling a dependency on a combined version + build
>number using an "odd" spelling, such as: '1.2.3#4567' or '1.2.3 at 4567'?

And if we make that special string "dev" or "post", then we get 
backwards compatibility and upgrade options for setuptools users, 
too.  Great idea.  ;-)


>This spelling would be *disallowed* for "released" packages, but could
>still satisfy the folks who use such dependencies in internal-only
>development mode.

I don't see what usefulness such a disallowal would have.  PyPI 
already accepts all sorts of stupid versions, including ones even 
setuptools can't make any sense of.  And for packages installed on 
the machine, the build information is still going to need to be in 
any generated filenames, such as that of the .egg-info directory or 
the egg file itself.


From eric at trueblade.com  Fri Jun 12 03:58:48 2009
From: eric at trueblade.com (Eric Smith)
Date: Thu, 11 Jun 2009 21:58:48 -0400
Subject: [Distutils] post-release revisions
In-Reply-To: <20090612005109.0E6E63A402D@sparrow.telecommunity.com>
References: <6db0ea510906090058x76c8cadfj22e7d0dc619b56aa@mail.gmail.com>	<829FF863-51DC-4214-B2A7-9FDEE82A08D5@zooko.com>	<4A3114BA.1040009@trueblade.com>	<79990c6b0906110748h7cdab88k723b96d8797ba4cf@mail.gmail.com>	<20090611160116.9D0813A402D@sparrow.telecommunity.com>	<87ocsu8kfn.fsf@benfinney.id.au>
	<20090612005109.0E6E63A402D@sparrow.telecommunity.com>
Message-ID: <4A31B658.7030000@trueblade.com>

P.J. Eby wrote:
> I can *almost* see a case for dropping "post" - I personally have not 
> used it, and Trent has shown evidence that its usage is relatively 
> rare.  And I could *almost* see tacking on extra zeros to signify a post 
> tag, at the cost of readability, and the need for any tracked packages 
> (where the Python version of the project is versioned as patchlevels of 
> the foreign project) to maintain a consistent numbering scheme.
> 
> However, I do not see an equivalent case for dropping "dev", since it's 
> an important part of development workflows, and I don't see any way to 
> simulate it in an an otherwise-linear scheme.

I agree with both of these: you can simulate "post" by adding another 
dot and a number (or string), but we still need "dev". I'm okay with 
dropping "post", leaving "dev" as the only special case.

Eric.

From david.lyon at preisshare.net  Fri Jun 12 05:30:35 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 11 Jun 2009 23:30:35 -0400
Subject: [Distutils]
 =?utf-8?q?easy=5Finstall_self_upgrade/version_check_b?=
 =?utf-8?q?ug?=
In-Reply-To: <b77cc058255c5d3a518d29d02e4d1fcb@mail.cornsyrup.org>
References: <b77cc058255c5d3a518d29d02e4d1fcb@mail.cornsyrup.org>
Message-ID: <2bc383fc9dcb58a5befd87d3b12f11b2@preisshare.net>

On Thu, 11 Jun 2009 13:42:23 -0700, jason <jason at mail.cornsyrup.org> wrote:
> The required version of setuptools (>=0.6c9) is not available, and
> can't be installed while this script is running. Please install
>  a more recent version first, using 'easy_install -U setuptools'.

Hi Jason,

I get this problem all the time under windows....

What I do is use the Python Package Manager to remove setuptools
(select setuptools 0.6c9 - click remove) and I also remove all of the
easyinstall scripts from the scripts directory manually.

After that, I proceed to install the package again. In the process
easyinstall gets installed again which reinstalls setuptools.

Regards

David

From trentm at gmail.com  Fri Jun 12 09:27:34 2009
From: trentm at gmail.com (Trent Mick)
Date: Fri, 12 Jun 2009 00:27:34 -0700
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
	".postNNN"
In-Reply-To: <20090612014516.089413A402D@sparrow.telecommunity.com>
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>
	<20090611155023.368A83A402D@sparrow.telecommunity.com>
	<b654cd2e0906110903p6ef3c5c2la560216b7246c9c7@mail.gmail.com>
	<6db0ea510906111105y79c346e3jf60e282172de16ad@mail.gmail.com>
	<94bdd2610906111252m39e6c20bsee5df434f8d7823e@mail.gmail.com>
	<6db0ea510906111338n3b7b32a1w430f3a592e722388@mail.gmail.com>
	<20090611211327.8E0EF3A411E@sparrow.telecommunity.com>
	<6db0ea510906111444y5a4d91b6p7bf770fb436f443d@mail.gmail.com>
	<h0sa0a$g7b$1@ger.gmane.org>
	<20090612014516.089413A402D@sparrow.telecommunity.com>
Message-ID: <6db0ea510906120027o199d23f7s9b263e33cc70f3cf@mail.gmail.com>

On Thu, Jun 11, 2009 at 6:48 PM, P.J. Eby<pje at telecommunity.com> wrote:
> I don't see what usefulness such a disallowal would have.

The sanity of other people browsing through and trying to grok the
versions of packages
released to PyPI.

>  ?PyPI already
> accepts all sorts of stupid versions, including ones even setuptools can't
> make any sense of.

That's not a feature. :)



Trent

-- 
Trent Mick
trentm at gmail.com

From floris.bruynooghe at gmail.com  Fri Jun 12 09:53:38 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Fri, 12 Jun 2009 08:53:38 +0100
Subject: [Distutils] version scheme: a case for dropping ".devNNN"
	and	".postNNN"
In-Reply-To: <4A316E4C.30408@nasa.gov>
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>
	<20090611135255.EFA2F3A402D@sparrow.telecommunity.com>
	<79990c6b0906110716q11b3a280l3065107d4ef95e76@mail.gmail.com>
	<20090611155023.368A83A402D@sparrow.telecommunity.com>
	<b654cd2e0906110903p6ef3c5c2la560216b7246c9c7@mail.gmail.com>
	<6db0ea510906111105y79c346e3jf60e282172de16ad@mail.gmail.com>
	<94bdd2610906111252m39e6c20bsee5df434f8d7823e@mail.gmail.com>
	<6db0ea510906111338n3b7b32a1w430f3a592e722388@mail.gmail.com>
	<4A316E4C.30408@nasa.gov>
Message-ID: <20090612075338.GA4213@laurie.devork>

On Thu, Jun 11, 2009 at 04:51:24PM -0400, Stephen Waterbury wrote:
> Trent Mick wrote:
>> I've been thinking from the p.o.v. of what releases get up on PyPI....
>> and I gather that those releases are the ones that lead to potential
>> packaging in RPM and .deb repositories.
>
> I think that is not necessarily true -- I seem to recall some discussion
> in the PyCon sessions about OS distro folks sometimes having to
> package dev versions ... hopefully there are some distro packagers
> here who can chime in on this ...

True, some packages have versions like 1.2.3+hg20090612-1 in which
case there didn't even need to be a development release.  Of course
1.2.3.dev456-1 would be fine too (the last "-1" would be debian's
version, ignore that).

Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From floris.bruynooghe at gmail.com  Fri Jun 12 10:02:14 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Fri, 12 Jun 2009 09:02:14 +0100
Subject: [Distutils] version scheme: a case for dropping ".devNNN"
	and	".postNNN"
In-Reply-To: <h0sa0a$g7b$1@ger.gmane.org>
References: <20090611135255.EFA2F3A402D@sparrow.telecommunity.com>
	<79990c6b0906110716q11b3a280l3065107d4ef95e76@mail.gmail.com>
	<20090611155023.368A83A402D@sparrow.telecommunity.com>
	<b654cd2e0906110903p6ef3c5c2la560216b7246c9c7@mail.gmail.com>
	<6db0ea510906111105y79c346e3jf60e282172de16ad@mail.gmail.com>
	<94bdd2610906111252m39e6c20bsee5df434f8d7823e@mail.gmail.com>
	<6db0ea510906111338n3b7b32a1w430f3a592e722388@mail.gmail.com>
	<20090611211327.8E0EF3A411E@sparrow.telecommunity.com>
	<6db0ea510906111444y5a4d91b6p7bf770fb436f443d@mail.gmail.com>
	<h0sa0a$g7b$1@ger.gmane.org>
Message-ID: <20090612080214.GB4213@laurie.devork>

On Thu, Jun 11, 2009 at 09:11:38PM -0400, Tres Seaver wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Trent Mick wrote:
> >>> - when I specify a dependency against a particular build_number of a
> >>> package, I don't care if that build_number happened to be a released
> >>> version or a dev version
> >> However, to specify that dependency you're going to need a way to represent
> >> the build number as part of a requirement string, at which point we're right
> >> back where we started.
> > 
> > Perhaps. I'm wondering if the separation of "version" (does not
> > include the ".dev") and "build_number" helps clear up some of the
> > cases.
> > 
> > If "1.2.3.dev456"-type version strings don't appear in packages
> > released to PyPI, then the job of the downstream RPM/.deb packagers is
> > easier (they then don't need to care about the spelling of the version
> > with the build number). Have a "RationalReleaseVersion()" that is just
> > the non-dev part of the proposal.
> > 
> > Yes, as you say, the requirement/dependency fields (presumably they
> > will be strings) need a way to spell the "build_number" part. However,
> > the large set of setup.py authors that don't need to understand or use
> > dependency strings don't need to see that.
> 
> Assuming that we add the requisite 'build_number' field to PKG_INFO,
> could we allow spelling a dependency on a combined version + build
> number using an "odd" spelling, such as: '1.2.3#4567' or '1.2.3 at 4567'?
> This spelling would be *disallowed* for "released" packages, but could
> still satisfy the folks who use such dependencies in internal-only
> development mode.

PyPI needs to allow any version anyway, so these things will end up
being used there and make the situation only worse.  The .devX and
.postX allow for both people counting up to and counting away from
releases and if combinations (.postX.devY) are disallowed I think it's
quite clear and straight forward.  Bear in mind no-one forces you to
use those suffixes.  If you never want to see them just don't use
them.  You can happily cope with just normal alpha, beta and release
candidate releases.

Regards
Floris


-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From h.goebel at goebel-consult.de  Fri Jun 12 10:36:35 2009
From: h.goebel at goebel-consult.de (Hartmut Goebel)
Date: Fri, 12 Jun 2009 10:36:35 +0200
Subject: [Distutils] Using setuptools as downloader for .msi/.exe
Message-ID: <4A321393.4030306@goebel-consult.de>

Hello,

how can I use setuptools for downloading (and installing) .msi and .exe
on Windows? I need this for extension modules like pygtk, lxml, psycopg2.

Or at least for finding the files within a html page, like easy_install
does.

Background:
The Tryton ERP system requires quite a lot of dependencies. The
installation is an imposition to the user when following the
installation instructions at
<http://code.google.com/p/tryton/wiki/InstallationWindows>.

So I want to write an installation script which bootstraps setuptools
(this part is already done), installs requires eggs (done) and downloads
.msi or .exe where eggs are not available. The later includes GTK+,
Postgresql and some extension modules like pygtk.

A normal Windows installer packages is not what I need here. The script
is indented for a) developers who want to set up their own development
environment and b) users who need to write extensions for their
installation.

Since the filenames may vary by version and python-version, (e.g.
pygtk-2.12.1-1.win32-py2.5.exe) I do not want to hard-code these. Esp.
since this would require changing the script quite often. easy_install
already has a feature for finding the best-matching version

-- 
Sch?nen Gru? - Regards
Hartmut Goebel
Dipl.-Informatiker (univ.), CISSP, CSSLP

Goebel Consult
Spezialist f?r IT-Sicherheit in komplexen Umgebungen
http://www.goebel-consult.de


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 4732 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090612/c292b699/attachment.bin>

From a.cavallo at cavallinux.eu  Fri Jun 12 10:49:07 2009
From: a.cavallo at cavallinux.eu (A. Cavallo)
Date: Fri, 12 Jun 2009 09:49:07 +0100
Subject: [Distutils] Python svn builds
Message-ID: <200906120949.08105.a.cavallo@cavallinux.eu>

Python svn builds (http://pyvm.sourceforge.net) is a project to provide 
python binary builds ready to use.

The main features for these builds are:

+ install under /opt/opt-python-svnXXXXX-2.7a0 (XXXXX is the svn revision)
+ multiple interpreters can be installed side by side 
+ installed in parallel to the system python (administrator friendly)
+ they're build for several different platforms (rpm based)
+ the build host is provided by the OpenSUSE buildserver    
+ the builds can be deployed using yum/yast/urpmi (OpenSUSE buildserver)
+ each build runs a small smoke test and it is unattented
+ the test results are reported


The aim is:

+ implementing a continuous integration practice 
+ to provide python developer with reliable and repeatable builds
+ modules developer can test their modules against the latest python

News for this version:

Few modules (imaging, numpy, pexpect and setuptools) have been added and 
they're build for each interpreter.
These are still in testing but the aim here is to have a full complete build 
chain with a test suite (next step is providing a test driven development).

Download and documentation from:

http://pyvm.sourceforge.net




Regards,
Antonio

References:
OpenSUSE buildserver: https://build.opensuse.org


From jason at mail.cornsyrup.org  Fri Jun 12 11:34:55 2009
From: jason at mail.cornsyrup.org (jason)
Date: Fri, 12 Jun 2009 02:34:55 -0700
Subject: [Distutils]
 =?utf-8?q?easy=5Finstall_self_upgrade/version_check_b?=
 =?utf-8?q?_ug?=
In-Reply-To: <20090612004211.4AF563A402D@sparrow.telecommunity.com>
References: <b77cc058255c5d3a518d29d02e4d1fcb@mail.cornsyrup.org>
	<20090612004211.4AF563A402D@sparrow.telecommunity.com>
Message-ID: <f0a97c011d920a1e076a893c9d686a9d@mail.cornsyrup.org>


I omitted it from the output because I showed my ~/.pydistutils.cfg
install_lib and PYTHONPATH pointing to ~/python which is where it correctly
installed setuptools 0.6c9. easy_install went to ~/bin, which I also
specified in ~/.pydistutils.cfg.

On Thu, 11 Jun 2009 20:45:04 -0400, "P.J. Eby" <pje at telecommunity.com>
wrote:
> At 01:42 PM 6/11/2009 -0700, jason wrote:
>>(Currently using setuptools 0.6c5 (/usr/lib/python2.4/site-packages))
>>error: Setup script exited with 2
>>$ easy_install -U setuptools
>>...
>>Processing dependencies for setuptools
> 
> You omitted the one part of the output that would tell us what the
> problem is; i.e. where the "easy_install -U setuptools" installed
> itself to.  (Apparently, it's not /usr/lib/python2.4/site-packages!)

From p.f.moore at gmail.com  Fri Jun 12 11:49:10 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 12 Jun 2009 10:49:10 +0100
Subject: [Distutils] recollections of Pycon distutils versioning
	discussion (part 2)
In-Reply-To: <87d49a8h3w.fsf@benfinney.id.au>
References: <6db0ea510906110042x4d31ab1dr35a7865315638daf@mail.gmail.com>
	<87bpovat2b.fsf@benfinney.id.au> <87d49a8h3w.fsf@benfinney.id.au>
Message-ID: <79990c6b0906120249s3c615473oee500e5659753f7a@mail.gmail.com>

2009/6/12 Ben Finney <ben+python at benfinney.id.au>:
> I realise now that this has an unintended effect: that version strings
> which have letters in differing case will compare ASCIIbetically, which
> may be non-obvious:
>
> ? ?1.2.C1
> ? ?1.2.D1
> ? ?1.2.REV876
> ? ?1.2.a1
> ? ?1.2.b1
> ? ?1.2.rev543
>
> I hereby simplify the above specification and its semantics, by
> declaring upper-case letters outside the scope of a version string. A
> component can have characters from the set [0-9a-z], removing the above
> cases of non-obvious comparison.
>
> ? ?1.2.a1
> ? ?1.2.b1
> ? ?1.2.c1
> ? ?1.2.d1
> ? ?1.2.rev543
> ? ?1.2.rev876

One other aspect of standard practice that I just realised your rules
don't cover is where version strings differ in length. The normal
lexicographic "shortest is earliest" rule doesn't work properly:

1.2a1 vs 1.2 (I hope everyone agrees that 1.2a1 is earlier)

Even adding a dot, 1.2.a1 vs 1.2 compares wrongly (and gets worse when
you add in 1.2.1...)

Here's an alternative suggestion:

* Versions are treated as dot-separated tuples
* Comparison is component-by-component, exactly as Python tuples compare
* Components must have the form [a-z]*[0-9]+([a-z][0-9]+)? (ie,
optional leading alphas, an integer, and an optional "letter-integer"
suffix)
* Call the 3 parts "prefix" ([a-z]*), "number" ([0-9]+), "suffix" ([a-z][0-9]+)
* Components compare as follows:
  - Components with differing prefixes are incomparable[1]. Otherwise,
ignore the prefix.
  - Within this, sort by the number part (as a number, not as text)
  - Within this, components with a suffix sort BEFORE those without,
in the obvious letter-then-number order.

That's a little messy, but I think it follows people's intuition,
allows for most of the variations people want, and most importantly
(to my mind) isolates the complexity to how *components* sort against
each other (the high-level rule is "like tuples", which is simple).

[1] Note that I see the "prefix" as cosmetic. I would expect real
projects to use a fixed prefix on a component-by-component basis -
1.2.r34567 or 1.2.dev5 or whatever, but never a mix of 1.2.3,
1.2.r1234 and 1.2.dev5. Hence, I have said that mixed prefixes are
incomparable. If this causes an outcry, the following rule could be
used instead:

  - Components with a prefix sort before components without, in
alphabetic order of prefix

but in my view it adds unnecessary complexity (and hence I'd like to
see real-world, justified use cases).

Hmm, this doesn't allow for a component which is a SHA ID (something
like a Mercurial revision ID). Given that these aren't ordered, I
think that's OK as they don't make usable version numbers in any case.

Paul.

From pje at telecommunity.com  Fri Jun 12 19:49:24 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 12 Jun 2009 13:49:24 -0400
Subject: [Distutils] Using setuptools as downloader for .msi/.exe
In-Reply-To: <4A321393.4030306@goebel-consult.de>
References: <4A321393.4030306@goebel-consult.de>
Message-ID: <20090612174631.C695C3A410E@sparrow.telecommunity.com>

At 10:36 AM 6/12/2009 +0200, Hartmut Goebel wrote:
>Hello,
>
>how can I use setuptools for downloading (and installing) .msi and .exe
>on Windows? I need this for extension modules like pygtk, lxml, psycopg2.
>
>Or at least for finding the files within a html page, like easy_install
>does.

See the setuptools.package_index module.  You probably want to 
subclass PackageIndex, and use your own versions of the 
distros_for_location() or interpret_distro_name() functions.


From pje at telecommunity.com  Fri Jun 12 19:51:54 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 12 Jun 2009 13:51:54 -0400
Subject: [Distutils] easy_install self upgrade/version check b ug
In-Reply-To: <f0a97c011d920a1e076a893c9d686a9d@mail.cornsyrup.org>
References: <b77cc058255c5d3a518d29d02e4d1fcb@mail.cornsyrup.org>
	<20090612004211.4AF563A402D@sparrow.telecommunity.com>
	<f0a97c011d920a1e076a893c9d686a9d@mail.cornsyrup.org>
Message-ID: <20090612174902.071A53A410E@sparrow.telecommunity.com>

At 02:34 AM 6/12/2009 -0700, jason wrote:

>I omitted it from the output because I showed my ~/.pydistutils.cfg
>install_lib and PYTHONPATH pointing to ~/python which is where it correctly
>installed setuptools 0.6c9. easy_install went to ~/bin, which I also
>specified in ~/.pydistutils.cfg.

It doesn't matter why you omitted it.  The point is, the place where 
the old version is installed (the one that's actually getting used) 
is clearly in site-packages:

> > At 01:42 PM 6/11/2009 -0700, jason wrote:
> >>(Currently using setuptools 0.6c5 (/usr/lib/python2.4/site-packages))


From kevin at bud.ca  Fri Jun 12 22:08:37 2009
From: kevin at bud.ca (Kevin Teague)
Date: Fri, 12 Jun 2009 13:08:37 -0700 (PDT)
Subject: [Distutils] PEP 376 APIs reworked (was: Re: PEP 376 comments)
In-Reply-To: <94bdd2610906110559n7d5a9c8ak5b75a0ebdbe21540@mail.gmail.com>
References: <94bdd2610906110558n5b3cf17ciec43a5987b4b75de@mail.gmail.com> 
	<94bdd2610906110559n7d5a9c8ak5b75a0ebdbe21540@mail.gmail.com>
Message-ID: <d367ce64-b565-4112-98aa-747dfe0bd565@v35g2000pro.googlegroups.com>

The code is much more readable with the Distribution root name in the
classes now I think.

I've made a small tweak the code and changed the _DistributionMetadata
class to a _parse_distribution_metadata function instead. I think the
code is a bit simpler this way, since then the
DistributionDirectory.metadata attribute is just a straight
distutils.dist.DistributionMetadata class.

http://bitbucket.org/wheat/pep376-minor-refactoring/changeset/c71fa8f3bca5/

(and I hope I've got the bit-bucketery going on correctly here ... I'm
a bitbucket noob)

(oh, and that changeset also contains a t

From kevin at bud.ca  Fri Jun 12 23:36:12 2009
From: kevin at bud.ca (Kevin Teague)
Date: Fri, 12 Jun 2009 14:36:12 -0700 (PDT)
Subject: [Distutils] PEP 376 APIs reworked (was: Re: PEP 376 comments)
In-Reply-To: <d367ce64-b565-4112-98aa-747dfe0bd565@v35g2000pro.googlegroups.com>
References: <94bdd2610906110558n5b3cf17ciec43a5987b4b75de@mail.gmail.com> 
	<94bdd2610906110559n7d5a9c8ak5b75a0ebdbe21540@mail.gmail.com> 
	<d367ce64-b565-4112-98aa-747dfe0bd565@v35g2000pro.googlegroups.com>
Message-ID: <36043de9-ab49-44bc-94a9-52b108c1b359@y33g2000prg.googlegroups.com>

(oops, my message got truncated)

>
> (oh, and that changeset also contains a t

... tiny bug fix to line 393 where the variable 'project_name' should
be named just 'name'.)

From ziade.tarek at gmail.com  Sat Jun 13 10:08:33 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 13 Jun 2009 10:08:33 +0200
Subject: [Distutils] PEP 376 APIs reworked (was: Re: PEP 376 comments)
In-Reply-To: <d367ce64-b565-4112-98aa-747dfe0bd565@v35g2000pro.googlegroups.com>
References: <94bdd2610906110558n5b3cf17ciec43a5987b4b75de@mail.gmail.com>
	<94bdd2610906110559n7d5a9c8ak5b75a0ebdbe21540@mail.gmail.com>
	<d367ce64-b565-4112-98aa-747dfe0bd565@v35g2000pro.googlegroups.com>
Message-ID: <94bdd2610906130108v5ca14468ye1e73984be189bda@mail.gmail.com>

On Fri, Jun 12, 2009 at 10:08 PM, Kevin Teague<kevin at bud.ca> wrote:
> The code is much more readable with the Distribution root name in the
> classes now I think.
>
> I've made a small tweak the code and changed the _DistributionMetadata
> class to a _parse_distribution_metadata function instead. I think the
> code is a bit simpler this way, since then the
> DistributionDirectory.metadata attribute is just a straight
> distutils.dist.DistributionMetadata class.

Well, when PEP 376 will be applied in Distutils and pkgutil, I would
rather have a DistributionMetadata class
that handles writing *and* reading of PKG-INFO files, so it feels more
natural to me to add the reading
feature within the class rather than in a function wrapper.

Plus, it makes it possible for third-party tools to instanciate
DistributionMetadata to work with this file,
which would be impossible if the code we add is a private function.

Also, Phillip proposed to move this class to pkgutil and remove it
from Distutils, which makes sense,
so maybe we should take the whole class and put it in pkgutil, maybe
in its own "metadata" module,
e.g. make pkgutil become a package so pkgutil.py doesn't grow too much


> (oh, and that changeset also contains a t

A cool, that was a bad renaming, I'll apply that change thx

I am adding you as a developer in bitbucket to simplify things

Thanks
Tarek



> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | http://ziade.org

From ben+python at benfinney.id.au  Sun Jun 14 01:46:17 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 14 Jun 2009 09:46:17 +1000
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
	".postNNN"
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>
	<20090611135255.EFA2F3A402D@sparrow.telecommunity.com>
	<79990c6b0906110716q11b3a280l3065107d4ef95e76@mail.gmail.com>
	<20090611155023.368A83A402D@sparrow.telecommunity.com>
	<b654cd2e0906110903p6ef3c5c2la560216b7246c9c7@mail.gmail.com>
	<6db0ea510906111105y79c346e3jf60e282172de16ad@mail.gmail.com>
	<94bdd2610906111252m39e6c20bsee5df434f8d7823e@mail.gmail.com>
	<6db0ea510906111338n3b7b32a1w430f3a592e722388@mail.gmail.com>
	<20090611211327.8E0EF3A411E@sparrow.telecommunity.com>
	<6db0ea510906111444y5a4d91b6p7bf770fb436f443d@mail.gmail.com>
	<87hbym8jep.fsf@benfinney.id.au> <4A31A133.9030100@trueblade.com>
Message-ID: <87vdmz66k6.fsf@benfinney.id.au>

Eric Smith <eric at trueblade.com> writes:

> Ben Finney wrote:
> > If the version comparison semantics are such that simple ?compare
> > non-digits per ASCII, compare digit sequences as integers? works
> > within a component, I'm not aware of any distribution downstream
> > that can't just use them as-is. What specific problems can you see
> > with that?
> 
> I think the "count up to" a release version is the problem, and the
> only one I can think of. Before Twisted releases 8.2.2, do they really
> need to call all of the versions 8.2.1.99.1, 8.2.1.99.2, etc., to
> avoid confusion with a possible 8.2.1 release? Don't you think some
> information is lost by not conveying "this will become 8.2.2 when it
> is the release version"?

Why is it necessary for a version string to convey a prediction about
what *future* version strings will mean? That seems to me to be an
overloading that costs a significant amount of complexity, and is
unnecessary since a different version string can be chosen that meets
the intentional sequence position anyway.


"P.J. Eby" <pje at telecommunity.com> writes:

> Please explain how you intend to designate "development" versions that
> compare less than the equivalent non-development versions in your
> scheme.

    >>> (V("0.9")
    ... < V("0.9.0")
    ... < V("0.9.1")
    ... < V("0.9.a")
    ... < V("0.9.a2")
    ... < V("0.9.post123")
    ... < V("0.9b")
    ... < V("0.10")
    ... < V("0.999.0.dev-r925")
    ... < V("0.999.0.dev-r1357")
    ... < V("0.999.0.dev-r2468")
    ... < V("0.999.0.dev-r3579")
    ... < V("0.999.alpha1")
    ... < V("0.999.alpha2")
    ... < V("0.999.beta1")
    ... < V("0.999.rc1")
    ... < V("0.999.rc2")
    ... < V("1.0"))
    ...
    True

"P.J. Eby" <pje at telecommunity.com> writes:

> I can *almost* see a case for dropping "post" [?]
> 
> However, I do not see an equivalent case for dropping "dev", since
> it's an important part of development workflows, and I don't see any
> way to simulate it in an an otherwise-linear scheme.

I don't understand. You see that there's a scheme for generating version
strings that compare in a linear sequence, yet you can't see how to
generate a version string that will compare at a specific point in that
linear sequence?

For my part, I don't see what it is about these workflows that requires
a special-cased token in the version string. Why can't the workflow
simply use version strings that compare linearly as specified?


Paul Moore <p.f.moore at gmail.com> writes:

> One other aspect of standard practice that I just realised your rules
> don't cover is where version strings differ in length. The normal
> lexicographic "shortest is earliest" rule doesn't work properly:
> 
> 1.2a1 vs 1.2 (I hope everyone agrees that 1.2a1 is earlier)

No, I don't agree. Those each represent two components, where the first
is identical and the second is different; and when comparing the two
different components, I would expect ?2? to be earlier than ?2a1?.

-- 
 \         ?Please leave your values at the front desk.? ?hotel, Paris |
  `\                                                                   |
_o__)                                                                  |
Ben Finney


From ben+python at benfinney.id.au  Sun Jun 14 01:55:18 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 14 Jun 2009 09:55:18 +1000
Subject: [Distutils] recollections of Pycon distutils versioning
	discussion (part 2)
References: <6db0ea510906110042x4d31ab1dr35a7865315638daf@mail.gmail.com>
	<87bpovat2b.fsf@benfinney.id.au> <87d49a8h3w.fsf@benfinney.id.au>
	<79990c6b0906120249s3c615473oee500e5659753f7a@mail.gmail.com>
Message-ID: <87ljnv6655.fsf@benfinney.id.au>

Paul Moore <p.f.moore at gmail.com> writes:

> Here's an alternative suggestion:
> 
> * Versions are treated as dot-separated tuples
> * Comparison is component-by-component, exactly as Python tuples
> compare

Agreed so far (unsurprisingly, because so far it matches the algorithm I
outlined).

> * Components must have the form [a-z]*[0-9]+([a-z][0-9]+)? (ie,
> optional leading alphas, an integer, and an optional "letter-integer"
> suffix)
> * Call the 3 parts "prefix" ([a-z]*), "number" ([0-9]+), "suffix" ([a-z][0-9]+)
> * Components compare as follows:
>   - Components with differing prefixes are incomparable[1]. Otherwise,
> ignore the prefix.
>   - Within this, sort by the number part (as a number, not as text)
>   - Within this, components with a suffix sort BEFORE those without,
> in the obvious letter-then-number order.
> 
> That's a little messy

More than a little. That's not something I'd expect people to keep in
their head without needing to look at the specification frequently; or,
worse, make a guess and often get it wrong.

> but I think it follows people's intuition,

I think it's far from obvious that this represents an intuitive
comparison scheme. It's yet another set of special cases for certain
tokens, as far as I can see; which leads us back to the point that as
soon as we get into those, there's far less consensus about how they
should work.

> allows for most of the variations people want, and most importantly
> (to my mind) isolates the complexity to how *components* sort against
> each other (the high-level rule is "like tuples", which is simple).

Yes, I've no disagreement about version strings being sorted like tuples
at the component level. I don't see how you claim that as a
distinguishing characteristic of this suggestion.

> [1] Note that I see the "prefix" as cosmetic. I would expect real
> projects to use a fixed prefix on a component-by-component basis -
> 1.2.r34567 or 1.2.dev5 or whatever, but never a mix of 1.2.3,
> 1.2.r1234 and 1.2.dev5.

Why not? We've already had people talking about a mix of ?a123?,
?post123?, ?b123?, ?r123?, ?dev123?, etc. I think any version comparison
scheme needs to allow a definite statement to be made about the
sequencing of *any* possible version strings, with only the answers
{equal, less-than, greater-than} possible.

> Hence, I have said that mixed prefixes are incomparable. If this
> causes an outcry,

The cry is a simple ?What does that mean I should do with two version
strings that are incomparable??.

> the following rule could be used instead:
> 
>   - Components with a prefix sort before components without, in
> alphabetic order of prefix
> 
> but in my view it adds unnecessary complexity (and hence I'd like to
> see real-world, justified use cases).

Actually, that seems *simpler*. It's such a simplification, in fact,
that it makes this rule redundant: it's covered already by the existing
rules (AFAICT).


You've essentially got components within components, but some components
sort differently from others by non-obvious rules, and worst of all some
components ?are incomparable?; what does *that* mean when I need to
compare them for sequence?

I think ?can keep the whole specification in one's head easily? is an
important criterion for any version comparison scheme that we promote
for the standard library. People should be able to learn it once, then
be able to look at any two version strings in the future and quickly
know what sequence Python will put them in, without going back to the
specification again for special cases and differing comparison rules.

-- 
 \             ?I have never imputed to Nature a purpose or a goal, or |
  `\    anything that could be understood as anthropomorphic.? ?Albert |
_o__)                                    Einstein, unsent letter, 1955 |
Ben Finney


From exarkun at divmod.com  Sun Jun 14 07:15:54 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Sun, 14 Jun 2009 01:15:54 -0400
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
 ".postNNN"
In-Reply-To: <87vdmz66k6.fsf@benfinney.id.au>
Message-ID: <20090614051554.22176.1668639488.divmod.quotient.5254@henry.divmod.com>

On Sun, 14 Jun 2009 09:46:17 +1000, Ben Finney <ben+python at benfinney.id.au> wrote:
>Eric Smith <eric at trueblade.com> writes:
>
>> Ben Finney wrote:
>> > If the version comparison semantics are such that simple ?compare
>> > non-digits per ASCII, compare digit sequences as integers? works
>> > within a component, I'm not aware of any distribution downstream
>> > that can't just use them as-is. What specific problems can you see
>> > with that?
>>
>> I think the "count up to" a release version is the problem, and the
>> only one I can think of. Before Twisted releases 8.2.2, do they really
>> need to call all of the versions 8.2.1.99.1, 8.2.1.99.2, etc., to
>> avoid confusion with a possible 8.2.1 release? Don't you think some
>> information is lost by not conveying "this will become 8.2.2 when it
>> is the release version"?
>
>Why is it necessary for a version string to convey a prediction about
>what *future* version strings will mean? That seems to me to be an
>overloading that costs a significant amount of complexity, and is
>unnecessary since a different version string can be chosen that meets
>the intentional sequence position anyway.
>

I think there was some confusion here.  Twisted doesn't use the "count up
to" a release scheme (which is the .devNNN scheme).  It uses the "count
away from" a release scheme (ie, the equivalent of the .postNNN scheme,
if I understand correctly).

The current "in development version" of Twisted is 8.2.0r27007.  The last
release was Twisted 8.2.0.  It is expected, but not guaranteed, that the
next release of Twisted will be 9.0.0.

Projects that use the .devNNN scheme, though, rather than something like
the .postNNN scheme, do have to deal with this, though.  It seems to me
that this is pretty common, as release candidates will necessarily fall
into this category (however, it is far past my bed time, so if I have
forgotten about some corner of things, apologies for the noise).

> [snip]
>
>Paul Moore <p.f.moore at gmail.com> writes:
>
>> One other aspect of standard practice that I just realised your rules
>> don't cover is where version strings differ in length. The normal
>> lexicographic "shortest is earliest" rule doesn't work properly:
>>
>> 1.2a1 vs 1.2 (I hope everyone agrees that 1.2a1 is earlier)
>
>No, I don't agree. Those each represent two components, where the first
>is identical and the second is different; and when comparing the two
>different components, I would expect ?2? to be earlier than ?2a1?.
>

This seems to be the crux of the use case.  1.2a1 obviously should come
before 1.2. :)  Forcing the first alpha release of 1.2 to be 1.1.999.a1
or something like that would be very unfortunate, even though from some
perspectives it's simpler than the reverse.

Jean-Paul

From p.f.moore at gmail.com  Sun Jun 14 18:44:40 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 14 Jun 2009 17:44:40 +0100
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
	".postNNN"
In-Reply-To: <87vdmz66k6.fsf@benfinney.id.au>
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>
	<b654cd2e0906110903p6ef3c5c2la560216b7246c9c7@mail.gmail.com>
	<6db0ea510906111105y79c346e3jf60e282172de16ad@mail.gmail.com>
	<94bdd2610906111252m39e6c20bsee5df434f8d7823e@mail.gmail.com>
	<6db0ea510906111338n3b7b32a1w430f3a592e722388@mail.gmail.com>
	<20090611211327.8E0EF3A411E@sparrow.telecommunity.com>
	<6db0ea510906111444y5a4d91b6p7bf770fb436f443d@mail.gmail.com>
	<87hbym8jep.fsf@benfinney.id.au> <4A31A133.9030100@trueblade.com>
	<87vdmz66k6.fsf@benfinney.id.au>
Message-ID: <79990c6b0906140944y4a854c7aif071a05518b5259c@mail.gmail.com>

2009/6/14 Ben Finney <ben+python at benfinney.id.au>:
> Paul Moore <p.f.moore at gmail.com> writes:
>
>> One other aspect of standard practice that I just realised your rules
>> don't cover is where version strings differ in length. The normal
>> lexicographic "shortest is earliest" rule doesn't work properly:
>>
>> 1.2a1 vs 1.2 (I hope everyone agrees that 1.2a1 is earlier)
>
> No, I don't agree. Those each represent two components, where the first
> is identical and the second is different; and when comparing the two
> different components, I would expect ?2? to be earlier than ?2a1?.

I don't think it's possible to get away with 1.2a1 sorting after 1.2.

At the bare minimum, the Python core version numbering needs to be
supported IMHO. (Not because it's going to be used frequently in this
context, but because it's the base model that all Python extension
authors will be familiar with, and so most likely to be what they base
any further extensions on).

Paul.

From tseaver at palladion.com  Mon Jun 15 02:20:44 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Sun, 14 Jun 2009 20:20:44 -0400
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
	".postNNN"
In-Reply-To: <87vdmz66k6.fsf@benfinney.id.au>
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>	<20090611135255.EFA2F3A402D@sparrow.telecommunity.com>	<79990c6b0906110716q11b3a280l3065107d4ef95e76@mail.gmail.com>	<20090611155023.368A83A402D@sparrow.telecommunity.com>	<b654cd2e0906110903p6ef3c5c2la560216b7246c9c7@mail.gmail.com>	<6db0ea510906111105y79c346e3jf60e282172de16ad@mail.gmail.com>	<94bdd2610906111252m39e6c20bsee5df434f8d7823e@mail.gmail.com>	<6db0ea510906111338n3b7b32a1w430f3a592e722388@mail.gmail.com>	<20090611211327.8E0EF3A411E@sparrow.telecommunity.com>	<6db0ea510906111444y5a4d91b6p7bf770fb436f443d@mail.gmail.com>	<87hbym8jep.fsf@benfinney.id.au>
	<4A31A133.9030100@trueblade.com> <87vdmz66k6.fsf@benfinney.id.au>
Message-ID: <h14450$bng$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ben Finney wrote:
> Eric Smith <eric at trueblade.com> writes:
> 
>> Ben Finney wrote:
>>> If the version comparison semantics are such that simple ?compare
>>> non-digits per ASCII, compare digit sequences as integers? works
>>> within a component, I'm not aware of any distribution downstream
>>> that can't just use them as-is. What specific problems can you see
>>> with that?
>> I think the "count up to" a release version is the problem, and the
>> only one I can think of. Before Twisted releases 8.2.2, do they really
>> need to call all of the versions 8.2.1.99.1, 8.2.1.99.2, etc., to
>> avoid confusion with a possible 8.2.1 release? Don't you think some
>> information is lost by not conveying "this will become 8.2.2 when it
>> is the release version"?
> 
> Why is it necessary for a version string to convey a prediction about
> what *future* version strings will mean? That seems to me to be an
> overloading that costs a significant amount of complexity, and is
> unnecessary since a different version string can be chosen that meets
> the intentional sequence position anyway.
> 
> 
> "P.J. Eby" <pje at telecommunity.com> writes:
> 
>> Please explain how you intend to designate "development" versions that
>> compare less than the equivalent non-development versions in your
>> scheme.
> 
>     >>> (V("0.9")
>     ... < V("0.9.0")
>     ... < V("0.9.1")
>     ... < V("0.9.a")
>     ... < V("0.9.a2")
>     ... < V("0.9.post123")
>     ... < V("0.9b")
>     ... < V("0.10")
>     ... < V("0.999.0.dev-r925")
>     ... < V("0.999.0.dev-r1357")
>     ... < V("0.999.0.dev-r2468")
>     ... < V("0.999.0.dev-r3579")
>     ... < V("0.999.alpha1")
>     ... < V("0.999.alpha2")
>     ... < V("0.999.beta1")
>     ... < V("0.999.rc1")
>     ... < V("0.999.rc2")
>     ... < V("1.0"))
>     ...
>     True
> 
> "P.J. Eby" <pje at telecommunity.com> writes:
> 
>> I can *almost* see a case for dropping "post" [?]
>>
>> However, I do not see an equivalent case for dropping "dev", since
>> it's an important part of development workflows, and I don't see any
>> way to simulate it in an an otherwise-linear scheme.
> 
> I don't understand. You see that there's a scheme for generating version
> strings that compare in a linear sequence, yet you can't see how to
> generate a version string that will compare at a specific point in that
> linear sequence?
> 
> For my part, I don't see what it is about these workflows that requires
> a special-cased token in the version string. Why can't the workflow
> simply use version strings that compare linearly as specified?
> 
> 
> Paul Moore <p.f.moore at gmail.com> writes:
> 
>> One other aspect of standard practice that I just realised your rules
>> don't cover is where version strings differ in length. The normal
>> lexicographic "shortest is earliest" rule doesn't work properly:
>>
>> 1.2a1 vs 1.2 (I hope everyone agrees that 1.2a1 is earlier)
> 
> No, I don't agree. Those each represent two components, where the first
> is identical and the second is different; and when comparing the two
> different components, I would expect ?2? to be earlier than ?2a1?.

- -1:  "practicality beats purity" here:  there is *no* case in which an
alpha version should *ever* sort after the final release with the
corresponding number.  If we specify the "simple pure" scheme you
propose, nobody will use it, period.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFKNZPc+gerLs4ltQ4RAlYOAJ4mtGSe0Iadd6fyiilVO2OVgjwpjwCgth2n
WXOpSORFnStyLWmXPTOP41w=
=hUtJ
-----END PGP SIGNATURE-----


From ben+python at benfinney.id.au  Mon Jun 15 12:10:05 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 15 Jun 2009 20:10:05 +1000
Subject: [Distutils] A tale of two conventions: Incompatibility of version
	comparison semantics (was: version scheme: a case for
	dropping ".devNNN" and ".postNNN")
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>
	<20090611135255.EFA2F3A402D@sparrow.telecommunity.com>
	<79990c6b0906110716q11b3a280l3065107d4ef95e76@mail.gmail.com>
	<20090611155023.368A83A402D@sparrow.telecommunity.com>
	<b654cd2e0906110903p6ef3c5c2la560216b7246c9c7@mail.gmail.com>
	<6db0ea510906111105y79c346e3jf60e282172de16ad@mail.gmail.com>
	<94bdd2610906111252m39e6c20bsee5df434f8d7823e@mail.gmail.com>
	<6db0ea510906111338n3b7b32a1w430f3a592e722388@mail.gmail.com>
	<20090611211327.8E0EF3A411E@sparrow.telecommunity.com>
	<6db0ea510906111444y5a4d91b6p7bf770fb436f443d@mail.gmail.com>
	<87hbym8jep.fsf@benfinney.id.au> <4A31A133.9030100@trueblade.com>
	<87vdmz66k6.fsf@benfinney.id.au> <h14450$bng$1@ger.gmane.org>
Message-ID: <871vpl4xky.fsf_-_@benfinney.id.au>

Tres Seaver <tseaver at palladion.com> writes:

> -1:  "practicality beats purity" here:  there is *no* case in which an
> alpha version should *ever* sort after the final release with the
> corresponding number.  If we specify the "simple pure" scheme you
> propose, nobody will use it, period.

Well then, I don't see a way forward on the issue of helping
distributors to manage version numbers sanely. I don't know of any
operating system package manager that returns different comparison
results depending on what specific letters are used in the version
string.

All this specialness-of-certain-letters may make sense to Python core
developers, but it's just going to result in nonsensical version
progressions as far as operating system distributors are concerned. What
will be the resolution there?

-- 
 \         ?Pinky, are you pondering what I'm pondering?? ?I think so, |
  `\     Brain, but me and Pippi Longstocking ? I mean, what would the |
_o__)                      children look like?? ?_Pinky and The Brain_ |
Ben Finney
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090615/beae6987/attachment.pgp>

From david at ar.media.kyoto-u.ac.jp  Mon Jun 15 12:11:27 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Mon, 15 Jun 2009 19:11:27 +0900
Subject: [Distutils] version scheme: a case for dropping ".devNNN" and
 ".postNNN"
In-Reply-To: <h14450$bng$1@ger.gmane.org>
References: <6db0ea510906110229m291fcacbvcff487ff30391826@mail.gmail.com>	<20090611135255.EFA2F3A402D@sparrow.telecommunity.com>	<79990c6b0906110716q11b3a280l3065107d4ef95e76@mail.gmail.com>	<20090611155023.368A83A402D@sparrow.telecommunity.com>	<b654cd2e0906110903p6ef3c5c2la560216b7246c9c7@mail.gmail.com>	<6db0ea510906111105y79c346e3jf60e282172de16ad@mail.gmail.com>	<94bdd2610906111252m39e6c20bsee5df434f8d7823e@mail.gmail.com>	<6db0ea510906111338n3b7b32a1w430f3a592e722388@mail.gmail.com>	<20090611211327.8E0EF3A411E@sparrow.telecommunity.com>	<6db0ea510906111444y5a4d91b6p7bf770fb436f443d@mail.gmail.com>	<87hbym8jep.fsf@benfinney.id.au>	<4A31A133.9030100@trueblade.com>
	<87vdmz66k6.fsf@benfinney.id.au> <h14450$bng$1@ger.gmane.org>
Message-ID: <4A361E4F.6090502@ar.media.kyoto-u.ac.jp>

Tres Seaver wrote:
> Ben Finney wrote:
> > Eric Smith <eric at trueblade.com> writes:
>
> >> Ben Finney wrote:
> >>> If the version comparison semantics are such that simple 
compare
> >>> non-digits per ASCII, compare digit sequences as integers
 works
> >>> within a component, I'm not aware of any distribution downstream
> >>> that can't just use them as-is. What specific problems can you see
> >>> with that?
> >> I think the "count up to" a release version is the problem, and the
> >> only one I can think of. Before Twisted releases 8.2.2, do they really
> >> need to call all of the versions 8.2.1.99.1, 8.2.1.99.2, etc., to
> >> avoid confusion with a possible 8.2.1 release? Don't you think some
> >> information is lost by not conveying "this will become 8.2.2 when it
> >> is the release version"?
> > Why is it necessary for a version string to convey a prediction about
> > what *future* version strings will mean? That seems to me to be an
> > overloading that costs a significant amount of complexity, and is
> > unnecessary since a different version string can be chosen that meets
> > the intentional sequence position anyway.
>
>
> > "P.J. Eby" <pje at telecommunity.com> writes:
>
> >> Please explain how you intend to designate "development" versions that
> >> compare less than the equivalent non-development versions in your
> >> scheme.
> >     >>> (V("0.9")
> >     ... < V("0.9.0")
> >     ... < V("0.9.1")
> >     ... < V("0.9.a")
> >     ... < V("0.9.a2")
> >     ... < V("0.9.post123")
> >     ... < V("0.9b")
> >     ... < V("0.10")
> >     ... < V("0.999.0.dev-r925")
> >     ... < V("0.999.0.dev-r1357")
> >     ... < V("0.999.0.dev-r2468")
> >     ... < V("0.999.0.dev-r3579")
> >     ... < V("0.999.alpha1")
> >     ... < V("0.999.alpha2")
> >     ... < V("0.999.beta1")
> >     ... < V("0.999.rc1")
> >     ... < V("0.999.rc2")
> >     ... < V("1.0"))
> >     ...
> >     True
>
> > "P.J. Eby" <pje at telecommunity.com> writes:
>
> >> I can *almost* see a case for dropping "post" [&]
> >>
> >> However, I do not see an equivalent case for dropping "dev", since
> >> it's an important part of development workflows, and I don't see any
> >> way to simulate it in an an otherwise-linear scheme.
> > I don't understand. You see that there's a scheme for generating version
> > strings that compare in a linear sequence, yet you can't see how to
> > generate a version string that will compare at a specific point in that
> > linear sequence?
>
> > For my part, I don't see what it is about these workflows that requires
> > a special-cased token in the version string. Why can't the workflow
> > simply use version strings that compare linearly as specified?
>
>
> > Paul Moore <p.f.moore at gmail.com> writes:
>
> >> One other aspect of standard practice that I just realised your rules
> >> don't cover is where version strings differ in length. The normal
> >> lexicographic "shortest is earliest" rule doesn't work properly:
> >>
> >> 1.2a1 vs 1.2 (I hope everyone agrees that 1.2a1 is earlier)
> > No, I don't agree. Those each represent two components, where the first
> > is identical and the second is different; and when comparing the two
> > different components, I would expect 
2
 to be earlier than 
2a1
.
>
> -1:  "practicality beats purity" here:  there is *no* case in which an
> alpha version should *ever* sort after the final release with the
> corresponding number.  

I thought improved support for OS packagers was one of the main
rationale for this version scheme ? In that case, the above scheme is
totally incompatible with what at least dpkg, and I would assume rpm,
use for version comparison:

http://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-Version

I am a bit confused, because I thought several packagers were there at
PyCon to discuss about this ?

David

From brian at vanguardistas.net  Mon Jun 15 13:10:36 2009
From: brian at vanguardistas.net (Brian Sutherland)
Date: Mon, 15 Jun 2009 13:10:36 +0200
Subject: [Distutils] Utility to mirror an egg index from a
	Debian	distribution
In-Reply-To: <20090515162252.81B003A4061@sparrow.telecommunity.com>
References: <20090515110041.GC66338@Boo.lan>
	<20090515162252.81B003A4061@sparrow.telecommunity.com>
Message-ID: <20090615111036.GA14197@Boo.lan>

On Fri, May 15, 2009 at 12:25:31PM -0400, P.J. Eby wrote:
> At 01:00 PM 5/15/2009 +0200, Brian Sutherland wrote:
>> Hi,
>>
>> It may seem like a backwards way of doing things, but I have a need for
>> a utility that can maintain a python package index mirror of a Debian
>> repository.
>>
>> The basic idea is to extract the tarballs of Python packages from the
>> Debian repository and rename them to the original setuptools name. It
>> should also create a buildout-compatible versions file of the versions
>> in the repository.
>>
>> My current implementation idea is to unpack the tarball and use the
>> egg-metadata to figure out what the "egg" name of the tarball should be.
>
> Running "setup.py --name --version" will dump out the name and version, 
> whether you use distutils or setuptools.  If you want a  
> setuptools-compatible name and version, you'll need to postprocess those 
> strings with pkg_resources.safe_name() and safe_version(), then escape 
> them with to_filename() if you're using them as components of a sdist or 
> egg filename.

Thanks, I've just released a prototype:

    http://pypi.python.org/pypi/van.reposync

However, as I did not want to actually execute code contained in the
tarball, I did something like this: 

    basedir = os.path.dirname(egg_info)
    metadata = PathMetadata(basedir, egg_info)
    dist_name = os.path.splitext(os.path.basename(egg_info))[0]
    dist = Distribution(basedir, project_name=dist_name, metadata=metadata)

Unfortunately I had to play some guessing games to find where the
.egg-info directory was.

-- 
Brian Sutherland

From floris.bruynooghe at gmail.com  Mon Jun 15 13:43:03 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Mon, 15 Jun 2009 12:43:03 +0100
Subject: [Distutils] A tale of two conventions: Incompatibility
	of	version comparison semantics (was: version scheme: a case
	for	dropping ".devNNN" and ".postNNN")
In-Reply-To: <871vpl4xky.fsf_-_@benfinney.id.au>
References: <6db0ea510906111105y79c346e3jf60e282172de16ad@mail.gmail.com>
	<94bdd2610906111252m39e6c20bsee5df434f8d7823e@mail.gmail.com>
	<6db0ea510906111338n3b7b32a1w430f3a592e722388@mail.gmail.com>
	<20090611211327.8E0EF3A411E@sparrow.telecommunity.com>
	<6db0ea510906111444y5a4d91b6p7bf770fb436f443d@mail.gmail.com>
	<87hbym8jep.fsf@benfinney.id.au> <4A31A133.9030100@trueblade.com>
	<87vdmz66k6.fsf@benfinney.id.au> <h14450$bng$1@ger.gmane.org>
	<871vpl4xky.fsf_-_@benfinney.id.au>
Message-ID: <20090615114303.GA12556@laurie.devork>

On Mon, Jun 15, 2009 at 08:10:05PM +1000, Ben Finney wrote:
> Tres Seaver <tseaver at palladion.com> writes:
> 
> > -1:  "practicality beats purity" here:  there is *no* case in which an
> > alpha version should *ever* sort after the final release with the
> > corresponding number.  If we specify the "simple pure" scheme you
> > propose, nobody will use it, period.
> 
> Well then, I don't see a way forward on the issue of helping
> distributors to manage version numbers sanely. I don't know of any
> operating system package manager that returns different comparison
> results depending on what specific letters are used in the version
> string.
> 
> All this specialness-of-certain-letters may make sense to Python core
> developers, but it's just going to result in nonsensical version
> progressions as far as operating system distributors are concerned. What
> will be the resolution there?

Alpha, beta and release candidate releases have been around for ages
and I imagine most distributors have figured out a way of coping with
this.

In Debian's case 1.0a1 would become 1.0~a1 for example (IIRC), which
sorts before 1.0.

Regards
Floris


-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From ben+python at benfinney.id.au  Mon Jun 15 15:08:33 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 15 Jun 2009 23:08:33 +1000
Subject: [Distutils] A tale of two conventions: Incompatibility
	of	version comparison semantics
References: <6db0ea510906111105y79c346e3jf60e282172de16ad@mail.gmail.com>
	<94bdd2610906111252m39e6c20bsee5df434f8d7823e@mail.gmail.com>
	<6db0ea510906111338n3b7b32a1w430f3a592e722388@mail.gmail.com>
	<20090611211327.8E0EF3A411E@sparrow.telecommunity.com>
	<6db0ea510906111444y5a4d91b6p7bf770fb436f443d@mail.gmail.com>
	<87hbym8jep.fsf@benfinney.id.au> <4A31A133.9030100@trueblade.com>
	<87vdmz66k6.fsf@benfinney.id.au> <h14450$bng$1@ger.gmane.org>
	<871vpl4xky.fsf_-_@benfinney.id.au>
	<20090615114303.GA12556@laurie.devork>
Message-ID: <87ws7d3ar2.fsf@benfinney.id.au>

Floris Bruynooghe <floris.bruynooghe at gmail.com> writes:

> On Mon, Jun 15, 2009 at 08:10:05PM +1000, Ben Finney wrote:
> > Well then, I don't see a way forward on the issue of helping
> > distributors to manage version numbers sanely. I don't know of any
> > operating system package manager that returns different comparison
> > results depending on what specific letters are used in the version
> > string.
>
> Alpha, beta and release candidate releases have been around for ages
> and I imagine most distributors have figured out a way of coping with
> this.

That doesn't answer the point raised, though. Isn't a major reason for
all this work to help the downstream distributors to help migrate *away*
from munging version strings and toward version strings that compare
sanely as-is?

> In Debian's case 1.0a1 would become 1.0~a1 for example (IIRC), which
> sorts before 1.0.

That's exactly the sort of hack I would hope would no longer be
necessary for version strings formed under an improved version
comparison specification.

Forgive me for saying so, but this looks strongly like abandoning the
criterion of ?make it easier for distributors than it is at present?.

-- 
 \       ?Are you pondering what I'm pondering?? ?Umm, I think so, Don |
  `\          Cerebro, but, umm, why would Sophia Loren do a musical?? |
_o__)                                           ?_Pinky and The Brain_ |
Ben Finney


From exarkun at divmod.com  Mon Jun 15 15:24:07 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Mon, 15 Jun 2009 09:24:07 -0400
Subject: [Distutils] A tale of two conventions: Incompatibility
 of	version comparison semantics
In-Reply-To: <87ws7d3ar2.fsf@benfinney.id.au>
Message-ID: <20090615132407.22176.185550323.divmod.quotient.5567@henry.divmod.com>

On Mon, 15 Jun 2009 23:08:33 +1000, Ben Finney <ben+python at benfinney.id.au> wrote:
>Floris Bruynooghe <floris.bruynooghe at gmail.com> writes:
>
>> On Mon, Jun 15, 2009 at 08:10:05PM +1000, Ben Finney wrote:
>> > Well then, I don't see a way forward on the issue of helping
>> > distributors to manage version numbers sanely. I don't know of any
>> > operating system package manager that returns different comparison
>> > results depending on what specific letters are used in the version
>> > string.
>>
>> Alpha, beta and release candidate releases have been around for ages
>> and I imagine most distributors have figured out a way of coping with
>> this.
>
>That doesn't answer the point raised, though. Isn't a major reason for
>all this work to help the downstream distributors to help migrate *away*
>from munging version strings and toward version strings that compare
>sanely as-is?
>
>> In Debian's case 1.0a1 would become 1.0~a1 for example (IIRC), which
>> sorts before 1.0.
>
>That's exactly the sort of hack I would hope would no longer be
>necessary for version strings formed under an improved version
>comparison specification.
>
>Forgive me for saying so, but this looks strongly like abandoning the
>criterion of ?make it easier for distributors than it is at present?.

By having this standard, distributors do have an easier time.  They can
transform these version numbers automatically, just as they do many other
things to upstream packages automatically to turn them into a distro
package.

So no, the goal of making it easier for distributors is not being
abandoned.

Jean-Paul

From ziade.tarek at gmail.com  Mon Jun 15 15:31:10 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 15 Jun 2009 15:31:10 +0200
Subject: [Distutils] A tale of two conventions: Incompatibility of
	version comparison semantics
In-Reply-To: <20090615132407.22176.185550323.divmod.quotient.5567@henry.divmod.com>
References: <87ws7d3ar2.fsf@benfinney.id.au>
	<20090615132407.22176.185550323.divmod.quotient.5567@henry.divmod.com>
Message-ID: <94bdd2610906150631v2db8c1c9hcab680928e1de80b@mail.gmail.com>

On Mon, Jun 15, 2009 at 3:24 PM, Jean-Paul Calderone<exarkun at divmod.com> wrote:
>> Forgive me for saying so, but this looks strongly like abandoning the
>> criterion of ?make it easier for distributors than it is at present?.
>
> By having this standard, distributors do have an easier time. ?They can
> transform these version numbers automatically, just as they do many other
> things to upstream packages automatically to turn them into a distro
> package.
>
> So no, the goal of making it easier for distributors is not being
> abandoned.

Absolutely,

I
let's not forget that this proposal will provide a standard for people to ve


-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Mon Jun 15 15:40:40 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 15 Jun 2009 15:40:40 +0200
Subject: [Distutils] A tale of two conventions: Incompatibility of
	version comparison semantics
In-Reply-To: <94bdd2610906150631v2db8c1c9hcab680928e1de80b@mail.gmail.com>
References: <87ws7d3ar2.fsf@benfinney.id.au>
	<20090615132407.22176.185550323.divmod.quotient.5567@henry.divmod.com>
	<94bdd2610906150631v2db8c1c9hcab680928e1de80b@mail.gmail.com>
Message-ID: <94bdd2610906150640h359fdb2dofe0e34d8fa6da0b8@mail.gmail.com>

On Mon, Jun 15, 2009 at 3:31 PM, Tarek Ziad?<ziade.tarek at gmail.com> wrote:
> On Mon, Jun 15, 2009 at 3:24 PM, Jean-Paul Calderone<exarkun at divmod.com> wrote:
>>> Forgive me for saying so, but this looks strongly like abandoning the
>>> criterion of ?make it easier for distributors than it is at present?.
>>
>> By having this standard, distributors do have an easier time. ?They can
>> transform these version numbers automatically, just as they do many other
>> things to upstream packages automatically to turn them into a distro
>> package.
>>
>> So no, the goal of making it easier for distributors is not being
>> abandoned.
>
> Absolutely,
>
> I
> let's not forget that this proposal will provide a standard for people to ve
>

oups, sent too early

I won't comment on this week mails yet, I lost track of the
discussions so I need to read them.
But I wanted to react on this:

Let's not forget that this proposal will provide a standard for people
to version
their packages in a way that is directly compatible for distributions out there,

The final goal is to write some statement like this in the python distutils doc:

"Hey, if you want to have a distributions-friendly package, you might
want to use this tool for
 your version numbers, it's proven to work on most packaging systems out there"

People will be free not to follow it of course if they don't
distribute their packages,
and this will not change their lives.

Regards
Tarek

From pje at telecommunity.com  Mon Jun 15 17:08:22 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 15 Jun 2009 11:08:22 -0400
Subject: [Distutils] Utility to mirror an egg index from a Debian
 distribution
In-Reply-To: <20090615111036.GA14197@Boo.lan>
References: <20090515110041.GC66338@Boo.lan>
	<20090515162252.81B003A4061@sparrow.telecommunity.com>
	<20090615111036.GA14197@Boo.lan>
Message-ID: <20090615150532.D74A53A4060@sparrow.telecommunity.com>

At 01:10 PM 6/15/2009 +0200, Brian Sutherland wrote:
>Thanks, I've just released a prototype:
>
>     http://pypi.python.org/pypi/van.reposync
>
>However, as I did not want to actually execute code contained in the
>tarball, I did something like this:
>
>     basedir = os.path.dirname(egg_info)
>     metadata = PathMetadata(basedir, egg_info)
>     dist_name = os.path.splitext(os.path.basename(egg_info))[0]
>     dist = Distribution(basedir, project_name=dist_name, metadata=metadata)
>
>Unfortunately I had to play some guessing games to find where the
>.egg-info directory was.

Actually, if all you want is the name and version, I suppose you 
could parse the PKG-INFO, which is always in a set location in an 
sdist tarball or zip.


From sridharr at activestate.com  Mon Jun 15 19:43:23 2009
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Mon, 15 Jun 2009 10:43:23 -0700
Subject: [Distutils] Utility to mirror an egg index from a Debian
	distribution
In-Reply-To: <20090515162252.81B003A4061@sparrow.telecommunity.com>
References: <20090515110041.GC66338@Boo.lan>
	<20090515162252.81B003A4061@sparrow.telecommunity.com>
Message-ID: <4A36883B.3080502@activestate.com>

On 09-05-15 09:25 AM, P.J. Eby wrote:
>> My current implementation idea is to unpack the tarball and use the
>> egg-metadata to figure out what the "egg" name of the tarball should be.
>
> Running "setup.py --name --version" will dump out the name and version,
> whether you use distutils or setuptools.  If you want a
> setuptools-compatible name and version, you'll need to postprocess those
> strings with pkg_resources.safe_name() and safe_version(), then escape
> them with to_filename() if you're using them as components of a sdist or
> egg filename.

There are two issues with relying on 'setup.py --<field>' to find the 
value of <field>:

1) setuptools prints warning messages to stdout: 
http://bugs.python.org/setuptools/issue73 (does `safe_name()` handle this?)

2) Some packages in PyPI prints unexpected messages depending upon the 
environment (even if you pass --name to setup.py). How do you suggest to 
find name/version under such cases?

From brian at vanguardistas.net  Tue Jun 16 10:43:26 2009
From: brian at vanguardistas.net (Brian Sutherland)
Date: Tue, 16 Jun 2009 10:43:26 +0200
Subject: [Distutils] Utility to mirror an egg index from a
	Debian	distribution
In-Reply-To: <20090615150532.D74A53A4060@sparrow.telecommunity.com>
References: <20090515110041.GC66338@Boo.lan>
	<20090515162252.81B003A4061@sparrow.telecommunity.com>
	<20090615111036.GA14197@Boo.lan>
	<20090615150532.D74A53A4060@sparrow.telecommunity.com>
Message-ID: <20090616084326.GA48696@Boo.lan>

On Mon, Jun 15, 2009 at 11:08:22AM -0400, P.J. Eby wrote:
> At 01:10 PM 6/15/2009 +0200, Brian Sutherland wrote:
>> Thanks, I've just released a prototype:
>>
>>     http://pypi.python.org/pypi/van.reposync
>>
>> However, as I did not want to actually execute code contained in the
>> tarball, I did something like this:
>>
>>     basedir = os.path.dirname(egg_info)
>>     metadata = PathMetadata(basedir, egg_info)
>>     dist_name = os.path.splitext(os.path.basename(egg_info))[0]
>>     dist = Distribution(basedir, project_name=dist_name, metadata=metadata)
>>
>> Unfortunately I had to play some guessing games to find where the
>> .egg-info directory was.
>
> Actually, if all you want is the name and version, I suppose you could 
> parse the PKG-INFO, which is always in a set location in an sdist tarball 
> or zip.

Yep, I changed to that method and managed to remove the guessing games.

-- 
Brian Sutherland

From ed.zwart at gmail.com  Tue Jun 16 21:23:28 2009
From: ed.zwart at gmail.com (Ed Zwart)
Date: Tue, 16 Jun 2009 12:23:28 -0700 (PDT)
Subject: [Distutils] add new compiler to distutils
In-Reply-To: <49F60580.8040905@well.com>
References: <49F60580.8040905@well.com>
Message-ID: <28d777ee-a158-4f85-ad90-3930fcd4d3e1@h2g2000yqg.googlegroups.com>

Did you ever get this figured out?

I am having trouble just using zc.buildout with non-KGS packages, but
mingw seems to do its thing just fine with the KGS.

I get the following error:

Installing app.
Getting distribution for 'zope.security'.
error: Setup script exited with error: don't know how to compile C/C++
code on platform 'nt' with 'mingw32-gcc-3.4.5' compiler
An error occured when trying to install zope.security 3.7.0.Look above
this message for any errors thatwere output by easy_install.
While:
  Installing app.
  Getting distribution for 'zope.security'.
Error: Couldn't install: zope.security 3.7.0



On Apr 27, 12:20?pm, patrick flaherty <p... at well.com> wrote:
> Hi,
>
> Relatively new to python (and very new to distutils) but experienced
> programmer.
>
> I'm trying to do a 64 bit build ofhttp://sourceforge.net/projects/pyopenssl/
>
> I'm on Windows Server 2008 R2. ?First had to acquaint myself with
> mingw32. ?That built but the DLLs (or rather PYDs) that are created
> won't load at runtime. ?I transferred the build to a 32 bit machine; the
> PYDs loaded; and the program ran successfully. ?Thus, as I said, I'm
> looking to rebuild in 64 bits.
>
> It's seems that there's a 64 bit version of mingw in development on
> sourceforge: ?http://sourceforge.net/projects/mingw-w64. ?You download a
> toolchain; add that to your Windows PATH; ?and I'm able to build a
> sample hello.c successfully.
>
> The problem is with Python and setup.py. ?When I try to specifycompiler
> i86_64-mingw32 or some variant to setup.py, I get the following:
>
> > L:\pf\Python\pyOpenSSL>setup.py build_ext -Ic:\openssl\include -c
> > x86_64-pc-mingw32
> > running build_ext
> > error: don't know how to compile C/C++ code on platform 'nt' with
> > 'x86_64-pc-mingw32'compiler
>
> So after fumbling around a bit, where I've arrived at is that in the
> distutils package ( c:\python26\lib\distutils ) there are various files
> such as msvccompiler.py, msvccompiler9.py (I have VS 2008 installed),
> ccompiler.py and cygwincompiler.py that would seem to indicate that
> 'new' compilers have to have support built into distutils.
>
> Is this correct, and if so, how does one go about the task of adding a
> newcompiler?
>
> thanx - pat
>
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-... at python.orghttp://mail.python.org/mailman/listinfo/distutils-sig

From dave.pawson at gmail.com  Wed Jun 17 13:47:35 2009
From: dave.pawson at gmail.com (Dave Pawson)
Date: Wed, 17 Jun 2009 12:47:35 +0100
Subject: [Distutils] python setup tools
Message-ID: <711a73df0906170447j2d258a0byadcb8d47ae044944@mail.gmail.com>

At
http://pypi.python.org/pypi/setuptools#rpm-based-systems
I downloaded setuptools-0.6c9-py2.6.egg
as root, chmod +x setup...

ran it and received:

[root at marge2 python]# ./setuptools-0.6c9-py2.6.egg
error: invalid Python installation: unable to open
/usr/lib/python2.6/config/Makefile (No such file or directory)


Running python
[dpawson at marge2 python]$ python --version
Python 2.6

on Fedora core 11.

Any suggestions please, I'd like to use easy_install.


TIA

-- 
Dave Pawson
XSLT XSL-FO FAQ.
Docbook FAQ.
http://www.dpawson.co.uk

From ndbecker2 at gmail.com  Wed Jun 17 14:56:14 2009
From: ndbecker2 at gmail.com (Neal Becker)
Date: Wed, 17 Jun 2009 08:56:14 -0400
Subject: [Distutils] python setup tools
References: <711a73df0906170447j2d258a0byadcb8d47ae044944@mail.gmail.com>
Message-ID: <h1ap5e$5cn$1@ger.gmane.org>

Dave Pawson wrote:

> At
> http://pypi.python.org/pypi/setuptools#rpm-based-systems
> I downloaded setuptools-0.6c9-py2.6.egg
> as root, chmod +x setup...
> 
> ran it and received:
> 
> [root at marge2 python]# ./setuptools-0.6c9-py2.6.egg
> error: invalid Python installation: unable to open
> /usr/lib/python2.6/config/Makefile (No such file or directory)
> 
> 
> Running python
> [dpawson at marge2 python]$ python --version
> Python 2.6
> 
> on Fedora core 11.
> 
> Any suggestions please, I'd like to use easy_install.
> 
> 
> TIA
> 
Why not just do:
yum install python-setuptools-devel



From ziade.tarek at gmail.com  Wed Jun 17 16:34:43 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 17 Jun 2009 16:34:43 +0200
Subject: [Distutils] sdist default archive format
Message-ID: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>

Hi,

Michael reported that sdist produces a zip file under windows and a
tar file under posix by default, which is inconsistent.

I've changed the sdist command some time ago, so it uses the sdtilb
tarfile module, instead of the "tar" command,
meaning that we could switch to the tar format under windows by
default as well now, not requiring the "tar" program anymore

But some people may prefer the zip format.

What are your opinion on this ?

You can respond here, or in the issue we have created
http://bugs.python.org/issue6296

Thanks

Cheers
Tarek

-- 
Tarek Ziad? | http://ziade.org

From pje at telecommunity.com  Wed Jun 17 18:32:30 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 17 Jun 2009 12:32:30 -0400
Subject: [Distutils] python setup tools
In-Reply-To: <711a73df0906170447j2d258a0byadcb8d47ae044944@mail.gmail.co
 m>
References: <711a73df0906170447j2d258a0byadcb8d47ae044944@mail.gmail.com>
Message-ID: <20090617162937.700F13A404D@sparrow.telecommunity.com>

At 12:47 PM 6/17/2009 +0100, Dave Pawson wrote:
>[root at marge2 python]# ./setuptools-0.6c9-py2.6.egg
>error: invalid Python installation: unable to open
>/usr/lib/python2.6/config/Makefile (No such file or directory)

This means you don't have the python-dev or python-devel package 
installed on your system.  Some OS distributors actually split Python 
into two packages, and you can't run the distutils properly without 
the second one.


>on Fedora core 11.

You can probably also just install the Fedora package for setuptools. 


From trentm at gmail.com  Wed Jun 17 23:07:30 2009
From: trentm at gmail.com (Trent Mick)
Date: Wed, 17 Jun 2009 14:07:30 -0700
Subject: [Distutils] sdist default archive format
In-Reply-To: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
Message-ID: <6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>

> I've changed the sdist command some time ago, so it uses the sdtilb
> tarfile module, instead of the "tar" command,
> meaning that we could switch to the tar format under windows by
> default as well now, not requiring the "tar" program anymore
>
> But some people may prefer the zip format.
>
> What are your opinion on this ?

My preferences is for .zip (that is what I use for my packages), the
main reason is that Windows users can always unpack a .zip file. Often
that is not true for .tar.gz or .tar.bz2 files.

This reference is ancient, but the argument is valid, IMO:
http://www.37signals.com/svn/archives2/some_notes_on_the_building_of_codezoo.php
> ...
> Where there were multiple versions available, we chose the most recent, stable version and made that the default. Where there were multiple packages, we chose the Zip format (which is widely available on any platform) and got rid or tar.gz, tar.bz, .exe, and everything else we could. If there were many parts of the component, we defaulted to the one file that gave the user the most value in a single download. Other files, versions, and packages fell to the bottom of the page -- available for the users that absolutely need it, but far from distracting for the users that just want to get their work done. Reducing choices makes the site more useful, not less.
> ...


The tradeoff is that .zip files are compressed as well. But I don't
believe that difference in compression size between
.zip/.tar.gz/.tar.bz2 is that big of a concern in *most* cases because
most Python sdist's are small, are they not?


Cheers,
Trent

-- 
Trent Mick
trentm at gmail.com

From david.lyon at preisshare.net  Thu Jun 18 01:29:11 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 17 Jun 2009 19:29:11 -0400
Subject: [Distutils] sdist default archive format
In-Reply-To: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
Message-ID: <e52dd8cdbd8e3f5744f6e92db038ef06@preisshare.net>

On Wed, 17 Jun 2009 16:34:43 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
wrote:
> I've changed the sdist command some time ago, so it uses the sdtilb
> tarfile module, instead of the "tar" command,
> meaning that we could switch to the tar format under windows by
> default as well now, not requiring the "tar" program anymore
> 
> But some people may prefer the zip format.

Sounds like useful work Tarek,

tarfiles aren't a problem at all on windows if you have the
software to read them....

I'm personally a fan of the "black-box" approach...

Where it just works.... 

I say this just that my personal experience with python so
far is that .tar.gz packages rarely seem to install properly
under windows. Or if they do... easyinstall reports an error
on them....

(try - easyinstall buildbot... or pydbf to see what i mean)

I guess that is a side issue...

My point is whatever you do to try to help developers build
for multiple platforms from their own platform is worthwhile.

When I get a chance I'll actually try to start using
distutils to bundle my own application. Which is as you
might know a package manager for python that is supposed
to be delivered to all platforms on which python runs.

After that, I'm having thoughts of doing a gui interface
for distutils... to make the whole thing easier from a
developers point of view...

but I can only make observations on distutils from a
bit of a distance... but it looks better the more I
learn about it...

David 








From santagada at gmail.com  Thu Jun 18 02:06:28 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Wed, 17 Jun 2009 21:06:28 -0300
Subject: [Distutils] sdist default archive format
In-Reply-To: <6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>
Message-ID: <56988003-61B7-425D-AADD-B229D5CF09B6@gmail.com>


On Jun 17, 2009, at 6:07 PM, Trent Mick wrote:

>> I've changed the sdist command some time ago, so it uses the sdtilb
>> tarfile module, instead of the "tar" command,
>> meaning that we could switch to the tar format under windows by
>> default as well now, not requiring the "tar" program anymore
>>
>> But some people may prefer the zip format.
>>
>> What are your opinion on this ?
>
> My preferences is for .zip (that is what I use for my packages), the
> main reason is that Windows users can always unpack a .zip file. Often
> that is not true for .tar.gz or .tar.bz2 files.
>
> This reference is ancient, but the argument is valid, IMO:
> http://www.37signals.com/svn/archives2/some_notes_on_the_building_of_codezoo.php
>> ...
>> Where there were multiple versions available, we chose the most  
>> recent, stable version and made that the default. Where there were  
>> multiple packages, we chose the Zip format (which is widely  
>> available on any platform) and got rid or tar.gz, tar.bz, .exe, and  
>> everything else we could. If there were many parts of the  
>> component, we defaulted to the one file that gave the user the most  
>> value in a single download. Other files, versions, and packages  
>> fell to the bottom of the page -- available for the users that  
>> absolutely need it, but far from distracting for the users that  
>> just want to get their work done. Reducing choices makes the site  
>> more useful, not less.
>> ...
>
>
> The tradeoff is that .zip files are compressed as well. But I don't
> believe that difference in compression size between
> .zip/.tar.gz/.tar.bz2 is that big of a concern in *most* cases because
> most Python sdist's are small, are they not?

Zip files don't support symbolic links(and maybe unix permissions/ 
ownership). Maybe tarek could include a python script in tools to  
uncompress tar files so you can always be sure that there is support  
for it even on windows.

Is support for symbolic links(unix permissions/ownership) important?

--
Leonardo Santagada
santagada at gmail.com




From pje at telecommunity.com  Thu Jun 18 03:39:35 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 17 Jun 2009 21:39:35 -0400
Subject: [Distutils] sdist default archive format
In-Reply-To: <e52dd8cdbd8e3f5744f6e92db038ef06@preisshare.net>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
	<e52dd8cdbd8e3f5744f6e92db038ef06@preisshare.net>
Message-ID: <20090618013641.874743A404D@sparrow.telecommunity.com>

At 07:29 PM 6/17/2009 -0400, David Lyon wrote:
>(try - easyinstall buildbot... or pydbf to see what i mean)

I don't see any errors here.  What errors are you seeing? 


From ben+python at benfinney.id.au  Thu Jun 18 03:57:57 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 18 Jun 2009 11:57:57 +1000
Subject: [Distutils] sdist default archive format
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>
Message-ID: <874oue2ti2.fsf@benfinney.id.au>

Trent Mick <trentm at gmail.com> writes:

> My preferences is for .zip (that is what I use for my packages), the
> main reason is that Windows users can always unpack a .zip file. Often
> that is not true for .tar.gz or .tar.bz2 files.

The reverse is true on Unix. The tar-and-gzip format is extensively used
so it is pretty much guaranteed to be readable on any Unix system; the
Zip format does not have anywhere near the same level of guarantee.

I would think the best solution is to use the tar format (since, as
pointed out elsewhere, it has better support for symlinks and permission
flags that are often important when distributing works), and ensure that
distutils will include a tool for unpacking them with pure Python on
those platforms without external support for the format.

-- 
 \          ?When we call others dogmatic, what we really object to is |
  `\   their holding dogmas that are different from our own.? ?Charles |
_o__)                                                           Issawi |
Ben Finney


From david.lyon at preisshare.net  Thu Jun 18 04:29:40 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 17 Jun 2009 22:29:40 -0400
Subject: [Distutils] sdist default archive format
In-Reply-To: <874oue2ti2.fsf@benfinney.id.au>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>
	<874oue2ti2.fsf@benfinney.id.au>
Message-ID: <da0c5c919fe9fb9fb9d8fb2a6370b79f@preisshare.net>

On Thu, 18 Jun 2009 11:57:57 +1000, Ben Finney <ben+python at benfinney.id.au>
wrote:
> The reverse is true on Unix. The tar-and-gzip format is extensively used
> so it is pretty much guaranteed to be readable on any Unix system; the
> Zip format does not have anywhere near the same level of guarantee.
> 
> I would think the best solution is to use the tar format (since, as
> pointed out elsewhere, it has better support for symlinks and permission
> flags that are often important when distributing works), and ensure that
> distutils will include a tool for unpacking them with pure Python on
> those platforms without external support for the format.

Symlinks shouldn't be used.. within python packages... they are
totally unneccessary....

As soon as the package gets to a windows system with symlinks it
will break and we are back to where we started - a divided community.

If you can give one example within a cross-platform package where 
a symlink would be required (outside of what can be created by distutils
from
the metadata) I would be interested... but I don't believe there are any...

I already thought we had eggs....

With zip, the unzipper code is within python (I believe). So anything
where the code for the format is already buried within python is going
to be the best for python developers.

If tar code is built in within python... then it should be a flick of
the coin for people to decide....

There's nothing bad about tar... as long as symlinks stay well out of
them....

David





From david.lyon at preisshare.net  Thu Jun 18 04:57:56 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 17 Jun 2009 22:57:56 -0400
Subject: [Distutils] sdist default archive format
In-Reply-To: <20090618013641.874743A404D@sparrow.telecommunity.com>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
	<e52dd8cdbd8e3f5744f6e92db038ef06@preisshare.net>
	<20090618013641.874743A404D@sparrow.telecommunity.com>
Message-ID: <4d509a164a3cd00205b1b94bc18afaf6@preisshare.net>

On Wed, 17 Jun 2009 21:39:35 -0400, "P.J. Eby" <pje at telecommunity.com>
wrote:
> At 07:29 PM 6/17/2009 -0400, David Lyon wrote:
>>(try - easyinstall buildbot... or pydbf to see what i mean)
> 
> I don't see any errors here.  What errors are you seeing?

Python 2.5, Windows XP

Running installer ... C:\Python25\scripts\easy_install.exe buildbot
Searching for buildbot
Reading http://pypi.python.org/simple/buildbot/

Reading http://buildbot.sourceforge.net/

Reading http://buildbot.net/

Reading
https://sourceforge.net/project/showfiles.php?group_id=73177&package_id=73212&release_id=665379

Best match: buildbot 0.7.10p1
Downloading
http://pypi.python.org/packages/source/b/buildbot/buildbot-0.7.10p1.zip#md5=51f744937d65635810aa6496eb2bf078

Processing buildbot-0.7.10p1.zip

Running buildbot-0.7.10p1\setup.py -q bdist_egg --dist-dir
c:\docume~1\david\locals~1\temp\easy_install-xnvlp8\buildbot-0.7.10p1\egg-dist-tmp-cxmna9

zip_safe flag not set; analyzing archive contents...

buildbot.clients.debug: module references __file__
buildbot.scripts.runner: module references __file__

buildbot.status.web.baseweb: module references __file__
buildbot.status.web.grid: module references __file__
buildbot.test.runutils: module references __file__
buildbot.test.test_maildir: module references __file__
buildbot.test.test_mailparse: module references __file__
buildbot.test.test_shell: module references __file__
buildbot.test.test_slavecommand: module references __file__

No eggs found in
c:\docume~1\david\locals~1\temp\easy_install-xnvlp8\buildbot-0.7.10p1\egg-dist-tmp-cxmna9
(setup script problem?)


From trentm at gmail.com  Thu Jun 18 06:56:45 2009
From: trentm at gmail.com (Trent Mick)
Date: Wed, 17 Jun 2009 21:56:45 -0700
Subject: [Distutils] sdist default archive format
In-Reply-To: <874oue2ti2.fsf@benfinney.id.au>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>
	<874oue2ti2.fsf@benfinney.id.au>
Message-ID: <6db0ea510906172156m2028b0d1o89653f125999d930@mail.gmail.com>

>> My preferences is for .zip (that is what I use for my packages), the
>> main reason is that Windows users can always unpack a .zip file. Often
>> that is not true for .tar.gz or .tar.bz2 files.
>
> The reverse is true on Unix. The tar-and-gzip format is extensively used
> so it is pretty much guaranteed to be readable on any Unix system; the
> Zip format does not have anywhere near the same level of guarantee.

On what modern Linux and Mac OSes is "/usr/bin/unzip" not installed by default?

Trent

-- 
Trent Mick
trentm at gmail.com

From trentm at gmail.com  Thu Jun 18 06:59:25 2009
From: trentm at gmail.com (Trent Mick)
Date: Wed, 17 Jun 2009 21:59:25 -0700
Subject: [Distutils] sdist default archive format
In-Reply-To: <da0c5c919fe9fb9fb9d8fb2a6370b79f@preisshare.net>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>
	<874oue2ti2.fsf@benfinney.id.au>
	<da0c5c919fe9fb9fb9d8fb2a6370b79f@preisshare.net>
Message-ID: <6db0ea510906172159x63be0fd1v540a13a631baf97a@mail.gmail.com>

> There's nothing bad about tar... as long as symlinks stay well out of
> them....

Tar on solaris is often broken by design. :) For compat reasons
Solaris' default 'tar' is one that has file length problems
(http://www.mikehan.com/rant/solaris-tools.html).

Trent

-- 
Trent Mick
trentm at gmail.com

From santagada at gmail.com  Thu Jun 18 07:32:10 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Thu, 18 Jun 2009 02:32:10 -0300
Subject: [Distutils] sdist default archive format
In-Reply-To: <da0c5c919fe9fb9fb9d8fb2a6370b79f@preisshare.net>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>
	<874oue2ti2.fsf@benfinney.id.au>
	<da0c5c919fe9fb9fb9d8fb2a6370b79f@preisshare.net>
Message-ID: <7EB1EF41-2863-4FF4-B6E5-3D7F33FBE18C@gmail.com>


On Jun 17, 2009, at 11:29 PM, David Lyon wrote:

> With zip, the unzipper code is within python (I believe). So anything
> where the code for the format is already buried within python is going
> to be the best for python developers.
>
> If tar code is built in within python... then it should be a flick of
> the coin for people to decide....


The tar code is built in within python (there have been problems with  
early versions of the tar module though).

What about unix permissions are those supported on zip, and are they  
important in a source package? I would think they are importante (and  
not supported in zip) but I'm not sure.


--
Leonardo Santagada
santagada at gmail.com




From ben+python at benfinney.id.au  Thu Jun 18 08:45:55 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 18 Jun 2009 16:45:55 +1000
Subject: [Distutils] sdist default archive format
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>
	<874oue2ti2.fsf@benfinney.id.au>
	<6db0ea510906172156m2028b0d1o89653f125999d930@mail.gmail.com>
Message-ID: <87zlc611lo.fsf@benfinney.id.au>

Trent Mick <trentm at gmail.com> writes:

> > The reverse is true on Unix. The tar-and-gzip format is extensively
> > used so it is pretty much guaranteed to be readable on any Unix
> > system; the Zip format does not have anywhere near the same level of
> > guarantee.
> 
> On what modern Linux and Mac OSes is "/usr/bin/unzip" not installed by
> default?

You ask as though there were a single default for such systems. For GNU
and BSD, at least, there is no such thing.

To answer the question you may have instead been wanting answered: Any
Unix-like system where ?get a minimal installation first and then I'll
install just what's needed to support the task this machine should
perform? will not have an ?unzip? utility except by mere chance
coincidence with the desired task.

It's certainly not a valid assumption that ?I want to set up this
machine for a purpose that needs Python packages? and ?I want the
ability to read Zip format files? will necessarily co-incide. On the
other hand, it *is* a valid assumption that any such system will have
the ability to read tar-and-gzip archives.

-- 
 \          ?Friendship is born at that moment when one person says to |
  `\    another, ?What! You too? I thought I was the only one!?? ?C.S. |
_o__)                                                            Lewis |
Ben Finney


From ben+python at benfinney.id.au  Thu Jun 18 08:49:15 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 18 Jun 2009 16:49:15 +1000
Subject: [Distutils] sdist default archive format
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>
	<874oue2ti2.fsf@benfinney.id.au>
	<da0c5c919fe9fb9fb9d8fb2a6370b79f@preisshare.net>
	<7EB1EF41-2863-4FF4-B6E5-3D7F33FBE18C@gmail.com>
Message-ID: <87vdmu11g4.fsf@benfinney.id.au>

Leonardo Santagada <santagada at gmail.com> writes:

> What about unix permissions are those supported on zip, and are they
> important in a source package? I would think they are importante (and
> not supported in zip) but I'm not sure.

The classic one is the ?execute? permission on files. It's very
important to preserve that (whether on or off) in distribution of works
for use on Unix; and the Zip format doesn't support it AFAIK.

-- 
 \                   ?The best ad-libs are rehearsed.? ?Graham Kennedy |
  `\                                                                   |
_o__)                                                                  |
Ben Finney


From david.lyon at preisshare.net  Thu Jun 18 09:12:13 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 18 Jun 2009 03:12:13 -0400
Subject: [Distutils] sdist default archive format
In-Reply-To: <7EB1EF41-2863-4FF4-B6E5-3D7F33FBE18C@gmail.com>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>
	<874oue2ti2.fsf@benfinney.id.au>
	<da0c5c919fe9fb9fb9d8fb2a6370b79f@preisshare.net>
	<7EB1EF41-2863-4FF4-B6E5-3D7F33FBE18C@gmail.com>
Message-ID: <fa22b74c12a708377b988b14aca1a5ca@preisshare.net>

On Thu, 18 Jun 2009 02:32:10 -0300, Leonardo Santagada
<santagada at gmail.com>
wrote:
> The tar code is built in within python (there have been problems with  
> early versions of the tar module though).

Ok - well maybe it is fixed now...

> What about unix permissions are those supported on zip, and are they  
> important in a source package? I would think they are importante (and  
> not supported in zip) but I'm not sure.

True.

But we are back to coin tossing... :-)

If you think about it, the "scripts" get specified in the metadata.

Distutils itself, if running on *nix, can chmod the scripts with +x
as required. (As in, when "setup.py install" is run...

That way, nothing is being done that is "naughty"......

It means people can have their scripts and have them +x'ed too....

(but only after they arrive... ie no sneaky +x scripts can
 be shipped.)

Then the issue should come back to which archive format has the
best support within python (core).. ie fastest.. most robust..

Whatever is the best black box... should be the winner...

David




From david at ar.media.kyoto-u.ac.jp  Thu Jun 18 09:32:51 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Thu, 18 Jun 2009 16:32:51 +0900
Subject: [Distutils] sdist default archive format
In-Reply-To: <87vdmu11g4.fsf@benfinney.id.au>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>	<874oue2ti2.fsf@benfinney.id.au>	<da0c5c919fe9fb9fb9d8fb2a6370b79f@preisshare.net>	<7EB1EF41-2863-4FF4-B6E5-3D7F33FBE18C@gmail.com>
	<87vdmu11g4.fsf@benfinney.id.au>
Message-ID: <4A39EDA3.6020906@ar.media.kyoto-u.ac.jp>

Ben Finney wrote:
> Leonardo Santagada <santagada at gmail.com> writes:
>
>   
>> What about unix permissions are those supported on zip, and are they
>> important in a source package? I would think they are importante (and
>> not supported in zip) but I'm not sure.
>>     
>
> The classic one is the ?execute? permission on files. It's very
> important to preserve that (whether on or off) in distribution of works
> for use on Unix; and the Zip format doesn't support it AFAIK.
>   

I think the problem of execute permission should be handled on its own
(where the format is part of the problem, of course). There are problems
with e.g. setuptools which do things differently than distutils with
respect to the execute permission on some platforms, and that causes
some distributions problems (like nose not finding tests at some
locations), not just for sdist command.

Whatever the solution, it should be as consistent as possible on every
platform.

cheers,

David


From david.lyon at preisshare.net  Thu Jun 18 10:17:31 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 18 Jun 2009 04:17:31 -0400
Subject: [Distutils] more easyinstall error messages - haha
In-Reply-To: <4A39EDA3.6020906@ar.media.kyoto-u.ac.jp>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>	<874oue2ti2.fsf@benfinney.id.au>	<da0c5c919fe9fb9fb9d8fb2a6370b79f@preisshare.net>	<7EB1EF41-2863-4FF4-B6E5-3D7F33FBE18C@gmail.com>
	<87vdmu11g4.fsf@benfinney.id.au>
	<4A39EDA3.6020906@ar.media.kyoto-u.ac.jp>
Message-ID: <56dfbddc42a9e15ffbe3f67937b25e21@preisshare.net>


This is kind of typical of what I get all the time...

Platform: XP-Python 2.5

-----------------------------------------------------------------------------
Running installer ... C:\Python25\scripts\easy_install.exe asciiporn
Searching for asciiporn
Reading http://pypi.python.org/simple/asciiporn/

Reading http://www-rcf.usc.edu/~kaizhu/work/asciiporn

Best match: asciiporn 2009.05.01
Downloading
http://pypi.python.org/packages/source/a/asciiporn/asciiporn-2009.05.01.tar.gz#md5=4ee8276f6e0bb3f3b7fb9b089d4a26e2

ERRORS: Traceback (most recent call last):
  File "C:\Python25\scripts\easy_install-script.py", line 8, in <module>
    load_entry_point('setuptools==0.6c9', 'console_scripts',
'easy_install')()
  File
"C:\Python25\lib\site-packages\setuptools-0.6c9-py2.5.egg\setuptools\command\easy_install.py",
line 1671, in main
  File
"C:\Python25\lib\site-packages\setuptools-0.6c9-py2.5.egg\setuptools\command\easy_install.py",
line 1659, in with_ei_usage
  File
"C:\Python25\lib\site-packages\setuptools-0.6c9-py2.5.egg\setuptools\command\easy_install.py",
line 1675, in <lambda>
  File "C:\Python25\lib\distutils\core.py", line 151, in setup
    dist.run_commands()
  File "C:\Python25\lib\distutils\dist.py", line 974, in run_commands
    self.run_command(cmd)
  File "C:\Python25\lib\distutils\dist.py", line 994, in run_command
    cmd_obj.run()
  File
"C:\Python25\lib\site-packages\setuptools-0.6c9-py2.5.egg\setuptools\command\easy_install.py",
line 211, in run
  File
"C:\Python25\lib\site-packages\setuptools-0.6c9-py2.5.egg\setuptools\command\easy_install.py",
line 446, in easy_install
  File
"C:\Python25\lib\site-packages\setuptools-0.6c9-py2.5.egg\setuptools\command\easy_install.py",
line 476, in install_item
  File
"C:\Python25\lib\site-packages\setuptools-0.6c9-py2.5.egg\setuptools\command\easy_install.py",
line 655, in install_eggs
  File
"C:\Python25\lib\site-packages\setuptools-0.6c9-py2.5.egg\setuptools\command\easy_install.py",
line 930, in build_and_install
  File
"C:\Python25\lib\site-packages\setuptools-0.6c9-py2.5.egg\setuptools\command\easy_install.py",
line 919, in run_setup
  File
"C:\Python25\lib\site-packages\setuptools-0.6c9-py2.5.egg\setuptools\sandbox.py",
line 27, in run_setup
  File
"C:\Python25\lib\site-packages\setuptools-0.6c9-py2.5.egg\setuptools\sandbox.py",
line 63, in run
  File
"C:\Python25\lib\site-packages\setuptools-0.6c9-py2.5.egg\setuptools\sandbox.py",
line 29, in <lambda>
  File "setup.py", line 440, in <module>
    
Exception: asciiporn requires Python 2.6 or higher

Processing asciiporn-2009.05.01.tar.gz
Running asciiporn-2009.05.01\setup.py -q bdist_egg --dist-dir
c:\docume~1\david\locals~1\temp\easy_install-pnlek8\asciiporn-2009.05.01\egg-dist-tmp-8p44m6



From david.lyon at preisshare.net  Thu Jun 18 10:26:37 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 18 Jun 2009 04:26:37 -0400
Subject: [Distutils] imho Distutils should stop this from happening to poor
	users....
In-Reply-To: <4A39EDA3.6020906@ar.media.kyoto-u.ac.jp>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>	<874oue2ti2.fsf@benfinney.id.au>	<da0c5c919fe9fb9fb9d8fb2a6370b79f@preisshare.net>	<7EB1EF41-2863-4FF4-B6E5-3D7F33FBE18C@gmail.com>
	<87vdmu11g4.fsf@benfinney.id.au>
	<4A39EDA3.6020906@ar.media.kyoto-u.ac.jp>
Message-ID: <e3371a6c1a56725af62c4c7f364523d5@preisshare.net>


Hi All,

Here's a package installation error message that's just plain scary
for the average user.

imho Distutils should have checks to stock packages requiring a C
compiler from being uploaded. It isn't "right" to expect that
from users. ie They should get the .pyd files.

-----------------------------------------------------------------------------
Running installer ... C:\Python25\scripts\easy_install.exe appscript
Searching for appscript
Reading http://pypi.python.org/simple/appscript/

Reading http://freespace.virgin.net/hamish.sanderson/appscript.html

Reading http://appscript.sourceforge.net

Best match: appscript 0.19.0
Downloading
http://pypi.python.org/packages/source/a/appscript/appscript-0.19.0.tar.gz#md5=d234c1f1b92290da9eb866f78f9a7343

Processing appscript-0.19.0.tar.gz

Running appscript-0.19.0\setup.py -q bdist_egg --dist-dir
c:\docume~1\david\locals~1\temp\easy_install-lte00e\appscript-0.19.0\egg-dist-tmp-jw2svy

ERRORS: error: Setup script exited with error: Python was built with Visual
Studio 2003;
extensions must be built with a compiler than can generate compatible
binaries.
Visual Studio 2003 was not found on this system. If you have Cygwin
installed,
you can try compiling with MingW32, by passing "-c mingw32" to setup.py.



From ziade.tarek at gmail.com  Thu Jun 18 10:41:16 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 18 Jun 2009 10:41:16 +0200
Subject: [Distutils] more easyinstall error messages - haha
In-Reply-To: <56dfbddc42a9e15ffbe3f67937b25e21@preisshare.net>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>
	<874oue2ti2.fsf@benfinney.id.au>
	<da0c5c919fe9fb9fb9d8fb2a6370b79f@preisshare.net>
	<7EB1EF41-2863-4FF4-B6E5-3D7F33FBE18C@gmail.com>
	<87vdmu11g4.fsf@benfinney.id.au>
	<4A39EDA3.6020906@ar.media.kyoto-u.ac.jp>
	<56dfbddc42a9e15ffbe3f67937b25e21@preisshare.net>
Message-ID: <94bdd2610906180141r45903c9drffb89e2f8d91a6fd@mail.gmail.com>

On Thu, Jun 18, 2009 at 10:17 AM, David Lyon <david.lyon at preisshare.net>wrote:

>
> This is kind of typical of what I get all the time...
>
> Platform: XP-Python 2.5
>
>
> -----------------------------------------------------------------------------
> Running installer ... C:\Python25\scripts\easy_install.exe asciiporn
> Searching for asciiporn
> ...
>
> Exception: asciiporn requires Python 2.6 or higher
>
> Processing asciiporn-2009.05.01.tar.gz
> Running asciiporn-2009.05.01\setup.py -q bdist_egg --dist-dir
>
> c:\docume~1\david\locals~1\temp\easy_install-pnlek8\asciiporn-2009.05.01\egg-dist-tmp-8p44m6
>

What's wrong with that ?

If you look at ascii porn setup.py file, it has an explicit test on the
python version:

...
if sys.version_info[1] < 6: raise Exception("asciiporn requires Python 2.6
or higher")
...

So you have to easy_install it on Python 2.6, not Python2.5, or to ask
asciiporn maintainer
to backport it to 2.5
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090618/0bc6f011/attachment.htm>

From ziade.tarek at gmail.com  Thu Jun 18 10:45:58 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 18 Jun 2009 10:45:58 +0200
Subject: [Distutils] imho Distutils should stop this from happening to
	poor users....
In-Reply-To: <e3371a6c1a56725af62c4c7f364523d5@preisshare.net>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>
	<874oue2ti2.fsf@benfinney.id.au>
	<da0c5c919fe9fb9fb9d8fb2a6370b79f@preisshare.net>
	<7EB1EF41-2863-4FF4-B6E5-3D7F33FBE18C@gmail.com>
	<87vdmu11g4.fsf@benfinney.id.au>
	<4A39EDA3.6020906@ar.media.kyoto-u.ac.jp>
	<e3371a6c1a56725af62c4c7f364523d5@preisshare.net>
Message-ID: <94bdd2610906180145g1f411858je8010e9eff018886@mail.gmail.com>

On Thu, Jun 18, 2009 at 10:26 AM, David Lyon <david.lyon at preisshare.net>wrote:

>
> Hi All,
>
> Here's a package installation error message that's just plain scary
> for the average user.
>
> imho Distutils should have checks to stock packages requiring a C
> compiler from being uploaded. It isn't "right" to expect that
> from users. ie They should get the .pyd files.


That's not distutils fault, neither easy_install one.

"appscript" should provide a binary distribution for Windows (it does for
Mac OS if you look at the pypi repo)
Since it doesn't, easy_install grabs the source distro, which is the best
behavior I can think of,
then tries to compile it.

The error message is pretty explicit by the way

Tarek
-- 
Tarek Ziad? | http://ziade.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090618/bde0dec4/attachment.htm>

From ziade.tarek at gmail.com  Thu Jun 18 11:02:30 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 18 Jun 2009 11:02:30 +0200
Subject: [Distutils] sdist default archive format
In-Reply-To: <874oue2ti2.fsf@benfinney.id.au>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>
	<874oue2ti2.fsf@benfinney.id.au>
Message-ID: <94bdd2610906180202u20e51baeo5cafaa44d412daa2@mail.gmail.com>

On Thu, Jun 18, 2009 at 3:57 AM, Ben Finney<ben+python at benfinney.id.au> wrote:
> Trent Mick <trentm at gmail.com> writes:
>
>> My preferences is for .zip (that is what I use for my packages), the
>> main reason is that Windows users can always unpack a .zip file. Often
>> that is not true for .tar.gz or .tar.bz2 files.
>
> The reverse is true on Unix. The tar-and-gzip format is extensively used
> so it is pretty much guaranteed to be readable on any Unix system; the
> Zip format does not have anywhere near the same level of guarantee.
>
> I would think the best solution is to use the tar format (since, as
> pointed out elsewhere, it has better support for symlinks and permission
> flags that are often important when distributing works), and ensure that
> distutils will include a tool for unpacking them with pure Python on
> those platforms without external support for the format.
>

+1

A end-user that uses easy_install ends up calling the stdlib tarfile
module to unpack the archives,
so it works out of the box.  I guess pip is also using that code
(unpack_archive in setuptools.archive_util)

Although, I would rather see a new high-level function in tarfile
itself, callable from the __main__ section,
to pack/unpack an archive provided in the arguments. (like the doctest
modules has).

Cheers
Tarek

From ziade.tarek at gmail.com  Thu Jun 18 11:05:08 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 18 Jun 2009 11:05:08 +0200
Subject: [Distutils] zip vs tar.gz at PyPI
Message-ID: <94bdd2610906180205s47513652t5f84dc4000e4e640@mail.gmail.com>

Hi Martin,

We are talking about the "preferred" default archive format in Distutils-SIG.

I was wondering if you could give us some statistics on the various
archives format used at PyPI.

Thanks

Regards
Tarek
-- 
Tarek Ziad? | http://ziade.org

From ronaldoussoren at mac.com  Thu Jun 18 11:05:25 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Thu, 18 Jun 2009 11:05:25 +0200
Subject: [Distutils] imho Distutils should stop this from happening
 to	poor users....
In-Reply-To: <94bdd2610906180145g1f411858je8010e9eff018886@mail.gmail.com>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>
	<874oue2ti2.fsf@benfinney.id.au>
	<da0c5c919fe9fb9fb9d8fb2a6370b79f@preisshare.net>
	<7EB1EF41-2863-4FF4-B6E5-3D7F33FBE18C@gmail.com>
	<87vdmu11g4.fsf@benfinney.id.au>
	<4A39EDA3.6020906@ar.media.kyoto-u.ac.jp>
	<e3371a6c1a56725af62c4c7f364523d5@preisshare.net>
	<94bdd2610906180145g1f411858je8010e9eff018886@mail.gmail.com>
Message-ID: <FA9B86AC-52E6-4F96-8F0C-5F9E9A039EC7@mac.com>


On 18 Jun, 2009, at 10:45, Tarek Ziad? wrote:

>
>
> On Thu, Jun 18, 2009 at 10:26 AM, David Lyon <david.lyon at preisshare.net 
> > wrote:
>
> Hi All,
>
> Here's a package installation error message that's just plain scary
> for the average user.
>
> imho Distutils should have checks to stock packages requiring a C
> compiler from being uploaded. It isn't "right" to expect that
> from users. ie They should get the .pyd files.
>
> That's not distutils fault, neither easy_install one.
>
> "appscript" should provide a binary distribution for Windows (it  
> does for Mac OS if you look at the pypi repo)
> Since it doesn't, easy_install grabs the source distro, which is the  
> best behavior I can think of,
> then tries to compile it.

I don't know why it or any other package "should" provide a binary  
distribution for Windows, having one would be nice but shouldn't be a  
hard requirement, ever.

This does point to an missing feature in distutils or setuptools  
though: neither distutils nor setuptools has a way to specify the  
supported platforms, at least not in a way that is actually used  
during installation.  It would be nice if the appscript author could  
add some metadata to his setup.py file that would result in a clear  
error-message when you try to install it on a platform other than  
MacOSX.

The only way to achieve that at the moment is to add some code to  
setup.py that explicitly raises an exception when you try to build it  
on an unsupported platform.

The same is true for the python version, pyobjc currently explicitly  
tests for python >= 2.3,<3.0 because other versions aren't supported  
and cause compile errors (at best, IIRC it will compile with python  
2.2 but cause interpreter crashes when you try to use it).

Ronald

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090618/90e38b1b/attachment-0001.htm>

From ziade.tarek at gmail.com  Thu Jun 18 11:22:47 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 18 Jun 2009 11:22:47 +0200
Subject: [Distutils] imho Distutils should stop this from happening to
	poor users....
In-Reply-To: <FA9B86AC-52E6-4F96-8F0C-5F9E9A039EC7@mac.com>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>
	<874oue2ti2.fsf@benfinney.id.au>
	<da0c5c919fe9fb9fb9d8fb2a6370b79f@preisshare.net>
	<7EB1EF41-2863-4FF4-B6E5-3D7F33FBE18C@gmail.com>
	<87vdmu11g4.fsf@benfinney.id.au>
	<4A39EDA3.6020906@ar.media.kyoto-u.ac.jp>
	<e3371a6c1a56725af62c4c7f364523d5@preisshare.net>
	<94bdd2610906180145g1f411858je8010e9eff018886@mail.gmail.com>
	<FA9B86AC-52E6-4F96-8F0C-5F9E9A039EC7@mac.com>
Message-ID: <94bdd2610906180222k2608fac9t5c574d21cf83635b@mail.gmail.com>

On Thu, Jun 18, 2009 at 11:05 AM, Ronald Oussoren<ronaldoussoren at mac.com> wrote:
> The only way to achieve that at the moment is to add some code to setup.py
> that explicitly raises an exception when you try to build it on an
> unsupported platform.

Or to provide binary releases because python version and os info are
set in the archive name.
But it's hackish.

> The same is true for the python version, pyobjc currently explicitly tests
> for python >= 2.3,<3.0 because other versions aren't supported and cause
> compile errors (at best, IIRC it will compile with python 2.2 but cause
> interpreter crashes when you try to use it).

Some Trove classifiers have been added for Python 3 for example,

Martin uses it to provide a "Python 3 packages" page at Pypi

it's "Programming Language :: Python :: 3" (You have other Python
versions as well)

There are also an "Operating System ::" classifier.

I don't like this very much though, I find it cryptic, and I don't
think  we can force people+installers to use those classifiers.

The other solution is to use the "Requires-Python" Metadadata in PEP
345 and add a "Requires-Platform" when someone
wants to mark a source distribution compatibility. And the RPC
Interface at pypi provides a method to retrieve them without
downloading the package

it's : release_data(package_name, version)

It would be up to easy_install and such tools, to test those values,
prior downloading the package.


-- 
Tarek Ziad? | http://ziade.org

From ronaldoussoren at mac.com  Thu Jun 18 11:25:25 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Thu, 18 Jun 2009 11:25:25 +0200
Subject: [Distutils] imho Distutils should stop this from happening
 to	poor users....
In-Reply-To: <94bdd2610906180222k2608fac9t5c574d21cf83635b@mail.gmail.com>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>
	<874oue2ti2.fsf@benfinney.id.au>
	<da0c5c919fe9fb9fb9d8fb2a6370b79f@preisshare.net>
	<7EB1EF41-2863-4FF4-B6E5-3D7F33FBE18C@gmail.com>
	<87vdmu11g4.fsf@benfinney.id.au>
	<4A39EDA3.6020906@ar.media.kyoto-u.ac.jp>
	<e3371a6c1a56725af62c4c7f364523d5@preisshare.net>
	<94bdd2610906180145g1f411858je8010e9eff018886@mail.gmail.com>
	<FA9B86AC-52E6-4F96-8F0C-5F9E9A039EC7@mac.com>
	<94bdd2610906180222k2608fac9t5c574d21cf83635b@mail.gmail.com>
Message-ID: <F70C37FD-2E85-48D6-BF6E-B7BD1C54FA0A@mac.com>


On 18 Jun, 2009, at 11:22, Tarek Ziad? wrote:

> On Thu, Jun 18, 2009 at 11:05 AM, Ronald Oussoren<ronaldoussoren at mac.com 
> > wrote:
>> The only way to achieve that at the moment is to add some code to  
>> setup.py
>> that explicitly raises an exception when you try to build it on an
>> unsupported platform.
>
> Or to provide binary releases because python version and os info are
> set in the archive name.
> But it's hackish.

A binary release that does nothing would indeed be hackish, and would  
be worse than the existing error because a binary package gives the  
impression that the platform is actually supported.

Ronald


From ziade.tarek at gmail.com  Thu Jun 18 11:29:23 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 18 Jun 2009 11:29:23 +0200
Subject: [Distutils] imho Distutils should stop this from happening to
	poor users....
In-Reply-To: <F70C37FD-2E85-48D6-BF6E-B7BD1C54FA0A@mac.com>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
	<da0c5c919fe9fb9fb9d8fb2a6370b79f@preisshare.net>
	<7EB1EF41-2863-4FF4-B6E5-3D7F33FBE18C@gmail.com>
	<87vdmu11g4.fsf@benfinney.id.au>
	<4A39EDA3.6020906@ar.media.kyoto-u.ac.jp>
	<e3371a6c1a56725af62c4c7f364523d5@preisshare.net>
	<94bdd2610906180145g1f411858je8010e9eff018886@mail.gmail.com>
	<FA9B86AC-52E6-4F96-8F0C-5F9E9A039EC7@mac.com>
	<94bdd2610906180222k2608fac9t5c574d21cf83635b@mail.gmail.com>
	<F70C37FD-2E85-48D6-BF6E-B7BD1C54FA0A@mac.com>
Message-ID: <94bdd2610906180229k5298b4cx3874964938e49aa@mail.gmail.com>

On Thu, Jun 18, 2009 at 11:25 AM, Ronald Oussoren<ronaldoussoren at mac.com> wrote:
>
> On 18 Jun, 2009, at 11:22, Tarek Ziad? wrote:
>
>> On Thu, Jun 18, 2009 at 11:05 AM, Ronald Oussoren<ronaldoussoren at mac.com>
>> wrote:
>>>
>>> The only way to achieve that at the moment is to add some code to
>>> setup.py
>>> that explicitly raises an exception when you try to build it on an
>>> unsupported platform.
>>
>> Or to provide binary releases because python version and os info are
>> set in the archive name.
>> But it's hackish.
>
> A binary release that does nothing would indeed be hackish, and would be
> worse than the existing error because a binary package gives the impression
> that the platform is actually supported.
>

IIRC if you provide only binary releases for supported platforms,
easy_install will  end up saying it didn't find the appropriate
distribution.

I'm not saying it's the way to go, but I know some people distribute
only bdist_egg to make sure their packages are
used with the right Python version.



> Ronald
>
>



-- 
Tarek Ziad? | http://ziade.org

From david.lyon at preisshare.net  Thu Jun 18 11:42:28 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 18 Jun 2009 05:42:28 -0400
Subject: [Distutils] more easyinstall error messages - haha
In-Reply-To: <94bdd2610906180141r45903c9drffb89e2f8d91a6fd@mail.gmail.com>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>
	<874oue2ti2.fsf@benfinney.id.au>
	<da0c5c919fe9fb9fb9d8fb2a6370b79f@preisshare.net>
	<7EB1EF41-2863-4FF4-B6E5-3D7F33FBE18C@gmail.com>
	<87vdmu11g4.fsf@benfinney.id.au>
	<4A39EDA3.6020906@ar.media.kyoto-u.ac.jp>
	<56dfbddc42a9e15ffbe3f67937b25e21@preisshare.net>
	<94bdd2610906180141r45903c9drffb89e2f8d91a6fd@mail.gmail.com>
Message-ID: <3a8fae1ee5d0263afeb5c0e2fb08f050@preisshare.net>

On Thu, 18 Jun 2009 10:41:16 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
wrote:
>> Exception: asciiporn requires Python 2.6 or higher
>>
>> Processing asciiporn-2009.05.01.tar.gz
>> Running asciiporn-2009.05.01\setup.py -q bdist_egg --dist-dir
>>
>>
c:\docume~1\david\locals~1\temp\easy_install-pnlek8\asciiporn-2009.05.01\egg-dist-tmp-8p44m6
>>
> 
> What's wrong with that ?

Well, for a start it probably shouldn't get downloaded....

I admit - I am picking nits...

But what's wrong with doing QA on what comes out of distutils...?

Why can't Distutils check that a package will work on multiple python
versions before uploading it?

Then users could have the packages on their platform of choice right?

(Just thinking of the users)

David


 

From david.lyon at preisshare.net  Thu Jun 18 11:48:05 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 18 Jun 2009 05:48:05 -0400
Subject: [Distutils] imho Distutils should stop this from happening to
 poor users....
In-Reply-To: <94bdd2610906180145g1f411858je8010e9eff018886@mail.gmail.com>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>
	<874oue2ti2.fsf@benfinney.id.au>
	<da0c5c919fe9fb9fb9d8fb2a6370b79f@preisshare.net>
	<7EB1EF41-2863-4FF4-B6E5-3D7F33FBE18C@gmail.com>
	<87vdmu11g4.fsf@benfinney.id.au>
	<4A39EDA3.6020906@ar.media.kyoto-u.ac.jp>
	<e3371a6c1a56725af62c4c7f364523d5@preisshare.net>
	<94bdd2610906180145g1f411858je8010e9eff018886@mail.gmail.com>
Message-ID: <f1f9d30c9882eab66e068573aa19d213@preisshare.net>

On Thu, 18 Jun 2009 10:45:58 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
wrote:
> That's not distutils fault, neither easy_install one.
> 
> "appscript" should provide a binary distribution for Windows (it does for
> Mac OS if you look at the pypi repo)
> Since it doesn't, easy_install grabs the source distro, which is the best
> behavior I can think of,
> then tries to compile it.
> 
> The error message is pretty explicit by the way

It's good that you are seeing the challenges from a user perspective.

These are only tests I am doing on random packages and it doesn't worry
me too much if they don't work.

But I think there is much to be done to smooth things out...

Assigning all blame to the developers isn't what I'm doing. As I am
finding it hard to build and test on all platforms myself. Needless
to say it is something that distutils imho should be aiming for.

David


From ziade.tarek at gmail.com  Thu Jun 18 11:52:02 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 18 Jun 2009 11:52:02 +0200
Subject: [Distutils] more easyinstall error messages - haha
In-Reply-To: <3a8fae1ee5d0263afeb5c0e2fb08f050@preisshare.net>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>
	<874oue2ti2.fsf@benfinney.id.au>
	<da0c5c919fe9fb9fb9d8fb2a6370b79f@preisshare.net>
	<7EB1EF41-2863-4FF4-B6E5-3D7F33FBE18C@gmail.com>
	<87vdmu11g4.fsf@benfinney.id.au>
	<4A39EDA3.6020906@ar.media.kyoto-u.ac.jp>
	<56dfbddc42a9e15ffbe3f67937b25e21@preisshare.net>
	<94bdd2610906180141r45903c9drffb89e2f8d91a6fd@mail.gmail.com>
	<3a8fae1ee5d0263afeb5c0e2fb08f050@preisshare.net>
Message-ID: <94bdd2610906180252l706fae3fj563191db0dfb1cc9@mail.gmail.com>

On Thu, Jun 18, 2009 at 11:42 AM, David Lyon<david.lyon at preisshare.net> wrote:
>
> Why can't Distutils check that a package will work on multiple python
> versions before uploading it?

you mean before uploading a distribution at pypi ? how do you automate that ?

From david.lyon at preisshare.net  Thu Jun 18 11:59:39 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 18 Jun 2009 05:59:39 -0400
Subject: [Distutils] more easyinstall error messages - haha
In-Reply-To: <94bdd2610906180252l706fae3fj563191db0dfb1cc9@mail.gmail.com>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>
	<874oue2ti2.fsf@benfinney.id.au>
	<da0c5c919fe9fb9fb9d8fb2a6370b79f@preisshare.net>
	<7EB1EF41-2863-4FF4-B6E5-3D7F33FBE18C@gmail.com>
	<87vdmu11g4.fsf@benfinney.id.au>
	<4A39EDA3.6020906@ar.media.kyoto-u.ac.jp>
	<56dfbddc42a9e15ffbe3f67937b25e21@preisshare.net>
	<94bdd2610906180141r45903c9drffb89e2f8d91a6fd@mail.gmail.com>
	<3a8fae1ee5d0263afeb5c0e2fb08f050@preisshare.net>
	<94bdd2610906180252l706fae3fj563191db0dfb1cc9@mail.gmail.com>
Message-ID: <3676fcccbf03027c21b2302fafb05b7c@preisshare.net>

On Thu, 18 Jun 2009 11:52:02 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
wrote:
> On Thu, Jun 18, 2009 at 11:42 AM, David Lyon>
> wrote:
>>
>> Why can't Distutils check that a package will work on multiple python
>> versions before uploading it?
> 
> you mean before uploading a distribution at pypi ? how do you automate
that
> ?

Easily(*).....

in a script... 

for pyver in installed_python_versions:
    run_all_tests_on(pyver)

eg.. A 2.6 python can fire off a 2.2 python script.... which can run lots
of
tests on the 2.2 system....

David



From ziade.tarek at gmail.com  Thu Jun 18 12:19:13 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 18 Jun 2009 12:19:13 +0200
Subject: [Distutils] more easyinstall error messages - haha
In-Reply-To: <3676fcccbf03027c21b2302fafb05b7c@preisshare.net>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
	<da0c5c919fe9fb9fb9d8fb2a6370b79f@preisshare.net>
	<7EB1EF41-2863-4FF4-B6E5-3D7F33FBE18C@gmail.com>
	<87vdmu11g4.fsf@benfinney.id.au>
	<4A39EDA3.6020906@ar.media.kyoto-u.ac.jp>
	<56dfbddc42a9e15ffbe3f67937b25e21@preisshare.net>
	<94bdd2610906180141r45903c9drffb89e2f8d91a6fd@mail.gmail.com>
	<3a8fae1ee5d0263afeb5c0e2fb08f050@preisshare.net>
	<94bdd2610906180252l706fae3fj563191db0dfb1cc9@mail.gmail.com>
	<3676fcccbf03027c21b2302fafb05b7c@preisshare.net>
Message-ID: <94bdd2610906180319t6ba53ff7l2611efda8835d71b@mail.gmail.com>

On Thu, Jun 18, 2009 at 11:59 AM, David Lyon<david.lyon at preisshare.net> wrote:
> On Thu, 18 Jun 2009 11:52:02 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
> wrote:
>> On Thu, Jun 18, 2009 at 11:42 AM, David Lyon>
>> wrote:
>>>
>>> Why can't Distutils check that a package will work on multiple python
>>> versions before uploading it?
>>
>> you mean before uploading a distribution at pypi ? how do you automate
> that
>> ?
>
> Easily(*).....
>
> in a script...
>
> for pyver in installed_python_versions:
> ? ?run_all_tests_on(pyver)
>
> eg.. A 2.6 python can fire off a 2.2 python script.... which can run lots
> of
> tests on the 2.2 system....

I'm lost now :)

What do you want to do exaclty ?

Let's suppose you have all your code 100% covered in tests (wich is
not the case in 99.99% of the packages).
So you run the test using all Python versions (which supposes you have
*all* Python versions installed, which is not the case on 99% of the
people systems out ther)

Then what happens ? Do you store the result in the metadata ? Like, by
setting up the Requires-Python field with the output ?

Frankly, you could just explicitely mark the Python version your code
is suppose to run with,

From david.lyon at preisshare.net  Thu Jun 18 13:54:35 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 18 Jun 2009 07:54:35 -0400
Subject: [Distutils] more easyinstall error messages - haha
In-Reply-To: <94bdd2610906180319t6ba53ff7l2611efda8835d71b@mail.gmail.com>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
	<da0c5c919fe9fb9fb9d8fb2a6370b79f@preisshare.net>
	<7EB1EF41-2863-4FF4-B6E5-3D7F33FBE18C@gmail.com>
	<87vdmu11g4.fsf@benfinney.id.au>
	<4A39EDA3.6020906@ar.media.kyoto-u.ac.jp>
	<56dfbddc42a9e15ffbe3f67937b25e21@preisshare.net>
	<94bdd2610906180141r45903c9drffb89e2f8d91a6fd@mail.gmail.com>
	<3a8fae1ee5d0263afeb5c0e2fb08f050@preisshare.net>
	<94bdd2610906180252l706fae3fj563191db0dfb1cc9@mail.gmail.com>
	<3676fcccbf03027c21b2302fafb05b7c@preisshare.net>
	<94bdd2610906180319t6ba53ff7l2611efda8835d71b@mail.gmail.com>
Message-ID: <43fbc0ede50839cb4122067af9e80cb4@preisshare.net>

On Thu, 18 Jun 2009 12:19:13 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
wrote:
> Let's suppose you have all your code 100% covered in tests (wich is
> not the case in 99.99% of the packages).

Sure. Ok, developers don't write enough tests...

They don't have time... it's not fun.... can't be bothered... 

But lets be realistic, there's too many python versions now...

too many platforms...

why can't we do automated builds/tests to cut down the work for
developers...

> So you run the test using all Python versions (which supposes you have
> *all* Python versions installed, which is not the case on 99% of the
> people systems out ther)

Think of it like this....

We give python 3.1 the crown....

and write the code for that, that can regressively test any code
backwards a package on all the old python versions..

thus, our latest python version... has way more power than any 
previous version...

> Then what happens ? Do you store the result in the metadata ? Like, by
> setting up the Requires-Python field with the output ?

I didn't think of that... it's an excellent idea...

Think of how much work that could save...

> Frankly, you could just explicitely mark the Python version your code
> is suppose to run with,

but it would be more fun as you say if it was determined programmatically
by distutils based on running some analysis of the actual code....

David

From cz at gocept.com  Thu Jun 18 15:20:31 2009
From: cz at gocept.com (Christian Zagrodnick)
Date: Thu, 18 Jun 2009 15:20:31 +0200
Subject: [Distutils] Setuptools + Subversion 1.6 - new setuptools
	release necessary
References: <49F6FBC1.8080001@simplistix.co.uk> <49F6FD73.10809@zopyx.com>
	<4A013739.6020105@zopyx.com>
	<99d67a00-c550-4b8f-9914-a7348a587b1e@k19g2000prh.googlegroups.com>
Message-ID: <h1deuv$bcu$1@ger.gmane.org>

On 2009-05-07 22:18:35 +0200, Wheat <wheatix at gmail.com> said:

> 
> 
> On May 6, 12:07?am, Andreas Jung <li... at zopyx.com> wrote:
>> On 28.04.09 14:58, Andreas Jung wrote:
>> 
>>> On 28.04.2009 14:51 Uhr, Chris Withers wrote:
>> 
>>>> Andreas Jung wrote:
>> 
>>>>> it is known that the latest setuptools version produces broken
>>>>> packages with SVN 1.6 checkouts. Could we get a fixed setuptools
>>>>> version asap - fixing this issue is essential
>>>>> (there is some patch floating around).
>> 
>> There is obviously a bugreport and patches available for this issue:
>> 
>> http://bugs.python.org/setuptools/issue64
>> 
>> Any chance to see this patch being merged and released together
>> with a new setuptools version soon?
>> 
> 
> *bump*, *bump*
> 
> Yeah, I spent a good amount of time yesterday banging my head against
> broken sdists after recently upgrading to SVN 1.6. A new setuptools
> release would be much appreciated!

Indeed. Always the same story. svn 1.6 was released in *March*.


-- 
Christian Zagrodnick ? cz at gocept.com
gocept gmbh & co. kg ? forsterstra?e 29 ? 06112 halle (saale) ? germany
http://gocept.com ? tel +49 345 1229889 4 ? fax +49 345 1229889 1
Zope and Plone consulting and development



From santagada at gmail.com  Thu Jun 18 17:02:24 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Thu, 18 Jun 2009 12:02:24 -0300
Subject: [Distutils] Autobuild packages using snakebite
Message-ID: <7158858D-8587-4EF7-A18B-D7450A4A2953@gmail.com>

I remember when snakebite was announced someone had the idea to have a  
kind of build bot go into pypi and get all packages sent to it and  
automatically compile them on snakebite for all major platforms and  
automatically run tests and submit the compiled packages to pypi. Is  
anyone working on this?

The biggest problem I see is security, but if people are really  
interested in this we could at least try it no?

Automatic testing in many python versions, conversion using 2to3 and  
building packages for windows and macosx seems like a very interesting  
service.

--
Leonardo Santagada
santagada at gmail.com




From a.cavallo at mailsnare.com  Thu Jun 18 16:27:33 2009
From: a.cavallo at mailsnare.com (A. Cavallo)
Date: Thu, 18 Jun 2009 15:27:33 +0100
Subject: [Distutils] more easyinstall error messages - haha
In-Reply-To: <43fbc0ede50839cb4122067af9e80cb4@preisshare.net>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
	<94bdd2610906180319t6ba53ff7l2611efda8835d71b@mail.gmail.com>
	<43fbc0ede50839cb4122067af9e80cb4@preisshare.net>
Message-ID: <200906181527.33566.a.cavallo@mailsnare.com>

Please have a look to my pet project:

http://pyvm.sourceforge.net

It solves all the problem you could possibly imagine (famine included!):

* provides binary build - ADMINISTRATOR FRIENDLY
* could guarantee a reproducible build
* integrates smoke tests (just the begin of a continuous integration system)


Regards,
Antonio


> On Thu, 18 Jun 2009 12:19:13 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
>
> wrote:
> > Let's suppose you have all your code 100% covered in tests (wich is
> > not the case in 99.99% of the packages).
>
> Sure. Ok, developers don't write enough tests...
>
> They don't have time... it's not fun.... can't be bothered...
>
> But lets be realistic, there's too many python versions now...
>
> too many platforms...
>
> why can't we do automated builds/tests to cut down the work for
> developers...
>
> > So you run the test using all Python versions (which supposes you have
> > *all* Python versions installed, which is not the case on 99% of the
> > people systems out ther)
>
> Think of it like this....
>
> We give python 3.1 the crown....
>
> and write the code for that, that can regressively test any code
> backwards a package on all the old python versions..
>
> thus, our latest python version... has way more power than any
> previous version...
>
> > Then what happens ? Do you store the result in the metadata ? Like, by
> > setting up the Requires-Python field with the output ?
>
> I didn't think of that... it's an excellent idea...
>
> Think of how much work that could save...
>
> > Frankly, you could just explicitely mark the Python version your code
> > is suppose to run with,
>
> but it would be more fun as you say if it was determined programmatically
> by distutils based on running some analysis of the actual code....
>
> David
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig

From a.cavallo at mailsnare.com  Thu Jun 18 17:18:04 2009
From: a.cavallo at mailsnare.com (A. Cavallo)
Date: Thu, 18 Jun 2009 16:18:04 +0100
Subject: [Distutils] Autobuild packages using snakebite
In-Reply-To: <7158858D-8587-4EF7-A18B-D7450A4A2953@gmail.com>
References: <7158858D-8587-4EF7-A18B-D7450A4A2953@gmail.com>
Message-ID: <200906181618.04235.a.cavallo@mailsnare.com>

I'm currently working on an automated build system with similar goals:

http://pyvm.sourceforge.net

Regards,
Antonio
> I remember when snakebite was announced someone had the idea to have a
> kind of build bot go into pypi and get all packages sent to it and
> automatically compile them on snakebite for all major platforms and
> automatically run tests and submit the compiled packages to pypi. Is
> anyone working on this?
>
> The biggest problem I see is security, but if people are really
> interested in this we could at least try it no?
>
> Automatic testing in many python versions, conversion using 2to3 and
> building packages for windows and macosx seems like a very interesting
> service.
>
> --
> Leonardo Santagada
> santagada at gmail.com
>
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig

From santagada at gmail.com  Thu Jun 18 18:43:35 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Thu, 18 Jun 2009 13:43:35 -0300
Subject: [Distutils] Autobuild packages using snakebite
In-Reply-To: <200906181618.04235.a.cavallo@mailsnare.com>
References: <7158858D-8587-4EF7-A18B-D7450A4A2953@gmail.com>
	<200906181618.04235.a.cavallo@mailsnare.com>
Message-ID: <BE717DD5-EFF7-4FCA-9430-84A20A9DF4F1@gmail.com>



On Jun 18, 2009, at 12:18 PM, A. Cavallo wrote:

>> I remember when snakebite was announced someone had the idea to  
>> have a
>> kind of build bot go into pypi and get all packages sent to it and
>> automatically compile them on snakebite for all major platforms and
>> automatically run tests and submit the compiled packages to pypi. Is
>> anyone working on this?
>>
>> The biggest problem I see is security, but if people are really
>> interested in this we could at least try it no?
>>
>> Automatic testing in many python versions, conversion using 2to3 and
>> building packages for windows and macosx seems like a very  
>> interesting
>> service.

> I'm currently working on an automated build system with similar goals:
>
> http://pyvm.sourceforge.net
>
> Regards,
> Antonio


What I proposed was a autobuild for pytho packages on pypi not cpython  
builds. The system could certanly use your tool as a way to have many  
python versions, but in the end I think people would want packages  
built by cpython binaries distributed at python.org and third party  
distros (like ubuntu, mac os x, macports, etc).

--
Leonardo Santagada
santagada at gmail.com




From ziade.tarek at gmail.com  Thu Jun 18 18:54:31 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 18 Jun 2009 18:54:31 +0200
Subject: [Distutils] Autobuild packages using snakebite
In-Reply-To: <7158858D-8587-4EF7-A18B-D7450A4A2953@gmail.com>
References: <7158858D-8587-4EF7-A18B-D7450A4A2953@gmail.com>
Message-ID: <94bdd2610906180954j3aebf1c3s4a58016f2faee085@mail.gmail.com>

On Thu, Jun 18, 2009 at 5:02 PM, Leonardo Santagada <santagada at gmail.com>wrote:

> I remember when snakebite was announced someone had the idea to have a kind
> of build bot go into pypi and get all packages sent to it and automatically
> compile them on snakebite for all major platforms and automatically run
> tests and submit the compiled packages to pypi. Is anyone working on this?


I mentioned that need a while ago and work on a project see (
http://tarekziade.wordpress.com/2009/05/28/smoke-testing-in-distutils), and
I made a buildbot doing smoke tests : I have it running for a few packages
here http://buildbot.ziade.org/waterfall on one slave so far.

I helps me on making sure I don't break distutils :)

I know Titus wants to do something similar as well, and I think Antonio
project is adressing another need.


>
> The biggest problem I see is security, but if people are really interested
> in this we could at least try it no?


I have a white list of packages so far. A complete system needs to run in a
virtual environment and scratch it everytime
a package is tested + cut the network access.


>
> Automatic testing in many python versions, conversion using 2to3 and
> building packages for windows and macosx seems like a very interesting
> service.
>


I am calling "python setup.py sdist" and "python setup.py bdist" so far and
compare the output on various python versions,
but any command would work. If you are interested in writing some scripts,
you are welcome to join, it's an open project
located here http://bitbucket.org/tarek/distutils-builbot/src/

Tarek

-- 
Tarek Ziad? | http://ziade.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090618/68b34cc1/attachment.htm>

From sridharr at activestate.com  Thu Jun 18 20:40:00 2009
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Thu, 18 Jun 2009 11:40:00 -0700
Subject: [Distutils] zip vs tar.gz at PyPI
In-Reply-To: <94bdd2610906180205s47513652t5f84dc4000e4e640@mail.gmail.com>
References: <94bdd2610906180205s47513652t5f84dc4000e4e640@mail.gmail.com>
Message-ID: <4A3A8A00.2030302@activestate.com>


On 09-06-18 02:05 AM, Tarek Ziad? wrote:
> Hi Martin,
>
> We are talking about the "preferred" default archive format in Distutils-SIG.
>
> I was wondering if you could give us some statistics on the various
> archives format used at PyPI.

I have a local PyPI mirror .. and it seems there are about 15000 
tar.gz/tgz files .. while only 2000 zip files.

From zooko at zooko.com  Thu Jun 18 22:02:05 2009
From: zooko at zooko.com (Zooko Wilcox-O'Hearn)
Date: Thu, 18 Jun 2009 14:02:05 -0600
Subject: [Distutils] sdist default archive format
In-Reply-To: <6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>
Message-ID: <A84E15AC-9524-4D52-B62B-E2BDB8D0DBAE@zooko.com>

On Jun 17, 2009, at 15:07 PM, Trent Mick wrote:

> The tradeoff is that .zip files are compressed as well. But I don't  
> believe that difference in compression size  
> between .zip/.tar.gz/.tar.bz2 is that big of a concern in *most*  
> cases because most Python sdist's are small, are they not?

My project is 2.5 MB in .zip, 2.25 MB in .tar.bz2, and 1.8 MB  
in .tar.xz.  I would be happy to save that 700 KB by moving from .zip  
to .tar.xz (once there is LZMA-tar in Python), but I'm more concerned  
about the permissions issues, which are still an unsolved problem in  
setuptools, e.g.: http://bugs.python.org/setuptools/issue6

Regards,

Zooko

  $ ls -lS
-rw-rw-r-- 1 zooko zooko 8355840 2009-06-17 09:48 allmydata- 
tahoe-1.4.1-r3916.tar
-rw-rw-r-- 1 zooko zooko 2562835 2009-06-17 09:51 allmydata- 
tahoe-1.4.1-r3916.zip
-rw-rw-r-- 1 zooko zooko 2383653 2009-06-17 09:48 allmydata- 
tahoe-1.4.1-r3916.tar.gz
-rw-rw-r-- 1 zooko zooko 2250149 2009-06-17 09:49 allmydata- 
tahoe-1.4.1-r3916.tar.bz2
-rw-rw-r-- 1 zooko zooko 2223425 2009-06-17 09:49 allmydata- 
tahoe-1.4.1-r3916.tar.rz
-rw-rw-r-- 1 zooko zooko 1818466 2009-06-17 09:52 allmydata- 
tahoe-1.4.1-r3916.7z
-rw-rw-r-- 1 zooko zooko 1811698 2009-06-17 09:50 allmydata- 
tahoe-1.4.1-r3916.tar.xz

From sridharr at activestate.com  Thu Jun 18 22:27:47 2009
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Thu, 18 Jun 2009 13:27:47 -0700
Subject: [Distutils] PEP 376 - .egg-info directory name separator
Message-ID: <4A3AA343.6030000@activestate.com>

Hi Tarek,

While I was reading this PEP, I noticed that it suggests the '-' 
character as separator (between name, version, etc..) .. naturally 
introducing '_' *within* names and versions.

   'python_ldap-2.5.a_5.egg_info'

This decision looks odd to me, as several package systems I know of use 
the '_' character as seperator. In Debian,

   'python-ldap_2.3.5-1ubuntu1_i386.deb'

Besides, it is very common for Python packages to use '-' in their names 
instead of '_'.

   [.. pypimirror]$ ls | grep "_" | wc -l
   200
   [.. pypimirror]$ ls | grep "-" | wc -l
   628

Why not use '_' as separator?

Cheers,
Sridhar

From a.cavallo at mailsnare.com  Fri Jun 19 00:31:00 2009
From: a.cavallo at mailsnare.com (A. Cavallo)
Date: Thu, 18 Jun 2009 23:31:00 +0100
Subject: [Distutils] Autobuild packages using snakebite
In-Reply-To: <94bdd2610906180954j3aebf1c3s4a58016f2faee085@mail.gmail.com>
References: <7158858D-8587-4EF7-A18B-D7450A4A2953@gmail.com>
	<94bdd2610906180954j3aebf1c3s4a58016f2faee085@mail.gmail.com>
Message-ID: <200906182331.01028.a.cavallo@mailsnare.com>

Ok,
I've been quiet for a little while fixing few things on the pyvm project...
so sorry for this following very long email.

Pyvm is my (personal at the moment) attempt to use what opensuse team has come 
up with: the opensuse build server.

This service (free) provides a continuous integration system as many 
commercial projects are actively using: having some first hand experience I 
can tell you is quite an effective method.

The basics are very simple:

1. the user/developer puts some files in a repository (actually a "svn" 
repository)

2. The scheduler builds a xen system image in a well known status (so 
guaranteed repeatable) 

3. The scheduler then runs the build using the provided files including the 
smoke tests, collects the resulting rpms and publish them on line

The advantage on buildbot are many fold:

1. There is a guarantee the resulting binaries are untampered (using the 
standard libraries provided by the linux distributions not the one the 
developer has installed on his box) 

2. Hands off build, so no funny hacks that work on some machines/distributions 
but not on other (yes, it enforces a development discipline)

3. It supports a variety of linux distribution in parallel (so far I've 
addressed the lsb standard so using the rpm)

4. It make possible for an administrator to have many python interpreter 
versions side by side (believe or not there're companies still using python 
2.3 because is the one shipped with red hat enterprise linux 4!)

5. The system images are network disconnected

While build bot is fine for personal use, opensuse is a step ahead providing 
build that can be used by the users and, by the way, this system is the one 
that builds opensuse.

Osc (the susebuild client) can provide also a local build in the same spirit 
of buildbot: the only missing bit (at the moment) is the reporting part.


Now the pyvm project.
--------------------------------

There are really two different subprojects: the intepreters and the modules.

Now the interpreters are taken from the svn trunk and rebuild for a pletora of 
linux distributions and architectures. There's a minimal smoke test and few 
patches applied.
A user can point its yum/yast package manager and download the python 
intepreter for each subversion checkout (or doing it by downloading the rpms).

Then there is the more interesting part for developers: the modules.
Basically they are few python packages build against each python subversion 
check out so the building chain is:

 opt-python-svn73450 ->  setuptools-latest-svn73450 -> sphinx-0.6.1-svn73450

If something is broken everything will fail: this process is automatic(-ish) 
so I can manage a failry large set of package already.

My use case for this is:

A module developer creates his masterpiece then, if it is worth, gets an 
account on opensuse.
He sends the tarball, a spec file and the eventual patch files and waits for 
the build to happen. 
If fails he has to repeat it again: the first time will be painful, but with 
time it will improve and the overall quality will be much better.

There are many more use cases pyvm address, but it will bore people too much.

I hope this clarifies a little bit what is my aim with pyvm.
I wish this approach will be adopted because I can see clearly the benefits of 
it in terms of quality control (I'm not promoting here pyvm but the service 
opensuse is providing for free).

Regards,
Antonio













> On Thu, Jun 18, 2009 at 5:02 PM, Leonardo Santagada 
<santagada at gmail.com>wrote:
> > I remember when snakebite was announced someone had the idea to have a
> > kind of build bot go into pypi and get all packages sent to it and
> > automatically compile them on snakebite for all major platforms and
> > automatically run tests and submit the compiled packages to pypi. Is
> > anyone working on this?
>
> I mentioned that need a while ago and work on a project see (
> http://tarekziade.wordpress.com/2009/05/28/smoke-testing-in-distutils), and
> I made a buildbot doing smoke tests : I have it running for a few packages
> here http://buildbot.ziade.org/waterfall on one slave so far.
>
> I helps me on making sure I don't break distutils :)
>
> I know Titus wants to do something similar as well, and I think Antonio
> project is adressing another need.
>
> > The biggest problem I see is security, but if people are really
> > interested in this we could at least try it no?
>
> I have a white list of packages so far. A complete system needs to run in a
> virtual environment and scratch it everytime
> a package is tested + cut the network access.
>
> > Automatic testing in many python versions, conversion using 2to3 and
> > building packages for windows and macosx seems like a very interesting
> > service.
>
> I am calling "python setup.py sdist" and "python setup.py bdist" so far and
> compare the output on various python versions,
> but any command would work. If you are interested in writing some scripts,
> you are welcome to join, it's an open project
> located here http://bitbucket.org/tarek/distutils-builbot/src/
>
> Tarek

From david.lyon at preisshare.net  Fri Jun 19 01:18:21 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 18 Jun 2009 19:18:21 -0400
Subject: [Distutils] sdist default archive format
In-Reply-To: <A84E15AC-9524-4D52-B62B-E2BDB8D0DBAE@zooko.com>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>
	<A84E15AC-9524-4D52-B62B-E2BDB8D0DBAE@zooko.com>
Message-ID: <e4dd18fa8673ee0d8eb9f19bbd60fe34@preisshare.net>

On Thu, 18 Jun 2009 14:02:05 -0600, Zooko Wilcox-O'Hearn <zooko at zooko.com>
wrote:
> On Jun 17, 2009, at 15:07 PM, Trent Mick wrote:
> 
>> The tradeoff is that .zip files are compressed as well. But I don't  
>> believe that difference in compression size  
>> between .zip/.tar.gz/.tar.bz2 is that big of a concern in *most*  
>> cases because most Python sdist's are small, are they not?
> 
> My project is 2.5 MB in .zip, 2.25 MB in .tar.bz2, and 1.8 MB  
> in .tar.xz.  I would be happy to save that 700 KB by moving from .zip  
> to .tar.xz (once there is LZMA-tar in Python), but I'm more concerned  
> about the permissions issues, which are still an unsolved problem in  
> setuptools, e.g.: http://bugs.python.org/setuptools/issue6

But setup tools looks like it is trying to do the "right" thing
and getting it wrong...

10% difference in library size is nothing.

Somebody obviously needs to look into the bug...

David

From pje at telecommunity.com  Fri Jun 19 01:36:05 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 18 Jun 2009 19:36:05 -0400
Subject: [Distutils] PEP 376 - .egg-info directory name separator
In-Reply-To: <4A3AA343.6030000@activestate.com>
References: <4A3AA343.6030000@activestate.com>
Message-ID: <20090618233310.457C03A404D@sparrow.telecommunity.com>

At 01:27 PM 6/18/2009 -0700, Sridhar Ratnakumar wrote:
>Why not use '_' as separator?

Backward compatibility. 


From david.lyon at preisshare.net  Fri Jun 19 01:57:58 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 18 Jun 2009 19:57:58 -0400
Subject: [Distutils] Autobuild packages using snakebite
In-Reply-To: <7158858D-8587-4EF7-A18B-D7450A4A2953@gmail.com>
References: <7158858D-8587-4EF7-A18B-D7450A4A2953@gmail.com>
Message-ID: <a0984a0a1818c4e27567bbda4e5198fd@preisshare.net>

On Thu, 18 Jun 2009 12:02:24 -0300, Leonardo Santagada
<santagada at gmail.com>
wrote:
> I remember when snakebite was announced someone had the idea to have a  
> kind of build bot go into pypi and get all packages sent to it and  
> automatically compile them on snakebite for all major platforms and  
> automatically run tests and submit the compiled packages to pypi. Is  
> anyone working on this?

I think I suggested that.. 

I followed it up with Snakebite. They said their facilities weren't
operational and they suggested there might be a cost to doing it.. 

They gave me the impression that they wanted to control the coding...

> The biggest problem I see is security, but if people are really  
> interested in this we could at least try it no?

It's not that. It's that the whole thing starts to get really
huge. Really fast...

And in my mind, it starts to look like a commercial service at
some point. Because perphaps developers might find it worthwhile
to donate $20 to test their apps on multiple platforms. I say
this because not many mac users are happy to lend or run
scripts from "those other people for free". And fair enough
too because those machines are not cheap.

Then you have the "free" side... yes.. I still believe that
someone somewhere should be running tests on pypi on the
packages. But that shouldn't cost anybody else. It should
just be built in to the pypi facilities.

Firebird, an open source dbms (http://www.firebirdsql.org/) do 
their open source project very well via a non-profit foundation. 
They just take donations and then direct the funds to paying for
whatever they want done.

Last but not least is the programming... which isn't such a
big deal. But it takes time, effort and dedication. Even though
I regard it as easy* it isn't trivial... :-)

> Automatic testing in many python versions, conversion using 2to3 and  
> building packages for windows and macosx seems like a very interesting  
> service.

yep... sure is...

David



From stefan_ml at behnel.de  Fri Jun 19 07:24:21 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 19 Jun 2009 07:24:21 +0200
Subject: [Distutils] Autobuild packages using snakebite
In-Reply-To: <7158858D-8587-4EF7-A18B-D7450A4A2953@gmail.com>
References: <7158858D-8587-4EF7-A18B-D7450A4A2953@gmail.com>
Message-ID: <h1f7e6$kg4$1@ger.gmane.org>

Leonardo Santagada wrote:
> The biggest problem I see is security, but if people are really
> interested in this we could at least try it no?

Security certainly is a major issue here. Anyone can upload packages to
PyPI, so you can run arbitrary code on tons of machines, just by pushing
some well-forged setup.py script there.

Stefan


From david at ar.media.kyoto-u.ac.jp  Fri Jun 19 07:17:34 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Fri, 19 Jun 2009 14:17:34 +0900
Subject: [Distutils] Autobuild packages using snakebite
In-Reply-To: <h1f7e6$kg4$1@ger.gmane.org>
References: <7158858D-8587-4EF7-A18B-D7450A4A2953@gmail.com>
	<h1f7e6$kg4$1@ger.gmane.org>
Message-ID: <4A3B1F6E.907@ar.media.kyoto-u.ac.jp>

Stefan Behnel wrote:
> Leonardo Santagada wrote:
>   
>> The biggest problem I see is security, but if people are really
>> interested in this we could at least try it no?
>>     
>
> Security certainly is a major issue here. Anyone can upload packages to
> PyPI, so you can run arbitrary code on tons of machines, just by pushing
> some well-forged setup.py script there.
>   

Since it would be inside a VM, the major risk would be running some kind
of malicious server of some kind inside the setup script - but it should
be relatively easy to make sure the vm prevents that from happening ?

It is a major issue, but I would guess it has been solved by the build
service system (which is really great BTW, I think it is a very
significant advancement, under-rated project for open source software
deployment).

cheers,

David

From david.lyon at preisshare.net  Fri Jun 19 16:03:40 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Fri, 19 Jun 2009 10:03:40 -0400
Subject: [Distutils] Autobuild packages using snakebite
In-Reply-To: <h1f7e6$kg4$1@ger.gmane.org>
References: <7158858D-8587-4EF7-A18B-D7450A4A2953@gmail.com>
	<h1f7e6$kg4$1@ger.gmane.org>
Message-ID: <1ebcb2879ff07da133846cfd42b606f4@preisshare.net>

On Fri, 19 Jun 2009 07:24:21 +0200, Stefan Behnel <stefan_ml at behnel.de>
wrote:
> Leonardo Santagada wrote:
>> The biggest problem I see is security, but if people are really
>> interested in this we could at least try it no?
> 
> Security certainly is a major issue here. Anyone can upload packages to
> PyPI, so you can run arbitrary code on tons of machines, just by pushing
> some well-forged setup.py script there.

Doesn't a chroot jail stop this? (on unix anyway)



From humanist.steve at gmail.com  Thu Jun 18 23:55:57 2009
From: humanist.steve at gmail.com (Steve Bonner)
Date: Thu, 18 Jun 2009 17:55:57 -0400
Subject: [Distutils] setuptools for Python 3
Message-ID: <6e6ae6530906181455k17e3883cy1a311b854506c8e6@mail.gmail.com>

Hello.  Could you please tell me what your schedule is for
upgrading setuptools and easy_install to support Python 3?
Do you know what version of those packages will support
Python 3, and about what timeframe?

Thanks,
Steve Bonner
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090618/26fd5207/attachment.htm>

From stefan_ml at behnel.de  Fri Jun 19 16:28:48 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 19 Jun 2009 16:28:48 +0200 (CEST)
Subject: [Distutils] Autobuild packages using snakebite
In-Reply-To: <1ebcb2879ff07da133846cfd42b606f4@preisshare.net>
References: <7158858D-8587-4EF7-A18B-D7450A4A2953@gmail.com>
	<h1f7e6$kg4$1@ger.gmane.org>
	<1ebcb2879ff07da133846cfd42b606f4@preisshare.net>
Message-ID: <7e3853f2854fe6848d2c4825ec0c147e.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Hi,

this is really getting off-topic, BTW.

David Lyon wrote:
> On Fri, 19 Jun 2009 07:24:21 +0200, Stefan Behnel wrote:
>> Leonardo Santagada wrote:
>>> The biggest problem I see is security, but if people are really
>>> interested in this we could at least try it no?
>>
>> Security certainly is a major issue here. Anyone can upload packages to
>> PyPI, so you can run arbitrary code on tons of machines, just by pushing
>> some well-forged setup.py script there.
>
> Doesn't a chroot jail stop this? (on unix anyway)

Stop it from running arbitrary code? How would it do that?

Imagine your setup.py downloads Seti at Home jobs from a web site and
calculates them. You can't really ban web access, as the setup.py script
may use setuptools to download dependencies. You can't really stop it from
doing calculations, as it might actually be doing real build stuff like
source code generation. You can kill it after, say, an hour (or a half if
you think that no build should take longer than that), but I guess I can
do a lot of calculations in that time, on a lot of your computers. And if
you ban my package, I'll just upload a different one with a different
name. And if you ban my PyPI account, I'll grab a new one. And if you ban
web access from your machines, well, I'll just upload a package that
contains the tasks in the archive. And if you use hash codes to check for
malicious packages, well, I'll just start obfuscating my code and my data.
And be assured that what I'm really doing is not calculating Seti at Home
jobs.

There's really no way you can stop someone from misusing your
infrastructure if you go for building any arbitrary package that gets
uploaded to PyPI.

Stefan


From zooko at zooko.com  Fri Jun 19 18:38:39 2009
From: zooko at zooko.com (Zooko Wilcox-O'Hearn)
Date: Fri, 19 Jun 2009 10:38:39 -0600
Subject: [Distutils] sdist default archive format
In-Reply-To: <e4dd18fa8673ee0d8eb9f19bbd60fe34@preisshare.net>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>
	<A84E15AC-9524-4D52-B62B-E2BDB8D0DBAE@zooko.com>
	<e4dd18fa8673ee0d8eb9f19bbd60fe34@preisshare.net>
Message-ID: <7FD6E55A-1652-4024-85CF-C0CFA69F0F56@zooko.com>

On Jun 18, 2009, at 17:18 PM, David Lyon wrote:

> But setup tools looks like it is trying to do the "right" thing and  
> getting it wrong...

It isn't so easy for setuptools to know which things ought to have +x  
and which things ought not just based on their pathnames or other  
metadata.  There was a bug in setuptools on cygwin years ago where  
something didn't have +x that needed it, and that was fixed in  
setuptools-0.6c6 by adding +x to a variety of files.  That led to  
this new bug where something has +x that shouldn't:

http://bugs.python.org/setuptools/issue6

I can imagine that in general there might be a package which has two  
files named "foo/bar" and "foo/quux" where the package really needs  
bar to have +x and quux not to have +x.

Having an archive format that preserves such bits would probably be a  
good way to solve all such problems -- by making it the packagers  
problems to set the bits before packaging rather than setuptools's  
problem to figure out which bits ought to be set after installation.

> 10% difference in library size is nothing.

1.8 / 2.5 == 72%, so 28% of the .zip file size is saved by switching  
to .tar.xz.  Of course, some packages may gain more or less than 28%  
(compared to ZIP compression) from LZMA compression.

Regards,

Zooko

From santagada at gmail.com  Fri Jun 19 20:43:36 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Fri, 19 Jun 2009 15:43:36 -0300
Subject: [Distutils] Autobuild packages using snakebite
In-Reply-To: <7e3853f2854fe6848d2c4825ec0c147e.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
References: <7158858D-8587-4EF7-A18B-D7450A4A2953@gmail.com>
	<h1f7e6$kg4$1@ger.gmane.org>
	<1ebcb2879ff07da133846cfd42b606f4@preisshare.net>
	<7e3853f2854fe6848d2c4825ec0c147e.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
Message-ID: <82CDDF2F-F2CE-4353-959C-3701E3A2E806@gmail.com>


On Jun 19, 2009, at 11:28 AM, Stefan Behnel wrote:

> Hi,
>
> this is really getting off-topic, BTW.
>
> David Lyon wrote:
>> On Fri, 19 Jun 2009 07:24:21 +0200, Stefan Behnel wrote:
>>> Leonardo Santagada wrote:
>>>> The biggest problem I see is security, but if people are really
>>>> interested in this we could at least try it no?
>>>
>>> Security certainly is a major issue here. Anyone can upload  
>>> packages to
>>> PyPI, so you can run arbitrary code on tons of machines, just by  
>>> pushing
>>> some well-forged setup.py script there.
>>
>> Doesn't a chroot jail stop this? (on unix anyway)
>
> Stop it from running arbitrary code? How would it do that?
>
> Imagine your setup.py downloads Seti at Home jobs from a web site and
> calculates them. You can't really ban web access, as the setup.py  
> script
> may use setuptools to download dependencies. You can't really stop  
> it from
> doing calculations, as it might actually be doing real build stuff  
> like
> source code generation. You can kill it after, say, an hour (or a  
> half if
> you think that no build should take longer than that), but I guess I  
> can
> do a lot of calculations in that time, on a lot of your computers.  
> And if
> you ban my package, I'll just upload a different one with a different
> name. And if you ban my PyPI account, I'll grab a new one. And if  
> you ban
> web access from your machines, well, I'll just upload a package that
> contains the tasks in the archive. And if you use hash codes to  
> check for
> malicious packages, well, I'll just start obfuscating my code and my  
> data.
> And be assured that what I'm really doing is not calculating Seti at Home
> jobs.
>
> There's really no way you can stop someone from misusing your
> infrastructure if you go for building any arbitrary package that gets
> uploaded to PyPI.


I think you are completely right... so lets not do that. We should  
whitelist some pgp signatures from people we trust (and some of those  
people should whitelist other people) and then we can have simpler  
virtualization methods that only garantee isolation from one another.  
If we caught someone cheating we remove their signature and all the  
signatures from everyone this person permitted into the autobuild  
farm... and maybe we should ask the person that gave permission to  
this guy.

Or something on these lines.

--
Leonardo Santagada
santagada at gmail.com




From eric at trueblade.com  Fri Jun 19 21:54:53 2009
From: eric at trueblade.com (Eric Smith)
Date: Fri, 19 Jun 2009 15:54:53 -0400
Subject: [Distutils] sdist default archive format
In-Reply-To: <7FD6E55A-1652-4024-85CF-C0CFA69F0F56@zooko.com>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>	<A84E15AC-9524-4D52-B62B-E2BDB8D0DBAE@zooko.com>	<e4dd18fa8673ee0d8eb9f19bbd60fe34@preisshare.net>
	<7FD6E55A-1652-4024-85CF-C0CFA69F0F56@zooko.com>
Message-ID: <4A3BED0D.5070407@trueblade.com>

> It isn't so easy for setuptools to know which things ought to have +x 
> and which things ought not just based on their pathnames or other 
> metadata.

Unless you specify explicitly it in a metadata file.

> Having an archive format that preserves such bits would probably be a 
> good way to solve all such problems -- by making it the packagers 
> problems to set the bits before packaging rather than setuptools's 
> problem to figure out which bits ought to be set after installation.

How would you create such an archive on Windows? (A serious question, 
not trying to be argumentative.)

Eric.


From zooko at zooko.com  Fri Jun 19 22:38:37 2009
From: zooko at zooko.com (Zooko Wilcox-O'Hearn)
Date: Fri, 19 Jun 2009 14:38:37 -0600
Subject: [Distutils] sdist default archive format
In-Reply-To: <4A3BED0D.5070407@trueblade.com>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>	<A84E15AC-9524-4D52-B62B-E2BDB8D0DBAE@zooko.com>	<e4dd18fa8673ee0d8eb9f19bbd60fe34@preisshare.net>
	<7FD6E55A-1652-4024-85CF-C0CFA69F0F56@zooko.com>
	<4A3BED0D.5070407@trueblade.com>
Message-ID: <5C878C3E-E510-4197-BC0E-81E4A7EB58E2@zooko.com>

On Jun 19, 2009, at 13:54 PM, Eric Smith wrote:

>> It isn't so easy for setuptools to know which things ought to have  
>> +x and which things ought not just based on their pathnames or  
>> other metadata.
>
> Unless you specify explicitly it in a metadata file.

My argument is that using a metadata-preserving archive format allows  
a programmer to control that stuff and also allows setuptools (or  
distutils, I guess) to ignore that stuff.  Inventing a way to pass  
that information through the metadata requires the programmer to  
learn and use an extra configuration on top of what they already do,  
and requires setuptools (distutils) to pay attention to that stuff.

>> Having an archive format that preserves such bits would probably  
>> be a good way to solve all such problems -- by making it the  
>> packagers problems to set the bits before packaging rather than  
>> setuptools's problem to figure out which bits ought to be set  
>> after installation.
>
> How would you create such an archive on Windows? (A serious  
> question, not trying to be argumentative.)


You mean a bdist?  You can't "cross-compile" and build a bdist for  
Linux on Windows (can you?), or vice versa.  I think we're only  
talking about "non-cross-compiled" bdists here.

A metadata-preserving archive format wouldn't make this harder, and  
it might make it easier.

Regards,

Zooko

From santagada at gmail.com  Fri Jun 19 23:19:52 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Fri, 19 Jun 2009 18:19:52 -0300
Subject: [Distutils] sdist default archive format
In-Reply-To: <5C878C3E-E510-4197-BC0E-81E4A7EB58E2@zooko.com>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>	<A84E15AC-9524-4D52-B62B-E2BDB8D0DBAE@zooko.com>	<e4dd18fa8673ee0d8eb9f19bbd60fe34@preisshare.net>
	<7FD6E55A-1652-4024-85CF-C0CFA69F0F56@zooko.com>
	<4A3BED0D.5070407@trueblade.com>
	<5C878C3E-E510-4197-BC0E-81E4A7EB58E2@zooko.com>
Message-ID: <2B74CE66-45BD-4DF1-9D1B-C478F9EEBA9B@gmail.com>


On Jun 19, 2009, at 5:38 PM, Zooko Wilcox-O'Hearn wrote:

> On Jun 19, 2009, at 13:54 PM, Eric Smith wrote:
>
>>> It isn't so easy for setuptools to know which things ought to have  
>>> +x and which things ought not just based on their pathnames or  
>>> other metadata.
>>
>> Unless you specify explicitly it in a metadata file.
>
> My argument is that using a metadata-preserving archive format  
> allows a programmer to control that stuff and also allows setuptools  
> (or distutils, I guess) to ignore that stuff.  Inventing a way to  
> pass that information through the metadata requires the programmer  
> to learn and use an extra configuration on top of what they already  
> do, and requires setuptools (distutils) to pay attention to that  
> stuff.
>
>>> Having an archive format that preserves such bits would probably  
>>> be a good way to solve all such problems -- by making it the  
>>> packagers problems to set the bits before packaging rather than  
>>> setuptools's problem to figure out which bits ought to be set  
>>> after installation.
>>
>> How would you create such an archive on Windows? (A serious  
>> question, not trying to be argumentative.)
>
>
> You mean a bdist?  You can't "cross-compile" and build a bdist for  
> Linux on Windows (can you?), or vice versa.  I think we're only  
> talking about "non-cross-compiled" bdists here.
>
> A metadata-preserving archive format wouldn't make this harder, and  
> it might make it easier.


I think he mean how to do a sdist on windows that has +x on executable  
python scripts? I don't know the answer but it might be a good case  
for metadata external to the filesystem.


--
Leonardo Santagada
santagada at gmail.com


From eric at trueblade.com  Fri Jun 19 23:23:51 2009
From: eric at trueblade.com (Eric Smith)
Date: Fri, 19 Jun 2009 17:23:51 -0400
Subject: [Distutils] sdist default archive format
In-Reply-To: <5C878C3E-E510-4197-BC0E-81E4A7EB58E2@zooko.com>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>	<A84E15AC-9524-4D52-B62B-E2BDB8D0DBAE@zooko.com>	<e4dd18fa8673ee0d8eb9f19bbd60fe34@preisshare.net>
	<7FD6E55A-1652-4024-85CF-C0CFA69F0F56@zooko.com>
	<4A3BED0D.5070407@trueblade.com>
	<5C878C3E-E510-4197-BC0E-81E4A7EB58E2@zooko.com>
Message-ID: <4A3C01E7.6070702@trueblade.com>

Zooko Wilcox-O'Hearn wrote:
>>> Having an archive format that preserves such bits would probably be a 
>>> good way to solve all such problems -- by making it the packagers 
>>> problems to set the bits before packaging rather than setuptools's 
>>> problem to figure out which bits ought to be set after installation.
>>
>> How would you create such an archive on Windows? (A serious question, 
>> not trying to be argumentative.)
> 
> You mean a bdist?  You can't "cross-compile" and build a bdist for Linux 
> on Windows (can you?), or vice versa.  I think we're only talking about 
> "non-cross-compiled" bdists here.

I mean an sdist, which I hope is what we're still talking about, in the 
context of using a .zip or .tar file. How do I specify +x on Windows? I 
use cygwin, but what about those who don't? (Again, this is a serious 
question. I don't use straight Windows enough to know the answer.)

> A metadata-preserving archive format wouldn't make this harder, and it 
> might make it easier.

As long as the metadata is cross-platform, which I'm not sure it is. 
We're trying to create a cross-platform source distribution, after all.

Eric.


From david.lyon at preisshare.net  Sat Jun 20 03:43:19 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Fri, 19 Jun 2009 21:43:19 -0400
Subject: [Distutils] sdist default archive format
In-Reply-To: <4A3C01E7.6070702@trueblade.com>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>	<A84E15AC-9524-4D52-B62B-E2BDB8D0DBAE@zooko.com>	<e4dd18fa8673ee0d8eb9f19bbd60fe34@preisshare.net>
	<7FD6E55A-1652-4024-85CF-C0CFA69F0F56@zooko.com>
	<4A3BED0D.5070407@trueblade.com>
	<5C878C3E-E510-4197-BC0E-81E4A7EB58E2@zooko.com>
	<4A3C01E7.6070702@trueblade.com>
Message-ID: <a4863b91f6ace639c41292963e1ea1d8@preisshare.net>

On Fri, 19 Jun 2009 17:23:51 -0400, Eric Smith <eric at trueblade.com> wrote:
> How do I specify +x on Windows? I 
> use cygwin, but what about those who don't? (Again, this is a serious 
> question. I don't use straight Windows enough to know the answer.)

There's no such thing in the same way. Python programs are associated
with the python interpreter via the windows registry (system config)
and their .py suffix.

What this means is that the .py files are always executeable by
default on XP and beyond.

imho - if scripts need to be made executable on landing, then it
should be via the python subsystem with a chmod +x. Otherwise
there's just too much temptation for things to be done (with
permissions) that can cause things to go horribly wrong.

The end results of symlinks and permissions is that they tend
to make the package only work on *nix. Thats my basic objection
to them.

David




From david.lyon at preisshare.net  Sat Jun 20 03:53:20 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Fri, 19 Jun 2009 21:53:20 -0400
Subject: [Distutils] sdist default archive format
In-Reply-To: <7FD6E55A-1652-4024-85CF-C0CFA69F0F56@zooko.com>
References: <94bdd2610906170734o74e33d8fn2f21eb1ae9fe8d74@mail.gmail.com>
	<6db0ea510906171407i3595c13r121a9faf5bc65bb5@mail.gmail.com>
	<A84E15AC-9524-4D52-B62B-E2BDB8D0DBAE@zooko.com>
	<e4dd18fa8673ee0d8eb9f19bbd60fe34@preisshare.net>
	<7FD6E55A-1652-4024-85CF-C0CFA69F0F56@zooko.com>
Message-ID: <37310c7fc53a6ff4feac18ae0f6d4635@preisshare.net>

On Fri, 19 Jun 2009 10:38:39 -0600, Zooko Wilcox-O'Hearn <zooko at zooko.com>
wrote:
> It isn't so easy for setuptools to know which things ought to have +x  
> and which things ought not just based on their pathnames or other  
> metadata.  

It is very easy.... it already knows...

http://docs.python.org/distutils/setupscript.html#installing-scripts

The scripts option simply is a list of files to be handled in this way.
>From the PyXML setup script:

setup(...,
      scripts=['scripts/xmlproc_parse', 'scripts/xmlproc_val']
      )

> Having an archive format that preserves such bits would probably be a  
> good way to solve all such problems -- by making it the packagers  
> problems to set the bits before packaging rather than setuptools's  
> problem to figure out which bits ought to be set after installation.

It's totally impossible on windows...

Because you are talking about a unix specific problem...

Why not just solve the problem rather than spreading it to
more platforms?

> 1.8 / 2.5 == 72%, so 28% of the .zip file size is saved by switching  
> to .tar.xz.  Of course, some packages may gain more or less than 28%  
> (compared to ZIP compression) from LZMA compression.

Well maybe its good. Its a new one to me though...

David


From david at ar.media.kyoto-u.ac.jp  Sat Jun 20 12:17:12 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Sat, 20 Jun 2009 19:17:12 +0900
Subject: [Distutils] Autobuild packages using snakebite
In-Reply-To: <7e3853f2854fe6848d2c4825ec0c147e.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
References: <7158858D-8587-4EF7-A18B-D7450A4A2953@gmail.com>	<h1f7e6$kg4$1@ger.gmane.org>	<1ebcb2879ff07da133846cfd42b606f4@preisshare.net>
	<7e3853f2854fe6848d2c4825ec0c147e.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
Message-ID: <4A3CB728.4030404@ar.media.kyoto-u.ac.jp>

Stefan Behnel wrote:
> Hi,
>
> this is really getting off-topic, BTW.
>
> David Lyon wrote:
>   
>> On Fri, 19 Jun 2009 07:24:21 +0200, Stefan Behnel wrote:
>>     
>>> Leonardo Santagada wrote:
>>>       
>>>> The biggest problem I see is security, but if people are really
>>>> interested in this we could at least try it no?
>>>>         
>>> Security certainly is a major issue here. Anyone can upload packages to
>>> PyPI, so you can run arbitrary code on tons of machines, just by pushing
>>> some well-forged setup.py script there.
>>>       
>> Doesn't a chroot jail stop this? (on unix anyway)
>>     
>
> Stop it from running arbitrary code? How would it do that?
>
> Imagine your setup.py downloads Seti at Home jobs from a web site and
> calculates them. You can't really ban web access, as the setup.py script
> may use setuptools to download dependencies. You can't really stop it from
> doing calculations, as it might actually be doing real build stuff like
> source code generation. You can kill it after, say, an hour (or a half if
> you think that no build should take longer than that), but I guess I can
> do a lot of calculations in that time, on a lot of your computers. And if
> you ban my package, I'll just upload a different one with a different
> name. And if you ban my PyPI account, I'll grab a new one. And if you ban
> web access from your machines, well, I'll just upload a package that
> contains the tasks in the archive. And if you use hash codes to check for
> malicious packages, well, I'll just start obfuscating my code and my data.
> And be assured that what I'm really doing is not calculating Seti at Home
> jobs.
>
> There's really no way you can stop someone from misusing your
> infrastructure if you go for building any arbitrary package that gets
> uploaded to PyPI.
>   

Forbidding any network access from the vm used to build would solve most
of those problems. Controlling CPU/memory can be done from the host OS.
I would be surprised if the openSuse build system worked in a
fundamentally different way: rpm .spec files can also execute arbitrary
code.

cheers,

David

From david.lyon at preisshare.net  Sat Jun 20 13:03:10 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Sat, 20 Jun 2009 07:03:10 -0400
Subject: [Distutils] Autobuild packages using snakebite
In-Reply-To: <4A3CB728.4030404@ar.media.kyoto-u.ac.jp>
References: <7158858D-8587-4EF7-A18B-D7450A4A2953@gmail.com>	<h1f7e6$kg4$1@ger.gmane.org>	<1ebcb2879ff07da133846cfd42b606f4@preisshare.net>
	<7e3853f2854fe6848d2c4825ec0c147e.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<4A3CB728.4030404@ar.media.kyoto-u.ac.jp>
Message-ID: <0f534aaa59a4d196e5de3299d244adbb@preisshare.net>

On Sat, 20 Jun 2009 19:17:12 +0900, David Cournapeau

>> There's really no way you can stop someone from misusing your
>> infrastructure if you go for building any arbitrary package that gets
>> uploaded to PyPI.

But CPU cycles can be counted and likewise network usage...

> Forbidding any network access from the vm used to build would solve most
> of those problems. Controlling CPU/memory can be done from the host OS.
> I would be surprised if the openSuse build system worked in a
> fundamentally different way: rpm .spec files can also execute arbitrary
> code.

I struggle with seeing how that would work. My easyinstall is always
downloading additional packages (as it should). And how would you
get packages onto the infastructure from pypi in the first place if 
there was no internet access?

David


From martin at v.loewis.de  Sat Jun 20 14:29:47 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 20 Jun 2009 14:29:47 +0200
Subject: [Distutils] zip vs tar.gz at PyPI
In-Reply-To: <94bdd2610906180205s47513652t5f84dc4000e4e640@mail.gmail.com>
References: <94bdd2610906180205s47513652t5f84dc4000e4e640@mail.gmail.com>
Message-ID: <4A3CD63B.5010606@v.loewis.de>

> We are talking about the "preferred" default archive format in Distutils-SIG.
> 
> I was wondering if you could give us some statistics on the various
> archives format used at PyPI.

Here are some statistics:

total number of uploaded files:
- .tar.gz or .tgz files: 16667
- .tar.bz2 files:          547
- .zip files:             2053
- .egg files:            10189
- .exe files:             1115
- .rpm files:               77
- .dmg files:                4
- .deb files:                1

total number of such files uploaded as source:
- .tar.gz or .tgz: 16525
- .tar.bz2: 543
- .zip: 2046
- other source files: 6

number of packages that have uploaded such source files:
- .tar.gz or .tgz: 4229
- .tar.bz2:         151
- .zip:             706

Some packages release more than one source file per release;
it's difficult to compute statistics here, because these are
sometimes just repackagings (e.g. Amara: both .tar.gz and .tar.bz2)
and sometimes different sets of sources (e.g. 4Suite-XML: both
program source, and -docs).

HTH,
Martin

From david at ar.media.kyoto-u.ac.jp  Sat Jun 20 14:24:39 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Sat, 20 Jun 2009 21:24:39 +0900
Subject: [Distutils] Autobuild packages using snakebite
In-Reply-To: <0f534aaa59a4d196e5de3299d244adbb@preisshare.net>
References: <7158858D-8587-4EF7-A18B-D7450A4A2953@gmail.com>	<h1f7e6$kg4$1@ger.gmane.org>	<1ebcb2879ff07da133846cfd42b606f4@preisshare.net>
	<7e3853f2854fe6848d2c4825ec0c147e.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<4A3CB728.4030404@ar.media.kyoto-u.ac.jp>
	<0f534aaa59a4d196e5de3299d244adbb@preisshare.net>
Message-ID: <4A3CD507.20509@ar.media.kyoto-u.ac.jp>

David Lyon wrote:
>
> I struggle with seeing how that would work. My easyinstall is always
> downloading additional packages (as it should).

the answer is obvious: it should not download anything from the wild.
That's one of the big reason why so many people literally hate this
setuptools feature, BTW.

>  And how would you
> get packages onto the infastructure from pypi in the first place if 
> there was no internet access?
>   

that's the job of the infrastructure. All those problems have been
solved by the opensuse build service, which is open source (in Perl for
the server part, though). Before stating what is possible and not, it
would be wise to study other similar services:

http://en.opensuse.org/Build_Service

cheers,

David


From david.lyon at preisshare.net  Sat Jun 20 16:10:10 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Sat, 20 Jun 2009 10:10:10 -0400
Subject: [Distutils] Autobuild packages using snakebite
In-Reply-To: <4A3CD507.20509@ar.media.kyoto-u.ac.jp>
References: <7158858D-8587-4EF7-A18B-D7450A4A2953@gmail.com>	<h1f7e6$kg4$1@ger.gmane.org>	<1ebcb2879ff07da133846cfd42b606f4@preisshare.net>
	<7e3853f2854fe6848d2c4825ec0c147e.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<4A3CB728.4030404@ar.media.kyoto-u.ac.jp>
	<0f534aaa59a4d196e5de3299d244adbb@preisshare.net>
	<4A3CD507.20509@ar.media.kyoto-u.ac.jp>
Message-ID: <a18d48ebbc7946520a8edb63be73b59e@preisshare.net>

On Sat, 20 Jun 2009 21:24:39 +0900, David Cournapeau

> that's the job of the infrastructure. All those problems have been
> solved by the opensuse build service, which is open source (in Perl for
> the server part, though). Before stating what is possible and not, it
> would be wise to study other similar services:
> 
> http://en.opensuse.org/Build_Service

Thanks David. I will look into it.

Looks very fascinating at first glance...

:-)


From rupert.thurner at gmail.com  Sat Jun 20 23:10:58 2009
From: rupert.thurner at gmail.com (rupert.thurner)
Date: Sat, 20 Jun 2009 14:10:58 -0700 (PDT)
Subject: [Distutils] easy_install moinmoin does not work, (very) old url
Message-ID: <9c87d479-cbc1-4333-b6d2-d3acbea350fd@s21g2000vbb.googlegroups.com>

there is http://pypi.python.org/pypi/MoinMoin/1.8.4, and
http://pypi.python.org/pypi?:action=doap&name=MoinMoin&version=1.8.4.
but instlaling moin somehow does not work here. what might be the
reason for this? the download links easy_install displays seem
outdated.

# easy_install moin
Searching for moin
Reading http://pypi.python.org/simple/moin/
Reading http://moin.sf.net/
No local packages or download links found for moin
error: Could not find suitable distribution for Requirement.parse
('moin')


# easy_install MoinMoin
Searching for MoinMoin
Reading http://pypi.python.org/simple/MoinMoin/
Reading http://moinmoin.wikiwikiweb.de/
Reading http://moinmoin.wikiwikiweb.de/MoinMoinDownload
No local packages or download links found for MoinMoin
error: Could not find suitable distribution for Requirement.parse
('MoinMoin')

From pje at telecommunity.com  Sun Jun 21 05:11:05 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Sat, 20 Jun 2009 23:11:05 -0400
Subject: [Distutils] easy_install moinmoin does not work, (very)  old url
In-Reply-To: <9c87d479-cbc1-4333-b6d2-d3acbea350fd@s21g2000vbb.googlegro
	ups.com>
References: <9c87d479-cbc1-4333-b6d2-d3acbea350fd@s21g2000vbb.googlegroups.com>
Message-ID: <20090621030810.1C4763A404D@sparrow.telecommunity.com>

At 02:10 PM 6/20/2009 -0700, rupert.thurner wrote:
>there is http://pypi.python.org/pypi/MoinMoin/1.8.4, and

The distribution filename is 'moin-1.8.4', but the package name 
registered on Pypi is "MoinMoin".  This won't work.

You may be successful by running:

"easy_install http://static.moinmo.in/files/moin-1.8.4.tar.gz"

The problem is that the package is mis-registered on PyPI; its 
setup.py declares its name "moin", so that is its distribution 
filename.  However, on PyPI it is indexed as MoinMoin.  You may wish 
to inform the maintainers of this, so they can update either the 
setup.py to use MoinMoin as the name, or change the Pypi registration 
to use "moin".  Either should work, assuming that the distribution is 
able to work with setuptools.  (You'll have to try it and see.)


From ziade.tarek at gmail.com  Sun Jun 21 17:57:30 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 21 Jun 2009 17:57:30 +0200
Subject: [Distutils] PEP 376 - Anything else ?
Message-ID: <94bdd2610906210857x39380348m4ae8df248461a4@mail.gmail.com>

Hello

I've added Zip support for PEP 376, as asked by Ronald some time ago.

I think I have included everything people reported in the code
prototype and the PEP text,

Any other feedback before I resend it back to python-dev, to propose
its inclusion for 2.7/3.2 ?

pep : http://svn.python.org/projects/peps/trunk/pep-0376.txt
code prototype : http://bitbucket.org/tarek/pep376/src/tip/pkgutil.py

Regards
Tarek
-- 
Tarek Ziad? | http://ziade.org

From barry at python.org  Tue Jun 23 00:09:48 2009
From: barry at python.org (Barry Warsaw)
Date: Mon, 22 Jun 2009 18:09:48 -0400
Subject: [Distutils] setup_requires not caching downloaded packages?
Message-ID: <FD2D95F2-9362-4BF0-BEB0-6F759577476A@python.org>

We have a suite of tools in the 'lazr' namespace that we've begun to  
release as open source.  All of these packages are maintained under  
bzr and it's really nice to be able to add this to our setup.py:

setup_requires=['setuptools_bzr']

This extends setuptools so that we can use bzr instead of the built-in  
revision control systems.  With that, 'python setup.py sdist' pulls in  
the right stuff.  All of our packages are buildout based.

The problem is that when we buildout the package, the download cache  
is never consulted and setuptools_bzr is /always/ downloaded.  This is  
a non-starter for our production environments, which are not allowed  
to hit the external 'net during build time.  Because of this, we've  
had to disable the setup_requires.  We can kind of make things work  
with MANIFEST.ins but I'd prefer to be able to use setup_requires.   
Otherwise, um, what's the point of having this plugin?

I'm not sure whether this is a bug in setuptools, distutils, or  
buildout, but since y'all are here, we should be able to figure it  
out.  Or do you not think this is a bug?

If you agree it's a bug, and we can agree where it's a bug, I'd be  
willing to put some time into fixing this, if we can get a timely  
update released in the affected package.

Thanks,
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090622/23bec5ab/attachment.pgp>

From pje at telecommunity.com  Tue Jun 23 05:40:28 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 22 Jun 2009 23:40:28 -0400
Subject: [Distutils] setup_requires not caching downloaded packages?
In-Reply-To: <FD2D95F2-9362-4BF0-BEB0-6F759577476A@python.org>
References: <FD2D95F2-9362-4BF0-BEB0-6F759577476A@python.org>
Message-ID: <20090623033733.38DC63A40D7@sparrow.telecommunity.com>

At 06:09 PM 6/22/2009 -0400, Barry Warsaw wrote:
>We have a suite of tools in the 'lazr' namespace that we've begun to
>release as open source.  All of these packages are maintained under
>bzr and it's really nice to be able to add this to our setup.py:
>
>setup_requires=['setuptools_bzr']
>
>This extends setuptools so that we can use bzr instead of the built-in
>revision control systems.  With that, 'python setup.py sdist' pulls in
>the right stuff.  All of our packages are buildout based.
>
>The problem is that when we buildout the package, the download cache
>is never consulted and setuptools_bzr is /always/ downloaded.

I don't know what the "download cache" is, so I can't speak to 
that.  However, what I can say is that if you're using "sdist" for 
distribution, you probably don't need that setup_requires line in the 
first place.

When you build an sdist using setuptools, it includes a pre-generated 
file manifest, along with a setup.cfg that forces the right revision 
tag information in the project's version number (if 
applicable).  This is specifically designed so that the person 
building your sdist doesn't need the same revision control system installed.

In short, the only person who needs setuptools_bzr installed is the 
person who's generating the sdists.  So, feel free to drop it from 
setup_requires and be happy.  ;-)


From zooko at zooko.com  Tue Jun 23 17:42:32 2009
From: zooko at zooko.com (Zooko Wilcox-O'Hearn)
Date: Tue, 23 Jun 2009 09:42:32 -0600
Subject: [Distutils] setup_requires not caching downloaded packages?
In-Reply-To: <FD2D95F2-9362-4BF0-BEB0-6F759577476A@python.org>
References: <FD2D95F2-9362-4BF0-BEB0-6F759577476A@python.org>
Message-ID: <FB844080-DA20-42F7-AFAE-C02301A03B6A@zooko.com>

Barry:

What is the download cache?

I have a guess.  If you are installing with the setuptools "install"  
command, then the things which are "install_requires" get installed  
into some target location, but things which are build_requires get  
installed into the PWD.  (If you don't specify a target, then it is  
the system site-packages.  In my build system [1], we typically don't  
use "install" and instead use "develop" with a --prefix argument  
which specifies the target.  I assume virtualenv does something  
similar.)

Therefore, a later build will have its install_requires already  
satisfied (if it is using the same installation target), but its  
build_requirements will not be satisfied.

If that's what's going on then a solution might involve pre- 
installing setuptools_bzr onto the build system, or patching  
setuptools to specify a location for build requirements and re-using  
that location.

Regards,

Zooko

[1] http://allmydata.org/trac/tahoe/browser/setup.cfg

From thomas at thomas-lotze.de  Wed Jun 24 19:04:41 2009
From: thomas at thomas-lotze.de (Thomas Lotze)
Date: Wed, 24 Jun 2009 19:04:41 +0200
Subject: [Distutils] [Buildout] Download API and extends caching -
	please	review
References: <pan.2009.06.11.06.55.07.449140@ID-174572.user.uni-berlin.de>
Message-ID: <pan.2009.06.24.15.51.01.221130@ID-174572.user.uni-berlin.de>

Thomas Lotze wrote:

> As discussed before on this list, I've been working on an internal
> download API for zc.buildout that takes care of handling the download
> cache, offline mode, MD5 checks etc and is used for fetching configuration
> files (as specified by the buildout:extends option). I consider the
> feature finished and I'd be grateful if someone could review it before it
> gets merged to the trunk. The branch I've been developing this on is
> <svn://svn.zope.org/repos/main/zc.buildout/branches/tlotze-download-api>.

Since I wrote that message, I've been working on using the download API in
zc.recipe.cmmi. The recipe is now able to perform MD5 checksum
verification on downloaded source archives and patches and I've used this
opportunity of seeing the new download code in action to improve it
further.

I'd like to ask again for a code review of both projects' feature branches
(the recipe's being at
<svn://svn.zope.org/repos/main/zc.buildout/branches/tlotze-download-api>)
before merging them to the respective trunks. Jim, as you've responded to
questions on the subject before, would you be willing and find some time
to have a look at the code?

-- 
Thomas




From Mike_Moerk at us.xyratex.com  Wed Jun 24 18:45:02 2009
From: Mike_Moerk at us.xyratex.com (Mike Moerk)
Date: Wed, 24 Jun 2009 09:45:02 -0700
Subject: [Distutils] SVN URLs in setuptools / EasyInstall?
Message-ID: <9F94CC89AA0F524CBA8A619D43C4AED8E762B8@XYUS-EX22.xyus.xyratex.com>

I am trying to use setuptools and EasyInstall to handle dependency
management for me, using a local subversion server.  The svn server uses
svnserve, not http.  I have the following in my setup call:

       install_requires = ["myModule"],
       dependency_links = ["svn://myServer.myCompany.com/myModule"],

But setuptools does not like the svn:// URL.  I get the following error:

    Searching for myModule
    Reading svn://myServer.myCompany.com/myModule
    Download error: unknown url type: svn -- Some packages may not be
found!

I have also tried using find_links in the setup.cfg, rather than
dependency_links in the setup.py.  I get the same result.  Here is my
setup.cfg:

    [easy_install]
    allow_hosts = *.myCompany.com
    find_links = svn://myServer.myCompany.com/myModule

The release notes for EasyInstall 0.3a2 say that download support was
added for svn: URLs, which is what makes me think this is supposed to
work.

Is this a bug, or am I doing something wrong, or am I just
misinterpreting what the tool can do?

Thanks,
Mike


______________________________________________________________________
This email may contain privileged or confidential information, which should only be used for the purpose for which it was sent by Xyratex. No further rights or licenses are granted to use such information. If you are not the intended recipient of this message, please notify the sender by return and delete it. You may not use, copy, disclose or rely on the information contained in it.

Internet email is susceptible to data corruption, interception and unauthorised amendment for which Xyratex does not accept liability. While we have taken reasonable precautions to ensure that this email is free of viruses, Xyratex does not accept liability for the presence of any computer viruses in this email, nor for any losses caused as a result of viruses.

Xyratex Technology Limited (03134912), Registered in England & Wales, Registered Office, Langstone Road, Havant, Hampshire, PO9 1SA.

The Xyratex group of companies also includes, Xyratex Ltd, registered in Bermuda, Xyratex International Inc, registered in California, Xyratex (Malaysia) Sdn Bhd registered in Malaysia, Xyratex Technology (Wuxi) Co Ltd registered in The People's Republic of China and Xyratex Japan Limited registered in Japan.
______________________________________________________________________

From jim at zope.com  Wed Jun 24 19:38:39 2009
From: jim at zope.com (Jim Fulton)
Date: Wed, 24 Jun 2009 13:38:39 -0400
Subject: [Distutils] [Buildout] Download API and extends caching -
	please	review
In-Reply-To: <pan.2009.06.24.15.51.01.221130@ID-174572.user.uni-berlin.de>
References: <pan.2009.06.11.06.55.07.449140@ID-174572.user.uni-berlin.de>
	<pan.2009.06.24.15.51.01.221130@ID-174572.user.uni-berlin.de>
Message-ID: <89FA5D8A-970A-49F2-95F9-E1C0B823B194@zope.com>


On Jun 24, 2009, at 1:04 PM, Thomas Lotze wrote:

> Thomas Lotze wrote:
>
>> As discussed before on this list, I've been working on an internal
>> download API for zc.buildout that takes care of handling the download
>> cache, offline mode, MD5 checks etc and is used for fetching  
>> configuration
>> files (as specified by the buildout:extends option). I consider the
>> feature finished and I'd be grateful if someone could review it  
>> before it
>> gets merged to the trunk. The branch I've been developing this on is
>> <svn://svn.zope.org/repos/main/zc.buildout/branches/tlotze-download-api 
>> >.
>
> Since I wrote that message, I've been working on using the download  
> API in
> zc.recipe.cmmi. The recipe is now able to perform MD5 checksum
> verification on downloaded source archives and patches and I've used  
> this
> opportunity of seeing the new download code in action to improve it
> further.
>
> I'd like to ask again for a code review of both projects' feature  
> branches
> (the recipe's being at
> <svn://svn.zope.org/repos/main/zc.buildout/branches/tlotze-download-api 
> >)
> before merging them to the respective trunks. Jim, as you've  
> responded to
> questions on the subject before, would you be willing and find some  
> time
> to have a look at the code?

Yup

Jim

--
Jim Fulton
Zope Corporation



From rupert.thurner at gmail.com  Wed Jun 24 21:54:33 2009
From: rupert.thurner at gmail.com (rupert.thurner)
Date: Wed, 24 Jun 2009 12:54:33 -0700 (PDT)
Subject: [Distutils] easy_install moinmoin does not work, (very) old url
In-Reply-To: <20090621030810.1C4763A404D@sparrow.telecommunity.com>
References: <9c87d479-cbc1-4333-b6d2-d3acbea350fd@s21g2000vbb.googlegroups.com>
	<20090621030810.1C4763A404D@sparrow.telecommunity.com>
Message-ID: <afbf66d0-0bcd-40b2-8c67-d012cea00d1f@n19g2000vba.googlegroups.com>

many thanks for your reply. there is now an issue as well at the
moinmoin site basically saying "make me a contact to a pypi
maintainter and i'll fix this": http://moinmo.in/4ct10n/edit/MoinMoinBugs/FixPyPIName
so i was wondering if somebody  could help out here?

On 21 Jun., 05:11, "P.J. Eby" <p... at telecommunity.com> wrote:
> At 02:10 PM 6/20/2009 -0700, rupert.thurner wrote:
>
> >there ishttp://pypi.python.org/pypi/MoinMoin/1.8.4, and
>
> The distribution filename is 'moin-1.8.4', but the package name
> registered on Pypi is "MoinMoin". ?This won't work.
>
> You may be successful by running:
>
> "easy_installhttp://static.moinmo.in/files/moin-1.8.4.tar.gz"
>
> The problem is that the package is mis-registered on PyPI; its
> setup.py declares its name "moin", so that is its distribution
> filename. ?However, on PyPI it is indexed asMoinMoin. ?You may wish
> to inform the maintainers of this, so they can update either the
> setup.py to useMoinMoinas the name, or change the Pypi registration
> to use "moin". ?Either should work, assuming that the distribution is
> able to work with setuptools. ?(You'll have to try it and see.)
>
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-... at python.orghttp://mail.python.org/mailman/listinfo/distutils-sig

From david.lyon at preisshare.net  Thu Jun 25 03:34:42 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 24 Jun 2009 21:34:42 -0400
Subject: [Distutils] Python 3 porting effort..
In-Reply-To: <afbf66d0-0bcd-40b2-8c67-d012cea00d1f@n19g2000vba.googlegroups.com>
References: <9c87d479-cbc1-4333-b6d2-d3acbea350fd@s21g2000vbb.googlegroups.com>
	<20090621030810.1C4763A404D@sparrow.telecommunity.com>
	<afbf66d0-0bcd-40b2-8c67-d012cea00d1f@n19g2000vba.googlegroups.com>
Message-ID: <1f549ed51a6742759f83d498bfcac4f2@preisshare.net>


Hi All,

Is anybody working on python 3 porting?

I'm volunteering to help out with the porting effort if there's nobody
else..

I have access to most of the platforms ie windows, linux but I'll need to
go 
buy a new mac and since it is tax time... I'm thinking of letting it be on 
the tax man... haha

If everybody helps out a little, we should get it done by december right?

Anybody know what the issues are?

David


From ziade.tarek at gmail.com  Thu Jun 25 11:08:35 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 25 Jun 2009 11:08:35 +0200
Subject: [Distutils] Python 3 porting effort..
In-Reply-To: <1f549ed51a6742759f83d498bfcac4f2@preisshare.net>
References: <9c87d479-cbc1-4333-b6d2-d3acbea350fd@s21g2000vbb.googlegroups.com>
	<20090621030810.1C4763A404D@sparrow.telecommunity.com>
	<afbf66d0-0bcd-40b2-8c67-d012cea00d1f@n19g2000vba.googlegroups.com>
	<1f549ed51a6742759f83d498bfcac4f2@preisshare.net>
Message-ID: <94bdd2610906250208g5080a96asb0a9daa0bb03601f@mail.gmail.com>

Hi David,
which project you are referring to ? I suppose you mean setuptools ?

Lennart Regebro worked on this, and Ronald Orrousen lately. You might want
to check with them

Tarek

On Thu, Jun 25, 2009 at 3:34 AM, David Lyon <david.lyon at preisshare.net>wrote:

>
> Hi All,
>
> Is anybody working on python 3 porting?
>
> I'm volunteering to help out with the porting effort if there's nobody
> else..
>
> I have access to most of the platforms ie windows, linux but I'll need to
> go
> buy a new mac and since it is tax time... I'm thinking of letting it be on
> the tax man... haha
>
> If everybody helps out a little, we should get it done by december right?
>
> Anybody know what the issues are?
>
> David
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | http://ziade.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090625/2dbba5de/attachment-0001.htm>

From dkdropbox at gmail.com  Fri Jun 26 01:20:02 2009
From: dkdropbox at gmail.com (David Kim)
Date: Thu, 25 Jun 2009 19:20:02 -0400
Subject: [Distutils] newbie bootstrap question
Message-ID: <f5014d110906251620x29b7ccecq9123fdbd1587253a@mail.gmail.com>

Sorry to ask, but I'm having with the first step of the buildout process,
the bootstrap.py script. I get the following error:

> Traceback (most recent call last):

  File "bootstrap.py", line 76, in <module>

    ws.find(pkg_resources.Requirement.parse('setuptools')).location

AttributeError: 'NoneType' object has no attribute 'location'


Unfortunately, I have no idea why this error is occurring. I am running on
OS X Leopard and was just trying to bootstrap an empty buildout.cfg file to
get things started. Tried googling the error, but nothing came up.

Any pointers?


-- 
morenotestoself.wordpress.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090625/48921509/attachment.htm>

From jim at zope.com  Fri Jun 26 11:23:28 2009
From: jim at zope.com (Jim Fulton)
Date: Fri, 26 Jun 2009 05:23:28 -0400
Subject: [Distutils] newbie bootstrap question
In-Reply-To: <f5014d110906251620x29b7ccecq9123fdbd1587253a@mail.gmail.com>
References: <f5014d110906251620x29b7ccecq9123fdbd1587253a@mail.gmail.com>
Message-ID: <0040C94A-BC62-4F3E-AD61-32A7D624EB12@zope.com>


On Jun 25, 2009, at 7:20 PM, David Kim wrote:

> Sorry to ask, but I'm having with the first step of the buildout  
> process, the bootstrap.py script. I get the following error:
>
> Traceback (most recent call last):   File "bootstrap.py", line 76,  
> in <module>      
> ws.find(pkg_resources.Requirement.parse('setuptools')).location  
> AttributeError: 'NoneType' object has no attribute 'location'
>
> Unfortunately, I have no idea why this error is occurring. I am  
> running on OS X Leopard and was just trying to bootstrap an empty  
> buildout.cfg file to get things started. Tried googling the error,  
> but nothing came up.
>
> Any pointers?


My guess is that you have setuptools installed in your python that  
wasn't installed as an egg.   What is the output of running the  
following Python script:

   import pkg_resources
   ws  = pkg_resources.working_set
   print ws.find(pkg_resources.Requirement.parse('setuptools'))

?

The bootstrap script has a try/except that tries to import pkg_utils.   
If the import succeeds, it assumes that setiuptools is already  
installed.  Try replacing the try/except with the contents of the  
except.

Jim

--
Jim Fulton
Zope Corporation



From dkdropbox at gmail.com  Fri Jun 26 17:04:56 2009
From: dkdropbox at gmail.com (David Kim)
Date: Fri, 26 Jun 2009 11:04:56 -0400
Subject: [Distutils] newbie bootstrap question
In-Reply-To: <0040C94A-BC62-4F3E-AD61-32A7D624EB12@zope.com>
References: <f5014d110906251620x29b7ccecq9123fdbd1587253a@mail.gmail.com> 
	<0040C94A-BC62-4F3E-AD61-32A7D624EB12@zope.com>
Message-ID: <f5014d110906260804p6744da31n6cc9e6fe82fec6a5@mail.gmail.com>

Thanks Jim, I think you are correct. The result of the script you provided
returned 'None'. Unfortunately, I commented out the "try" code...

> #try:

#    import pkg_resources

#except ImportError:


and left the "except" code...

ez = {}

exec urllib2.urlopen('http://peak.telecommunity.com/dist/ez_setup.py'

                         ).read() in ez

ez['use_setuptools'](to_dir=tmpeggs, download_delay=0)


> import pkg_resources


...but got the same error.

I am running the OS X EPD distribution, which uses Enstaller (built on
setuptools). Could this have something to do with it?

Many thanks for the help,

DK


On Fri, Jun 26, 2009 at 5:23 AM, Jim Fulton <jim at zope.com> wrote:

>
> On Jun 25, 2009, at 7:20 PM, David Kim wrote:
>
>  Sorry to ask, but I'm having with the first step of the buildout process,
>> the bootstrap.py script. I get the following error:
>>
>> Traceback (most recent call last):   File "bootstrap.py", line 76, in
>> <module>     ws.find(pkg_resources.Requirement.parse('setuptools')).location
>> AttributeError: 'NoneType' object has no attribute 'location'
>>
>> Unfortunately, I have no idea why this error is occurring. I am running on
>> OS X Leopard and was just trying to bootstrap an empty buildout.cfg file to
>> get things started. Tried googling the error, but nothing came up.
>>
>> Any pointers?
>>
>
>
> My guess is that you have setuptools installed in your python that wasn't
> installed as an egg.   What is the output of running the following Python
> script:
>
>  import pkg_resources
>  ws  = pkg_resources.working_set
>  print ws.find(pkg_resources.Requirement.parse('setuptools'))
>
> ?
>
> The bootstrap script has a try/except that tries to import pkg_utils.  If
> the import succeeds, it assumes that setiuptools is already installed.  Try
> replacing the try/except with the contents of the except.
>
> Jim
>
> --
> Jim Fulton
> Zope Corporation
>
>
>


-- 
morenotestoself.wordpress.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090626/f3e06277/attachment.htm>

From dpeterson at enthought.com  Fri Jun 26 22:05:09 2009
From: dpeterson at enthought.com (Dave Peterson)
Date: Fri, 26 Jun 2009 15:05:09 -0500
Subject: [Distutils] newbie bootstrap question
In-Reply-To: <f5014d110906260804p6744da31n6cc9e6fe82fec6a5@mail.gmail.com>
References: <f5014d110906251620x29b7ccecq9123fdbd1587253a@mail.gmail.com>
	<0040C94A-BC62-4F3E-AD61-32A7D624EB12@zope.com>
	<f5014d110906260804p6744da31n6cc9e6fe82fec6a5@mail.gmail.com>
Message-ID: <4A4529F5.6080708@enthought.com>

David Kim wrote:
> I am running the OS X EPD distribution, which uses Enstaller (built on 
> setuptools). Could this have something to do with it?

Sorry, but yes, this is exactly the problem.   EPD comes with Enstaller 
instead of setuptools, and thus there is no installed project called 
'setuptools', so pkg_resources.require('setuptools') isn't going to find 
anything.

As a quick fix, you could re-write the code you have as 
pkg_resources.require('enstaller') and be on your way.

I think a better, and longer term, fix is for us to patch the 'require' 
method in Enstaller's version of pkg_resources so that it catches a 
request for setuptools and maps it to Enstaller.   This is probably safe 
because you can't have both Enstaller and setuptools installed without 
getting errors, plus Enstaller includes a patched version of setuptools. 


-- Dave


From barry at python.org  Fri Jun 26 23:02:22 2009
From: barry at python.org (Barry Warsaw)
Date: Fri, 26 Jun 2009 17:02:22 -0400
Subject: [Distutils] setup_requires not caching downloaded packages?
In-Reply-To: <20090623033733.38DC63A40D7@sparrow.telecommunity.com>
References: <FD2D95F2-9362-4BF0-BEB0-6F759577476A@python.org>
	<20090623033733.38DC63A40D7@sparrow.telecommunity.com>
Message-ID: <85E22C58-7797-4157-BA35-20CB92629865@python.org>

Sorry for the delay in responding and the lack of clarity in the  
original message...

On Jun 22, 2009, at 11:40 PM, P.J. Eby wrote:

> I don't know what the "download cache" is, so I can't speak to  
> that.  However, what I can say is that if you're using "sdist" for  
> distribution, you probably don't need that setup_requires line in  
> the first place.

download cache is a buildout thing.

> When you build an sdist using setuptools, it includes a pre- 
> generated file manifest, along with a setup.cfg that forces the  
> right revision tag information in the project's version number (if  
> applicable).  This is specifically designed so that the person  
> building your sdist doesn't need the same revision control system  
> installed.
>
> In short, the only person who needs setuptools_bzr installed is the  
> person who's generating the sdists.  So, feel free to drop it from  
> setup_requires and be happy.  ;-)

Here's what the setuptools docs say:

(Note: projects listed in setup_requires will NOT be automatically  
installed on the system where the setup script is being run. They are  
simply downloaded to the setup directory if they're not locally  
available already. If you want them to be installed, as well as being  
available when the setup script is run, you should add them to  
install_requires and setup_requires.)

So I get that setuptools is doing what it's supposed to do.  I also  
get that it's (usually) not too hard to write a MANIFEST.in that  
pretty much does what you need it to do.  I'm bothered by the fact  
that this puts us in a bind where we want to use setup_requires but  
really can't.  After further thought though, the "problem" is probably  
in buildout, because it shouldn't be downloading setup_requires  
packages if they are in its download cache.

-Barry


-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090626/c59bcb8d/attachment.pgp>

From barry at python.org  Fri Jun 26 23:08:24 2009
From: barry at python.org (Barry Warsaw)
Date: Fri, 26 Jun 2009 17:08:24 -0400
Subject: [Distutils] setup_requires not caching downloaded packages?
In-Reply-To: <FB844080-DA20-42F7-AFAE-C02301A03B6A@zooko.com>
References: <FD2D95F2-9362-4BF0-BEB0-6F759577476A@python.org>
	<FB844080-DA20-42F7-AFAE-C02301A03B6A@zooko.com>
Message-ID: <8480A170-AAA8-40BF-A014-5359ADE0DDB6@python.org>

Hi Zooko,

On Jun 23, 2009, at 11:42 AM, Zooko Wilcox-O'Hearn wrote:

> What is the download cache?

It's where buildout caches packages it downloads and it's very handy  
because we can tell buildout to use a pre-defined download cache and  
to never download anything from the intarwebs.  Except that this  
doesn't work for setup_requires packages, so that seems like a bug in  
buildout.  We really don't want to add those to install_requires  
because the clients of our libraries don't care.

> I have a guess.  If you are installing with the setuptools "install"  
> command, then the things which are "install_requires" get installed  
> into some target location, but things which are build_requires get  
> installed into the PWD.

Right, that's another thing I don't particularly like about  
setup_requires.  I don't like those packages polluting my pwd.

> (If you don't specify a target, then it is the system site- 
> packages.  In my build system [1], we typically don't use "install"  
> and instead use "develop" with a --prefix argument which specifies  
> the target.  I assume virtualenv does something similar.)

I think so.  We use buildout, which has the moral equivalent of  
'develop'.

> Therefore, a later build will have its install_requires already  
> satisfied (if it is using the same installation target), but its  
> build_requirements will not be satisfied.

Right.

> If that's what's going on then a solution might involve pre- 
> installing setuptools_bzr onto the build system, or patching  
> setuptools to specify a location for build requirements and re-using  
> that location.

We could pre-install setuptools_bzr if we built an Ubuntu package for  
it.  I think the right thing for us is to look into patching buildout  
so that it can get its setup_requires from its download cache.

Cheers,
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090626/acf569c9/attachment-0001.pgp>

From pje at telecommunity.com  Sat Jun 27 01:50:53 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 26 Jun 2009 19:50:53 -0400
Subject: [Distutils] setup_requires not caching downloaded packages?
In-Reply-To: <85E22C58-7797-4157-BA35-20CB92629865@python.org>
References: <FD2D95F2-9362-4BF0-BEB0-6F759577476A@python.org>
	<20090623033733.38DC63A40D7@sparrow.telecommunity.com>
	<85E22C58-7797-4157-BA35-20CB92629865@python.org>
Message-ID: <20090626234755.D856D3A4099@sparrow.telecommunity.com>

At 05:02 PM 6/26/2009 -0400, Barry Warsaw wrote:
>>When you build an sdist using setuptools, it includes a pre- 
>>generated file manifest, along with a setup.cfg that forces the
>>right revision tag information in the project's version number (if
>>applicable).  This is specifically designed so that the person
>>building your sdist doesn't need the same revision control system
>>installed.
>>
>>In short, the only person who needs setuptools_bzr installed is the
>>person who's generating the sdists.  So, feel free to drop it from
>>setup_requires and be happy.  ;-)
>
>Here's what the setuptools docs say:
>
>(Note: projects listed in setup_requires will NOT be automatically
>installed on the system where the setup script is being run. They are
>simply downloaded to the setup directory if they're not locally
>available already. If you want them to be installed, as well as being
>available when the setup script is run, you should add them to
>install_requires and setup_requires.)
>
>So I get that setuptools is doing what it's supposed to do.  I also
>get that it's (usually) not too hard to write a MANIFEST.in that
>pretty much does what you need it to do.  I'm bothered by the fact
>that this puts us in a bind where we want to use setup_requires but
>really can't.  After further thought though, the "problem" is probably
>in buildout, because it shouldn't be downloading setup_requires
>packages if they are in its download cache.

You seem to be ignoring the part where *you don't need 
setup_requires* in order to get a correct manifest on the target 
system, as long as they're using an sdist.

In other words, the only people who need setuptools_bzr are the 
people *making* sdists, not the people *using* sdists.  You can 
therefore omit it from your setup_requires, if I understand your 
current situation correctly.


From optilude+lists at gmail.com  Sat Jun 27 15:06:50 2009
From: optilude+lists at gmail.com (Martin Aspeli)
Date: Sat, 27 Jun 2009 21:06:50 +0800
Subject: [Distutils] Why is zc.buildout picking the wrong platform for
	binary eggs?
Message-ID: <h255hd$leh$1@ger.gmane.org>

Hi,

As some of you may know, using lxml on OSX is a big pain because of an 
incompatible system version of libxml2. The solution is to ship binary 
eggs, which we already do for Windows.

Now, as a first test, Stefan has uploaded this egg:

   lxml-2.2.1-py2.4-macosx-10.5-i386.egg

See http://pypi.python.org/pypi/lxml/2.2.1

Now, I'm on OSX 10.5, and I've requested this exact version.

In a virtualenv, this works. It downloads the given binary egg.

In a buildout, it's preferring to download and build the egg itself, and 
it calls it lxml-2.2.1-py2.4-macosx-10.3-i386.egg. That's weird, because 
this machine has never had OSX 10.3 on it. :(

My Python 2.4.6 is installed via darwinports.

Now, I can probably work around this, but we're trying to ensure that 
people have a sane lxml egg that will install cleanly on a number of 
platforms. It's therefore pretty important that the average user doesn't 
get this.

So, I'd like to try and figure out what's going on. How does buildout 
decide which egg to look for? Why is easy_install in a virtualenv 
behaving differently to my buildout?

Martin


-- 
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book


From optilude+lists at gmail.com  Sat Jun 27 15:13:08 2009
From: optilude+lists at gmail.com (Martin Aspeli)
Date: Sat, 27 Jun 2009 21:13:08 +0800
Subject: [Distutils] Why is zc.buildout picking the wrong platform for
	binary eggs?
In-Reply-To: <h255hd$leh$1@ger.gmane.org>
References: <h255hd$leh$1@ger.gmane.org>
Message-ID: <h255t4$leh$3@ger.gmane.org>

Martin Aspeli wrote:
> Hi,
> 
> As some of you may know, using lxml on OSX is a big pain because of an 
> incompatible system version of libxml2. The solution is to ship binary 
> eggs, which we already do for Windows.
> 
> Now, as a first test, Stefan has uploaded this egg:
> 
>    lxml-2.2.1-py2.4-macosx-10.5-i386.egg
> 
> See http://pypi.python.org/pypi/lxml/2.2.1
> 
> Now, I'm on OSX 10.5, and I've requested this exact version.
> 
> In a virtualenv, this works. It downloads the given binary egg.
> 
> In a buildout, it's preferring to download and build the egg itself, and 
> it calls it lxml-2.2.1-py2.4-macosx-10.3-i386.egg. That's weird, because 
> this machine has never had OSX 10.3 on it. :(
> 
> My Python 2.4.6 is installed via darwinports.
> 
> Now, I can probably work around this, but we're trying to ensure that 
> people have a sane lxml egg that will install cleanly on a number of 
> platforms. It's therefore pretty important that the average user doesn't 
> get this.
> 
> So, I'd like to try and figure out what's going on. How does buildout 
> decide which egg to look for? Why is easy_install in a virtualenv 
> behaving differently to my buildout?

Okay - I just wiped my parts/, bin/ and .installed.cfg and ran bootstrap 
again. This time, it's building an egg called 10.5, but it's still 
building its own from the source rather than downloading the binary egg. :(

Martin

-- 
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book


From jim at zope.com  Sat Jun 27 15:29:29 2009
From: jim at zope.com (Jim Fulton)
Date: Sat, 27 Jun 2009 09:29:29 -0400
Subject: [Distutils] Why is zc.buildout picking the wrong platform for
	binary eggs?
In-Reply-To: <h255t4$leh$3@ger.gmane.org>
References: <h255hd$leh$1@ger.gmane.org> <h255t4$leh$3@ger.gmane.org>
Message-ID: <460CA60B-E768-4E40-9C36-F78A46AF4FA7@zope.com>

I plan to add an option to force source or binary eggs to be used.   
Something like:

   egg-type = any|binary|source

where any is the default.  I haven't gotten to this yet. :)

It would probably be nice to specify this at the part level.

Jim

On Jun 27, 2009, at 9:13 AM, Martin Aspeli wrote:

> Martin Aspeli wrote:
>> Hi,
>> As some of you may know, using lxml on OSX is a big pain because of  
>> an incompatible system version of libxml2. The solution is to ship  
>> binary eggs, which we already do for Windows.
>> Now, as a first test, Stefan has uploaded this egg:
>>   lxml-2.2.1-py2.4-macosx-10.5-i386.egg
>> See http://pypi.python.org/pypi/lxml/2.2.1
>> Now, I'm on OSX 10.5, and I've requested this exact version.
>> In a virtualenv, this works. It downloads the given binary egg.
>> In a buildout, it's preferring to download and build the egg  
>> itself, and it calls it lxml-2.2.1-py2.4-macosx-10.3-i386.egg.  
>> That's weird, because this machine has never had OSX 10.3 on it. :(
>> My Python 2.4.6 is installed via darwinports.
>> Now, I can probably work around this, but we're trying to ensure  
>> that people have a sane lxml egg that will install cleanly on a  
>> number of platforms. It's therefore pretty important that the  
>> average user doesn't get this.
>> So, I'd like to try and figure out what's going on. How does  
>> buildout decide which egg to look for? Why is easy_install in a  
>> virtualenv behaving differently to my buildout?
>
> Okay - I just wiped my parts/, bin/ and .installed.cfg and ran  
> bootstrap again. This time, it's building an egg called 10.5, but  
> it's still building its own from the source rather than downloading  
> the binary egg. :(
>
> Martin
>
> -- 
> Author of `Professional Plone Development`, a book for developers who
> want to work with Plone. See http://martinaspeli.net/plone-book
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig

--
Jim Fulton
Zope Corporation



From optilude+lists at gmail.com  Sat Jun 27 15:32:24 2009
From: optilude+lists at gmail.com (Martin Aspeli)
Date: Sat, 27 Jun 2009 21:32:24 +0800
Subject: [Distutils] Why is zc.buildout picking the wrong platform for
	binary eggs?
In-Reply-To: <460CA60B-E768-4E40-9C36-F78A46AF4FA7@zope.com>
References: <h255hd$leh$1@ger.gmane.org> <h255t4$leh$3@ger.gmane.org>
	<460CA60B-E768-4E40-9C36-F78A46AF4FA7@zope.com>
Message-ID: <h25718$o2p$1@ger.gmane.org>

Jim Fulton wrote:
> I plan to add an option to force source or binary eggs to be used.   
> Something like:
> 
>    egg-type = any|binary|source
> 
> where any is the default.  I haven't gotten to this yet. :)
> 
> It would probably be nice to specify this at the part level.

Agree.

In the meantime, I think I found the problem: I have a shared download 
cache, and the .tgz was cached there. When I removed it, buildout 
started preferring the egg on PyPI.

I still have no idea why it was building an osx-10.3 egg before I re-did 
bootstrap, though. Note that I'd upgraded my MacPorts supplied Python 
2.4 from 2.4.5 to 2.4.6 in between setting up the buildout initially, 
and doing the wipe-and-re-bootstrap.

Martin

-- 
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book


From h.goebel at goebel-consult.de  Sat Jun 27 16:20:52 2009
From: h.goebel at goebel-consult.de (Hartmut Goebel)
Date: Sat, 27 Jun 2009 16:20:52 +0200
Subject: [Distutils] Using setuptools as downloader for .msi/.exe
In-Reply-To: <20090612174631.C695C3A410E@sparrow.telecommunity.com>
References: <4A321393.4030306@goebel-consult.de>
	<20090612174631.C695C3A410E@sparrow.telecommunity.com>
Message-ID: <4A462AC4.6040108@goebel-consult.de>

P.J. Eby schrieb:

> See the setuptools.package_index module.  You probably want to
> subclass
> PackageIndex, and use your own versions of the distros_for_location()
> or interpret_distro_name() functions.

Thanks for the tips.

The task requires monkey-patching a "decorated" distros_for_location, a
new version of parse_bdist_winexe (not monkey-patched) and a new
function parse_bdist_winexe.

I decided against subclassing PackageIndex, since this would require
much more code duplication. (Some of the required methods are quite
large and call distros_for_location at the and anyway.)

For the records I'll post the download link here, as soon as the modules
is published.

-- 
Sch?nen Gru? - Regards
Hartmut Goebel
Dipl.-Informatiker (univ.), CISSP, CSSLP

Goebel Consult
Spezialist f?r IT-Sicherheit in komplexen Umgebungen
http://www.goebel-consult.de

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 4732 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090627/086c8613/attachment-0001.bin>

From h.goebel at goebel-consult.de  Sat Jun 27 16:25:33 2009
From: h.goebel at goebel-consult.de (Hartmut Goebel)
Date: Sat, 27 Jun 2009 16:25:33 +0200
Subject: [Distutils] easy_install fails on sourceforge?
Message-ID: <4A462BDD.1090101@goebel-consult.de>

Hi,

I'm struggling with easy_install to download packages from sourceforge.

Eg. pyOpenSSL can not be downloaded for windows. even if adding
<http://sourceforge.net/project/showfiles.php?group_id=31249&package_id=90289>
as find-link.

Any hints how I can work around this issue?

-- 
Sch?nen Gru? - Regards
Hartmut Goebel
Dipl.-Informatiker (univ.), CISSP, CSSLP

Goebel Consult
Spezialist f?r IT-Sicherheit in komplexen Umgebungen
http://www.goebel-consult.de

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 4732 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090627/921bb9c2/attachment.bin>

From dkdropbox at gmail.com  Sat Jun 27 17:36:40 2009
From: dkdropbox at gmail.com (David Kim)
Date: Sat, 27 Jun 2009 11:36:40 -0400
Subject: [Distutils] newbie bootstrap question
In-Reply-To: <4A4529F5.6080708@enthought.com>
References: <f5014d110906251620x29b7ccecq9123fdbd1587253a@mail.gmail.com> 
	<0040C94A-BC62-4F3E-AD61-32A7D624EB12@zope.com>
	<f5014d110906260804p6744da31n6cc9e6fe82fec6a5@mail.gmail.com> 
	<4A4529F5.6080708@enthought.com>
Message-ID: <f5014d110906270836o2709ae9u1904fe7b920e2111@mail.gmail.com>

Thanks Dave, appreciate the tip.

Dave Peterson <dpeterson at enthought.com> wrote:

> As a quick fix, you could re-write the code you have as
> pkg_resources.require('enstaller') and be on your way.
>

I tried replacing 'setuptools' with 'enstaller' in the code and got the
following result:

> Retrieved config from: /Users/davidkim/.enstallerrc

Retrieved config from: /Users/davidkim/.enstallerrc

There is no difference between the current state and the most recent saved
> state in the enstaller.cache, so the current state does not need to be
> saved.

There is no difference between the current state and the most recent saved
> state in the enstaller.cache, so the current state does not need to be
> saved.

Traceback (most recent call last):

  File "bootstrap.py", line 91, in <module>

    import zc.buildout.buildout

  File
> "/private/var/folders/A4/A44P935JH0OrA7hGlWeH7E+++TI/-Tmp-/tmpRKD30Q/zc.buildout-1.3.0-py2.5.egg/zc/buildout/buildout.py",
> line 35, in <module>

    import zc.buildout.easy_install

  File
> "/private/var/folders/A4/A44P935JH0OrA7hGlWeH7E+++TI/-Tmp-/tmpRKD30Q/zc.buildout-1.3.0-py2.5.egg/zc/buildout/easy_install.py",
> line 64, in <module>

     pkg_resources.Requirement.parse('setuptools')

AttributeError: 'NoneType' object has no attribute 'location'


It's unclear to me where these files are and how I can get to them. I used
the following bootstrap.py code.

import os, shutil, sys, tempfile, urllib2


> tmpeggs = tempfile.mkdtemp()


> is_jython = sys.platform.startswith('java')


> ez = {}

exec urllib2.urlopen('http://peak.telecommunity.com/dist/ez_setup.py'

                     ).read() in ez

ez['use_setuptools'](to_dir=tmpeggs, download_delay=0)


> import pkg_resources


> if sys.platform == 'win32':

    def quote(c):

        if ' ' in c:

            return '"%s"' % c # work around spawn lamosity on windows

        else:

            return c

else:

    def quote (c):

        return c


> cmd = 'from setuptools.command.easy_install import main; main()'

ws  = pkg_resources.working_set


> if len(sys.argv) > 2 and sys.argv[1] == '--version':

    VERSION = '==%s' % sys.argv[2]

    args = sys.argv[3:] + ['bootstrap']

else:

    VERSION = ''

    args = sys.argv[1:] + ['bootstrap']


> if is_jython:

    import subprocess


>     assert subprocess.Popen([sys.executable] + ['-c', quote(cmd), '-mqNxd',

           quote(tmpeggs), 'zc.buildout' + VERSION],

           env=dict(os.environ,

               PYTHONPATH=


> ws.find(pkg_resources.Requirement.parse('enstaller')).location

               ),

           ).wait() == 0


> else:

    assert os.spawnle(

        os.P_WAIT, sys.executable, quote (sys.executable),

        '-c', quote (cmd), '-mqNxd', quote (tmpeggs), 'zc.buildout' +
> VERSION,

        dict(os.environ,

            PYTHONPATH=

            ws.find(pkg_resources.Requirement.parse('enstaller')).location

            ),

        ) == 0


> ws.add_entry(tmpeggs)

ws.require('zc.buildout' + VERSION)

import zc.buildout.buildout

zc.buildout.buildout.main(args)

shutil.rmtree(tmpeggs)


Apologies in advance if I'm being dense. Thanks so much for the help.

DK
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090627/d5c1ef2d/attachment-0001.htm>

From santagada at gmail.com  Sat Jun 27 17:56:01 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Sat, 27 Jun 2009 12:56:01 -0300
Subject: [Distutils] easy_install fails on sourceforge?
In-Reply-To: <4A462BDD.1090101@goebel-consult.de>
References: <4A462BDD.1090101@goebel-consult.de>
Message-ID: <88FAF1A8-34F4-4F78-BB1C-3DFFBA4A4551@gmail.com>


On Jun 27, 2009, at 11:25 AM, Hartmut Goebel wrote:

> Hi,
>
> I'm struggling with easy_install to download packages from  
> sourceforge.
>
> Eg. pyOpenSSL can not be downloaded for windows. even if adding
> <http://sourceforge.net/project/showfiles.php?group_id=31249&package_id=90289 
> >
> as find-link.
>
> Any hints how I can work around this issue?

A workaround is looking if pip supports it and start recomending it to  
your users.

I vagaly remembers downloading things from sf links with easy_install,  
but it was somewhat long ago, maybe sf or setuptools changed since that.

[]'s
--
Leonardo Santagada
santagada at gmail.com




From pje at telecommunity.com  Sat Jun 27 19:02:04 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Sat, 27 Jun 2009 13:02:04 -0400
Subject: [Distutils] easy_install fails on sourceforge?
In-Reply-To: <4A462BDD.1090101@goebel-consult.de>
References: <4A462BDD.1090101@goebel-consult.de>
Message-ID: <20090627165906.95F943A4060@sparrow.telecommunity.com>

At 04:25 PM 6/27/2009 +0200, Hartmut Goebel wrote:
>Hi,
>
>I'm struggling with easy_install to download packages from sourceforge.
>
>Eg. pyOpenSSL can not be downloaded for windows. even if adding
><http://sourceforge.net/project/showfiles.php?group_id=31249&package_id=90289>
>as find-link.

Using that as a find-link works fine for me; it downloads the 0.9 
source package, since I don't have Python 2.6.  (The windows files 
shown there are all for Python 2.6.)


From h.goebel at goebel-consult.de  Sun Jun 28 19:57:28 2009
From: h.goebel at goebel-consult.de (Hartmut Goebel)
Date: Sun, 28 Jun 2009 19:57:28 +0200
Subject: [Distutils] easy_install fails on sourceforge?
In-Reply-To: <20090627165906.95F943A4060@sparrow.telecommunity.com>
References: <4A462BDD.1090101@goebel-consult.de>
	<20090627165906.95F943A4060@sparrow.telecommunity.com>
Message-ID: <4A47AF08.4030504@goebel-consult.de>

P.J. Eby schrieb:

> Using that as a find-link works fine for me; it downloads the 0.9 source
> package, since I don't have Python 2.6.  (The windows files shown there
> are all for Python 2.6.)

So my explanation has bee a bit short, I see.

This is not what I want. Esp. the source is not found at sourceforge but
via pypi :-)

Try without using an index. I would expect to get
pyOpenSSL-0.8.winxp32-py2.5.exe  from SF, but is is not found:

easy_install -i file://NUL -f
'http://sourceforge.net/project/showfiles.php?group_id=31249&package_id=90289'
pyOpenSSL==0.8

I use the following script for testing. Setting loglevel to 5 would
report any link found, but in this case no link is reported (and package
is found).

....8<-------------
from setuptools.command import easy_install
from distutils import log
log.set_verbosity(5) # this would print any found link

args = """
-i file://NUL
-f
http://sourceforge.net/project/showfiles.php?group_id=31249&package_id=90289
pyOpenSSL==0.8""".split()

easy_install.main(args)
....8<-------------


-- 
Sch?nen Gru? - Regards
Hartmut Goebel
Dipl.-Informatiker (univ.), CISSP, CSSLP

Goebel Consult
Spezialist f?r IT-Sicherheit in komplexen Umgebungen
http://www.goebel-consult.de

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 4732 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090628/4cb8f867/attachment.bin>

From barry at python.org  Sun Jun 28 22:24:39 2009
From: barry at python.org (Barry Warsaw)
Date: Sun, 28 Jun 2009 16:24:39 -0400
Subject: [Distutils] setup_requires not caching downloaded packages?
In-Reply-To: <20090626234755.D856D3A4099@sparrow.telecommunity.com>
References: <FD2D95F2-9362-4BF0-BEB0-6F759577476A@python.org>
	<20090623033733.38DC63A40D7@sparrow.telecommunity.com>
	<85E22C58-7797-4157-BA35-20CB92629865@python.org>
	<20090626234755.D856D3A4099@sparrow.telecommunity.com>
Message-ID: <36CDEE5D-F011-40ED-9986-79E3F8759ACC@python.org>

On Jun 26, 2009, at 7:50 PM, P.J. Eby wrote:

> You seem to be ignoring the part where *you don't need  
> setup_requires* in order to get a correct manifest on the target  
> system, as long as they're using an sdist.
>
> In other words, the only people who need setuptools_bzr are the  
> people *making* sdists, not the people *using* sdists.  You can  
> therefore omit it from your setup_requires, if I understand your  
> current situation correctly.

I get that.  The problem is that there's nothing like an  
'sdist_requires' which would list packages needed to make the sdist.   
I'm not the only one making dists for upload, and I'd like to be able  
to specify this requirement in the setup.py so that other people  
making dists would have less dependency on their environment (i.e.  
repeatablility).  So maybe setup_requires isn't the right thing to  
use, but it's the only hook to hang this hat on (AFAICT).

But maybe there is and I've just overlooked it.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090628/b957c499/attachment.pgp>

From chris at simplistix.co.uk  Mon Jun 29 11:15:17 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Mon, 29 Jun 2009 10:15:17 +0100
Subject: [Distutils] what does buildout do with scripts?
Message-ID: <4A488625.6030904@simplistix.co.uk>

Hi All,

What does buildout do with scripts specified in the scripts parameter to 
distutils'/setuptools' setup function?

The xlrd package defines one such script and when specified as a 
required egg in a zc.recipe.egg part, this script doesn't seem to show 
up anywhere...

cheers,

Chris


From jim at zope.com  Mon Jun 29 11:36:27 2009
From: jim at zope.com (Jim Fulton)
Date: Mon, 29 Jun 2009 05:36:27 -0400
Subject: [Distutils] what does buildout do with scripts?
In-Reply-To: <4A488625.6030904@simplistix.co.uk>
References: <4A488625.6030904@simplistix.co.uk>
Message-ID: <2A04D60A-F005-4C03-9C26-3EB3A5CE40F9@zope.com>


On Jun 29, 2009, at 5:15 AM, Chris Withers wrote:

> Hi All,
>
> What does buildout do with scripts specified in the scripts  
> parameter to distutils'/setuptools' setup function?
>
> The xlrd package defines one such script and when specified as a  
> required egg in a zc.recipe.egg part, this script doesn't seem to  
> show up anywhere...


buildout only supports scripts provided as entry points.

Jim

--
Jim Fulton
Zope Corporation



From slray at dow.com  Mon Jun 29 19:54:11 2009
From: slray at dow.com (Ray, Scott (SL))
Date: Mon, 29 Jun 2009 13:54:11 -0400
Subject: [Distutils] Easy Install/setuptools installer for Python 2.6 on
	Windows
Message-ID: <988BFB859872FD4B997EBC59585536AC033E1CF0@USMDLMDOWX030.dow.com>

I recently installed Python 2.6.2 on MS Windows and would like to get
Easy Install/setuptools working.  The associated download page
(http://pypi.python.org/pypi/setuptools) provides MS Windows .EXE
installers for Python versions 2.3 - 2.5 only.  Are there any prospects
for a 2.6 installer?  Or is there another reasonably straightforward way
to get Easy Install/setuptools built and installed?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090629/a5ddb989/attachment.htm>

From p.f.moore at gmail.com  Mon Jun 29 23:52:21 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 29 Jun 2009 22:52:21 +0100
Subject: [Distutils] Easy Install/setuptools installer for Python 2.6 on
	Windows
In-Reply-To: <988BFB859872FD4B997EBC59585536AC033E1CF0@USMDLMDOWX030.dow.com>
References: <988BFB859872FD4B997EBC59585536AC033E1CF0@USMDLMDOWX030.dow.com>
Message-ID: <79990c6b0906291452l53f1fe8fi8e93e5adffcab467@mail.gmail.com>

2009/6/29 Ray, Scott (SL) <slray at dow.com>:
> I recently installed Python 2.6.2 on MS Windows and would like to get Easy
> Install/setuptools working.? The associated download page
> (http://pypi.python.org/pypi/setuptools) provides MS Windows .EXE installers
> for Python versions 2.3 - 2.5 only.? Are there any prospects for a 2.6
> installer?? Or is there another reasonably straightforward way to get Easy
> Install/setuptools built and installed?

python setup.py bdist_wininst

There's no C code, so this doesn't need anything beyond Python itself.
Of course, it's annoyingly unfriendly that a built exe isn't provided,
to save confusion for people struggling with the bootstrapping
problem. But it's not actually an issue...

Paul.

From david.lyon at preisshare.net  Tue Jun 30 00:26:16 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Mon, 29 Jun 2009 18:26:16 -0400
Subject: [Distutils] Easy Install/setuptools installer for Python 2.6 on
	Windows
In-Reply-To: <988BFB859872FD4B997EBC59585536AC033E1CF0@USMDLMDOWX030.dow.com>
References: <988BFB859872FD4B997EBC59585536AC033E1CF0@USMDLMDOWX030.dow.com>
Message-ID: <a0b30cab45a8502da0af7bc5c0f8f01f@preisshare.net>

On Mon, 29 Jun 2009 13:54:11 -0400, "Ray, Scott (SL)" <slray at dow.com>
wrote:
> I recently installed Python 2.6.2 on MS Windows and would like to get
> Easy Install/setuptools working.  The associated download page
> (http://pypi.python.org/pypi/setuptools) provides MS Windows .EXE
> installers for Python versions 2.3 - 2.5 only.  Are there any prospects
> for a 2.6 installer?  Or is there another reasonably straightforward way
> to get Easy Install/setuptools built and installed?

The easiest way is download the Python Package Manager at:

http://sourceforge.net/projects/pythonpkgmgr/

It extends easy_install and will automatically install the
correct version of setuptools/easyinstall when you go
to install a package.

It also provides pypi search capability and a deinstall function
which can come in handy.

Regards

David



From strawman at astraw.com  Tue Jun 30 03:26:39 2009
From: strawman at astraw.com (Andrew Straw)
Date: Mon, 29 Jun 2009 18:26:39 -0700
Subject: [Distutils] easy_install fails on sourceforge?
In-Reply-To: <4A47AF08.4030504@goebel-consult.de>
References: <4A462BDD.1090101@goebel-consult.de>	<20090627165906.95F943A4060@sparrow.telecommunity.com>
	<4A47AF08.4030504@goebel-consult.de>
Message-ID: <4A4969CF.7040309@astraw.com>

I'm getting a similar issue (easy_install failing on sourceforge) when
trying to download Pyro:

$ easy_install -eb. Pyro
/usr/lib/python2.6/dist-packages/Pyrex/Compiler/Errors.py:17:
DeprecationWarning: BaseException.message has been deprecated as of
Python 2.6
  self.message = message
Searching for Pyro
Reading http://pypi.python.org/simple/Pyro/
Reading http://pyro.sourceforge.net
Reading http://sourceforge.net/project/showfiles.php?group_id=18837
No local packages or download links found for a source distribution of Pyro
error: Could not find suitable distribution for Requirement.parse('Pyro')

The thing is, there is a source distribution of Pyro right there on the
SourceForge website. Why doesn't this grab the .tar.gz file and build
and install it? I thought setuptools did this automatically. Perhaps my
memory is failing me. (It wouldn't be the first time.)

-Andrew

From pje at telecommunity.com  Tue Jun 30 03:52:01 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 29 Jun 2009 21:52:01 -0400
Subject: [Distutils] easy_install fails on sourceforge?
In-Reply-To: <4A4969CF.7040309@astraw.com>
References: <4A462BDD.1090101@goebel-consult.de>
	<20090627165906.95F943A4060@sparrow.telecommunity.com>
	<4A47AF08.4030504@goebel-consult.de> <4A4969CF.7040309@astraw.com>
Message-ID: <20090630014902.766153A4109@sparrow.telecommunity.com>

At 06:26 PM 6/29/2009 -0700, Andrew Straw wrote:
>I'm getting a similar issue (easy_install failing on sourceforge) when
>trying to download Pyro:
>
>$ easy_install -eb. Pyro
>/usr/lib/python2.6/dist-packages/Pyrex/Compiler/Errors.py:17:
>DeprecationWarning: BaseException.message has been deprecated as of
>Python 2.6
>   self.message = message
>Searching for Pyro
>Reading http://pypi.python.org/simple/Pyro/
>Reading http://pyro.sourceforge.net
>Reading http://sourceforge.net/project/showfiles.php?group_id=18837
>No local packages or download links found for a source distribution of Pyro
>error: Could not find suitable distribution for Requirement.parse('Pyro')
>
>The thing is, there is a source distribution of Pyro right there on the
>SourceForge website. Why doesn't this grab the .tar.gz file and build
>and install it? I thought setuptools did this automatically. Perhaps my
>memory is failing me. (It wouldn't be the first time.)

Two things.  First, the correct URL for Pyro downloads is 
http://sourceforge.net/project/showfiles.php?group_id=18837&package_id=29259 .

Second, Sourceforge has changed their pages again so that they no 
longer give direct links to downloadable files; instead, they link to 
a "downloading.php" which setuptools does not recognize as a downloadable file.

I'm not sure when this changed on Sourceforge, but I do hope that SF 
users will complain long and loud about it.


From SridharR at activestate.com  Tue Jun 30 04:19:25 2009
From: SridharR at activestate.com (Sridhar Ratnakumar)
Date: Mon, 29 Jun 2009 19:19:25 -0700
Subject: [Distutils] easy_install fails on sourceforge?
In-Reply-To: <20090630014902.766153A4109@sparrow.telecommunity.com>
References: <4A462BDD.1090101@goebel-consult.de>
	<20090627165906.95F943A4060@sparrow.telecommunity.com>
	<4A47AF08.4030504@goebel-consult.de> <4A4969CF.7040309@astraw.com>
	<20090630014902.766153A4109@sparrow.telecommunity.com>
Message-ID: <op.uwbfenj7brrvlq@double>

On Mon, 29 Jun 2009 18:52:01 -0700, P.J. Eby <pje at telecommunity.com> wrote:

> The thing is, there is a source distribution of Pyro right there on the
> SourceForge website. Why doesn't this grab the .tar.gz file and build
> and install it? I thought setuptools did this automatically. Perhaps my
> memory is failing me. (It wouldn't be the first time.)
>  Two things.  First, the correct URL for Pyro downloads is  
> http://sourceforge.net/project/showfiles.php?group_id=18837&package_id=29259  
> .
>  Second, Sourceforge has changed their pages again so that they no  
> longer give direct links to downloadable files; instead, they link to a  
> "downloading.php" which setuptools does not recognize as a downloadable  
> file.
>  I'm not sure when this changed on Sourceforge, but I do hope that SF  
> users will complain long and loud about it.

Just throwing out some ideas:

Wouldn't it be better for package authors to register their packages in  
PYPI? They don't have to upload the entire tarball .. but even giving a  
/direct link/ to this would be sufficient. Would providing pypi plugin  
that uploads sdist's to SourceForge (or freshmeat, so on) make this  
process easier for package authors? Maybe then the network effect will  
make setuptools not rely too much on the unreliable download logic .. as  
eventually most SF packages would have a download link on PyPI for every  
subsequent release.

-srid

From david.lyon at preisshare.net  Tue Jun 30 06:33:24 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 30 Jun 2009 00:33:24 -0400
Subject: [Distutils] =?utf-8?q?easy=5Finstall_fails_on_sourceforge=3F?=
In-Reply-To: <op.uwbfenj7brrvlq@double>
References: <4A462BDD.1090101@goebel-consult.de>
	<20090627165906.95F943A4060@sparrow.telecommunity.com>
	<4A47AF08.4030504@goebel-consult.de> <4A4969CF.7040309@astraw.com>
	<20090630014902.766153A4109@sparrow.telecommunity.com>
	<op.uwbfenj7brrvlq@double>
Message-ID: <f4d246dff03846afd7eebf6d544a4970@preisshare.net>

On Mon, 29 Jun 2009 19:19:25 -0700, "Sridhar Ratnakumar"
> Just throwing out some ideas:
> 
> Wouldn't it be better for package authors to register their packages in  
> PYPI? 

It already exists (be it in command line form). See:

http://docs.python.org/distutils/uploading.html

> They don't have to upload the entire tarball .. but even giving a  
> /direct link/ to this would be sufficient. Would providing pypi plugin  
> that uploads sdist's to SourceForge (or freshmeat, so on) make this  
> process easier for package authors? Maybe then the network effect will  
> make setuptools not rely too much on the unreliable download logic .. as 

> eventually most SF packages would have a download link on PyPI for every 

> subsequent release.

imho the distutils core code is quite good. But it is a little unfriendly
to those who don't like command lines (like me - because I can't remember
them)

I've been tossing around doing a GUI interface for building packages
and doing the pypi upload. I think a lot can be done to improve the
process too.

But you too can play a part to imrove things being at ActiveState..

Why don't you have a look at building in the Python Package Manager into
ActivePython. After all, ActivePerl has a package manager.. why can't
the same thing be available to python users?

Regards

David



From SridharR at activestate.com  Tue Jun 30 09:25:33 2009
From: SridharR at activestate.com (Sridhar Ratnakumar)
Date: Tue, 30 Jun 2009 00:25:33 -0700
Subject: [Distutils] easy_install fails on sourceforge?
In-Reply-To: <f4d246dff03846afd7eebf6d544a4970@preisshare.net>
References: <4A462BDD.1090101@goebel-consult.de>
	<20090627165906.95F943A4060@sparrow.telecommunity.com>
	<4A47AF08.4030504@goebel-consult.de> <4A4969CF.7040309@astraw.com>
	<20090630014902.766153A4109@sparrow.telecommunity.com>
	<op.uwbfenj7brrvlq@double>
	<f4d246dff03846afd7eebf6d544a4970@preisshare.net>
Message-ID: <4A49BDED.9080109@activestate.com>

On 6/29/2009 9:33 PM, David Lyon wrote:
> On Mon, 29 Jun 2009 19:19:25 -0700, "Sridhar Ratnakumar"
>> Just throwing out some ideas:
>>
>> Wouldn't it be better for package authors to register their packages in
>> PYPI?
>
> It already exists (be it in command line form). See:
>
> http://docs.python.org/distutils/uploading.html

Yup, I'm aware of this. I was making a general comment that it is better 
for package authors to register their packages in PyPI in such a way 
that they have the 'Download URL' set properly to the actual tarball .. 
instead of the SF download page (eg: see 'Pyro' in pypi). The 
implication of this is that .. setuptools users need not rely much on 
unreliable URL parsing methods.

> Why don't you have a look at building in the Python Package Manager into
> ActivePython. After all, ActivePerl has a package manager.. why can't
> the same thing be available to python users?

I recall Trent mentioned a while ago in this list that we started 
working on PyPM (Python Package Manager).

In fact, last week .. there was a first internal alpha of the tool which 
at the moment can download and install packages from the internal binary 
repository which has over 4000 package releases for mac/lin/win. 
Although, some important packages (eg: scipy, numpy, pyqt, etc..) are 
not yet in the repository .. and we need to deal with these 
build/metadata issues before making a public release.

-srid

From kiorky at cryptelium.net  Tue Jun 30 11:24:59 2009
From: kiorky at cryptelium.net (kiorky)
Date: Tue, 30 Jun 2009 11:24:59 +0200
Subject: [Distutils] what does buildout do with scripts?
In-Reply-To: <4A488625.6030904@simplistix.co.uk>
References: <4A488625.6030904@simplistix.co.uk>
Message-ID: <4A49D9EB.9070302@cryptelium.net>

Chris Withers a ?crit :
> Hi All,
>
> What does buildout do with scripts specified in the scripts parameter
> to distutils'/setuptools' setup function?
>
> The xlrd package defines one such script and when specified as a
> required egg in a zc.recipe.egg part, this script doesn't seem to show
> up anywhere...
>
> cheers,
>
> Chris
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
You can use minitage.recipe:scripts recipe to install them.
This is an another implementation to install eggs.
The recipe has some interresting features you won't have with
zc.recipe.egg (like on the fly patching/generating isolated eggs with
those patches, installing from arbitrary urls, and better dependency
handling)
See http://pypi.python.org/pypi/minitage.recipe for documentation

If you have bugs with it, poke me, i ll be plased to help fix them.

-- 
--
Cordialement,
KiOrKY
GPG Key FingerPrint: 0x1A1194B7681112AF




From chris at simplistix.co.uk  Tue Jun 30 13:59:15 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 30 Jun 2009 12:59:15 +0100
Subject: [Distutils] introspecting package data
Message-ID: <4A49FE13.2050902@simplistix.co.uk>

Hi All,

I hope this is a silly question and I'm just missing something, but how 
do I introspect information from setup.py when I'm not using setuptools?

(ie: pkg_resources for those who refuse to drink the setuptools koolaid?)

cheers,

Chris

From ziade.tarek at gmail.com  Tue Jun 30 14:23:28 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 30 Jun 2009 14:23:28 +0200
Subject: [Distutils] introspecting package data
In-Reply-To: <4A49FE13.2050902@simplistix.co.uk>
References: <4A49FE13.2050902@simplistix.co.uk>
Message-ID: <94bdd2610906300523l39fc6189s55466cfcd12531d0@mail.gmail.com>

On Tue, Jun 30, 2009 at 1:59 PM, Chris Withers<chris at simplistix.co.uk> wrote:
> Hi All,
>
> I hope this is a silly question and I'm just missing something, but how do I
> introspect information from setup.py when I'm not using setuptools?
>
> (ie: pkg_resources for those who refuse to drink the setuptools koolaid?)
>
> cheers,

What do you want to do exactly ? And do you do it with setuptools ?

If it's about setup.py I'm guesssing it's done on a distribution
source, not an installed source ?

>
> Chris
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | http://ziade.org

From david.lyon at preisshare.net  Tue Jun 30 14:25:15 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 30 Jun 2009 08:25:15 -0400
Subject: [Distutils] =?utf-8?q?easy=5Finstall_fails_on_sourceforge=3F?=
In-Reply-To: <4A49BDED.9080109@activestate.com>
References: <4A462BDD.1090101@goebel-consult.de>
	<20090627165906.95F943A4060@sparrow.telecommunity.com>
	<4A47AF08.4030504@goebel-consult.de> <4A4969CF.7040309@astraw.com>
	<20090630014902.766153A4109@sparrow.telecommunity.com>
	<op.uwbfenj7brrvlq@double>
	<f4d246dff03846afd7eebf6d544a4970@preisshare.net>
	<4A49BDED.9080109@activestate.com>
Message-ID: <55b010d574960529aa35180590281f8d@preisshare.net>

On Tue, 30 Jun 2009 00:25:33 -0700, Sridhar Ratnakumar
<SridharR at activestate.com> wrote:
> Yup, I'm aware of this. I was making a general comment that it is better 
> for package authors to register their packages in PyPI in such a way 
> that they have the 'Download URL' set properly to the actual tarball .. 
>...

> In fact, last week .. there was a first internal alpha of the tool which 
> at the moment can download and install packages from the internal binary 
> repository which has over 4000 package releases for mac/lin/win. 

ok - so on the one hand you are suggesting that people use pypi and
then on the other you are suggesting that activestate will have
its own package repository....

> Although, some important packages (eg: scipy, numpy, pyqt, etc..) are 
> not yet in the repository .. and we need to deal with these 
> build/metadata issues before making a public release.

Aha - sounds good.

David



From chris at simplistix.co.uk  Tue Jun 30 15:24:30 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 30 Jun 2009 14:24:30 +0100
Subject: [Distutils] introspecting package data
In-Reply-To: <94bdd2610906300523l39fc6189s55466cfcd12531d0@mail.gmail.com>
References: <4A49FE13.2050902@simplistix.co.uk>
	<94bdd2610906300523l39fc6189s55466cfcd12531d0@mail.gmail.com>
Message-ID: <4A4A120E.7070801@simplistix.co.uk>

Tarek Ziad? wrote:
> On Tue, Jun 30, 2009 at 1:59 PM, Chris Withers<chris at simplistix.co.uk> wrote:
>> Hi All,
>>
>> I hope this is a silly question and I'm just missing something, but how do I
>> introspect information from setup.py when I'm not using setuptools?
>>
>> (ie: pkg_resources for those who refuse to drink the setuptools koolaid?)
>>
>> cheers,
> 
> What do you want to do exactly ? And do you do it with setuptools ?
> 
> If it's about setup.py I'm guesssing it's done on a distribution
> source, not an installed source ?

I'm building Sphinx documentation, but I want the conf.py to be driven 
right from the package it's in, rather than having to manually enter the 
information again...

Chris


From jim at zope.com  Tue Jun 30 15:37:45 2009
From: jim at zope.com (Jim Fulton)
Date: Tue, 30 Jun 2009 09:37:45 -0400
Subject: [Distutils] introspecting package data
In-Reply-To: <4A4A120E.7070801@simplistix.co.uk>
References: <4A49FE13.2050902@simplistix.co.uk>
	<94bdd2610906300523l39fc6189s55466cfcd12531d0@mail.gmail.com>
	<4A4A120E.7070801@simplistix.co.uk>
Message-ID: <CDE24769-8203-4478-9CDA-5FFE39768251@zope.com>


On Jun 30, 2009, at 9:24 AM, Chris Withers wrote:

> Tarek Ziad? wrote:
>> On Tue, Jun 30, 2009 at 1:59 PM, Chris  
>> Withers<chris at simplistix.co.uk> wrote:
>>> Hi All,
>>>
>>> I hope this is a silly question and I'm just missing something,  
>>> but how do I
>>> introspect information from setup.py when I'm not using setuptools?
>>>
>>> (ie: pkg_resources for those who refuse to drink the setuptools  
>>> koolaid?)
>>>
>>> cheers,
>> What do you want to do exactly ? And do you do it with setuptools ?
>> If it's about setup.py I'm guesssing it's done on a distribution
>> source, not an installed source ?
>
> I'm building Sphinx documentation, but I want the conf.py to be  
> driven right from the package it's in, rather than having to  
> manually enter the information again...


I recommend creating a data file that both setup.py and sphinx's  
config.py read.

Jim

--
Jim Fulton
Zope Corporation



From david.lyon at preisshare.net  Tue Jun 30 16:04:52 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 30 Jun 2009 10:04:52 -0400
Subject: [Distutils] introspecting package data
In-Reply-To: <4A4A120E.7070801@simplistix.co.uk>
References: <4A49FE13.2050902@simplistix.co.uk>
	<94bdd2610906300523l39fc6189s55466cfcd12531d0@mail.gmail.com>
	<4A4A120E.7070801@simplistix.co.uk>
Message-ID: <2815db19a2e18a2aa5d38756116be6cc@preisshare.net>

On Tue, 30 Jun 2009 14:24:30 +0100, Chris Withers <chris at simplistix.co.uk>
wrote:
> I'm building Sphinx documentation, but I want the conf.py to be driven 
> right from the package it's in, rather than having to manually enter the 
> information again...

It's such a good point...

Maybe I have it wrong and don't know something... but I take it
you're talking about the information we already have to put in
the setup.py.....

--setup.py----------------------------
#!/usr/bin/env python

from distutils.core import setup

setup(name='myproject',
      version='0.11',
      description='myPetProject',
      author='James Smith',
      author_email='smith at nowhere.net',
      url='http://www.yahoo.net/pet',
      scripts=['petproject.py'],
      data_files=[('bitmaps', ['bm/b1.gif', 'bm/b2.gif']),
                  ('config', ['cfg/data.cfg']),
                  ('/etc/init.d', ['init-script'])]
      classifiers=[
          'Development Status :: 4 - Beta',
          'Environment :: Console',
          'Environment :: Web Environment',
          'Intended Audience :: End Users/Desktop',
          'Intended Audience :: Developers',
          'Intended Audience :: System Administrators',
          'License :: OSI Approved :: Python Software Foundation License',
          'Operating System :: MacOS :: MacOS X',
          'Operating System :: Microsoft :: Windows',
          'Operating System :: POSIX',
          'Programming Language :: Python',
          'Topic :: Communications :: Email',
          'Topic :: Office/Business',
          'Topic :: Software Development :: Bug Tracking',
          ],
      
     )

--------------------------------------

Yes, it would be cool to be able to read this out and
autoupdate the build/version numbers.....

Regards

David


From chris at simplistix.co.uk  Tue Jun 30 16:31:01 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 30 Jun 2009 15:31:01 +0100
Subject: [Distutils] introspecting package data
In-Reply-To: <CDE24769-8203-4478-9CDA-5FFE39768251@zope.com>
References: <4A49FE13.2050902@simplistix.co.uk>
	<94bdd2610906300523l39fc6189s55466cfcd12531d0@mail.gmail.com>
	<4A4A120E.7070801@simplistix.co.uk>
	<CDE24769-8203-4478-9CDA-5FFE39768251@zope.com>
Message-ID: <4A4A21A5.7010102@simplistix.co.uk>

Jim Fulton wrote:
> 
>>> What do you want to do exactly ? And do you do it with setuptools ?
>>> If it's about setup.py I'm guesssing it's done on a distribution
>>> source, not an installed source ?
>>
>> I'm building Sphinx documentation, but I want the conf.py to be driven 
>> right from the package it's in, rather than having to manually enter 
>> the information again...
> 
> I recommend creating a data file that both setup.py and sphinx's 
> config.py read.

Declarative specification of package metadata in a non-python format?

Heresy! ;-)

Chris


From chris at simplistix.co.uk  Tue Jun 30 16:45:56 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 30 Jun 2009 15:45:56 +0100
Subject: [Distutils] introspecting package data
In-Reply-To: <2815db19a2e18a2aa5d38756116be6cc@preisshare.net>
References: <4A49FE13.2050902@simplistix.co.uk>	<94bdd2610906300523l39fc6189s55466cfcd12531d0@mail.gmail.com>	<4A4A120E.7070801@simplistix.co.uk>
	<2815db19a2e18a2aa5d38756116be6cc@preisshare.net>
Message-ID: <4A4A2524.8000706@simplistix.co.uk>

David Lyon wrote:
> Yes, it would be cool to be able to read this out and
> autoupdate the build/version numbers.....

Bingo.

I find it pretty surprising, given Sphinx's acceptance in the python 
world, that no-one appears to have done this this!

Chris


From david.lyon at preisshare.net  Tue Jun 30 17:06:08 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 30 Jun 2009 11:06:08 -0400
Subject: [Distutils] introspecting package data
In-Reply-To: <4A4A2524.8000706@simplistix.co.uk>
References: <4A49FE13.2050902@simplistix.co.uk>	<94bdd2610906300523l39fc6189s55466cfcd12531d0@mail.gmail.com>	<4A4A120E.7070801@simplistix.co.uk>
	<2815db19a2e18a2aa5d38756116be6cc@preisshare.net>
	<4A4A2524.8000706@simplistix.co.uk>
Message-ID: <2c860aa5414e6733735cf05facf5dd49@preisshare.net>

On Tue, 30 Jun 2009 15:45:56 +0100, Chris Withers
wrote:
>> Yes, it would be cool to be able to read this out and
>> autoupdate the build/version numbers.....
> 
> Bingo.
> 
> I find it pretty surprising, given Sphinx's acceptance in the python 
> world, that no-one appears to have done this this!

Well, there could be many reasons for that....

I'd say that writing the code to accomplish what you want is trivial.

We could just write a script that:

 - reads the setup.py, 
 - takes out the "from distutils.core.." 
 - replace it with something else...
 - use the metadata (already in memory) to
 - update whatever it is in sphinx (?conf.py)
 - write that file as an 'updatesphinx.py' or whatever
 - run it and bingo

Surely at some point, integrating sphinx into distutils has got to
get on the roadmap.... (???)

What project doesn't need better documentation?

What better python tool is there than sphinx?

How good would it be if all projects had sphinx based documentation.

So it leaves me with the question - why not? (maybe nobody is listening -
haha)

David







From ziade.tarek at gmail.com  Tue Jun 30 17:18:45 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 30 Jun 2009 17:18:45 +0200
Subject: [Distutils] introspecting package data
In-Reply-To: <4A4A21A5.7010102@simplistix.co.uk>
References: <4A49FE13.2050902@simplistix.co.uk>
	<94bdd2610906300523l39fc6189s55466cfcd12531d0@mail.gmail.com>
	<4A4A120E.7070801@simplistix.co.uk>
	<CDE24769-8203-4478-9CDA-5FFE39768251@zope.com>
	<4A4A21A5.7010102@simplistix.co.uk>
Message-ID: <94bdd2610906300818g20c82215g48601ad54767c985@mail.gmail.com>

2009/6/30 Chris Withers <chris at simplistix.co.uk>:
> Jim Fulton wrote:
>>
>>>> What do you want to do exactly ? And do you do it with setuptools ?
>>>> If it's about setup.py I'm guesssing it's done on a distribution
>>>> source, not an installed source ?
>>>
>>> I'm building Sphinx documentation, but I want the conf.py to be driven
>>> right from the package it's in, rather than having to manually enter the
>>> information again...
>>
>> I recommend creating a data file that both setup.py and sphinx's config.py
>> read.
>
> Declarative specification of package metadata in a non-python format?
>
> Heresy! ;-)

Yes,  as long as the metadata are in a python file, you're in trouble
to automate anything like you are mentioning.
But a static metadata system needs to provide a registery mechanism,
so we can call some arbitrary code
when some metadata are generated at installation time for instance.

In any case, one of the goal we have is to move to a static-based
metadata solution.

This work has been initiated at Pycon by Tres and Matthias, see the notes here

http://wiki.python.org/moin/Distutils/StaticMetadata

if you guys want to start filling your ideas in this page, below
Tres's notes, this would be helpfull

This is the "next" big topic I guess, after PEP 376, PEP 386, and PEP
345 extension.

Tarek

From chris at simplistix.co.uk  Tue Jun 30 18:49:13 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 30 Jun 2009 17:49:13 +0100
Subject: [Distutils] setuptools v. subversion 1.6
Message-ID: <4A4A4209.4000905@simplistix.co.uk>

Hi All,

I see setuptools still doesn't like subversion 1.6 working copies.
Is that likely to change this century? :-(

Worse still, I tried using:

http://svn.collab.net/repos/svn/trunk/tools/client-side/change-svn-wc-format.py

..to downgrade the working copy with:

python ../change-svn_wc_format.py --verbose . 1.4

However, setuptools still whines:

$ bin/buildout setup setup.py sdist
Running setup script 'setup.py'.
running sdist
running egg_info
writing requirements to mypackage.egg-info/requires.txt
writing mypackage.egg-info/PKG-INFO
writing top-level names to mypackage.egg-info/top_level.txt
writing dependency_links to mypackage.egg-info/dependency_links.txt
unrecognized .svn/entries format in

...uselessly: it does't actually tell me where the entries file is that 
it has problems with.

What am I doing wrong? Is this a bug in change-svn-wc-format.py?
How can I work around this?

cheers,

Chris

From jim at zope.com  Tue Jun 30 19:34:56 2009
From: jim at zope.com (Jim Fulton)
Date: Tue, 30 Jun 2009 13:34:56 -0400
Subject: [Distutils] setuptools v. subversion 1.6
In-Reply-To: <4A4A4209.4000905@simplistix.co.uk>
References: <4A4A4209.4000905@simplistix.co.uk>
Message-ID: <B98C64D0-5154-4CCE-8293-14D6EDE0DC43@zope.com>


IMO, include_package_data is an attractive nuisance. For newer  
projects, I just use package_data with a few globs.

Jim

On Jun 30, 2009, at 12:49 PM, Chris Withers wrote:

> Hi All,
>
> I see setuptools still doesn't like subversion 1.6 working copies.
> Is that likely to change this century? :-(
>
> Worse still, I tried using:
>
> http://svn.collab.net/repos/svn/trunk/tools/client-side/change-svn-wc-format.py
>
> ..to downgrade the working copy with:
>
> python ../change-svn_wc_format.py --verbose . 1.4
>
> However, setuptools still whines:
>
> $ bin/buildout setup setup.py sdist
> Running setup script 'setup.py'.
> running sdist
> running egg_info
> writing requirements to mypackage.egg-info/requires.txt
> writing mypackage.egg-info/PKG-INFO
> writing top-level names to mypackage.egg-info/top_level.txt
> writing dependency_links to mypackage.egg-info/dependency_links.txt
> unrecognized .svn/entries format in
>
> ...uselessly: it does't actually tell me where the entries file is  
> that it has problems with.
>
> What am I doing wrong? Is this a bug in change-svn-wc-format.py?
> How can I work around this?
>
> cheers,
>
> Chris
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig

--
Jim Fulton
Zope Corporation



From SridharR at activestate.com  Tue Jun 30 19:37:18 2009
From: SridharR at activestate.com (Sridhar Ratnakumar)
Date: Tue, 30 Jun 2009 10:37:18 -0700
Subject: [Distutils] easy_install fails on sourceforge?
In-Reply-To: <55b010d574960529aa35180590281f8d@preisshare.net>
References: <4A462BDD.1090101@goebel-consult.de>
	<20090627165906.95F943A4060@sparrow.telecommunity.com>
	<4A47AF08.4030504@goebel-consult.de> <4A4969CF.7040309@astraw.com>
	<20090630014902.766153A4109@sparrow.telecommunity.com>
	<op.uwbfenj7brrvlq@double>
	<f4d246dff03846afd7eebf6d544a4970@preisshare.net>
	<4A49BDED.9080109@activestate.com>
	<55b010d574960529aa35180590281f8d@preisshare.net>
Message-ID: <4A4A4D4E.7090504@activestate.com>

On 6/30/2009 5:25 AM, David Lyon wrote:
> ok - so on the one hand you are suggesting that people use pypi and
> then on the other you are suggesting that activestate will have
> its own package repository....

ActiveState uses PyPI to get package tarballs in first place. So, for 
example, if John uploads "Foo 0.1" to PyPI (at least register an entry 
.. and point the download URL of tarball), then it will become 
automatically available in the ActiveState repository.

Just like how ppm works with CPAN.

I am also adding some functionality to z3c.pypimirror with respect to 
its external download features .. with the intent to submit patches back 
to the developers.

-srid

From chris at simplistix.co.uk  Tue Jun 30 19:44:52 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 30 Jun 2009 18:44:52 +0100
Subject: [Distutils] setuptools v. subversion 1.6
In-Reply-To: <B98C64D0-5154-4CCE-8293-14D6EDE0DC43@zope.com>
References: <4A4A4209.4000905@simplistix.co.uk>
	<B98C64D0-5154-4CCE-8293-14D6EDE0DC43@zope.com>
Message-ID: <4A4A4F14.7050408@simplistix.co.uk>

Jim Fulton wrote:
> 
> IMO, include_package_data is an attractive nuisance. For newer projects, 
> I just use package_data with a few globs.

It's all I've ever known :-S

Where are the docs for what I should use instead of the packages and 
incude_package_data parameters to setuptools's setup function and the 
find_packages function?

cheers,

Chris


From jim at zope.com  Tue Jun 30 19:53:28 2009
From: jim at zope.com (Jim Fulton)
Date: Tue, 30 Jun 2009 13:53:28 -0400
Subject: [Distutils] setuptools v. subversion 1.6
In-Reply-To: <4A4A4F14.7050408@simplistix.co.uk>
References: <4A4A4209.4000905@simplistix.co.uk>
	<B98C64D0-5154-4CCE-8293-14D6EDE0DC43@zope.com>
	<4A4A4F14.7050408@simplistix.co.uk>
Message-ID: <366EB75A-9810-44E0-AC00-AD9BA90A4529@zope.com>


On Jun 30, 2009, at 1:44 PM, Chris Withers wrote:

> Jim Fulton wrote:
>> IMO, include_package_data is an attractive nuisance. For newer  
>> projects, I just use package_data with a few globs.
>
> It's all I've ever known :-S
>
> Where are the docs for what I should use instead of the packages and  
> incude_package_data parameters to setuptools's setup function and  
> the find_packages function?

The packages function just takes a list of packages. I doubt that  
find_packages depends on subversion. It doesn't buy you much in any  
case, as the expected value is simply a list of the packages to  
include in the distribution.

As far as package data, see:

   http://docs.python.org/distutils/setupscript.html#installing-package-data

Jim

--
Jim Fulton
Zope Corporation



From h.goebel at goebel-consult.de  Tue Jun 30 20:01:57 2009
From: h.goebel at goebel-consult.de (Hartmut Goebel)
Date: Tue, 30 Jun 2009 20:01:57 +0200
Subject: [Distutils] easy_install fails on sourceforge?
In-Reply-To: <op.uwbfenj7brrvlq@double>
References: <4A462BDD.1090101@goebel-consult.de>	<20090627165906.95F943A4060@sparrow.telecommunity.com>	<4A47AF08.4030504@goebel-consult.de>
	<4A4969CF.7040309@astraw.com>	<20090630014902.766153A4109@sparrow.telecommunity.com>
	<op.uwbfenj7brrvlq@double>
Message-ID: <4A4A5315.9040909@goebel-consult.de>

Hi,

Sridhar Ratnakumar schrieb:

> Wouldn't it be better for package authors to register their packages in
> PYPI? They don't have to upload the entire tarball .. but even giving a
> /direct link/ to this would be sufficient. Would providing pypi plugin

AFAIK this would require the package owner to give all the opload-links.
This fails for the common case of prebuild Windows binaries, which are
often provided by independent project. Examples are GTK (which in
addition provided some different packages) and psycopg2.

-- 
Sch?nen Gru? - Regards
Hartmut Goebel
Dipl.-Informatiker (univ.), CISSP, CSSLP

Goebel Consult
Spezialist f?r IT-Sicherheit in komplexen Umgebungen
http://www.goebel-consult.de

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 4732 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090630/bc03600e/attachment.bin>

From chris at simplistix.co.uk  Tue Jun 30 20:04:28 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 30 Jun 2009 19:04:28 +0100
Subject: [Distutils] setuptools v. subversion 1.6
In-Reply-To: <366EB75A-9810-44E0-AC00-AD9BA90A4529@zope.com>
References: <4A4A4209.4000905@simplistix.co.uk>
	<B98C64D0-5154-4CCE-8293-14D6EDE0DC43@zope.com>
	<4A4A4F14.7050408@simplistix.co.uk>
	<366EB75A-9810-44E0-AC00-AD9BA90A4529@zope.com>
Message-ID: <4A4A53AC.2030707@simplistix.co.uk>

Jim Fulton wrote:
> 
> As far as package data, see:
> 
>   http://docs.python.org/distutils/setupscript.html#installing-package-data

Ah cool...

Is package data included in source *and* egg/other distributions?
If the answer is "yes", is there an equivalent where the answer is "no, 
the specified files are only included in the source distro"? ;-)

I'm asking as the answer might well be a solution to this question:
http://groups.google.com/group/sphinx-dev/browse_thread/thread/903dcbfe04f898

cheers,

Chris


From h.goebel at goebel-consult.de  Tue Jun 30 20:05:29 2009
From: h.goebel at goebel-consult.de (Hartmut Goebel)
Date: Tue, 30 Jun 2009 20:05:29 +0200
Subject: [Distutils] easy_install fails on sourceforge?
In-Reply-To: <20090630014902.766153A4109@sparrow.telecommunity.com>
References: <4A462BDD.1090101@goebel-consult.de>	<20090627165906.95F943A4060@sparrow.telecommunity.com>	<4A47AF08.4030504@goebel-consult.de>
	<4A4969CF.7040309@astraw.com>
	<20090630014902.766153A4109@sparrow.telecommunity.com>
Message-ID: <4A4A53E9.3080702@goebel-consult.de>

Hi,

P.J. Eby schrieb:

> Second, Sourceforge has changed their pages again so that they no longer
> give direct links to downloadable files; instead, they link to a
> "downloading.php" which setuptools does not recognize as a downloadable
> file.

For my usecase I appended this to PackageIndex.process_url():

SF_ONCLICK =
re.compile(r"""onClick\s*=\s*"\s*init_download\(\s*'(.*?)'\);\s*\"""", re.I)

        # handle sourceforge links
        for match in SF_ONCLICK.finditer(page):
            link = urlparse.urljoin(base, htmldecode(match.group(1)))
            self.process_url(link)

This checks the onClick attribute.

Not nice, but works.

-- 
Sch?nen Gru? - Regards
Hartmut Goebel
Dipl.-Informatiker (univ.), CISSP, CSSLP

Goebel Consult
Spezialist f?r IT-Sicherheit in komplexen Umgebungen
http://www.goebel-consult.de

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 4732 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090630/3eaf2fda/attachment.bin>

From tseaver at palladion.com  Tue Jun 30 20:47:54 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 30 Jun 2009 14:47:54 -0400
Subject: [Distutils] introspecting package data
In-Reply-To: <4A49FE13.2050902@simplistix.co.uk>
References: <4A49FE13.2050902@simplistix.co.uk>
Message-ID: <h2dmkr$1id$2@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Chris Withers wrote:

> I hope this is a silly question and I'm just missing something, but how 
> do I introspect information from setup.py when I'm not using setuptools?
> 
> (ie: pkg_resources for those who refuse to drink the setuptools koolaid?)

This package might help:

  http://pypi.python.org/pypi/pkginfo

It uses the PKG-INFO files in "develop eggs," built distributions and /
or installed packages (Python 2.6+), rather than setup.py.



Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFKSl3a+gerLs4ltQ4RApYTAKDJfZ/vtwQZ8lF0MvXopYDMFMHAFQCfbBkh
1UqiyOMKyMRpqLWXSijJHYo=
=Ywzx
-----END PGP SIGNATURE-----