From manlio.perillo at gmail.com  Thu Mar  3 23:05:44 2011
From: manlio.perillo at gmail.com (Manlio Perillo)
Date: Thu, 03 Mar 2011 23:05:44 +0100
Subject: [Distutils] issues with namespace packages on Debian Squeeze and
	Python 2.6
Message-ID: <4D7010B8.4070408@gmail.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi.

It seems I'm having some trobles with setuptools and namespace packages
under Debian Squeeze and Python 2.6.

One thing to say about Python 2.6 on Debian Squeeze is that "local"
packages are installed under /usr/local/lib/python2.6/dist-packages/.

I have tried with system default python 2.5, and namespace packages
works. The same when I create a virtual environment with python 2.6.
The problem seems to just be with system default python 2.6.

I'm using the original setuptools (installed from sources), and not the
one provided by Debian.

Before trying to dig more into the issue, I would like to know if
someone else is experiencing the same problem.


Thanks   Manlio
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk1wELgACgkQscQJ24LbaUTtNwCfe5tj21cHc2IpDbDQShoJ+7zb
ml8AoI0h9IKuScKE/mZIkKZGbFC5ILka
=0sg2
-----END PGP SIGNATURE-----

From prologic at shortcircuit.net.au  Fri Mar  4 05:12:57 2011
From: prologic at shortcircuit.net.au (James Mills)
Date: Fri, 4 Mar 2011 14:12:57 +1000
Subject: [Distutils] Hello all :)
Message-ID: <AANLkTinkcAJbumRfOEzEUqFNmy7GzsWMf+X+ZjKD2kV2@mail.gmail.com>

Hi, I'm James Mills.
Very interested in how packaging and installation
is going wrt Python 3.

I re-ported pip to Python 3 (1) but it doesn't seem to
be very useful in general as other folks want to remain
backwards and forward compatible with a single codebase.

Anyway just introducing myself,

cheers
James

1. http://bitbucket.org/prologic/pip-py3k/

-- 
-- James Mills
--
-- "Problems are solved by method"

From vinay_sajip at yahoo.co.uk  Fri Mar  4 16:04:44 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 4 Mar 2011 15:04:44 +0000 (UTC)
Subject: [Distutils] Hello all :)
References: <AANLkTinkcAJbumRfOEzEUqFNmy7GzsWMf+X+ZjKD2kV2@mail.gmail.com>
Message-ID: <loom.20110304T154445-210@post.gmane.org>

James Mills <prologic <at> shortcircuit.net.au> writes:

> Hi, I'm James Mills.
> Very interested in how packaging and installation
> is going wrt Python 3.
> 
> I re-ported pip to Python 3 (1) but it doesn't seem to
> be very useful in general as other folks want to remain
> backwards and forward compatible with a single codebase.
> 

Hi James,

I just joined this list myself, as I have recently become actively interested in
pip, virtualenv and running them on py3k. I've also recently forked pip to make
a Py3K port (as an experiment) and am making what seems like good progress. From
my post to the virtualenv list:

A pip Py3k port needs to work with a virtualenv Py3k port. A
common codebase should IMO be maintained for Python 3.x and 2.x. My
experiments seem to indicate that this is feasible. An experimental
port of pip, virtualenv and virtualenvwrapper to Py3k is showing
promise.

Forks are here and only a few days old, so up to date:

https://bitbucket.org/vinay.sajip/virtualenv
https://bitbucket.org/vinay.sajip/virtualenvwrapper

With the above, I can create virtualenvs using either Python 3.2 or Python 2.7.1
(other versions not tested yet).

My pip fork is at

https://bitbucket.org/vinay.sajip/pip

and my pip changes are not yet public (still doing tests), but results are
encouraging. The aim for all Py3k porting that I do is to have a single code
base for 2.x and 3.x - which seems perfectly feasible for pip, virtualenv and
virtualenvwrapper. I've posted test results in this gist:

https://gist.github.com/852552

which can be summarised as

Vanilla pip trunk - tests with Python 2.7.1 - 102 run, 4  failures
Ported  pip trunk - tests with Python 2.7.1 - 102 run, 3  failures
Ported  pip trunk - tests with Python 3.2   - 101 run, 12 failures

All tests are with my virtualenv fork on a Ubuntu machine.

One issue which came up for me is that distutils-0.6.14 does not work with
Python 3.2, because of the abiflags configuration parameter which is new with
Python 3.2. I got around this using a tarball created from distribute trunk
("distribute-0.6.15dev.tar.gz") which I use with my virtualenv port.

Did you come across this issue? Are you using virtualenv3 with your pip fork?
How are your test results looking?

I think a single codebase for 2.x and 3.x is the way to go here - anything else
will be too much of a pain to keep in sync.  So far, I see no reasons why this
should not be feasible.

I'm waiting for Ian Bicking's comments: I made a pull request on my virtualenv
changes (which are IMO uncontroversial) but I guess he might be busy, what with
PyCon being next week.

If one of the distribute devs could give some idea on this list of when a
distribute-0.6.15 might be produced, that would be good. I can then use the
official distribute-0.6.15.tar.gz :-)

All in all, this porting exercise has not been as hard as I feared! I must be
missing something.

Regards,

Vinay Sajip


From regebro at gmail.com  Sun Mar  6 09:46:44 2011
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 6 Mar 2011 09:46:44 +0100
Subject: [Distutils] Name the software! Package quality tester.
Message-ID: <AANLkTim02Dv=J9RmQdVhWXRdML8rsvRNzRe-A6BP0290@mail.gmail.com>

I've started working on a little utility to give a quality rating on
packages, expressed in 0-10 points, and also in cheese types,
according to smellyness.

It's going to check for things like that it has all meta data it
should have, such as author_email, specifies Python versions via the
trove classifiers (currently works) and that it specifies all
dependencies (still todo). It will support both checking on a package
(works currently) a distribution file and PyPI (still to do).

It's not a uniqe idea, it overlaps with Andreas Jungs
zopyx.trashfinder in scope, and it will also in the case of checking a
package on PyPI check that there are several people that have owner
access, and hence include the functionality of mr.parker. (In fact
when checking on PyPI it will also check if there are documentation on
packages.python.org, that the distribution files are uploaded to PyPI,
etc, but this is all still todo). But I didn't find anything else, and
I wanted bigger scopes than both these in what to check in and which
cases.

But, before I move this to a public repository and upload it to PyPI,
there is one important thing to be determined: What should it be
called? Currently I'm calling it "pypilib.quality". I don't mind this
kind of boring names, but there is currently not a pypilib namespace,
and I don't want to just create top level namespaces left and right
for no reason. So other names are welcome. It doesn't have to have a
namespace either.

In the long run I would not mind to see this utility integrated into a
general pypi/cheeseshop script with other utility commands, which even
could include installing and removing, thusly giving Perl people what
they think they want a "CPAN" for Python. :-)

-- 
Lennart Regebro: http://regebro.wordpress.com/
The Python 3 Porting book is out: http://python3porting.com/
+33 661 58 14 64

From benji at benjiyork.com  Sun Mar  6 15:38:41 2011
From: benji at benjiyork.com (Benji York)
Date: Sun, 6 Mar 2011 09:38:41 -0500
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <AANLkTim02Dv=J9RmQdVhWXRdML8rsvRNzRe-A6BP0290@mail.gmail.com>
References: <AANLkTim02Dv=J9RmQdVhWXRdML8rsvRNzRe-A6BP0290@mail.gmail.com>
Message-ID: <AANLkTi=gHVqSLofbU3q_aC26ETCez4anpYU5D0u+GDpK@mail.gmail.com>

On Mar 6, 2011 3:47 AM, "Lennart Regebro" <regebro at gmail.com> wrote:
>
> I've started working on a little utility to give a quality rating on
> packages, expressed in 0-10 points, and also in cheese types,
> according to smellyness.

How about "cheese inspecter"?

--
Benji York
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110306/4b824019/attachment.html>

From jim at zope.com  Sun Mar  6 16:34:01 2011
From: jim at zope.com (Jim Fulton)
Date: Sun, 6 Mar 2011 10:34:01 -0500
Subject: [Distutils] Proposed change to buildout configuration-file handling
Message-ID: <AANLkTi=whFcCFq4uUnXadx3dS7tf9az+dz1g8TT2JvE7@mail.gmail.com>

An annoyance in buildout configurations, resulting from it's use of
ConfigParser, is that configuration data lines are stripped of leading
spaces. This means configurations like this:

[paste.ini]
recipe = zc.recipe.deployment:configuration
text =
  [app:main]
  use = egg:bobo
  bobo_resources = cmsapp
  filter-with = zodb

  [filter:zodb]
  use = egg:zc.zodbwsgi
  configuration =
    <zodb main>
       <zeoclient>
          server ${main-db-server:address}
       </zeoclient>
    </zodb>
    <zodb index>
       <zeoclient>
          server ${index-db-server:address}
       </zeoclient>
    </zodb>

  [server:main]
  use = egg:Paste#http
  host = localhost
  port = 8080

Don't work because leading whitespace is stripped, forcing me to use
hacks like this::

[paste.ini]
recipe = zc.recipe.deployment:configuration
s =
text =
  ${:s}[app:main]
  ${:s}use = egg:bobo
  ${:s}bobo_resources = cmsapp
  ${:s}filter-with = zodb
  ${:s}
  ${:s}[filter:zodb]
  ${:s}use = egg:zc.zodbwsgi
  ${:s}configuration =
  ${:s}  <zodb main>
  ${:s}     <zeoclient>
  ${:s}        server ${main-db-server:address}
  ${:s}     </zeoclient>
  ${:s}  </zodb>
  ${:s}  <zodb index>
  ${:s}     <zeoclient>
  ${:s}        server ${index-db-server:address}
  ${:s}     </zeoclient>
  ${:s}  </zodb>
  ${:s}
  ${:s}[server:main]
  ${:s}use = egg:Paste#http
  ${:s}host = localhost
  ${:s}port = 8080

:(

I propose to give buildout it's own parser that begaves like
ConfigParser, with the exeption that leading whitespace won't be
stripped, but will be dedented with textwrap.dedent.

There is some potential for breakage as some option values without
leading whitespace before would get it now, but my sense is that this
would be unlikely.

Thoughts?  If I did that (and I really want to do this) I'd release a
beta to help people assess the impact.

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From bradallen137 at gmail.com  Sun Mar  6 16:51:38 2011
From: bradallen137 at gmail.com (Brad Allen)
Date: Sun, 6 Mar 2011 15:51:38 +0000
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <AANLkTi=gHVqSLofbU3q_aC26ETCez4anpYU5D0u+GDpK@mail.gmail.com>
References: <AANLkTim02Dv=J9RmQdVhWXRdML8rsvRNzRe-A6BP0290@mail.gmail.com>
	<AANLkTi=gHVqSLofbU3q_aC26ETCez4anpYU5D0u+GDpK@mail.gmail.com>
Message-ID: <AANLkTimHzQCJ7j_c7XZkPA4CTANV8epgFb_Hp6aarcEZ@mail.gmail.com>

On Sun, Mar 6, 2011 at 2:38 PM, Benji York <benji at benjiyork.com> wrote:
> On Mar 6, 2011 3:47 AM, "Lennart Regebro" <regebro at gmail.com> wrote:
>>
>> I've started working on a little utility to give a quality rating on
>> packages, expressed in 0-10 points, and also in cheese types,
>> according to smellyness.
>
> How about "cheese inspecter"?

More ideas:

cheeseshop.critic
pypi.stickler

From marius at pov.lt  Sun Mar  6 16:47:40 2011
From: marius at pov.lt (Marius Gedminas)
Date: Sun, 6 Mar 2011 17:47:40 +0200
Subject: [Distutils] Proposed change to buildout configuration-file
 handling
In-Reply-To: <AANLkTi=whFcCFq4uUnXadx3dS7tf9az+dz1g8TT2JvE7@mail.gmail.com>
References: <AANLkTi=whFcCFq4uUnXadx3dS7tf9az+dz1g8TT2JvE7@mail.gmail.com>
Message-ID: <20110306154740.GB7842@fridge.pov.lt>

On Sun, Mar 06, 2011 at 10:34:01AM -0500, Jim Fulton wrote:
> An annoyance in buildout configurations, resulting from it's use of
> ConfigParser, is that configuration data lines are stripped of leading
> spaces.
...
> I propose to give buildout it's own parser that begaves like
> ConfigParser, with the exeption that leading whitespace won't be
> stripped, but will be dedented with textwrap.dedent.
> 
> There is some potential for breakage as some option values without
> leading whitespace before would get it now, but my sense is that this
> would be unlikely.
> 
> Thoughts?  If I did that (and I really want to do this) I'd release a
> beta to help people assess the impact.

Woot!

+2

I've had to do contortions like this

    initialization =
      # haaaaack because zc.recipe.testrunner 1.4.0 produces an _insane_
      # bin/test that cannot be run with bin/coverage run bin/test, or even
      # bin/python bin/test
      import coverage, atexit
      c = coverage.coverage(data_file='../../../.coverage')
      def _when_done(c=c): c.stop(), c.save()
      atexit.register(_when_done)
      c.start()

because of that, and because the ${:s} trick never occurred to me.


There's also another -- perhaps minor -- reason to use a custom parser.
That way you could use += more than once in a section, i.e.

  develop = foo
  develop += bar
  develop += baz

as an alternative to

  develop = foo
            bar
            baz

(I've wanted to do that so that I could comment out each develop line
individually, to disable it) 

Marius Gedminas
-- 
#define QUESTION ((bb) || !(bb)) /* Shakespeare */
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 190 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110306/9bd31f29/attachment.pgp>

From ziade.tarek at gmail.com  Sun Mar  6 17:00:21 2011
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 6 Mar 2011 17:00:21 +0100
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <AANLkTim02Dv=J9RmQdVhWXRdML8rsvRNzRe-A6BP0290@mail.gmail.com>
References: <AANLkTim02Dv=J9RmQdVhWXRdML8rsvRNzRe-A6BP0290@mail.gmail.com>
Message-ID: <AANLkTinpQ=-Z0Jw2yx+BAeUj-5aD3wtPzJOZSOapNoCX@mail.gmail.com>

On Sun, Mar 6, 2011 at 9:46 AM, Lennart Regebro <regebro at gmail.com> wrote:
> I've started working on a little utility to give a quality rating on
> packages, expressed in 0-10 points, and also in cheese types,
> according to smellyness.
>
> It's going to check for things like that it has all meta data it
> should have, such as author_email, specifies Python versions via the
> trove classifiers (currently works) and that it specifies all
> dependencies (still todo). It will support both checking on a package
> (works currently) a distribution file and PyPI (still to do).
>
> It's not a uniqe idea, it overlaps with Andreas Jungs
> zopyx.trashfinder in scope, and it will also in the case of checking a
> package on PyPI check that there are several people that have owner
> access, and hence include the functionality of mr.parker. (In fact
> when checking on PyPI it will also check if there are documentation on
> packages.python.org, that the distribution files are uploaded to PyPI,
> etc, but this is all still todo). But I didn't find anything else, and
> I wanted bigger scopes than both these in what to check in and which
> cases.

Reminds me a bit about "CheeseCake"   http://pycheesecake.org/

>
> But, before I move this to a public repository and upload it to PyPI,
> there is one important thing to be determined: What should it be
> called? Currently I'm calling it "pypilib.quality". I don't mind this
> kind of boring names, but there is currently not a pypilib namespace,
> and I don't want to just create top level namespaces left and right
> for no reason. So other names are welcome. It doesn't have to have a
> namespace either.

Camembert ? :D


>
> In the long run I would not mind to see this utility integrated into a
> general pypi/cheeseshop script with other utility commands, which even
> could include installing and removing, thusly giving Perl people what
> they think they want a "CPAN" for Python. :-)
>
> --
> Lennart Regebro: http://regebro.wordpress.com/
> The Python 3 Porting book is out: http://python3porting.com/
> +33 661 58 14 64
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | http://ziade.org

From regebro at gmail.com  Sun Mar  6 17:23:16 2011
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 6 Mar 2011 17:23:16 +0100
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <AANLkTinpQ=-Z0Jw2yx+BAeUj-5aD3wtPzJOZSOapNoCX@mail.gmail.com>
References: <AANLkTim02Dv=J9RmQdVhWXRdML8rsvRNzRe-A6BP0290@mail.gmail.com>
	<AANLkTinpQ=-Z0Jw2yx+BAeUj-5aD3wtPzJOZSOapNoCX@mail.gmail.com>
Message-ID: <AANLkTi=a1=Z1cht4Xi=Tmbg1nEFeD14cAUh7NvV2yRXs@mail.gmail.com>

On Sun, Mar 6, 2011 at 17:00, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> Reminds me a bit about "CheeseCake" ? http://pycheesecake.org/

Indeed, it's a lot like CheeseCake, which I had forgot about and
didn't find either by google, pypi or asking on #plone and #python.
:-)
However, there are a significant difference. CheeseCake is focusing a
lot on code quality, while I'm only interested in how well it plays
with the pypi ecosystem.

The percentage of functions with docstrings seems to me to be highly
irrelevant, as well. That's more like the percentage of public vs
private functions, and not a measure of quality at all.
So CheeseCake can take care of code quality, and this will do package
quality. :-)


//Lennart

From barry at python.org  Sun Mar  6 18:51:55 2011
From: barry at python.org (Barry Warsaw)
Date: Sun, 6 Mar 2011 12:51:55 -0500
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <AANLkTimHzQCJ7j_c7XZkPA4CTANV8epgFb_Hp6aarcEZ@mail.gmail.com>
References: <AANLkTim02Dv=J9RmQdVhWXRdML8rsvRNzRe-A6BP0290@mail.gmail.com>
	<AANLkTi=gHVqSLofbU3q_aC26ETCez4anpYU5D0u+GDpK@mail.gmail.com>
	<AANLkTimHzQCJ7j_c7XZkPA4CTANV8epgFb_Hp6aarcEZ@mail.gmail.com>
Message-ID: <20110306125155.65781d44@limelight.wooz.org>

Excellent idea Lennart...

On Mar 06, 2011, at 03:51 PM, Brad Allen wrote:

>cheeseshop.critic
>pypi.stickler

I like the cheeseshop namespace used for this.  Or maybe wenslydale :)

http://orangecow.org/pythonet/sketches/cheese.htm

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110306/073d9fab/attachment.pgp>

From regebro at gmail.com  Sun Mar  6 20:40:41 2011
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 6 Mar 2011 20:40:41 +0100
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <AANLkTim02Dv=J9RmQdVhWXRdML8rsvRNzRe-A6BP0290@mail.gmail.com>
References: <AANLkTim02Dv=J9RmQdVhWXRdML8rsvRNzRe-A6BP0290@mail.gmail.com>
Message-ID: <AANLkTi=q-C-owzXnFHB1j+5apM-sXf+pGzJaxCiCXw-S@mail.gmail.com>

The winner is Wichert, with "pyroma".

I do like the "stickler" name, and the cheeseshop namespace, but since
there is nothing else in that namespace I'll wait with it. It can
easily be moved to a "cheeseshop.compliance" or whatever in the
future, but that the moment it's "pyroma". I'll check it in somewhere
soon, maybe work a bit more on the plane to PyCon and probably mention
it in a lightning talk.


//Lennart

On Sun, Mar 6, 2011 at 09:46, Lennart Regebro <regebro at gmail.com> wrote:
> I've started working on a little utility to give a quality rating on
> packages, expressed in 0-10 points, and also in cheese types,
> according to smellyness.
>
> It's going to check for things like that it has all meta data it
> should have, such as author_email, specifies Python versions via the
> trove classifiers (currently works) and that it specifies all
> dependencies (still todo). It will support both checking on a package
> (works currently) a distribution file and PyPI (still to do).
>
> It's not a uniqe idea, it overlaps with Andreas Jungs
> zopyx.trashfinder in scope, and it will also in the case of checking a
> package on PyPI check that there are several people that have owner
> access, and hence include the functionality of mr.parker. (In fact
> when checking on PyPI it will also check if there are documentation on
> packages.python.org, that the distribution files are uploaded to PyPI,
> etc, but this is all still todo). But I didn't find anything else, and
> I wanted bigger scopes than both these in what to check in and which
> cases.
>
> But, before I move this to a public repository and upload it to PyPI,
> there is one important thing to be determined: What should it be
> called? Currently I'm calling it "pypilib.quality". I don't mind this
> kind of boring names, but there is currently not a pypilib namespace,
> and I don't want to just create top level namespaces left and right
> for no reason. So other names are welcome. It doesn't have to have a
> namespace either.
>
> In the long run I would not mind to see this utility integrated into a
> general pypi/cheeseshop script with other utility commands, which even
> could include installing and removing, thusly giving Perl people what
> they think they want a "CPAN" for Python. :-)
>
> --
> Lennart Regebro: http://regebro.wordpress.com/
> The Python 3 Porting book is out: http://python3porting.com/
> +33 661 58 14 64
>

From benji at benjiyork.com  Sun Mar  6 20:46:12 2011
From: benji at benjiyork.com (Benji York)
Date: Sun, 6 Mar 2011 14:46:12 -0500
Subject: [Distutils] Proposed change to buildout configuration-file
	handling
In-Reply-To: <AANLkTi=whFcCFq4uUnXadx3dS7tf9az+dz1g8TT2JvE7@mail.gmail.com>
References: <AANLkTi=whFcCFq4uUnXadx3dS7tf9az+dz1g8TT2JvE7@mail.gmail.com>
Message-ID: <AANLkTimq7O57evP4ppZiyQUeUH+CfrdAVJxtu4SZ2qNH@mail.gmail.com>

On Sun, Mar 6, 2011 at 10:34 AM, Jim Fulton <jim at zope.com> wrote:
> An annoyance in buildout configurations, resulting from it's use of
> ConfigParser, is that configuration data lines are stripped of leading
> spaces.

+1

How should we handle the dedenting?

Would both of these assignments produce the same result? (I.e., no
leading whitespace at all):

foo =
    first line
    second line
    third line

foo = first line
    second line
    third line

-- 
Benji York

From cool-rr at cool-rr.com  Sun Mar  6 20:49:27 2011
From: cool-rr at cool-rr.com (cool-RR)
Date: Sun, 6 Mar 2011 21:49:27 +0200
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <AANLkTi=q-C-owzXnFHB1j+5apM-sXf+pGzJaxCiCXw-S@mail.gmail.com>
References: <AANLkTim02Dv=J9RmQdVhWXRdML8rsvRNzRe-A6BP0290@mail.gmail.com>
	<AANLkTi=q-C-owzXnFHB1j+5apM-sXf+pGzJaxCiCXw-S@mail.gmail.com>
Message-ID: <AANLkTi=GTEmixPv2A4x7909qxV8eeeh=U5wVYTzsXKsB@mail.gmail.com>

Speaking of names, I would rename PyPI to packages.python.org, maybe move
the existing documentation center to docs.python.org, and then move the docs
of Python itself to a `/python` folder...

But that's just me.

On Sun, Mar 6, 2011 at 9:40 PM, Lennart Regebro <regebro at gmail.com> wrote:

> The winner is Wichert, with "pyroma".
>
> I do like the "stickler" name, and the cheeseshop namespace, but since
> there is nothing else in that namespace I'll wait with it. It can
> easily be moved to a "cheeseshop.compliance" or whatever in the
> future, but that the moment it's "pyroma". I'll check it in somewhere
> soon, maybe work a bit more on the plane to PyCon and probably mention
> it in a lightning talk.
>
>
> //Lennart
>
> On Sun, Mar 6, 2011 at 09:46, Lennart Regebro <regebro at gmail.com> wrote:
> > I've started working on a little utility to give a quality rating on
> > packages, expressed in 0-10 points, and also in cheese types,
> > according to smellyness.
> >
> > It's going to check for things like that it has all meta data it
> > should have, such as author_email, specifies Python versions via the
> > trove classifiers (currently works) and that it specifies all
> > dependencies (still todo). It will support both checking on a package
> > (works currently) a distribution file and PyPI (still to do).
> >
> > It's not a uniqe idea, it overlaps with Andreas Jungs
> > zopyx.trashfinder in scope, and it will also in the case of checking a
> > package on PyPI check that there are several people that have owner
> > access, and hence include the functionality of mr.parker. (In fact
> > when checking on PyPI it will also check if there are documentation on
> > packages.python.org, that the distribution files are uploaded to PyPI,
> > etc, but this is all still todo). But I didn't find anything else, and
> > I wanted bigger scopes than both these in what to check in and which
> > cases.
> >
> > But, before I move this to a public repository and upload it to PyPI,
> > there is one important thing to be determined: What should it be
> > called? Currently I'm calling it "pypilib.quality". I don't mind this
> > kind of boring names, but there is currently not a pypilib namespace,
> > and I don't want to just create top level namespaces left and right
> > for no reason. So other names are welcome. It doesn't have to have a
> > namespace either.
> >
> > In the long run I would not mind to see this utility integrated into a
> > general pypi/cheeseshop script with other utility commands, which even
> > could include installing and removing, thusly giving Perl people what
> > they think they want a "CPAN" for Python. :-)
> >
> > --
> > Lennart Regebro: http://regebro.wordpress.com/
> > The Python 3 Porting book is out: http://python3porting.com/
> > +33 661 58 14 64
> >
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Sincerely,
Ram Rachum
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110306/4a0a9a99/attachment.html>

From jim at zope.com  Sun Mar  6 21:00:37 2011
From: jim at zope.com (Jim Fulton)
Date: Sun, 6 Mar 2011 15:00:37 -0500
Subject: [Distutils] Proposed change to buildout configuration-file
	handling
In-Reply-To: <AANLkTimq7O57evP4ppZiyQUeUH+CfrdAVJxtu4SZ2qNH@mail.gmail.com>
References: <AANLkTi=whFcCFq4uUnXadx3dS7tf9az+dz1g8TT2JvE7@mail.gmail.com>
	<AANLkTimq7O57evP4ppZiyQUeUH+CfrdAVJxtu4SZ2qNH@mail.gmail.com>
Message-ID: <AANLkTi=8x9Gode-AmyPxjV5PdHv4Brpk+w0sP6KQ53xy@mail.gmail.com>

On Sun, Mar 6, 2011 at 2:46 PM, Benji York <benji at benjiyork.com> wrote:
> On Sun, Mar 6, 2011 at 10:34 AM, Jim Fulton <jim at zope.com> wrote:
>> An annoyance in buildout configurations, resulting from it's use of
>> ConfigParser, is that configuration data lines are stripped of leading
>> spaces.
>
> +1
>
> How should we handle the dedenting?
>
> Would both of these assignments produce the same result? (I.e., no
> leading whitespace at all):
>
> foo =
> ? ?first line
> ? ?second line
> ? ?third line
>
> foo = first line
> ? ?second line
> ? ?third line

I'm thinking no. The first example would have no leading whitespace,
but the second would have leading whitespace on the send and third lines.

I can maybe see treating values that begin on the option line differently,
but fear that would make the mechanism too hard to explain.  I'm interested
in other folks opinions though.

Jim


-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From regebro at gmail.com  Sun Mar  6 21:10:12 2011
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 6 Mar 2011 21:10:12 +0100
Subject: [Distutils] pypi/packages/docs.python.org
Message-ID: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>

On Sun, Mar 6, 2011 at 20:49, cool-RR <cool-rr at cool-rr.com> wrote:
> Speaking of names, I would rename PyPI to packages.python.org, maybe move
> the existing documentation center to docs.python.org, and then move the docs
> of Python itself to a `/python` folder...
> But that's just me.

No, I do think docs.python.org should be for documentation of Python.
I agree having package documentation on packages.python.org may not be
very pedagogical, though. :-)

//Lennart

From cool-rr at cool-rr.com  Sun Mar  6 21:16:39 2011
From: cool-rr at cool-rr.com (cool-RR)
Date: Sun, 6 Mar 2011 22:16:39 +0200
Subject: [Distutils] pypi/packages/docs.python.org
In-Reply-To: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>
References: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>
Message-ID: <AANLkTikOv7PL82iNOAg=rhYOUxjq_212WqUYU=uUQpxX@mail.gmail.com>

On Sun, Mar 6, 2011 at 10:10 PM, Lennart Regebro <regebro at gmail.com> wrote:

> On Sun, Mar 6, 2011 at 20:49, cool-RR <cool-rr at cool-rr.com> wrote:
> > Speaking of names, I would rename PyPI to packages.python.org, maybe
> move
> > the existing documentation center to docs.python.org, and then move the
> docs
> > of Python itself to a `/python` folder...
> > But that's just me.
>
> No, I do think docs.python.org should be for documentation of Python.
> I agree having package documentation on packages.python.org may not be
> very pedagogical, though. :-)
>
> //Lennart
>

We must remember that what is obvious for us is *not* obvious for newbies.
For us it's obvious that Python packages are on PyPI and that
packages.python.org is documentation. But for, say, a Java programmer who's
just touching Python for a small project, this is a possible point of
confusion which can make him think, "Man, Python people are weird, what is
this 'pypi' thing?"


Ram.

-- 
Sincerely,
Ram Rachum
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110306/aab77e9c/attachment.html>

From ziade.tarek at gmail.com  Sun Mar  6 21:16:41 2011
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 6 Mar 2011 21:16:41 +0100
Subject: [Distutils] pypi/packages/docs.python.org
In-Reply-To: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>
References: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>
Message-ID: <AANLkTinv_ADD_fjQQBw+Ptp3MJPZapP64xvcXU+05Js8@mail.gmail.com>

On Sun, Mar 6, 2011 at 9:10 PM, Lennart Regebro <regebro at gmail.com> wrote:
> On Sun, Mar 6, 2011 at 20:49, cool-RR <cool-rr at cool-rr.com> wrote:
>> Speaking of names, I would rename PyPI to packages.python.org, maybe move
>> the existing documentation center to docs.python.org, and then move the docs
>> of Python itself to a `/python` folder...
>> But that's just me.
>
> No, I do think docs.python.org should be for documentation of Python.
> I agree having package documentation on packages.python.org may not be
> very pedagogical, though. :-)

One thing I find weird is the root page of packages.python.org -- This
warning is not super user friendly.

That said, maybe projects documentation could simply be displayed
under their pypi.python.org/pypi root page,
and packages.python.org made an alias to pypi.python.org



>
> //Lennart
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | http://ziade.org

From martin at v.loewis.de  Sun Mar  6 21:42:17 2011
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 06 Mar 2011 21:42:17 +0100
Subject: [Distutils] pypi/packages/docs.python.org
In-Reply-To: <AANLkTinv_ADD_fjQQBw+Ptp3MJPZapP64xvcXU+05Js8@mail.gmail.com>
References: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>
	<AANLkTinv_ADD_fjQQBw+Ptp3MJPZapP64xvcXU+05Js8@mail.gmail.com>
Message-ID: <4D73F1A9.1000207@v.loewis.de>

> One thing I find weird is the root page of packages.python.org -- This
> warning is not super user friendly.

What particular clause strikes you as particularly unfriendly?
Please understand that it may sound harsh, but is necessary - better
be safe than sorry.

> That said, maybe projects documentation could simply be displayed
> under their pypi.python.org/pypi root page,
> and packages.python.org made an alias to pypi.python.org

Not sure I understand the proposal. What URL would distribute
get (say)? (i.e. what is distribute's "pypi.python.org/pypi
root page"?) If "http://pypi.python.org/pypi/distribute", what
would you do with the content that is currently displayed there?

Regards,
Martin

From cool-rr at cool-rr.com  Sun Mar  6 21:47:24 2011
From: cool-rr at cool-rr.com (cool-RR)
Date: Sun, 6 Mar 2011 22:47:24 +0200
Subject: [Distutils] pypi/packages/docs.python.org
In-Reply-To: <4D73F1A9.1000207@v.loewis.de>
References: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>
	<AANLkTinv_ADD_fjQQBw+Ptp3MJPZapP64xvcXU+05Js8@mail.gmail.com>
	<4D73F1A9.1000207@v.loewis.de>
Message-ID: <AANLkTim1_1JhynAGQ83EeX85mqcBa_TLWiMuaGt_xbpk@mail.gmail.com>

On Sun, Mar 6, 2011 at 10:42 PM, "Martin v. L?wis" <martin at v.loewis.de>wrote:

> One thing I find weird is the root page of packages.python.org -- This
>> warning is not super user friendly.
>>
>
> What particular clause strikes you as particularly unfriendly?
> Please understand that it may sound harsh, but is necessary - better
> be safe than sorry.


I think it's less about the content and more about the form. I think this
warning belongs in a little `legal` section at the bottom, not as the first
thing that a visitor sees.



>  That said, maybe projects documentation could simply be displayed
>> under their pypi.python.org/pypi root page,
>> and packages.python.org made an alias to pypi.python.org
>>
>
> Not sure I understand the proposal. What URL would distribute
> get (say)? (i.e. what is distribute's "pypi.python.org/pypi
> root page"?) If "http://pypi.python.org/pypi/distribute", what
> would you do with the content that is currently displayed there?
>
> Regards,
> Martin
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Sincerely,
Ram Rachum
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110306/84a08dc9/attachment.html>

From martin at v.loewis.de  Sun Mar  6 21:53:06 2011
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 06 Mar 2011 21:53:06 +0100
Subject: [Distutils] pypi/packages/docs.python.org
In-Reply-To: <AANLkTim1_1JhynAGQ83EeX85mqcBa_TLWiMuaGt_xbpk@mail.gmail.com>
References: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>	<AANLkTinv_ADD_fjQQBw+Ptp3MJPZapP64xvcXU+05Js8@mail.gmail.com>	<4D73F1A9.1000207@v.loewis.de>
	<AANLkTim1_1JhynAGQ83EeX85mqcBa_TLWiMuaGt_xbpk@mail.gmail.com>
Message-ID: <4D73F432.5090208@v.loewis.de>

> I think it's less about the content and more about the form. I think
> this warning belongs in a little `legal` section at the bottom, not as
> the first thing that a visitor sees.

Are we talking about the same document? On

http://packages.python.org/

the warning *is* in the bottom of the page (not that the page has much
content in the first place). If you want it in a `legal` section at the
bottom - that is easy (I just did it).

Regards,
Martin

From regebro at gmail.com  Sun Mar  6 21:53:07 2011
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 6 Mar 2011 21:53:07 +0100
Subject: [Distutils] pypi/packages/docs.python.org
In-Reply-To: <4D73F1A9.1000207@v.loewis.de>
References: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>
	<AANLkTinv_ADD_fjQQBw+Ptp3MJPZapP64xvcXU+05Js8@mail.gmail.com>
	<4D73F1A9.1000207@v.loewis.de>
Message-ID: <AANLkTi=qB70wFODiHvW=eG64PgE+LY-bW=88FCM5nBO=@mail.gmail.com>

On Sun, Mar 6, 2011 at 21:42, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> One thing I find weird is the root page of packages.python.org -- This
>> warning is not super user friendly.
>
> What particular clause strikes you as particularly unfriendly?

It's rather the lack of design and just a link that seems a bit...
"raw" I guess.

Could we have a list of packages than have documentation, maybe? No,
scratch that, people will think that's all packages that exist. Also
not good.

> Not sure I understand the proposal. What URL would distribute
> get (say)? (i.e. what is distribute's "pypi.python.org/pypi
> root page"?) If "http://pypi.python.org/pypi/distribute", what
> would you do with the content that is currently displayed there?

I guess either http://pypi.python.org/pypi/distribute/docs or
http://pypi.python.org/docs/distribute would be acceptable.

Personally I don't think it's a big deal, but yes, I'm sure people go
to packages.python.org expecting what they get on pypi.

//Lennart

From cool-rr at cool-rr.com  Sun Mar  6 21:57:15 2011
From: cool-rr at cool-rr.com (cool-RR)
Date: Sun, 6 Mar 2011 22:57:15 +0200
Subject: [Distutils] pypi/packages/docs.python.org
In-Reply-To: <4D73F432.5090208@v.loewis.de>
References: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>
	<AANLkTinv_ADD_fjQQBw+Ptp3MJPZapP64xvcXU+05Js8@mail.gmail.com>
	<4D73F1A9.1000207@v.loewis.de>
	<AANLkTim1_1JhynAGQ83EeX85mqcBa_TLWiMuaGt_xbpk@mail.gmail.com>
	<4D73F432.5090208@v.loewis.de>
Message-ID: <AANLkTin2cTNBFcDNphYAVFGPLVm9BgxT63jAL0X=285x@mail.gmail.com>

On Sun, Mar 6, 2011 at 10:53 PM, "Martin v. L?wis" <martin at v.loewis.de>wrote:

> I think it's less about the content and more about the form. I think
>> this warning belongs in a little `legal` section at the bottom, not as
>> the first thing that a visitor sees.
>>
>
> Are we talking about the same document? On
>
> http://packages.python.org/
>
> the warning *is* in the bottom of the page (not that the page has much
> content in the first place).


Haha, well I guess it's technically at the bottom, but since the page is (a)
almost empty and (b) looks like a text file, it kinda looks like the main
content of the page.


> If you want it in a `legal` section at the
> bottom - that is easy (I just did it).
>

That's somewhat of an improvement. A link to a separate legal statement,
like on python.org, would be nice. But I understand that this will require
some work that you may not have time to do.


>
> Regards,
> Martin
>



-- 
Sincerely,
Ram Rachum
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110306/373f0e6c/attachment.html>

From ziade.tarek at gmail.com  Sun Mar  6 21:59:13 2011
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 6 Mar 2011 21:59:13 +0100
Subject: [Distutils] pypi/packages/docs.python.org
In-Reply-To: <4D73F1A9.1000207@v.loewis.de>
References: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>
	<AANLkTinv_ADD_fjQQBw+Ptp3MJPZapP64xvcXU+05Js8@mail.gmail.com>
	<4D73F1A9.1000207@v.loewis.de>
Message-ID: <AANLkTi=vxdv_MTa4JujZPB4=kLuc94P100KiqZFr_a8e@mail.gmail.com>

On Sun, Mar 6, 2011 at 9:42 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> One thing I find weird is the root page of packages.python.org -- This
>> warning is not super user friendly.
>
> What particular clause strikes you as particularly unfriendly?
> Please understand that it may sound harsh, but is necessary - better
> be safe than sorry.

Nothing wrong about the warning itself, but about landing on a plain
condensed text page.
I think we should make it a html page. And maybe display the last ten
packages doc updates ?

>
>> That said, maybe projects documentation could simply be displayed
>> under their pypi.python.org/pypi root page,
>> and packages.python.org made an alias to pypi.python.org
>
> Not sure I understand the proposal. What URL would distribute
> get (say)? (i.e. what is distribute's "pypi.python.org/pypi
> root page"?) If "http://pypi.python.org/pypi/distribute", what
> would you do with the content that is currently displayed there?

so Distribute has currently:  "http://pypi.python.org/pypi/distribute"

It's the Description field of the metadata.

And then we push our doc at: "http://packages.python.org/distribute/"

Since "packages.python.org" allow us to have a directory with HTML
files, those could be made accessible under:

http://pypi.python.org/pypi/distribute/docs, with a big link on the
top of http://pypi.python.org/pypi/distribute to go there.

So:

http://packages.python.org/PROJECT/  would become
http://pypi.python.org/pypi/PROJECT/doc

IOW, not changes but just avoiding two places -- without giving any
hint on each place about the existence of the other place.

Why Distribute would have two root pages ?
http://pypi.python.org/pypi/distribute is a fine one


Then, maybe http://pypi.python.org/pypi could simply become
http://packages.python.org, with:


- http://packages.python.org/PROJECT
- http://packages.python.org/PROJECT/docs
- http://packages.python.org/PROJECT/1.2
- etc..


>
> Regards,
> Martin
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | http://ziade.org

From martin at v.loewis.de  Sun Mar  6 22:05:23 2011
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 06 Mar 2011 22:05:23 +0100
Subject: [Distutils] pypi/packages/docs.python.org
In-Reply-To: <AANLkTi=qB70wFODiHvW=eG64PgE+LY-bW=88FCM5nBO=@mail.gmail.com>
References: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>
	<AANLkTinv_ADD_fjQQBw+Ptp3MJPZapP64xvcXU+05Js8@mail.gmail.com>
	<4D73F1A9.1000207@v.loewis.de>
	<AANLkTi=qB70wFODiHvW=eG64PgE+LY-bW=88FCM5nBO=@mail.gmail.com>
Message-ID: <4D73F713.20204@v.loewis.de>

> I guess either http://pypi.python.org/pypi/distribute/docs or
> http://pypi.python.org/docs/distribute would be acceptable.

The former won't work - it tries to get a version labeled "docs"
from "distribute". As for the latter, see

http://mail.python.org/pipermail/catalog-sig/2008-August/001723.html

I originally proposed what you are proposing now, and Ian Bicking
told me use what we have now because of XSS concerns (take and
remove a plural form).

Regards,
Martin

From cool-rr at cool-rr.com  Sun Mar  6 22:08:25 2011
From: cool-rr at cool-rr.com (cool-RR)
Date: Sun, 6 Mar 2011 23:08:25 +0200
Subject: [Distutils] pypi/packages/docs.python.org
In-Reply-To: <AANLkTi=vxdv_MTa4JujZPB4=kLuc94P100KiqZFr_a8e@mail.gmail.com>
References: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>
	<AANLkTinv_ADD_fjQQBw+Ptp3MJPZapP64xvcXU+05Js8@mail.gmail.com>
	<4D73F1A9.1000207@v.loewis.de>
	<AANLkTi=vxdv_MTa4JujZPB4=kLuc94P100KiqZFr_a8e@mail.gmail.com>
Message-ID: <AANLkTimtrhMKBRqX5rsrnM-pzRZoucuBKg9VE4xYuaLq@mail.gmail.com>

2011/3/6 Tarek Ziad? <ziade.tarek at gmail.com>

> On Sun, Mar 6, 2011 at 9:42 PM, "Martin v. L?wis" <martin at v.loewis.de>
> wrote:
> >> One thing I find weird is the root page of packages.python.org -- This
> >> warning is not super user friendly.
> >
> > What particular clause strikes you as particularly unfriendly?
> > Please understand that it may sound harsh, but is necessary - better
> > be safe than sorry.
>
> Nothing wrong about the warning itself, but about landing on a plain
> condensed text page.
> I think we should make it a html page. And maybe display the last ten
> packages doc updates ?
>
> >
> >> That said, maybe projects documentation could simply be displayed
> >> under their pypi.python.org/pypi root page,
> >> and packages.python.org made an alias to pypi.python.org
> >
> > Not sure I understand the proposal. What URL would distribute
> > get (say)? (i.e. what is distribute's "pypi.python.org/pypi
> > root page"?) If "http://pypi.python.org/pypi/distribute", what
> > would you do with the content that is currently displayed there?
>
> so Distribute has currently:  "http://pypi.python.org/pypi/distribute"
>
> It's the Description field of the metadata.
>
> And then we push our doc at: "http://packages.python.org/distribute/"
>
> Since "packages.python.org" allow us to have a directory with HTML
> files, those could be made accessible under:
>
> http://pypi.python.org/pypi/distribute/docs, with a big link on the
> top of http://pypi.python.org/pypi/distribute to go there.
>
> So:
>
> http://packages.python.org/PROJECT/  would become
> http://pypi.python.org/pypi/PROJECT/doc
>
> IOW, not changes but just avoiding two places -- without giving any
> hint on each place about the existence of the other place.
>
> Why Distribute would have two root pages ?
> http://pypi.python.org/pypi/distribute is a fine one
>
>
> Then, maybe http://pypi.python.org/pypi could simply become
> http://packages.python.org, with:
>
>
> - http://packages.python.org/PROJECT
> - http://packages.python.org/PROJECT/docs
> - http://packages.python.org/PROJECT/1.2
> - etc..
>
>
+1


Ram
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110306/7742e120/attachment-0001.html>

From greg.ewing at canterbury.ac.nz  Sun Mar  6 22:12:24 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 07 Mar 2011 10:12:24 +1300
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <AANLkTim02Dv=J9RmQdVhWXRdML8rsvRNzRe-A6BP0290@mail.gmail.com>
References: <AANLkTim02Dv=J9RmQdVhWXRdML8rsvRNzRe-A6BP0290@mail.gmail.com>
Message-ID: <4D73F8B8.6020703@canterbury.ac.nz>

Lennart Regebro wrote:

> But, before I move this to a public repository and upload it to PyPI,
> there is one important thing to be determined: What should it be
> called?

Inspector Tiger?

http://www.youtube.com/watch?v=UDdCghQYvDY

-- 
Greg

From regebro at gmail.com  Sun Mar  6 22:14:49 2011
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 6 Mar 2011 22:14:49 +0100
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <AANLkTi=q-C-owzXnFHB1j+5apM-sXf+pGzJaxCiCXw-S@mail.gmail.com>
References: <AANLkTim02Dv=J9RmQdVhWXRdML8rsvRNzRe-A6BP0290@mail.gmail.com>
	<AANLkTi=q-C-owzXnFHB1j+5apM-sXf+pGzJaxCiCXw-S@mail.gmail.com>
Message-ID: <AANLkTikVrLupcpYdF4q2LsGa01-fAC+RrJiJSCAg75vE@mail.gmail.com>

Since mercurial makes me annoyed I decided to use it. I'll have to
learn it someday anyway, so why not now?

https://bitbucket.org/regebro/pyroma

Helpers welcome (although you'll probably have to wait to after PyCon).

//Lennart

On Sun, Mar 6, 2011 at 20:40, Lennart Regebro <regebro at gmail.com> wrote:
> The winner is Wichert, with "pyroma".
>
> I do like the "stickler" name, and the cheeseshop namespace, but since
> there is nothing else in that namespace I'll wait with it. It can
> easily be moved to a "cheeseshop.compliance" or whatever in the
> future, but that the moment it's "pyroma". I'll check it in somewhere
> soon, maybe work a bit more on the plane to PyCon and probably mention
> it in a lightning talk.
>
>
> //Lennart
>
> On Sun, Mar 6, 2011 at 09:46, Lennart Regebro <regebro at gmail.com> wrote:
>> I've started working on a little utility to give a quality rating on
>> packages, expressed in 0-10 points, and also in cheese types,
>> according to smellyness.
>>
>> It's going to check for things like that it has all meta data it
>> should have, such as author_email, specifies Python versions via the
>> trove classifiers (currently works) and that it specifies all
>> dependencies (still todo). It will support both checking on a package
>> (works currently) a distribution file and PyPI (still to do).
>>
>> It's not a uniqe idea, it overlaps with Andreas Jungs
>> zopyx.trashfinder in scope, and it will also in the case of checking a
>> package on PyPI check that there are several people that have owner
>> access, and hence include the functionality of mr.parker. (In fact
>> when checking on PyPI it will also check if there are documentation on
>> packages.python.org, that the distribution files are uploaded to PyPI,
>> etc, but this is all still todo). But I didn't find anything else, and
>> I wanted bigger scopes than both these in what to check in and which
>> cases.
>>
>> But, before I move this to a public repository and upload it to PyPI,
>> there is one important thing to be determined: What should it be
>> called? Currently I'm calling it "pypilib.quality". I don't mind this
>> kind of boring names, but there is currently not a pypilib namespace,
>> and I don't want to just create top level namespaces left and right
>> for no reason. So other names are welcome. It doesn't have to have a
>> namespace either.
>>
>> In the long run I would not mind to see this utility integrated into a
>> general pypi/cheeseshop script with other utility commands, which even
>> could include installing and removing, thusly giving Perl people what
>> they think they want a "CPAN" for Python. :-)
>>
>> --
>> Lennart Regebro: http://regebro.wordpress.com/
>> The Python 3 Porting book is out: http://python3porting.com/
>> +33 661 58 14 64
>>
>

From martin at v.loewis.de  Sun Mar  6 22:17:50 2011
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 06 Mar 2011 22:17:50 +0100
Subject: [Distutils] pypi/packages/docs.python.org
In-Reply-To: <AANLkTi=vxdv_MTa4JujZPB4=kLuc94P100KiqZFr_a8e@mail.gmail.com>
References: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>	<AANLkTinv_ADD_fjQQBw+Ptp3MJPZapP64xvcXU+05Js8@mail.gmail.com>	<4D73F1A9.1000207@v.loewis.de>
	<AANLkTi=vxdv_MTa4JujZPB4=kLuc94P100KiqZFr_a8e@mail.gmail.com>
Message-ID: <4D73F9FE.2060506@v.loewis.de>

>> What particular clause strikes you as particularly unfriendly?
>> Please understand that it may sound harsh, but is necessary - better
>> be safe than sorry.
>
> Nothing wrong about the warning itself, but about landing on a plain
> condensed text page.
> I think we should make it a html page. And maybe display the last ten
> packages doc updates ?

It actually *is* a html page (and always was). It just doesn't use
any styling.

As for changing the style: please submit a html file to replace what is
there (I refuse to do any styling . As for displaying the last ten doc 
updates: either submit a tracker request, or provide a patch.

> Since "packages.python.org" allow us to have a directory with HTML
> files, those could be made accessible under:
>
> http://pypi.python.org/pypi/distribute/docs, with a big link on the
> top of http://pypi.python.org/pypi/distribute to go there.

[I wonder why "links" and "buttons" always have to be "big", and often
"red" :-]

See my response to Lennart for the former: distribute/docs would be
the "docs" release of "distribute". As for a big link: if you think
your page should have one, you are free to make it yourself already.

> http://packages.python.org/PROJECT/  would become
> http://pypi.python.org/pypi/PROJECT/doc
>
> IOW, not changes but just avoiding two places -- without giving any
> hint on each place about the existence of the other place.
>
> Why Distribute would have two root pages ?

See above: what you propose cannot work. Also, I don't think many
users care about the URLs of things. If your package home page
is pypi/distribute, that's perfectly fine. Put a documentation
link on that page, and be done.

> Then, maybe http://pypi.python.org/pypi could simply become
> http://packages.python.org, with:
>
>
> - http://packages.python.org/PROJECT
> - http://packages.python.org/PROJECT/docs
> - http://packages.python.org/PROJECT/1.2
> - etc..

That would break existing packages URLs, so -1. You cannot
lightly change URLs in the Web - replacing an existing URL
is a multi-year project. Individual package maintainers may
do, but we still get lots of hits on cheeseshop.python.org,
so that needs to be supported many years in the future. I'm
not looking forward to having a *second* transitional URL.

Regards,
Martin

From regebro at gmail.com  Sun Mar  6 22:19:16 2011
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 6 Mar 2011 22:19:16 +0100
Subject: [Distutils] pypi/packages/docs.python.org
In-Reply-To: <4D73F713.20204@v.loewis.de>
References: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>
	<AANLkTinv_ADD_fjQQBw+Ptp3MJPZapP64xvcXU+05Js8@mail.gmail.com>
	<4D73F1A9.1000207@v.loewis.de>
	<AANLkTi=qB70wFODiHvW=eG64PgE+LY-bW=88FCM5nBO=@mail.gmail.com>
	<4D73F713.20204@v.loewis.de>
Message-ID: <AANLkTinCgx48G1uPhGj6PJ=hipLxTpU0Ex8DyJvzHFoX@mail.gmail.com>

On Sun, Mar 6, 2011 at 22:05, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> I guess either http://pypi.python.org/pypi/distribute/docs or
>> http://pypi.python.org/docs/distribute would be acceptable.
>
> The former won't work - it tries to get a version labeled "docs"
> from "distribute". As for the latter, see
>
> http://mail.python.org/pipermail/catalog-sig/2008-August/001723.html
>
> I originally proposed what you are proposing now, and Ian Bicking
> told me use what we have now because of XSS concerns (take and
> remove a plural form).

Ah, right, of course. So it needs to be a separate hostname. It's then
only a question of what hostname.

//Lennart

From ziade.tarek at gmail.com  Sun Mar  6 22:26:54 2011
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 6 Mar 2011 22:26:54 +0100
Subject: [Distutils] pypi/packages/docs.python.org
In-Reply-To: <4D73F9FE.2060506@v.loewis.de>
References: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>
	<AANLkTinv_ADD_fjQQBw+Ptp3MJPZapP64xvcXU+05Js8@mail.gmail.com>
	<4D73F1A9.1000207@v.loewis.de>
	<AANLkTi=vxdv_MTa4JujZPB4=kLuc94P100KiqZFr_a8e@mail.gmail.com>
	<4D73F9FE.2060506@v.loewis.de>
Message-ID: <AANLkTi=98hs7oXnmdZNmxPQ2Cu4CBRdMKaTJ07PAk_uB@mail.gmail.com>

2011/3/6 "Martin v. L?wis" <martin at v.loewis.de>:
>>> What particular clause strikes you as particularly unfriendly?
>>> Please understand that it may sound harsh, but is necessary - better
>>> be safe than sorry.
>>
>> Nothing wrong about the warning itself, but about landing on a plain
>> condensed text page.
>> I think we should make it a html page. And maybe display the last ten
>> packages doc updates ?
>
> It actually *is* a html page (and always was). It just doesn't use
> any styling.
>
> As for changing the style: please submit a html file to replace what is
> there (I refuse to do any styling . As for displaying the last ten doc
> updates: either submit a tracker request, or provide a patch.

That could be a nice small sprint task at Pycon -- will see

>
>> Since "packages.python.org" allow us to have a directory with HTML
>> files, those could be made accessible under:
>>
>> http://pypi.python.org/pypi/distribute/docs, with a big link on the
>> top of http://pypi.python.org/pypi/distribute to go there.
>
> [I wonder why "links" and "buttons" always have to be "big", and often
> "red" :-]
>
> See my response to Lennart for the former: distribute/docs would be
> the "docs" release of "distribute".

Yes, good points.

> As for a big link: if you think your page should have one, you are free
> to make it yourself already.

Sure but,

1/ I have never asked for the "Downloads ?" link either, but the UI
did add it, and it's really more ergonomic.

2/ I have never asked for "Latest Version: 0.6.14" on this page : hit
ttp://pypi.python.org/pypi/distribute/0.6.9

and you also say for 2/  "if you think your page should have links to
older releases, you are free to make it yourself already"
But same remark: it makes browsing more friendly.

And I think a link to packages.python.org/distribute is at the same level

-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Sun Mar  6 22:28:03 2011
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 6 Mar 2011 22:28:03 +0100
Subject: [Distutils] pypi/packages/docs.python.org
In-Reply-To: <AANLkTi=98hs7oXnmdZNmxPQ2Cu4CBRdMKaTJ07PAk_uB@mail.gmail.com>
References: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>
	<AANLkTinv_ADD_fjQQBw+Ptp3MJPZapP64xvcXU+05Js8@mail.gmail.com>
	<4D73F1A9.1000207@v.loewis.de>
	<AANLkTi=vxdv_MTa4JujZPB4=kLuc94P100KiqZFr_a8e@mail.gmail.com>
	<4D73F9FE.2060506@v.loewis.de>
	<AANLkTi=98hs7oXnmdZNmxPQ2Cu4CBRdMKaTJ07PAk_uB@mail.gmail.com>
Message-ID: <AANLkTikasfG0hTezLoOFcvDJKuNaWd4zAUDNXzKnWrfr@mail.gmail.com>

2011/3/6 Tarek Ziad? <ziade.tarek at gmail.com>:
...
> and you also say for 2/ ?"if you think your page should have links to

s/ you also / you could have also /

(not clear otherwise)

From martin at v.loewis.de  Sun Mar  6 22:43:20 2011
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 06 Mar 2011 22:43:20 +0100
Subject: [Distutils] pypi/packages/docs.python.org
In-Reply-To: <AANLkTi=98hs7oXnmdZNmxPQ2Cu4CBRdMKaTJ07PAk_uB@mail.gmail.com>
References: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>	<AANLkTinv_ADD_fjQQBw+Ptp3MJPZapP64xvcXU+05Js8@mail.gmail.com>	<4D73F1A9.1000207@v.loewis.de>	<AANLkTi=vxdv_MTa4JujZPB4=kLuc94P100KiqZFr_a8e@mail.gmail.com>	<4D73F9FE.2060506@v.loewis.de>
	<AANLkTi=98hs7oXnmdZNmxPQ2Cu4CBRdMKaTJ07PAk_uB@mail.gmail.com>
Message-ID: <4D73FFF8.3020801@v.loewis.de>

>> As for a big link: if you think your page should have one, you are free
>> to make it yourself already.
>
> Sure but,
>
> 1/ I have never asked for the "Downloads ?" link either, but the UI
> did add it, and it's really more ergonomic.
>
> 2/ I have never asked for "Latest Version: 0.6.14" on this page : hit
> ttp://pypi.python.org/pypi/distribute/0.6.9
[...]
> And I think a link to packages.python.org/distribute is at the same level

I can sympathize with that view. Cc'ing catalog-sig here:

if anybody would *not* want to have a "Documentation" link automatically
generated that points to packages.python.org/<project>, please speak up.

Regards,
Martin

From cool-rr at cool-rr.com  Sun Mar  6 22:45:31 2011
From: cool-rr at cool-rr.com (cool-RR)
Date: Sun, 6 Mar 2011 23:45:31 +0200
Subject: [Distutils] pypi/packages/docs.python.org
In-Reply-To: <4D73F9FE.2060506@v.loewis.de>
References: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>
	<AANLkTinv_ADD_fjQQBw+Ptp3MJPZapP64xvcXU+05Js8@mail.gmail.com>
	<4D73F1A9.1000207@v.loewis.de>
	<AANLkTi=vxdv_MTa4JujZPB4=kLuc94P100KiqZFr_a8e@mail.gmail.com>
	<4D73F9FE.2060506@v.loewis.de>
Message-ID: <AANLkTikBYjW2i4UcdEYRsXPshTaOA_ncotME=RqEzFwe@mail.gmail.com>

2011/3/6 "Martin v. L?wis" <martin at v.loewis.de>

> What particular clause strikes you as particularly unfriendly?
>>> Please understand that it may sound harsh, but is necessary - better
>>> be safe than sorry.
>>>
>>
>> Nothing wrong about the warning itself, but about landing on a plain
>> condensed text page.
>> I think we should make it a html page. And maybe display the last ten
>> packages doc updates ?
>>
>
> It actually *is* a html page (and always was). It just doesn't use
> any styling.
>
> As for changing the style: please submit a html file to replace what is
> there (I refuse to do any styling . As for displaying the last ten doc
> updates: either submit a tracker request, or provide a patch.
>
>
>  Since "packages.python.org" allow us to have a directory with HTML
>> files, those could be made accessible under:
>>
>> http://pypi.python.org/pypi/distribute/docs, with a big link on the
>> top of http://pypi.python.org/pypi/distribute to go there.
>>
>
> [I wonder why "links" and "buttons" always have to be "big", and often
> "red" :-]
>
> See my response to Lennart for the former: distribute/docs would be
> the "docs" release of "distribute". As for a big link: if you think
> your page should have one, you are free to make it yourself already.
>
>
>  http://packages.python.org/PROJECT/  would become
>> http://pypi.python.org/pypi/PROJECT/doc
>>
>> IOW, not changes but just avoiding two places -- without giving any
>> hint on each place about the existence of the other place.
>>
>> Why Distribute would have two root pages ?
>>
>
> See above: what you propose cannot work. Also, I don't think many
> users care about the URLs of things. If your package home page
> is pypi/distribute, that's perfectly fine. Put a documentation
> link on that page, and be done.


>
>  Then, maybe http://pypi.python.org/pypi could simply become
>> http://packages.python.org, with:
>>
>>
>> - http://packages.python.org/PROJECT
>> - http://packages.python.org/PROJECT/docs
>> - http://packages.python.org/PROJECT/1.2
>> - etc..
>>
>
> That would break existing packages URLs, so -1. You cannot
> lightly change URLs in the Web - replacing an existing URL
> is a multi-year project.


Sigh... Could everyone please be more careful before they create URLs that
will have to be maintained for YEARS? This is so frustrating.

As a practical solution now: Which links are being broken? If someone goes
into http://packages.python.org/PROJECT expecting documentation, and then he
gets a general page for the project which links to the documentation, then I
think it's not that bad.

If someone's following a deep link into the documentation, we can give a 404
page which suggests adding a `/docs`, so people will slowly learn to update
their links.


> Individual package maintainers may
> do, but we still get lots of hits on cheeseshop.python.org,
> so that needs to be supported many years in the future. I'm
> not looking forward to having a *second* transitional URL.
>
>
> Regards,
> Martin
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Sincerely,
Ram Rachum
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110306/71f2b305/attachment.html>

From mal at egenix.com  Sun Mar  6 23:18:14 2011
From: mal at egenix.com (M.-A. Lemburg)
Date: Sun, 06 Mar 2011 23:18:14 +0100
Subject: [Distutils] [Catalog-sig]  pypi/packages/docs.python.org
In-Reply-To: <4D73FFF8.3020801@v.loewis.de>
References: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>	<AANLkTinv_ADD_fjQQBw+Ptp3MJPZapP64xvcXU+05Js8@mail.gmail.com>	<4D73F1A9.1000207@v.loewis.de>	<AANLkTi=vxdv_MTa4JujZPB4=kLuc94P100KiqZFr_a8e@mail.gmail.com>	<4D73F9FE.2060506@v.loewis.de>	<AANLkTi=98hs7oXnmdZNmxPQ2Cu4CBRdMKaTJ07PAk_uB@mail.gmail.com>
	<4D73FFF8.3020801@v.loewis.de>
Message-ID: <4D740826.3030207@egenix.com>

"Martin v. L?wis" wrote:
>>> As for a big link: if you think your page should have one, you are free
>>> to make it yourself already.
>>
>> Sure but,
>>
>> 1/ I have never asked for the "Downloads ?" link either, but the UI
>> did add it, and it's really more ergonomic.
>>
>> 2/ I have never asked for "Latest Version: 0.6.14" on this page : hit
>> ttp://pypi.python.org/pypi/distribute/0.6.9
> [...]
>> And I think a link to packages.python.org/distribute is at the same level
> 
> I can sympathize with that view. Cc'ing catalog-sig here:
> 
> if anybody would *not* want to have a "Documentation" link automatically
> generated that points to packages.python.org/<project>, please speak up.

That would only make sense if there's something uploaded to
the PyPI docs dir.

If PyPI can detect that, +1. Otherwise, I think it's better not
adding such an automatic link, since no link is better than one
going nowhere.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 06 2011)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From richard at python.org  Mon Mar  7 00:31:12 2011
From: richard at python.org (Richard Jones)
Date: Mon, 7 Mar 2011 10:31:12 +1100
Subject: [Distutils] [Catalog-sig]  pypi/packages/docs.python.org
In-Reply-To: <4D740826.3030207@egenix.com>
References: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>
	<AANLkTinv_ADD_fjQQBw+Ptp3MJPZapP64xvcXU+05Js8@mail.gmail.com>
	<4D73F1A9.1000207@v.loewis.de>
	<AANLkTi=vxdv_MTa4JujZPB4=kLuc94P100KiqZFr_a8e@mail.gmail.com>
	<4D73F9FE.2060506@v.loewis.de>
	<AANLkTi=98hs7oXnmdZNmxPQ2Cu4CBRdMKaTJ07PAk_uB@mail.gmail.com>
	<4D73FFF8.3020801@v.loewis.de> <4D740826.3030207@egenix.com>
Message-ID: <AANLkTino9Ttx++_gqQQo95eJ6mChuiRmR+R_S4rW7LPV@mail.gmail.com>

On Mon, Mar 7, 2011 at 9:18 AM, M.-A. Lemburg <mal at egenix.com> wrote:
> "Martin v. L?wis" wrote:
>> if anybody would *not* want to have a "Documentation" link automatically
>> generated that points to packages.python.org/<project>, please speak up.
>
> That would only make sense if there's something uploaded to
> the PyPI docs dir.
>
> If PyPI can detect that, +1. Otherwise, I think it's better not
> adding such an automatic link, since no link is better than one
> going nowhere.

Absolutely - it only generates the link if there's something to link
to (that is, there's been a documentation upload with an index.html
file at the root, or in a top-level "html" directory).


     Richard

From martin at v.loewis.de  Mon Mar  7 00:47:03 2011
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 07 Mar 2011 00:47:03 +0100
Subject: [Distutils] [Catalog-sig]  pypi/packages/docs.python.org
In-Reply-To: <4D740826.3030207@egenix.com>
References: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>	<AANLkTinv_ADD_fjQQBw+Ptp3MJPZapP64xvcXU+05Js8@mail.gmail.com>	<4D73F1A9.1000207@v.loewis.de>	<AANLkTi=vxdv_MTa4JujZPB4=kLuc94P100KiqZFr_a8e@mail.gmail.com>	<4D73F9FE.2060506@v.loewis.de>	<AANLkTi=98hs7oXnmdZNmxPQ2Cu4CBRdMKaTJ07PAk_uB@mail.gmail.com>
	<4D73FFF8.3020801@v.loewis.de> <4D740826.3030207@egenix.com>
Message-ID: <4D741CF7.7010807@v.loewis.de>

>> if anybody would *not* want to have a "Documentation" link automatically
>> generated that points to packages.python.org/<project>, please speak up.
>
> That would only make sense if there's something uploaded to
> the PyPI docs dir.
>
> If PyPI can detect that, +1.

Yes, the link would certainly be conditional.

Regards,
Martin

From merwok at netwok.org  Mon Mar  7 11:01:40 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Mon, 07 Mar 2011 11:01:40 +0100
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <AANLkTim02Dv=J9RmQdVhWXRdML8rsvRNzRe-A6BP0290@mail.gmail.com>
References: <AANLkTim02Dv=J9RmQdVhWXRdML8rsvRNzRe-A6BP0290@mail.gmail.com>
Message-ID: <4D74AD04.6040900@netwok.org>

> I've started working on a little utility to give a quality rating on
> packages

Please don?t call those things packages.  Let?s try to use ?package?
only for a directory that you can import from Python, and ?distribution?
for a bundle/archive of one version of a project.

http://tarekziade.wordpress.com/2010/01/07/fixing-packaging-terminology-confusion/
http://docs.python.org/dev/distutils/introduction#concepts-terminology

Regards

From mal at egenix.com  Mon Mar  7 11:03:58 2011
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 07 Mar 2011 11:03:58 +0100
Subject: [Distutils] [Catalog-sig]  pypi/packages/docs.python.org
In-Reply-To: <4D741CF7.7010807@v.loewis.de>
References: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>	<AANLkTinv_ADD_fjQQBw+Ptp3MJPZapP64xvcXU+05Js8@mail.gmail.com>	<4D73F1A9.1000207@v.loewis.de>	<AANLkTi=vxdv_MTa4JujZPB4=kLuc94P100KiqZFr_a8e@mail.gmail.com>	<4D73F9FE.2060506@v.loewis.de>	<AANLkTi=98hs7oXnmdZNmxPQ2Cu4CBRdMKaTJ07PAk_uB@mail.gmail.com>	<4D73FFF8.3020801@v.loewis.de>
	<4D740826.3030207@egenix.com> <4D741CF7.7010807@v.loewis.de>
Message-ID: <4D74AD8E.4020604@egenix.com>

"Martin v. L?wis" wrote:
>>> if anybody would *not* want to have a "Documentation" link automatically
>>> generated that points to packages.python.org/<project>, please speak up.
>>
>> That would only make sense if there's something uploaded to
>> the PyPI docs dir.
>>
>> If PyPI can detect that, +1.
> 
> Yes, the link would certainly be conditional.

Great. Thanks.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 07 2011)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From merwok at netwok.org  Mon Mar  7 11:18:30 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Mon, 07 Mar 2011 11:18:30 +0100
Subject: [Distutils] pypi/packages/docs.python.org
In-Reply-To: <4D73F9FE.2060506@v.loewis.de>
References: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>	<AANLkTinv_ADD_fjQQBw+Ptp3MJPZapP64xvcXU+05Js8@mail.gmail.com>	<4D73F1A9.1000207@v.loewis.de>	<AANLkTi=vxdv_MTa4JujZPB4=kLuc94P100KiqZFr_a8e@mail.gmail.com>
	<4D73F9FE.2060506@v.loewis.de>
Message-ID: <4D74B0F6.5080204@netwok.org>

Le 06/03/2011 22:17, "Martin v. L?wis" a ?crit :
> As for changing the style: please submit a html file to replace what is
> there (I refuse to do any styling .

Do you mean you don?t want to work on styling personally or that you
object to someone proposing a patch adding styling?  It it?s the latter,
could you explain why?

Regards

From regebro at gmail.com  Mon Mar  7 11:20:09 2011
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 7 Mar 2011 11:20:09 +0100
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <4D74AD04.6040900@netwok.org>
References: <AANLkTim02Dv=J9RmQdVhWXRdML8rsvRNzRe-A6BP0290@mail.gmail.com>
	<4D74AD04.6040900@netwok.org>
Message-ID: <AANLkTimbs7c9xVGxxoN+RypusqfCC9ACT2k=E-978BzD@mail.gmail.com>

On Mon, Mar 7, 2011 at 11:01, ?ric Araujo <merwok at netwok.org> wrote:
>> I've started working on a little utility to give a quality rating on
>> packages
>
> Please don?t call those things packages. ?Let?s try to use ?package?
> only for a directory that you can import from Python, and ?distribution?
> for a bundle/archive of one version of a project.
>
> http://tarekziade.wordpress.com/2010/01/07/fixing-packaging-terminology-confusion/
> http://docs.python.org/dev/distutils/introduction#concepts-terminology

The problem is that it's not distributions (although it will test them
too). It's "Things that has a setup.py" and "what has an entry on
PyPI".  This is generally called a package, as in Python *Package*
Index. I agree this is confusing. But saying that Pyroma tests
distributions would also be confusing. :-) If a decision comes to call
these "projects", then I'm fine with that, but currently they are
called packages.

/Lennart

From merwok at netwok.org  Mon Mar  7 11:23:22 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Mon, 07 Mar 2011 11:23:22 +0100
Subject: [Distutils] Proposed change to buildout configuration-file
	handling
In-Reply-To: <AANLkTi=whFcCFq4uUnXadx3dS7tf9az+dz1g8TT2JvE7@mail.gmail.com>
References: <AANLkTi=whFcCFq4uUnXadx3dS7tf9az+dz1g8TT2JvE7@mail.gmail.com>
Message-ID: <4D74B21A.5060702@netwok.org>

Le 06/03/2011 16:34, Jim Fulton a ?crit :
> An annoyance in buildout configurations, resulting from [its] use of
> ConfigParser, is that configuration data lines are stripped of leading
> spaces.

configparser is much better in 3.2 (I?m cc?ing the author of those
improvements and new module maintainer), and I already suggested a
standalone release on PyPI for older Pythons.  It would be nice to work
out whether your needs could be addressed directly at the source before
being forced to fork or monkey-patch.

Regards

From merwok at netwok.org  Mon Mar  7 11:40:29 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Mon, 07 Mar 2011 11:40:29 +0100
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <AANLkTimbs7c9xVGxxoN+RypusqfCC9ACT2k=E-978BzD@mail.gmail.com>
References: <AANLkTim02Dv=J9RmQdVhWXRdML8rsvRNzRe-A6BP0290@mail.gmail.com>
	<4D74AD04.6040900@netwok.org>
	<AANLkTimbs7c9xVGxxoN+RypusqfCC9ACT2k=E-978BzD@mail.gmail.com>
Message-ID: <4D74B61D.10400@netwok.org>

> The problem is that it's not distributions (although it will test them
> too). It's "Things that has a setup.py" and "what has an entry on
> PyPI".

Okay, so that?s called a project in distutils docs and PEPs (more info
on Tarek?s post I linked to, which links to a previous distutils-sig
thread).  The names of PyPI and packages.python.org are the only parts
of our official infrastructure that disagree, if I recall correctly.

(It has been suggested to backronym PyPI to Python Projects Index, but
this was rejected for reasons I don?t remember.)

Of course, people coming from Perl or using an operating system with a
concept of ?software packages? and ?package manager? naturally call the
things you can download from PyPI ?packages?.  People from the BSD and
linux world also already have a conflicting use for ?distribution?.

It?s probably too late to try to call those downloadable things
archives, parcels or bundles, and even more too late to rename Python
packages ?supermodules? or something else to free up the word ?package?.

My point is that if we have to live with this imperfect situation, we
should try to remain consistent as much as possible to prevent confusion
(see for example the thread about distribution/release confusion in PEP
345 that was opened *after* the PEP was thoroughly discussed and accepted).

Regards

From regebro at gmail.com  Mon Mar  7 11:46:01 2011
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 7 Mar 2011 11:46:01 +0100
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <4D74B61D.10400@netwok.org>
References: <AANLkTim02Dv=J9RmQdVhWXRdML8rsvRNzRe-A6BP0290@mail.gmail.com>
	<4D74AD04.6040900@netwok.org>
	<AANLkTimbs7c9xVGxxoN+RypusqfCC9ACT2k=E-978BzD@mail.gmail.com>
	<4D74B61D.10400@netwok.org>
Message-ID: <AANLkTi=Dk=q6VGZGqjRL8uW7UodE1eVeoCHGyhgD7XSE@mail.gmail.com>

2011/3/7 ?ric Araujo <merwok at netwok.org>:
>> The problem is that it's not distributions (although it will test them
>> too). It's "Things that has a setup.py" and "what has an entry on
>> PyPI".
>
> Okay, so that?s called a project in distutils docs

In distutils2, yes.

> My point is that if we have to live with this imperfect situation, we
> should try to remain consistent as much as possible to prevent confusion

I'll make a note of this in the documentation to clear it up.
Distutils2 is definitely in the minority at the moment when it comes
to calling them "projects".

//Lennart

From merwok at netwok.org  Mon Mar  7 11:53:18 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Mon, 07 Mar 2011 11:53:18 +0100
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <AANLkTi=Dk=q6VGZGqjRL8uW7UodE1eVeoCHGyhgD7XSE@mail.gmail.com>
References: <AANLkTim02Dv=J9RmQdVhWXRdML8rsvRNzRe-A6BP0290@mail.gmail.com>
	<4D74AD04.6040900@netwok.org>
	<AANLkTimbs7c9xVGxxoN+RypusqfCC9ACT2k=E-978BzD@mail.gmail.com>
	<4D74B61D.10400@netwok.org>
	<AANLkTi=Dk=q6VGZGqjRL8uW7UodE1eVeoCHGyhgD7XSE@mail.gmail.com>
Message-ID: <4D74B91E.1060108@netwok.org>

>> Okay, so that?s called a project in distutils docs
> In distutils2, yes.

You?re right, I checked again and found that ?packages? has slipped into
distutils docs, so it?s only the PEPs and distutils2 docs that try to be
sane.

Thanks for listening to my request. :)

Regards

From jim at zope.com  Mon Mar  7 12:35:30 2011
From: jim at zope.com (Jim Fulton)
Date: Mon, 7 Mar 2011 06:35:30 -0500
Subject: [Distutils] Proposed change to buildout configuration-file
	handling
In-Reply-To: <4D74B21A.5060702@netwok.org>
References: <AANLkTi=whFcCFq4uUnXadx3dS7tf9az+dz1g8TT2JvE7@mail.gmail.com>
	<4D74B21A.5060702@netwok.org>
Message-ID: <AANLkTinNz5zd_skoi2=nxVJwd37535eSGnve6=eNj6wd@mail.gmail.com>

On Mon, Mar 7, 2011 at 5:23 AM, ?ric Araujo <merwok at netwok.org> wrote:
> Le 06/03/2011 16:34, Jim Fulton a ?crit :
>> An annoyance in buildout configurations, resulting from [its] use of
>> ConfigParser, is that configuration data lines are stripped of leading
>> spaces.
>
> configparser is much better in 3.2 (I?m cc?ing the author of those
> improvements and new module maintainer), and I already suggested a
> standalone release on PyPI for older Pythons. ?It would be nice to work
> out whether your needs could be addressed directly at the source before
> being forced to fork or monkey-patch.

3.2 is irrelevant.

While I plan to port buildout to Python 3 at PyCon, I intent to
maintain buildout on Python 2 for some time, so there's really no
point in depending on changes made in 3.2.

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From jim at zope.com  Mon Mar  7 15:03:25 2011
From: jim at zope.com (Jim Fulton)
Date: Mon, 7 Mar 2011 09:03:25 -0500
Subject: [Distutils] Proposed change to buildout configuration-file
	handling
In-Reply-To: <2F186522-40C5-479A-9AD2-DF669C874625@langa.pl>
References: <AANLkTi=whFcCFq4uUnXadx3dS7tf9az+dz1g8TT2JvE7@mail.gmail.com>
	<4D74B21A.5060702@netwok.org>
	<AANLkTinNz5zd_skoi2=nxVJwd37535eSGnve6=eNj6wd@mail.gmail.com>
	<2F186522-40C5-479A-9AD2-DF669C874625@langa.pl>
Message-ID: <AANLkTimSnR38GAiiY5_AN4heNW-XoWeo8w4XAZp9JsWC@mail.gmail.com>

On Mon, Mar 7, 2011 at 6:42 AM, ?ukasz Langa <lukasz at langa.pl> wrote:
>
> Wiadomo?? napisana przez Jim Fulton w dniu 2011-03-07, o godz. 12:35:
>
> On Mon, Mar 7, 2011 at 5:23 AM, ?ric Araujo <merwok at netwok.org> wrote:
>
> Le 06/03/2011 16:34, Jim Fulton a ?crit :
>
> An annoyance in buildout configurations, resulting from [its] use of
>
> ConfigParser, is that configuration data lines are stripped of leading
>
> spaces.
>
> configparser is much better in 3.2 (I'm cc'ing the author of those
>
> improvements and new module maintainer), and I already suggested a
>
> standalone release on PyPI for older Pythons.  It would be nice to work
>
> out whether your needs could be addressed directly at the source before
>
> being forced to fork or monkey-patch.
>
> 3.2 is irrelevant.
>
> While I plan to port buildout to Python 3 at PyCon, I intent to
> maintain buildout on Python 2 for some time, so there's really no
> point in depending on changes made in 3.2.
>
>
> If that changes anything, you can expect a 2.4+ compatible version soon (how
> soon depends on when you need it), much like unittest2 ports Python 3.2
> unittest down.

How so? As a pypi release?

I'd consider it under those circumstances.  I'll take a look at the
new version before writing my own then (although writing my own is a
pretty trivial exercise of copying the existing _read method and
making a few minor tweaks).

Jim


-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From merwok at netwok.org  Mon Mar  7 15:08:21 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Mon, 07 Mar 2011 15:08:21 +0100
Subject: [Distutils] Proposed change to buildout configuration-file
	handling
In-Reply-To: <AANLkTimSnR38GAiiY5_AN4heNW-XoWeo8w4XAZp9JsWC@mail.gmail.com>
References: <AANLkTi=whFcCFq4uUnXadx3dS7tf9az+dz1g8TT2JvE7@mail.gmail.com>
	<4D74B21A.5060702@netwok.org>
	<AANLkTinNz5zd_skoi2=nxVJwd37535eSGnve6=eNj6wd@mail.gmail.com>
	<2F186522-40C5-479A-9AD2-DF669C874625@langa.pl>
	<AANLkTimSnR38GAiiY5_AN4heNW-XoWeo8w4XAZp9JsWC@mail.gmail.com>
Message-ID: <4D74E6D5.3040204@netwok.org>

Yes, on PyPI, as I hinted in my first message:

>> configparser is much better in 3.2 (I'm cc'ing the author of those
>> improvements and new module maintainer), and I already suggested a
>> standalone release on PyPI for older Pythons.

:)

From sienkiew at stsci.edu  Mon Mar  7 22:06:38 2011
From: sienkiew at stsci.edu (Mark Sienkiewicz)
Date: Mon, 07 Mar 2011 16:06:38 -0500
Subject: [Distutils] [Catalog-sig]  pypi/packages/docs.python.org
In-Reply-To: <AANLkTino9Ttx++_gqQQo95eJ6mChuiRmR+R_S4rW7LPV@mail.gmail.com>
References: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>	<AANLkTinv_ADD_fjQQBw+Ptp3MJPZapP64xvcXU+05Js8@mail.gmail.com>	<4D73F1A9.1000207@v.loewis.de>	<AANLkTi=vxdv_MTa4JujZPB4=kLuc94P100KiqZFr_a8e@mail.gmail.com>	<4D73F9FE.2060506@v.loewis.de>	<AANLkTi=98hs7oXnmdZNmxPQ2Cu4CBRdMKaTJ07PAk_uB@mail.gmail.com>	<4D73FFF8.3020801@v.loewis.de>
	<4D740826.3030207@egenix.com>
	<AANLkTino9Ttx++_gqQQo95eJ6mChuiRmR+R_S4rW7LPV@mail.gmail.com>
Message-ID: <4D7548DE.6040602@stsci.edu>

Richard Jones wrote:
> On Mon, Mar 7, 2011 at 9:18 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>   
>> "Martin v. L?wis" wrote:
>>     
>>> if anybody would *not* want to have a "Documentation" link automatically
>>> generated that points to packages.python.org/<project>, please speak up.
>>>       
>> That would only make sense if there's something uploaded to
>> the PyPI docs dir.
>>
>> If PyPI can detect that, +1. Otherwise, I think it's better not
>> adding such an automatic link, since no link is better than one
>> going nowhere.
>>     
>
> Absolutely - it only generates the link if there's something to link
> to (that is, there's been a documentation upload with an index.html
> file at the root, or in a top-level "html" directory).
>   


I suggest:

- Generate a link if there is documentation

- Generate the text "No documentation" in the place where the link would 
be if there is no documentation.

I have two reasons for suggesting this:

1. I'm thinking explicit is better than implicit here.  The link for the 
documentation should always be in the same place on the page, so having 
a placeholder stating "there's nothing here" will make the page easier 
to read.  You don't wonder if you missed it, or if something is wrong 
that the link disappeared -- it says right there what you want to know.

2. It might remind developers that it is possible to upload 
documentation with their package.  Currently, a lot of stuff on pypi has 
poor or nonexistent documentation; the resource as a whole would be far 
more valuable if every package had at least a minimal summary of what it 
does and how to use it.  (Even if someone has no need for your package, 
you still help them if they can determine that _quickly_.)



From pje at telecommunity.com  Mon Mar  7 22:15:27 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 07 Mar 2011 16:15:27 -0500
Subject: [Distutils] Name the software! Package quality tester.
Message-ID: <20110307211538.2777B3A4062@sparrow.telecommunity.com>

At 11:46 AM 3/7/2011 +0100, Lennart Regebro wrote:
>I'll make a note of this in the documentation to clear it up.
>Distutils2 is definitely in the minority at the moment when it comes
>to calling them "projects".

The term has been in use in setuptools since around 2005, but it 
hasn't caught on much outside of the small group of people who need 
to be able to speak precisely about the concept.  ;-)

If you search the sig archives, though, you will find that it gets 
proposed and mostly-approved every time the topic comes up, of what 
to call these "things we distribute releases of".  The more-or-less 
consensus terminology (for people who need a precise terminology):

package = thing you import in Python that contains modules
project = thing you make releases of
release = one version of a project
distribution = a file that embodies the release of a project (may be 
source or binary)

People who don't care about precision just call everything a package, 
pretty much.  Heck, a lot of times I find myself starting to type 
"package" when I'm really talking about a project, release, or 
distribution, despite promoted the more-precise terminology for half 
a decade or so.  ;-)


From martin at v.loewis.de  Mon Mar  7 22:25:12 2011
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 07 Mar 2011 22:25:12 +0100
Subject: [Distutils] pypi/packages/docs.python.org
In-Reply-To: <4D74B0F6.5080204@netwok.org>
References: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>	<AANLkTinv_ADD_fjQQBw+Ptp3MJPZapP64xvcXU+05Js8@mail.gmail.com>	<4D73F1A9.1000207@v.loewis.de>	<AANLkTi=vxdv_MTa4JujZPB4=kLuc94P100KiqZFr_a8e@mail.gmail.com>
	<4D73F9FE.2060506@v.loewis.de> <4D74B0F6.5080204@netwok.org>
Message-ID: <4D754D38.3080300@v.loewis.de>

Am 07.03.2011 11:18, schrieb ?ric Araujo:
> Le 06/03/2011 22:17, "Martin v. L?wis" a ?crit :
>> As for changing the style: please submit a html file to replace what is
>> there (I refuse to do any styling .
>
> Do you mean you don?t want to work on styling personally or that you
> object to someone proposing a patch adding styling?

The former. Consequentially, I accept any style changes that people
propose without discussion (if they are spelled out explicitly,
preferably as code).

Regards,
Martin

From martin at v.loewis.de  Mon Mar  7 22:26:27 2011
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 07 Mar 2011 22:26:27 +0100
Subject: [Distutils] [Catalog-sig]  pypi/packages/docs.python.org
In-Reply-To: <4D7548DE.6040602@stsci.edu>
References: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>	<AANLkTinv_ADD_fjQQBw+Ptp3MJPZapP64xvcXU+05Js8@mail.gmail.com>	<4D73F1A9.1000207@v.loewis.de>	<AANLkTi=vxdv_MTa4JujZPB4=kLuc94P100KiqZFr_a8e@mail.gmail.com>	<4D73F9FE.2060506@v.loewis.de>	<AANLkTi=98hs7oXnmdZNmxPQ2Cu4CBRdMKaTJ07PAk_uB@mail.gmail.com>	<4D73FFF8.3020801@v.loewis.de>	<4D740826.3030207@egenix.com>	<AANLkTino9Ttx++_gqQQo95eJ6mChuiRmR+R_S4rW7LPV@mail.gmail.com>
	<4D7548DE.6040602@stsci.edu>
Message-ID: <4D754D83.6070308@v.loewis.de>

> - Generate the text "No documentation" in the place where the link would
> be if there is no documentation.

-1. Saying "No documentation" would be wrong if the package, in fact,
does have documentation, but elsewhere.

Regards,
Martin

From eric at trueblade.com  Mon Mar  7 22:46:04 2011
From: eric at trueblade.com (Eric Smith)
Date: Mon, 07 Mar 2011 16:46:04 -0500
Subject: [Distutils] [Catalog-sig]  pypi/packages/docs.python.org
In-Reply-To: <4D754D83.6070308@v.loewis.de>
References: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>	<AANLkTinv_ADD_fjQQBw+Ptp3MJPZapP64xvcXU+05Js8@mail.gmail.com>	<4D73F1A9.1000207@v.loewis.de>	<AANLkTi=vxdv_MTa4JujZPB4=kLuc94P100KiqZFr_a8e@mail.gmail.com>	<4D73F9FE.2060506@v.loewis.de>	<AANLkTi=98hs7oXnmdZNmxPQ2Cu4CBRdMKaTJ07PAk_uB@mail.gmail.com>	<4D73FFF8.3020801@v.loewis.de>	<4D740826.3030207@egenix.com>	<AANLkTino9Ttx++_gqQQo95eJ6mChuiRmR+R_S4rW7LPV@mail.gmail.com>	<4D7548DE.6040602@stsci.edu>
	<4D754D83.6070308@v.loewis.de>
Message-ID: <4D75521C.8000705@trueblade.com>

On 3/7/2011 4:26 PM, "Martin v. L?wis" wrote:
>> - Generate the text "No documentation" in the place where the link would
>> be if there is no documentation.
>
> -1. Saying "No documentation" would be wrong if the package, in fact,
> does have documentation, but elsewhere.

"No documentation uploaded to PyPi", maybe? I can see some value in 
noting that it's possible, just not done for this release.

From lukasz at langa.pl  Mon Mar  7 12:42:12 2011
From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=)
Date: Mon, 7 Mar 2011 12:42:12 +0100
Subject: [Distutils] Proposed change to buildout configuration-file
	handling
In-Reply-To: <AANLkTinNz5zd_skoi2=nxVJwd37535eSGnve6=eNj6wd@mail.gmail.com>
References: <AANLkTi=whFcCFq4uUnXadx3dS7tf9az+dz1g8TT2JvE7@mail.gmail.com>
	<4D74B21A.5060702@netwok.org>
	<AANLkTinNz5zd_skoi2=nxVJwd37535eSGnve6=eNj6wd@mail.gmail.com>
Message-ID: <2F186522-40C5-479A-9AD2-DF669C874625@langa.pl>


Wiadomo?? napisana przez Jim Fulton w dniu 2011-03-07, o godz. 12:35:

> On Mon, Mar 7, 2011 at 5:23 AM, ?ric Araujo <merwok at netwok.org> wrote:
>> Le 06/03/2011 16:34, Jim Fulton a ?crit :
>>> An annoyance in buildout configurations, resulting from [its] use of
>>> ConfigParser, is that configuration data lines are stripped of leading
>>> spaces.
>> 
>> configparser is much better in 3.2 (I?m cc?ing the author of those
>> improvements and new module maintainer), and I already suggested a
>> standalone release on PyPI for older Pythons.  It would be nice to work
>> out whether your needs could be addressed directly at the source before
>> being forced to fork or monkey-patch.
> 
> 3.2 is irrelevant.
> 
> While I plan to port buildout to Python 3 at PyCon, I intent to
> maintain buildout on Python 2 for some time, so there's really no
> point in depending on changes made in 3.2.
> 

If that changes anything, you can expect a 2.4+ compatible version soon (how soon depends on when you need it), much like unittest2 ports Python 3.2 unittest down.

-- 
Pozdrawiam serdecznie,
?ukasz Langa
tel. +48 791 080 144
WWW http://lukasz.langa.pl/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110307/168c6235/attachment.html>

From jim at zope.com  Mon Mar  7 23:18:57 2011
From: jim at zope.com (Jim Fulton)
Date: Mon, 7 Mar 2011 17:18:57 -0500
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <20110307211538.2777B3A4062@sparrow.telecommunity.com>
References: <20110307211538.2777B3A4062@sparrow.telecommunity.com>
Message-ID: <AANLkTi=E0LGnM1XMOFYE15qmLppoAX0igC7HNGZqASfd@mail.gmail.com>

On Mon, Mar 7, 2011 at 4:15 PM, P.J. Eby <pje at telecommunity.com> wrote:
> At 11:46 AM 3/7/2011 +0100, Lennart Regebro wrote:
>>
>> I'll make a note of this in the documentation to clear it up.
>> Distutils2 is definitely in the minority at the moment when it comes
>> to calling them "projects".
>
> The term has been in use in setuptools since around 2005, but it hasn't
> caught on much outside of the small group of people who need to be able to
> speak precisely about the concept. ?;-)
>
> If you search the sig archives, though, you will find that it gets proposed
> and mostly-approved every time the topic comes up, of what to call these
> "things we distribute releases of". ?The more-or-less consensus terminology
> (for people who need a precise terminology):
>
> package = thing you import in Python that contains modules
> project = thing you make releases of
> release = one version of a project
> distribution = a file that embodies the release of a project (may be source
> or binary)

+1

> People who don't care about precision just call everything a package, pretty
> much. ?Heck, a lot of times I find myself starting to type "package" when
> I'm really talking about a project, release, or distribution, despite
> promoted the more-precise terminology for half a decade or so. ?;-)

The root of the problem, IMO, is Python's (mis)use of the name "package"
for what is really only a nested module. Heck:

>>> import distutils
>>> distutils
<module 'distutils' ...>

Given that Python 3 is a reboot, maybe it's time for the Python
community to start calling these what ``python`` calls them,
"modules".

If what we now call "packages" were called "modules", then we could
start using the term "package" the way everyone else does. I think
lots of people would be less confused.

Otherwise, I prefer we try hard to use the precise definitions
above. This topic can be confusing enough without making it more so
through sloppy terminology.

Jim

--
Jim Fulton
http://www.linkedin.com/in/jimfulton

From fdrake at acm.org  Mon Mar  7 23:31:04 2011
From: fdrake at acm.org (Fred Drake)
Date: Mon, 7 Mar 2011 17:31:04 -0500
Subject: [Distutils] Proposed change to buildout configuration-file
	handling
In-Reply-To: <2F186522-40C5-479A-9AD2-DF669C874625@langa.pl>
References: <AANLkTi=whFcCFq4uUnXadx3dS7tf9az+dz1g8TT2JvE7@mail.gmail.com>
	<4D74B21A.5060702@netwok.org>
	<AANLkTinNz5zd_skoi2=nxVJwd37535eSGnve6=eNj6wd@mail.gmail.com>
	<2F186522-40C5-479A-9AD2-DF669C874625@langa.pl>
Message-ID: <AANLkTink5GyR6JitV7snCevqnaHqbAW17bDhR+NTzT=x@mail.gmail.com>

2011/3/7 ?ukasz Langa <lukasz at langa.pl>:
> If that changes anything, you can expect a 2.4+ compatible version soon (how
> soon depends on when you need it), much like unittest2 ports Python 3.2
> unittest down.

And for Python 2 at least, it can even take it's real name,
configparser.  That won't work for Python 3.1, though.


? -Fred

--
Fred L. Drake, Jr.? ? <fdrake at acm.org>
"A storm broke loose in my mind."? --Albert Einstein

From richard at python.org  Mon Mar  7 23:52:56 2011
From: richard at python.org (Richard Jones)
Date: Tue, 8 Mar 2011 09:52:56 +1100
Subject: [Distutils] [Catalog-sig] pypi/packages/docs.python.org
In-Reply-To: <4D7548DE.6040602@stsci.edu>
References: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>
	<AANLkTinv_ADD_fjQQBw+Ptp3MJPZapP64xvcXU+05Js8@mail.gmail.com>
	<4D73F1A9.1000207@v.loewis.de>
	<AANLkTi=vxdv_MTa4JujZPB4=kLuc94P100KiqZFr_a8e@mail.gmail.com>
	<4D73F9FE.2060506@v.loewis.de>
	<AANLkTi=98hs7oXnmdZNmxPQ2Cu4CBRdMKaTJ07PAk_uB@mail.gmail.com>
	<4D73FFF8.3020801@v.loewis.de> <4D740826.3030207@egenix.com>
	<AANLkTino9Ttx++_gqQQo95eJ6mChuiRmR+R_S4rW7LPV@mail.gmail.com>
	<4D7548DE.6040602@stsci.edu>
Message-ID: <AANLkTinQ6wUQ8xPGwXwQFO1xNuPo3qZ+a4K4iC7g9wqa@mail.gmail.com>

On Tue, Mar 8, 2011 at 8:06 AM, Mark Sienkiewicz <sienkiew at stsci.edu> wrote:
> I suggest:
>
> - Generate a link if there is documentation

Right, we do this already.


> - Generate the text "No documentation" in the place where the link would be
> if there is no documentation.

I don't like this. The documentation space on pypi is offered as a
space for projects that's convenient or where they otherwise wouldn't
be able to host their own documentation website. It's not something I
wish to force people to use (just like package upload isn't something
we force people to do), and stating "no documentation" would be a bit
of a lie in a bunch of cases. Even with additional clarifying wording
it's still a negative statement that I don't think we need to make.


     Richard

From benji at benjiyork.com  Tue Mar  8 00:09:24 2011
From: benji at benjiyork.com (Benji York)
Date: Mon, 7 Mar 2011 18:09:24 -0500
Subject: [Distutils] [Catalog-sig] pypi/packages/docs.python.org
In-Reply-To: <AANLkTinQ6wUQ8xPGwXwQFO1xNuPo3qZ+a4K4iC7g9wqa@mail.gmail.com>
References: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>
	<AANLkTinv_ADD_fjQQBw+Ptp3MJPZapP64xvcXU+05Js8@mail.gmail.com>
	<4D73F1A9.1000207@v.loewis.de>
	<AANLkTi=vxdv_MTa4JujZPB4=kLuc94P100KiqZFr_a8e@mail.gmail.com>
	<4D73F9FE.2060506@v.loewis.de>
	<AANLkTi=98hs7oXnmdZNmxPQ2Cu4CBRdMKaTJ07PAk_uB@mail.gmail.com>
	<4D73FFF8.3020801@v.loewis.de> <4D740826.3030207@egenix.com>
	<AANLkTino9Ttx++_gqQQo95eJ6mChuiRmR+R_S4rW7LPV@mail.gmail.com>
	<4D7548DE.6040602@stsci.edu>
	<AANLkTinQ6wUQ8xPGwXwQFO1xNuPo3qZ+a4K4iC7g9wqa@mail.gmail.com>
Message-ID: <AANLkTi=29BX5-tJF9QuznzNoS3ZedmLUtoH0yzteq4zu@mail.gmail.com>

On Mon, Mar 7, 2011 at 5:52 PM, Richard Jones <richard at python.org> wrote:
> On Tue, Mar 8, 2011 at 8:06 AM, Mark Sienkiewicz <sienkiew at stsci.edu> wrote:
>> I suggest:
>>
>> - Generate a link if there is documentation
>
> Right, we do this already.
>
>
>> - Generate the text "No documentation" in the place where the link would be
>> if there is no documentation.
>
> I don't like this. The documentation space on pypi is offered as a
> space for projects that's convenient or where they otherwise wouldn't
> be able to host their own documentation website. It's not something I
> wish to force people to use (just like package upload isn't something
> we force people to do), and stating "no documentation" would be a bit
> of a lie in a bunch of cases. Even with additional clarifying wording
> it's still a negative statement that I don't think we need to make.

What about this:

1) have some sort of "disabled" link placeholder so the documentation
   link is always there, but not an attractive nuisance leading to a
   dead end when there are no docs available and

2) let project owners provide an alternative target for the link if they
   host their documentation elsewhere.
-- 
Benji York

From greg.ewing at canterbury.ac.nz  Tue Mar  8 00:12:09 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 08 Mar 2011 12:12:09 +1300
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <AANLkTimbs7c9xVGxxoN+RypusqfCC9ACT2k=E-978BzD@mail.gmail.com>
References: <AANLkTim02Dv=J9RmQdVhWXRdML8rsvRNzRe-A6BP0290@mail.gmail.com>
	<4D74AD04.6040900@netwok.org>
	<AANLkTimbs7c9xVGxxoN+RypusqfCC9ACT2k=E-978BzD@mail.gmail.com>
Message-ID: <4D756649.3000807@canterbury.ac.nz>

Lennart Regebro wrote:
> If a decision comes to call
> these "projects", then I'm fine with that,

+1 on 'project' as the official term.

A bonus is that we could decide that PyPI stands for
Python Project Index if we wanted.

-- 
Greg

From barry at python.org  Tue Mar  8 00:14:07 2011
From: barry at python.org (Barry Warsaw)
Date: Mon, 7 Mar 2011 18:14:07 -0500
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <AANLkTi=E0LGnM1XMOFYE15qmLppoAX0igC7HNGZqASfd@mail.gmail.com>
References: <20110307211538.2777B3A4062@sparrow.telecommunity.com>
	<AANLkTi=E0LGnM1XMOFYE15qmLppoAX0igC7HNGZqASfd@mail.gmail.com>
Message-ID: <20110307181407.1cb799b7@neurotica.wooz.org>

On Mar 07, 2011, at 05:18 PM, Jim Fulton wrote:

>Given that Python 3 is a reboot, maybe it's time for the Python
>community to start calling these what ``python`` calls them,
>"modules".

Very nice.  +1.  And "nested module" where the distinction is important.  Even
"namespace module" sounds okay to me.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110307/ee4f8451/attachment.pgp>

From greg.ewing at canterbury.ac.nz  Tue Mar  8 00:57:37 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 08 Mar 2011 12:57:37 +1300
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <AANLkTi=E0LGnM1XMOFYE15qmLppoAX0igC7HNGZqASfd@mail.gmail.com>
References: <20110307211538.2777B3A4062@sparrow.telecommunity.com>
	<AANLkTi=E0LGnM1XMOFYE15qmLppoAX0igC7HNGZqASfd@mail.gmail.com>
Message-ID: <4D7570F1.4000302@canterbury.ac.nz>

Jim Fulton wrote:

> If what we now call "packages" were called "modules", then we could
> start using the term "package" the way everyone else does.

But then we would not have a term for a module that corresponds
to a directory rather than a single .py file.

-- 
Greg

From regebro at gmail.com  Tue Mar  8 10:34:17 2011
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 8 Mar 2011 10:34:17 +0100
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <20110307204732.06E013A4077@sparrow.telecommunity.com>
References: <AANLkTim02Dv=J9RmQdVhWXRdML8rsvRNzRe-A6BP0290@mail.gmail.com>
	<4D74AD04.6040900@netwok.org>
	<AANLkTimbs7c9xVGxxoN+RypusqfCC9ACT2k=E-978BzD@mail.gmail.com>
	<4D74B61D.10400@netwok.org>
	<AANLkTi=Dk=q6VGZGqjRL8uW7UodE1eVeoCHGyhgD7XSE@mail.gmail.com>
	<20110307204732.06E013A4077@sparrow.telecommunity.com>
Message-ID: <AANLkTimxkrN=QZpRkSWA2P1g5XwB6bfgyGgvfTGiNPD7@mail.gmail.com>

On Mon, Mar 7, 2011 at 21:47, P.J. Eby <pje at telecommunity.com> wrote:
> The term has been in use in setuptools since around 2005, but it hasn't
> caught on much outside of the small group of people who need to be able to
> speak precisely about the concept. ?;-)

Well, I've changed the terminology in pyroma, to hopefully spread it
around a bit.

//Lennart

From regebro at gmail.com  Tue Mar  8 10:39:06 2011
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 8 Mar 2011 10:39:06 +0100
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <AANLkTi=E0LGnM1XMOFYE15qmLppoAX0igC7HNGZqASfd@mail.gmail.com>
References: <20110307211538.2777B3A4062@sparrow.telecommunity.com>
	<AANLkTi=E0LGnM1XMOFYE15qmLppoAX0igC7HNGZqASfd@mail.gmail.com>
Message-ID: <AANLkTinbfYnY7ZU=gfd9XaYD2_9B1dy=phy7iC-wabnE@mail.gmail.com>

On Mon, Mar 7, 2011 at 23:18, Jim Fulton <jim at zope.com> wrote:
> Given that Python 3 is a reboot, maybe it's time for the Python
> community to start calling these what ``python`` calls them,
> "modules".

Well, given that the term "project" hasn't gained widespread
acceptance, maybe we should adjust the terminology to how people
actually use it.
That could only happen in Python 3.3, though, and obviously needs to
be discussed widely. Is there a language summit this year?

//Lennart

From jim at zope.com  Tue Mar  8 11:38:40 2011
From: jim at zope.com (Jim Fulton)
Date: Tue, 8 Mar 2011 05:38:40 -0500
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <4D7570F1.4000302@canterbury.ac.nz>
References: <20110307211538.2777B3A4062@sparrow.telecommunity.com>
	<AANLkTi=E0LGnM1XMOFYE15qmLppoAX0igC7HNGZqASfd@mail.gmail.com>
	<4D7570F1.4000302@canterbury.ac.nz>
Message-ID: <AANLkTi=JU091RF2MzEYVGZb7vp9Sv24ML1xxYk5cGsn7@mail.gmail.com>

On Mon, Mar 7, 2011 at 6:57 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Jim Fulton wrote:
>
>> If what we now call "packages" were called "modules", then we could
>> start using the term "package" the way everyone else does.
>
> But then we would not have a term for a module that corresponds
> to a directory rather than a single .py file.

I question whether that distinction is important, but if and when it
is, then we could use an adjective to clarify.  Under the hood, the
object we call packages today are just modules.

Jim


-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From alexis at notmyidea.org  Tue Mar  8 14:13:52 2011
From: alexis at notmyidea.org (=?ISO-8859-1?Q?Alexis_M=E9taireau?=)
Date: Tue, 08 Mar 2011 13:13:52 +0000
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <AANLkTinbfYnY7ZU=gfd9XaYD2_9B1dy=phy7iC-wabnE@mail.gmail.com>
References: <20110307211538.2777B3A4062@sparrow.telecommunity.com>	<AANLkTi=E0LGnM1XMOFYE15qmLppoAX0igC7HNGZqASfd@mail.gmail.com>
	<AANLkTinbfYnY7ZU=gfd9XaYD2_9B1dy=phy7iC-wabnE@mail.gmail.com>
Message-ID: <4D762B90.9070009@notmyidea.org>

Le 08/03/2011 09:39, Lennart Regebro a ?crit :
> Well, given that the term "project" hasn't gained widespread
> acceptance, maybe we should adjust the terminology to how people
> actually use it.
> That could only happen in Python 3.3, though, and obviously needs to
> be discussed widely. Is there a language summit this year?

The sure thing is we have to agree on something different that what we
already have: packages for directories containing .py files and packages
for things we want to distribute, which should have different names.

I'm +1 with the idea of replacing "packages" by "modules", especially
because this "package" thing just add some fuss to the overall
comprehension. Why do we need to make a distinctions between python
packages and python modules ? Why a package does contains modules rather
thatn the other way around?

If you have some pointers to the discussion (if any) that leaded to
these choices, it could certainly help us to understand the rationale
leading to this choice, and make us able to chose the right names.

We really need to do something about those names *now*, especially
because it's not too late to change names in use in distutils2 (it will
probably be more complicated once it will be integrated in the stdlib).

Maybe can we even match the rename of PyPI into python project index
with the release of distutils2 and the "renew of python packaging" ?

Cheers,
Alex

From pje at telecommunity.com  Tue Mar  8 18:26:51 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 08 Mar 2011 12:26:51 -0500
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <AANLkTi=E0LGnM1XMOFYE15qmLppoAX0igC7HNGZqASfd@mail.gmail.c
 om>
References: <20110307211538.2777B3A4062@sparrow.telecommunity.com>
	<AANLkTi=E0LGnM1XMOFYE15qmLppoAX0igC7HNGZqASfd@mail.gmail.com>
Message-ID: <20110308172703.716583A4062@sparrow.telecommunity.com>

At 05:18 PM 3/7/2011 -0500, Jim Fulton wrote:
>If what we now call "packages" were called "modules", then we could
>start using the term "package" the way everyone else does. I think
>lots of people would be less confused.

It seems to me that in order to make that change, you have to get 
more people to change their terminology, since the set of people who 
need to refer to package[module] is larger than the set of people who 
need to refer to package[project].  (There is also a larger body of 
documentation associated with package[module].)

IOW, I think this proposal is a heavy uphill battle, both in the 
number of people to be convinced and the amount of documentation.  In 
addition, the people who are calling a project a package can more 
easily understand the need to call it a project, than the people who 
are calling a package a package, will understand the need to call it 
a module.  ;-)


>Otherwise, I prefer we try hard to use the precise definitions
>above. This topic can be confusing enough without making it more so
>through sloppy terminology.

I think that this approach is more achievable: it requires only an 
official blessing, a relatively small amount of documentation to be 
changed, and the renaming of PyPI et al.  (i.e. "Python Projects 
Index", projects.python.org, etc.)

("Projects Index" is a better name anyway, since some things on PyPI 
are not packages at all but applications, scripts, modules, plugins, etc.)


From greg.ewing at canterbury.ac.nz  Wed Mar  9 00:00:39 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 09 Mar 2011 12:00:39 +1300
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <AANLkTi=JU091RF2MzEYVGZb7vp9Sv24ML1xxYk5cGsn7@mail.gmail.com>
References: <20110307211538.2777B3A4062@sparrow.telecommunity.com>
	<AANLkTi=E0LGnM1XMOFYE15qmLppoAX0igC7HNGZqASfd@mail.gmail.com>
	<4D7570F1.4000302@canterbury.ac.nz>
	<AANLkTi=JU091RF2MzEYVGZb7vp9Sv24ML1xxYk5cGsn7@mail.gmail.com>
Message-ID: <4D76B517.2070400@canterbury.ac.nz>

Jim Fulton wrote:

> I question whether that distinction is important, but if and when it
> is, then we could use an adjective to clarify.  Under the hood, the
> object we call packages today are just modules.

I think the fact that we used the word "package" in the
first place testifies that it *is* important -- otherwise
we wouldn't have invented the term.

The word "package" covers much more than just the module
object that comes into being when you import it. It also
implies the associated directory structure, extra import
facilities and mechanisms, etc. It's not just a minor
variation on the concept of a module.

-- 
Greg

From thomas at thomas-lotze.de  Wed Mar  9 07:27:17 2011
From: thomas at thomas-lotze.de (Thomas Lotze)
Date: Wed, 09 Mar 2011 07:27:17 +0100
Subject: [Distutils] zc.buildout: new option for requiring MD5 sums for
	downloads
Message-ID: <pan.2011.03.09.06.27.16.865299@ID-174572.user.uni-berlin.de>

Hi,

I wonder whether it makes sense to introduce a new option to the buildout
section, named downloads-require-md5sum or similar. Recipes such as
zc.recipe.cmmi or gocept.download could then be made to require an MD5
checksum for every downloaded resource if that option is set to true, and
treat checksums as optional otherwise.

The responsibility for evaluating that option would have to lie with
recipes, not buildout's download API for two reasons: the download API
should stay as simple as possible, and it must remain possible to use it
for downloading resources such as base configurations (by the extends
option) whose MD5 checksum isn't known beforehand.

Any opinions on whether such functionality would be worth the added weight
of such an option, or maybe insights into why requiring checksums
depending on a buildout option might not be useful after all? Thank you.

-- 
Thomas




From jim at zope.com  Wed Mar  9 12:52:15 2011
From: jim at zope.com (Jim Fulton)
Date: Wed, 9 Mar 2011 06:52:15 -0500
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <4D76B517.2070400@canterbury.ac.nz>
References: <20110307211538.2777B3A4062@sparrow.telecommunity.com>
	<AANLkTi=E0LGnM1XMOFYE15qmLppoAX0igC7HNGZqASfd@mail.gmail.com>
	<4D7570F1.4000302@canterbury.ac.nz>
	<AANLkTi=JU091RF2MzEYVGZb7vp9Sv24ML1xxYk5cGsn7@mail.gmail.com>
	<4D76B517.2070400@canterbury.ac.nz>
Message-ID: <AANLkTinAVHFCwzo5_Xnf-s1cxCg5ixXfaEZuxgfUhqYS@mail.gmail.com>

On Tue, Mar 8, 2011 at 6:00 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Jim Fulton wrote:
>
>> I question whether that distinction is important, but if and when it
>> is, then we could use an adjective to clarify. ?Under the hood, the
>> object we call packages today are just modules.
>
> I think the fact that we used the word "package" in the
> first place testifies that it *is* important -- otherwise
> we wouldn't have invented the term.

We invented the term long before distutils was invented and without
much, of any, consideration for software distribution.  Just because
we've historiaclly uses a term, doesn't mean it's a good term.

> The word "package" covers much more than just the module
> object that comes into being when you import it. It also
> implies the associated directory structure, extra import
> facilities and mechanisms, etc. It's not just a minor
> variation on the concept of a module.

I agree to disagree. :)

Jim

--
Jim Fulton
http://www.linkedin.com/in/jimfulton

From jim at zope.com  Wed Mar  9 13:06:36 2011
From: jim at zope.com (Jim Fulton)
Date: Wed, 9 Mar 2011 07:06:36 -0500
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <20110308172703.716583A4062@sparrow.telecommunity.com>
References: <20110307211538.2777B3A4062@sparrow.telecommunity.com>
	<AANLkTi=E0LGnM1XMOFYE15qmLppoAX0igC7HNGZqASfd@mail.gmail.com>
	<20110308172703.716583A4062@sparrow.telecommunity.com>
Message-ID: <AANLkTim2TK7DvF0pxGNMEgJ4FDWQUG=D6SXdD41Kd863@mail.gmail.com>

On Tue, Mar 8, 2011 at 12:26 PM, P.J. Eby <pje at telecommunity.com> wrote:
> At 05:18 PM 3/7/2011 -0500, Jim Fulton wrote:
>>
>> If what we now call "packages" were called "modules", then we could
>> start using the term "package" the way everyone else does. I think
>> lots of people would be less confused.
>
> It seems to me that in order to make that change, you have to get more
> people to change their terminology, since the set of people who need to
> refer to package[module] is larger than the set of people who need to refer
> to package[project]. ?(There is also a larger body of documentation
> associated with package[module].)
>
> IOW, I think this proposal is a heavy uphill battle, both in the number of
> people to be convinced and the amount of documentation. ?In addition, the
> people who are calling a project a package can more easily understand the
> need to call it a project, than the people who are calling a package a
> package, will understand the need to call it a module. ?;-)
>
>
>> Otherwise, I prefer we try hard to use the precise definitions
>> above. This topic can be confusing enough without making it more so
>> through sloppy terminology.
>
> I think that this approach is more achievable: it requires only an official
> blessing, a relatively small amount of documentation to be changed, and the
> renaming of PyPI et al. ?(i.e. "Python Projects Index", projects.python.org,
> etc.)

The term "project" has has never stuck, despite being discussed
repeatedly.  I think given the historical use of the term "package" it
was reasonable to find another term.  OK, we tried. It didn't work.
We can pretent that if we work hard enough, we can make people adopt
our confusing terminology.  Good luck with that.

> ("Projects Index" is a better name anyway, since some things on PyPI are not
> packages at all but applications, scripts, modules, plugins, etc.)

They aren't "packages" a according to current Python definition of the
term. They *are* packages according to the common usage within the
industry.

They certainly aren't "projects" in any sense that most people would
understand.  They are arguably products of projects. Of course, the
term "product" has negative connotations for some folks.

Jim

--
Jim Fulton
http://www.linkedin.com/in/jimfulton

From pje at telecommunity.com  Wed Mar  9 17:43:59 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 09 Mar 2011 11:43:59 -0500
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <AANLkTim2TK7DvF0pxGNMEgJ4FDWQUG=D6SXdD41Kd863@mail.gmail.c
 om>
References: <20110307211538.2777B3A4062@sparrow.telecommunity.com>
	<AANLkTi=E0LGnM1XMOFYE15qmLppoAX0igC7HNGZqASfd@mail.gmail.com>
	<20110308172703.716583A4062@sparrow.telecommunity.com>
	<AANLkTim2TK7DvF0pxGNMEgJ4FDWQUG=D6SXdD41Kd863@mail.gmail.com>
Message-ID: <20110309164421.A18473A4062@sparrow.telecommunity.com>

At 07:06 AM 3/9/2011 -0500, Jim Fulton wrote:
>They certainly aren't "projects" in any sense that most people would
>understand.

I don't follow you.  Sourceforge hosts projects.  Freshmeat indexes 
projects. Mozdev.org hosts projects.  The Apache Foundation hosts 
projects.  "Project", IOW, is *exactly* the word people use for these 
things, in the larger world of software, and that's precisely why I 
chose it for my original terminology proposal.

Many PyPI listings also only describe a project, in the sense that 
there is no release information at all, or the code is only available 
via a revision control system, etc.


>The term "project" has has never stuck, despite being discussed
>repeatedly.

It's only been discussed here, on this list.  It hasn't been put in 
official documentation, or really blessed by anybody.  When it has 
been discussed, it's been considered a reasonable name, and when 
people have needed to make the distinction, they've tended to use it.

The primary reason it hasn't caught on in a larger sense is that 
people don't know about it, and have no motivation to learn it unless 
they end up in a situation where the distinction makes a difference...

And let's face it, it really only makes a difference if you are 
creating some kind of packaging or distribution tool, or if you have 
a corner case of using one.


>I think given the historical use of the term "package" it
>was reasonable to find another term.  OK, we tried. It didn't work.

I don't think that's an accurate assessment on two points.  First, 
"we" didn't try -- *I* did.  (It's possible that someone else has 
promoted it, but I don't recall any instances of that.)

So, there actually being a "we" behind it -- where "we" includes an 
approved PEP, documentation, etc., would be helpful.

Second, I don't think it's accurate to say "it didn't work".  It's 
not like people who need the distinction have objected to the word or 
proposed any alternatives.


>We can pretent that if we work hard enough, we can make people adopt
>our confusing terminology.  Good luck with that.

No matter which way we go (assuming there aren't any other 
alternatives), we will be trying to get some group of people to 
change their terminology.  I'm just pointing out that the group that 
would need to change to use "project" is both smaller *and* 
inherently more motivated to change their usage, than the group that 
would need to replace "package" with "module".

Thus, if getting people to use "project" is an uphill battle, getting 
people to stop using "package" is going to be a much higher vertical 
cliff.  ;-)

For one thing, if the distutils documentation simply starts out like:

"If you want to distribute your work to the larger Python community, 
you need to create a project for it.  In practical terms, this means 
having a directory with a setup.py and the code, data, or 
documentation you wish to distribute.  Your project will also need a 
unique name, if you want to make it accessible to others via the 
Python Project Index (PyPI)."  (replace setup.py w/setup.cfg for 
distutils2, of course)

This usage of project also maps onto common IDE usage of the term 
project -- a directory of stuff that you're going to edit and build.

And, it immediately introduces the term to a superset of the audience 
that will need to know it.  Having PyPI describe its contents as 
"projects" is pretty much the other half of the indoctrination needed.

In contrast, to make the package->module change, you'd have to not 
only change the official language reference and tutorial, but also 
every third-party book or tutorial about Python.

Sure, some of those third party references would also exist for 
"package" as "project", but that's simply an *imprecise* usage, 
rather than an *incorrect* one.


From glyph at twistedmatrix.com  Wed Mar  9 19:26:34 2011
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Wed, 9 Mar 2011 13:26:34 -0500
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <AANLkTim2TK7DvF0pxGNMEgJ4FDWQUG=D6SXdD41Kd863@mail.gmail.com>
References: <20110307211538.2777B3A4062@sparrow.telecommunity.com>
	<AANLkTi=E0LGnM1XMOFYE15qmLppoAX0igC7HNGZqASfd@mail.gmail.com>
	<20110308172703.716583A4062@sparrow.telecommunity.com>
	<AANLkTim2TK7DvF0pxGNMEgJ4FDWQUG=D6SXdD41Kd863@mail.gmail.com>
Message-ID: <4158B0E2-7E3B-4888-B73A-74DDB8955798@twistedmatrix.com>

On Mar 9, 2011, at 7:06 AM, Jim Fulton <jim at zope.com> wrote:

> They certainly aren't "projects" in any sense that most people would
> understand.  They are arguably products of projects. Of course, the
> term "product" has negative connotations for some folks.

Not for everybody!  As far as I am concerned, the whole Python packaging ecosystem (not to mention every Twisted-based plugin mechanism and extension point) is merely trying to re-ascend to the lofty heights once occupied by the beautiful completeness and usability of the zope2 "product" architecture :).  (Not kidding!  I loved those things.)

From carl at oddbird.net  Thu Mar 10 02:09:31 2011
From: carl at oddbird.net (Carl Meyer)
Date: Wed, 09 Mar 2011 20:09:31 -0500
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <20110309164421.A18473A4062@sparrow.telecommunity.com>
References: <20110307211538.2777B3A4062@sparrow.telecommunity.com>	<AANLkTi=E0LGnM1XMOFYE15qmLppoAX0igC7HNGZqASfd@mail.gmail.com>	<20110308172703.716583A4062@sparrow.telecommunity.com>	<AANLkTim2TK7DvF0pxGNMEgJ4FDWQUG=D6SXdD41Kd863@mail.gmail.com>
	<20110309164421.A18473A4062@sparrow.telecommunity.com>
Message-ID: <4D7824CB.1050008@oddbird.net>

FWIW, when I've been writing docs or answering packaging questions and
trying to use the approved, unambiguous terminology, using "project" for
"thing with a PyPI page" has never been a problem; I've had much more
difficulty in using "distribution" as the term for "a zipfile or tarball
or egg that you download" (also a "package" in colloquial usage). I
always have to awkwardly qualify it with "distribution tarball" or
"project distribution" due to the conflict with Linux distributions.

Carl

From martin at v.loewis.de  Thu Mar 10 04:22:13 2011
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 09 Mar 2011 22:22:13 -0500
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <20110309164421.A18473A4062@sparrow.telecommunity.com>
References: <20110307211538.2777B3A4062@sparrow.telecommunity.com>	<AANLkTi=E0LGnM1XMOFYE15qmLppoAX0igC7HNGZqASfd@mail.gmail.com>	<20110308172703.716583A4062@sparrow.telecommunity.com>	<AANLkTim2TK7DvF0pxGNMEgJ4FDWQUG=D6SXdD41Kd863@mail.gmail.com>
	<20110309164421.A18473A4062@sparrow.telecommunity.com>
Message-ID: <4D7843E5.406@v.loewis.de>

>> They certainly aren't "projects" in any sense that most people would
>> understand.
>
> I don't follow you.

Maybe we have lost the context here, but I think I agree with Jim.
Even though PyPI hosts "projects", they (the files you download)
aren't "projects" - they are "distributions" or "packages".

Regards,
Martin

From pje at telecommunity.com  Thu Mar 10 04:40:19 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 09 Mar 2011 22:40:19 -0500
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <4D7843E5.406@v.loewis.de>
References: <20110307211538.2777B3A4062@sparrow.telecommunity.com>
	<AANLkTi=E0LGnM1XMOFYE15qmLppoAX0igC7HNGZqASfd@mail.gmail.com>
	<20110308172703.716583A4062@sparrow.telecommunity.com>
	<AANLkTim2TK7DvF0pxGNMEgJ4FDWQUG=D6SXdD41Kd863@mail.gmail.com>
	<20110309164421.A18473A4062@sparrow.telecommunity.com>
	<4D7843E5.406@v.loewis.de>
Message-ID: <20110310034029.C0EC83A4062@sparrow.telecommunity.com>

At 10:22 PM 3/9/2011 -0500, Martin v. L?wis wrote:
>>>They certainly aren't "projects" in any sense that most people would
>>>understand.
>>
>>I don't follow you.
>
>Maybe we have lost the context here, but I think I agree with Jim.
>Even though PyPI hosts "projects", they (the files you download)
>aren't "projects" - they are "distributions" or "packages".

I think you have lost the context; here, project refers to the thing 
that you have versions of, which distributions in turn are discrete 
manifestations of.

That is, a "project" has "releases" which has "distributions".  For example,

   http://pypi.python.org/pypi/Trac

is a PyPI page for a project,

   http://pypi.python.org/pypi/Trac/0.12.2

is the PyPI page for release 0.12.2 of that project, and

   http://pypi.python.org/packages/source/T/Trac/Trac-0.12.2.tar.gz

is one of the distributions available for that release of that project.

PyPI allows one to host a project that has neither releases nor 
distributions, so in that sense it is certainly an index of projects, 
in much the same way that ASF, SourceForge, MozDev, and others are.


From regebro at gmail.com  Thu Mar 10 11:24:17 2011
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 10 Mar 2011 11:24:17 +0100
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <20110310034029.C0EC83A4062@sparrow.telecommunity.com>
References: <20110307211538.2777B3A4062@sparrow.telecommunity.com>
	<AANLkTi=E0LGnM1XMOFYE15qmLppoAX0igC7HNGZqASfd@mail.gmail.com>
	<20110308172703.716583A4062@sparrow.telecommunity.com>
	<AANLkTim2TK7DvF0pxGNMEgJ4FDWQUG=D6SXdD41Kd863@mail.gmail.com>
	<20110309164421.A18473A4062@sparrow.telecommunity.com>
	<4D7843E5.406@v.loewis.de>
	<20110310034029.C0EC83A4062@sparrow.telecommunity.com>
Message-ID: <AANLkTimZdSnJfXn=+Vk-pKzbPppuPH6tb=wQ7qMRgDy8@mail.gmail.com>

Who are going to PyCon? I feel an open space on this coming up. :-)

From jim at zope.com  Thu Mar 10 14:08:46 2011
From: jim at zope.com (Jim Fulton)
Date: Thu, 10 Mar 2011 08:08:46 -0500
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <20110309164421.A18473A4062@sparrow.telecommunity.com>
References: <20110307211538.2777B3A4062@sparrow.telecommunity.com>
	<AANLkTi=E0LGnM1XMOFYE15qmLppoAX0igC7HNGZqASfd@mail.gmail.com>
	<20110308172703.716583A4062@sparrow.telecommunity.com>
	<AANLkTim2TK7DvF0pxGNMEgJ4FDWQUG=D6SXdD41Kd863@mail.gmail.com>
	<20110309164421.A18473A4062@sparrow.telecommunity.com>
Message-ID: <AANLkTi=95iakjfrYsqxt+eF-0ynrytDy6jzaM-h_LoYZ@mail.gmail.com>

I really should let this rest ....... really ....

On Wed, Mar 9, 2011 at 11:43 AM, P.J. Eby <pje at telecommunity.com> wrote:
> At 07:06 AM 3/9/2011 -0500, Jim Fulton wrote:
>>
>> They certainly aren't "projects" in any sense that most people would
>> understand.
>
> I don't follow you. ?Sourceforge hosts projects. ?Freshmeat indexes
> projects. Mozdev.org hosts projects. ?The Apache Foundation hosts projects.
> ?"Project", IOW, is *exactly* the word people use for these things, in the
> larger world of software, and that's precisely why I chose it for my
> original terminology proposal.

But the things in PyPI are not projects. Projects have bug trackers,
and mailing lists, and source code repositories. Projects are
organizations of people -- not software distributions.

The things in PyPI that you call "projects" are things that get
produced by projects. There isn't a buildout.recipe.egg
"project". buildout.recipe.egg is just a um packaging of some software
components. It's not a "project" in any usual sense of the word.

>> The term "project" has has never stuck, despite being discussed
>> repeatedly.
>
> It's only been discussed here, on this list. ?It hasn't been put in official
> documentation, or really blessed by anybody. ?When it has been discussed,
> it's been considered a reasonable name, and when people have needed to make
> the distinction, they've tended to use it.

OK, fair enough. Distutils uses the name "package" for both Python
packages and for the things it makes distributions of.  The historical
precedent from official documentation is to use "package" for what you
call project. Let's stick with that. :)

> And let's face it, it really only makes a difference if you are creating
> some kind of packaging or distribution tool, or if you have a corner case of
> using one.

No. End users need to know about this. They need to know what these
names mean and that the "package name" (distutils) or "project name"
(setuptools) doesn't imply that the distribution will provide a python
package of the same name.  Trust me on this. Users care about this.

> For one thing, if the distutils documentation simply starts out like:
>
> "If you want to distribute your work to the larger Python community, you
> need to create a project for it. ?In practical terms, this means having a
> directory with a setup.py and the code, data, or documentation you wish to
> distribute. ?Your project will also need a unique name, if you want to make
> it accessible to others via the Python Project Index (PyPI)." ?(replace
> setup.py w/setup.cfg for distutils2, of course)

It does? Where? Can you give a link? I don't see this text anywhere.

> This usage of project also maps onto common IDE usage of the term project --
> a directory of stuff that you're going to edit and build.

That use of project is equivalent to "working directory" and not
relevant, IMO.

> And, it immediately introduces the term to a superset of the audience that
> will need to know it. ?Having PyPI describe its contents as "projects" is
> pretty much the other half of the indoctrination needed.

Only if you think indoctrination is needed. I'd rather make things
more obvious.

Jim

--
Jim Fulton
http://www.linkedin.com/in/jimfulton

From jim at zope.com  Thu Mar 10 14:58:13 2011
From: jim at zope.com (Jim Fulton)
Date: Thu, 10 Mar 2011 08:58:13 -0500
Subject: [Distutils] Buildout documentation sprint at PyCon
Message-ID: <AANLkTikD+A2jDLKD7tD0aB=bxmfJOCkkWnHBpYhi7Od+@mail.gmail.com>

I've started working on a buildout documentation project to provide
decent sphinx documentation for buildout. This documentation will have
the following structure:

Introduction
How-tos
   ...
Special Topics
   ...
Reference
   ...

This is planned as a collection of relatively small relatively
independent documents.  Each document will be first and foremost
**documentation**, however manuel will be used to try to assure that
examples work.

This effort lends itself to small contributions and seems well suited
to sprint work if anyone is interested.  This is a good opportunity to
contribute to buildout and to learn about sphinx and manuel.

I'll have the subversion project in place by the time the sprints start.

I'll be there for all 4 sprint days.

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From pje at telecommunity.com  Thu Mar 10 18:55:35 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 10 Mar 2011 12:55:35 -0500
Subject: [Distutils] Name the software! Package quality tester.
In-Reply-To: <AANLkTi=95iakjfrYsqxt+eF-0ynrytDy6jzaM-h_LoYZ@mail.gmail.c
 om>
References: <20110307211538.2777B3A4062@sparrow.telecommunity.com>
	<AANLkTi=E0LGnM1XMOFYE15qmLppoAX0igC7HNGZqASfd@mail.gmail.com>
	<20110308172703.716583A4062@sparrow.telecommunity.com>
	<AANLkTim2TK7DvF0pxGNMEgJ4FDWQUG=D6SXdD41Kd863@mail.gmail.com>
	<20110309164421.A18473A4062@sparrow.telecommunity.com>
	<AANLkTi=95iakjfrYsqxt+eF-0ynrytDy6jzaM-h_LoYZ@mail.gmail.com>
Message-ID: <20110310180549.61D743A4758@sparrow.telecommunity.com>

At 08:08 AM 3/10/2011 -0500, Jim Fulton wrote:
>I really should let this rest ....... really ....

I notice you seem to have already let rest defending your proposal, 
as opposed to opposing mine.  ;-)

That is, I don't see where you've included any counterargument for 
why convincing people to *stop* using "package" for "python package" 
isn't going to be a lot harder than simply *adding* the term 
"project" to existing docs, via simple explanations like this one:

"If you would like to distribute your python package or module, you 
need to create a project.  A project consists of a directory 
containing the package(s) or module(s) you want to distribute, along 
with a setup.py/.cfg, specifying the project's name, current version 
number, and other information.  If you wish to distribute your 
package via PyPI, then you must choose a unique project name and 
register it.   If you are only distributing one module or package, 
you can name the project after it, as long as it doesn't conflict 
with any other project names already registered on PyPI."

Context-specific explanations like this are easy to give, at a place 
where the recipient of the explanation *wants* to learn something.

In contrast, to convince everybody in the world to replace "package" 
with "module", you must send out a constant broadcast to people who 
don't really care.

(your other points are addressed below)


>On Wed, Mar 9, 2011 at 11:43 AM, P.J. Eby <pje at telecommunity.com> wrote:
> > At 07:06 AM 3/9/2011 -0500, Jim Fulton wrote:
> >>
> >> They certainly aren't "projects" in any sense that most people would
> >> understand.
> >
> > I don't follow you.  Sourceforge hosts projects.  Freshmeat indexes
> > projects. Mozdev.org hosts projects.  The Apache Foundation hosts projects.
> >  "Project", IOW, is *exactly* the word people use for these things, in the
> > larger world of software, and that's precisely why I chose it for my
> > original terminology proposal.
>
>But the things in PyPI are not projects. Projects have bug trackers,
>and mailing lists, and source code repositories.

And which of those things have to be hosted on the same site as the 
project's index page?  Freshmeat, for example, is not a project 
*hosting* system, it's a project *index*.  PyPI is in that sense, a 
project index.  Some index pages may link to all the stuff associated 
with that project, some may not.

(i.e. the set of all possible project features is not equal to the 
intersection of all projects' actual features: some project have 
mailing lists and not bug trackers, and vice versa.  This does not 
make them non-projects.)


>  Projects are organizations of people -- not software distributions.

I'm an organization of people; am I a project?  No, the various 
things I have listed on PyPI are my projects, and I distribute 
releases of them.


>The things in PyPI that you call "projects" are things that get
>produced by projects. There isn't a buildout.recipe.egg
>"project". buildout.recipe.egg is just a um packaging of some software
>components. It's not a "project" in any usual sense of the word.

I don't get that.  You work on it, and it is distinct from other 
works.  That makes it a project.


>No. End users need to know about this. They need to know what these
>names mean and that the "package name" (distutils) or "project name"
>(setuptools) doesn't imply that the distribution will provide a python
>package of the same name.  Trust me on this. Users care about this.

Actually, I believe distutils calls it a "distribution name", but I 
could be wrong about that.  ;-)


> > For one thing, if the distutils documentation simply starts out like:
> >
> > "If you want to distribute your work to the larger Python community, you
> > need to create a project for it.  In practical terms, this means having a
> > directory with a setup.py and the code, data, or documentation you wish to
> > distribute.  Your project will also need a unique name, if you want to make
> > it accessible to others via the Python Project Index (PyPI)."  (replace
> > setup.py w/setup.cfg for distutils2, of course)
>
>It does? Where? Can you give a link? I don't see this text anywhere.

Of course not; it was a proposal for a hypothetical 
introduction.  That's why it says "if" up there.  ;-)


> > This usage of project also maps onto common IDE usage of the term 
> project --
> > a directory of stuff that you're going to edit and build.
>
>That use of project is equivalent to "working directory" and not
>relevant, IMO.

Huh?  In an IDE, a project is a thing you build and distribute, 
usually with source control.  It's distinct from other projects, and 
it has a name.  It is usually not tied to a specific code unit such 
as a module or package (regardless of what the code units of the 
relevant programming language are called).  It has additional 
metadata, over and above the actual code units, and possibly includes 
non-code resources such as documentation, data, images, etc.

IOW, it's *exactly* the sort of project we're talking about 
here.   So, the term project is consistent with both the IDE usage of 
the word, *and* the mozdev/freshmeat/sourceforge et al meaning of project.

It is thus unlikely to be confusing or surprising in the least.

(Side question: were you *really* confused by the term project when 
you first began working with setuptools?  I don't remember you having 
any issues with it at the time, but perhaps I've forgotten?)

The hard part of introducing the term *here*, on this mailing list, 
is precisely the same part that is hard for your proposal: it is 
usually happening at a point where people are *not* as open to 
learning something.

When somebody is on the list and we need to explain the 
project/package distinction, it's noise to them because they're 
trying to solve some other problem, and the distinction doesn't seem important.

At the point, however, where someone is reading the distutils docs 
for the first time, or reading a "how to distribute your stuff" 
tutorial, that is precisely the place where that distinction will be 
perceived as valuable.

That is, they are asking, "what do I need to do to get my code out 
there?"  And the answer is, "make a project, register it, and upload 
some files."  This answer is sensible and satisfying, so the term 
slips right in.

Arguably, you could call it a "fooblymuffin" at that same point of 
introduction, and the "it's an answer to their question" argument 
mostly still applies.  But the pre-existing meaning of "project" from 
both IDE usage and project-sites usage means that anyone who's 
encountered the term in either of those uses will likely have an 
immediate "aha" or "of course" reaction, thereby making the learning 
and acceptance even easier.

Renaming package to module, on the other hand, has no similarly 
obvious point of motivation to learn, except for people who are *not 
yet python developers*...  which means that it's going to take a 
long, long time to get the terminology changed.


From martin at v.loewis.de  Fri Mar 11 10:11:14 2011
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 11 Mar 2011 04:11:14 -0500
Subject: [Distutils] pypi/packages/docs.python.org
In-Reply-To: <AANLkTi=vxdv_MTa4JujZPB4=kLuc94P100KiqZFr_a8e@mail.gmail.com>
References: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>	<AANLkTinv_ADD_fjQQBw+Ptp3MJPZapP64xvcXU+05Js8@mail.gmail.com>	<4D73F1A9.1000207@v.loewis.de>
	<AANLkTi=vxdv_MTa4JujZPB4=kLuc94P100KiqZFr_a8e@mail.gmail.com>
Message-ID: <4D79E732.3040907@v.loewis.de>


> http://pypi.python.org/pypi/distribute/docs, with a big link on the
> top of http://pypi.python.org/pypi/distribute to go there.

I have now added a link at the top. Such a link already was there at
the bottom, but now it's duplicated at the top also. Not sure whether
it's big enough :-)

Regards,
Martin

From ziade.tarek at gmail.com  Fri Mar 11 14:37:50 2011
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 11 Mar 2011 08:37:50 -0500
Subject: [Distutils] pypi/packages/docs.python.org
In-Reply-To: <4D79E732.3040907@v.loewis.de>
References: <AANLkTika61vqEXERV4JGRRDV5Az+21_VF0mkodUSBdsg@mail.gmail.com>
	<AANLkTinv_ADD_fjQQBw+Ptp3MJPZapP64xvcXU+05Js8@mail.gmail.com>
	<4D73F1A9.1000207@v.loewis.de>
	<AANLkTi=vxdv_MTa4JujZPB4=kLuc94P100KiqZFr_a8e@mail.gmail.com>
	<4D79E732.3040907@v.loewis.de>
Message-ID: <AANLkTikPO-4DmySGbdtV3qzDFTY700vq+Mqz5DeL-Akv@mail.gmail.com>

It's Perfect thanks

2011/3/11 "Martin v. L?wis" <martin at v.loewis.de>:
>
>> http://pypi.python.org/pypi/distribute/docs, with a big link on the
>> top of http://pypi.python.org/pypi/distribute to go there.
>
> I have now added a link at the top. Such a link already was there at
> the bottom, but now it's duplicated at the top also. Not sure whether
> it's big enough :-)
>
> Regards,
> Martin
>



-- 
Tarek Ziad? | http://ziade.org

From barry at python.org  Fri Mar 11 14:27:30 2011
From: barry at python.org (Barry Warsaw)
Date: Fri, 11 Mar 2011 08:27:30 -0500
Subject: [Distutils] issues with namespace packages on Debian Squeeze
 and Python 2.6
In-Reply-To: <4D7010B8.4070408@gmail.com>
References: <4D7010B8.4070408@gmail.com>
Message-ID: <20110311082730.20677686@neurotica>

On Mar 03, 2011, at 11:05 PM, Manlio Perillo wrote:

>It seems I'm having some trobles with setuptools and namespace packages
>under Debian Squeeze and Python 2.6.

Can you be more specific about what problems you're having?

>One thing to say about Python 2.6 on Debian Squeeze is that "local"
>packages are installed under /usr/local/lib/python2.6/dist-packages/.

I think I explained the reason for this about 5 times at Pycon yesterday,
meaning it's probably time to blog about it. :)

>I have tried with system default python 2.5, and namespace packages
>works. The same when I create a virtual environment with python 2.6.
>The problem seems to just be with system default python 2.6.
>
>I'm using the original setuptools (installed from sources), and not the
>one provided by Debian.

Yes, I think this is going to give you problems because it doesn't know about
dist-packages, and site-packages is not on sys.path by default in the system
Python.  Please try again with the Ubuntu setuptools package (which is really
distribute).  It's been modified to know about dist-packages.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110311/110c0ce1/attachment.pgp>

From thomas at thomas-lotze.de  Fri Mar 11 16:15:09 2011
From: thomas at thomas-lotze.de (Thomas Lotze)
Date: Fri, 11 Mar 2011 16:15:09 +0100
Subject: [Distutils] zc.buildout: new option for requiring MD5 sums for
 downloads
In-Reply-To: <7EF3C8D4-652E-49E5-9F03-D18146383D4D@j03.de>
References: <7EF3C8D4-652E-49E5-9F03-D18146383D4D@j03.de>
Message-ID: <20110311161509.5147401a@krusty.ws.whq.gocept.com>

"Filip M. Noetzel" <filip at j03.de> schrieb:

> (I'm replying out of band, [...]

I hope you don't mind if I send a copy of my reply back to the list,
though.

> I think wrote what you are describing in your post a few months ago:
> 
> http://pypi.python.org/pypi/buildout-md5sums (Source at https://github.com/peritus/buildout-md5sums )

It has a very similar purpose indeed. Nice to see that this is
something I'm not the only one to want to have. Thank you for pointing
it out!

> I'd love feedback on it (I use it on a day-to-day basis for my buildouts, but don't know other users).

The problems I see with your approach:

- Patching the download API is technically less than optimal.

- Anchoring MD5 enforcement that deeply within the mechanics means that
  client code cannot decide whether its associated configuration needs
  to honour the allow-picked-downloads flag. I'm not sure whether
  that's a good thing or bad - that's part of what I'm hoping to
  discuss. I could imagine that one wants to enforce checksums for,
  e.g., source packages downloaded by a cmmi recipe while avoiding them
  for base configuration files downloaded by buildout itself.

- As a less technical aspect, you might want to consider a more serious
  licence for your package if you hope for more wide-spread use.

Thomas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110311/f83f6c5d/attachment.pgp>

From vinay_sajip at yahoo.co.uk  Sat Mar 12 12:05:50 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 12 Mar 2011 11:05:50 +0000 (UTC)
Subject: [Distutils] Distribute and Python 3.2
Message-ID: <loom.20110312T120426-797@post.gmane.org>

I'm working on Python 3 versions of pip and virtualenv, with the idea of keeping
a common codebase for Python 2.x and 3.x so as to make maintenance easier. To
date, I've made good progress: My port of pip (working with my port of
virtualenv) passes 100 tests (2 are skipped) on Python 3.2 and 3.3a0. However,
one roadblock is that distribute 0.6.14 doesn't work with Python 3.2 and later,
because of the "abiflags" configuration parameter which is new in 3.2. Toshio
Kuratomi has a fix for this checked in, and my tests have used a tarball of the
distribute 0.6-maintenance tip with a name of distribute-0.6.15dev.tar.gz.

It would be good if an official 0.6.15 release of distribute were made,
incorporating that fix, now that 3.2 is out. Can someone tell me if there is a
plan to release a 0.6.15, and if so, when it will be?

Regards,

Vinay Sajip

P.S. IMO Toshio Kuratomi's fix could be better implemented as

self.config_vars['abiflags'] = getattr(sys, 'abiflags', '')

in the same block as all the other self.config_vars[...] assignments.


From jim at zope.com  Sat Mar 12 13:04:58 2011
From: jim at zope.com (Jim Fulton)
Date: Sat, 12 Mar 2011 07:04:58 -0500
Subject: [Distutils] Distribute and Python 3.2
In-Reply-To: <loom.20110312T120426-797@post.gmane.org>
References: <loom.20110312T120426-797@post.gmane.org>
Message-ID: <AANLkTi=ZP1bM_XYJbQMiEYw3u0h_bbu4O-rn0izUzAWu@mail.gmail.com>

On Sat, Mar 12, 2011 at 6:05 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> I'm working on Python 3 versions of pip and virtualenv, with the idea of keeping
> a common codebase for Python 2.x and 3.x so as to make maintenance easier. To
> date, I've made good progress: My port of pip (working with my port of
> virtualenv) passes 100 tests (2 are skipped) on Python 3.2 and 3.3a0.

Yay!

> However,
> one roadblock is that distribute 0.6.14 doesn't work with Python 3.2 and later,
> because of the "abiflags" configuration parameter which is new in 3.2. Toshio
> Kuratomi has a fix for this checked in, and my tests have used a tarball of the
> distribute 0.6-maintenance tip with a name of distribute-0.6.15dev.tar.gz.
>
> It would be good if an official 0.6.15 release of distribute were made,
> incorporating that fix, now that 3.2 is out.

+1

I'm going to be working on a buildout port this week and will need a
Python 3.2-compatible port too!

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From regebro at gmail.com  Sat Mar 12 13:46:04 2011
From: regebro at gmail.com (Lennart Regebro)
Date: Sat, 12 Mar 2011 07:46:04 -0500
Subject: [Distutils] Distribute and Python 3.2
In-Reply-To: <AANLkTi=ZP1bM_XYJbQMiEYw3u0h_bbu4O-rn0izUzAWu@mail.gmail.com>
References: <loom.20110312T120426-797@post.gmane.org>
	<AANLkTi=ZP1bM_XYJbQMiEYw3u0h_bbu4O-rn0izUzAWu@mail.gmail.com>
Message-ID: <AANLkTi=KgV=Nzph1zDpr1RrOHt1R6vvFOPJnapTKCQoX@mail.gmail.com>

On Sat, Mar 12, 2011 at 07:04, Jim Fulton <jim at zope.com> wrote:
> On Sat, Mar 12, 2011 at 6:05 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
>> I'm working on Python 3 versions of pip and virtualenv, with the idea of keeping
>> a common codebase for Python 2.x and 3.x so as to make maintenance easier. To
>> date, I've made good progress: My port of pip (working with my port of
>> virtualenv) passes 100 tests (2 are skipped) on Python 3.2 and 3.3a0.
>
> Yay!
>
>> However,
>> one roadblock is that distribute 0.6.14 doesn't work with Python 3.2 and later,
>> because of the "abiflags" configuration parameter which is new in 3.2. Toshio
>> Kuratomi has a fix for this checked in, and my tests have used a tarball of the
>> distribute 0.6-maintenance tip with a name of distribute-0.6.15dev.tar.gz.
>>
>> It would be good if an official 0.6.15 release of distribute were made,
>> incorporating that fix, now that 3.2 is out.
>
> +1
>
> I'm going to be working on a buildout port this week and will need a
> Python 3.2-compatible port too!

Also my previous effort for buildout exposed a bug, which I fixed, so
we need a new release anyway. :-) So a new distribute release would be
very appreciated.

From ziade.tarek at gmail.com  Sat Mar 12 14:54:14 2011
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 12 Mar 2011 08:54:14 -0500
Subject: [Distutils] Distribute and Python 3.2
In-Reply-To: <AANLkTi=KgV=Nzph1zDpr1RrOHt1R6vvFOPJnapTKCQoX@mail.gmail.com>
References: <loom.20110312T120426-797@post.gmane.org>
	<AANLkTi=ZP1bM_XYJbQMiEYw3u0h_bbu4O-rn0izUzAWu@mail.gmail.com>
	<AANLkTi=KgV=Nzph1zDpr1RrOHt1R6vvFOPJnapTKCQoX@mail.gmail.com>
Message-ID: <AANLkTik0+2zaG1A7gY9vBHionGKV9kwn5B50Pa0gsup3@mail.gmail.com>

I will cut a release during the sprint, unless someone involved in the
project would like to volunteer to do a release.

Cheers

On Sat, Mar 12, 2011 at 7:46 AM, Lennart Regebro <regebro at gmail.com> wrote:
> On Sat, Mar 12, 2011 at 07:04, Jim Fulton <jim at zope.com> wrote:
>> On Sat, Mar 12, 2011 at 6:05 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
>>> I'm working on Python 3 versions of pip and virtualenv, with the idea of keeping
>>> a common codebase for Python 2.x and 3.x so as to make maintenance easier. To
>>> date, I've made good progress: My port of pip (working with my port of
>>> virtualenv) passes 100 tests (2 are skipped) on Python 3.2 and 3.3a0.
>>
>> Yay!
>>
>>> However,
>>> one roadblock is that distribute 0.6.14 doesn't work with Python 3.2 and later,
>>> because of the "abiflags" configuration parameter which is new in 3.2. Toshio
>>> Kuratomi has a fix for this checked in, and my tests have used a tarball of the
>>> distribute 0.6-maintenance tip with a name of distribute-0.6.15dev.tar.gz.
>>>
>>> It would be good if an official 0.6.15 release of distribute were made,
>>> incorporating that fix, now that 3.2 is out.
>>
>> +1
>>
>> I'm going to be working on a buildout port this week and will need a
>> Python 3.2-compatible port too!
>
> Also my previous effort for buildout exposed a bug, which I fixed, so
> we need a new release anyway. :-) So a new distribute release would be
> very appreciated.
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | http://ziade.org

From a.badger at gmail.com  Sat Mar 12 15:49:24 2011
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Sat, 12 Mar 2011 06:49:24 -0800
Subject: [Distutils] Distribute and Python 3.2
In-Reply-To: <loom.20110312T120426-797@post.gmane.org>
References: <loom.20110312T120426-797@post.gmane.org>
Message-ID: <20110312144924.GC2893@unaka.lan>

On Sat, Mar 12, 2011 at 11:05:50AM +0000, Vinay Sajip wrote:
> 
> P.S. IMO Toshio Kuratomi's fix could be better implemented as
> 
> self.config_vars['abiflags'] = getattr(sys, 'abiflags', '')
> 
> in the same block as all the other self.config_vars[...] assignments.
> 
Committed as:

diff -r f64c2d57df43 setuptools/command/easy_install.py
--- a/setuptools/command/easy_install.py        Tue Feb 22 12:05:49 2011 -0800
+++ b/setuptools/command/easy_install.py        Sat Mar 12 06:47:42 2011 -0800
@@ -202,14 +202,10 @@
                             'prefix': prefix,
                             'sys_exec_prefix': exec_prefix,
                             'exec_prefix': exec_prefix,
+                            # Only python 3.2+ has abiflags
+                            'abiflags': getattr(sys, 'abiflags', ''),
                            }
 
-        try:
-            self.config_vars['abiflags'] = sys.abiflags
-        except AttributeError:
-            # Only python-3.2+ has sys.abiflags
-            self.config_vars['abiflags'] = ''
-
         if HAS_USER_SITE:
             self.config_vars['userbase'] = self.install_userbase
             self.config_vars['usersite'] = self.install_usersite

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110312/4c42020c/attachment.pgp>

From jim at zope.com  Sat Mar 12 21:43:11 2011
From: jim at zope.com (Jim Fulton)
Date: Sat, 12 Mar 2011 15:43:11 -0500
Subject: [Distutils] Distribute and Python 3.2
In-Reply-To: <AANLkTik0+2zaG1A7gY9vBHionGKV9kwn5B50Pa0gsup3@mail.gmail.com>
References: <loom.20110312T120426-797@post.gmane.org>
	<AANLkTi=ZP1bM_XYJbQMiEYw3u0h_bbu4O-rn0izUzAWu@mail.gmail.com>
	<AANLkTi=KgV=Nzph1zDpr1RrOHt1R6vvFOPJnapTKCQoX@mail.gmail.com>
	<AANLkTik0+2zaG1A7gY9vBHionGKV9kwn5B50Pa0gsup3@mail.gmail.com>
Message-ID: <AANLkTinivLtHWtUrnfA9=zuBn4Ov-X02hJax=_45gGeH@mail.gmail.com>

On Sat, Mar 12, 2011 at 8:54 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> I will cut a release during the sprint, unless someone involved in the
> project would like to volunteer to do a release.

Thanks.

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From vinay_sajip at yahoo.co.uk  Sat Mar 12 22:17:42 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 12 Mar 2011 21:17:42 +0000 (UTC)
Subject: [Distutils] Distribute and Python 3.2
References: <loom.20110312T120426-797@post.gmane.org>
	<20110312144924.GC2893@unaka.lan>
Message-ID: <loom.20110312T221519-570@post.gmane.org>

Toshio Kuratomi <a.badger <at> gmail.com> writes:

> +    # Only python 3.2+ has abiflags
> +    'abiflags': getattr(sys, 'abiflags', ''),

Thanks, Toshio, and Tarek, in advance, for the upcoming release.

Regards,

Vinay Sajip




From a3.schumann at olympus.net  Sun Mar 13 06:02:34 2011
From: a3.schumann at olympus.net (Ned Schumann)
Date: Sat, 12 Mar 2011 21:02:34 -0800
Subject: [Distutils] [distribute] distribute_setup.py fails on Linux
Message-ID: <2A4EA1A2-2C2D-4758-B066-06D5B0046775@olympus.net>

Hello Arve,

Running python2.6 distribute_setup.py works on a vanilla 64 bit CentOS 5 installation but fails on a default vanilla EC2 AMI Linux . 

Both Pythons were built with 'make altinstall'. Your note suggests that there was a problem with your Python installation. It appears that I too have an installation problem. I installed with 'make altinstall' but how could that go wrong? 

The python-2.6.6 regression tests for the EC2 version reveal ...

test test_zipfile failed -- multiple errors occurred; run in verbose mode for details
test_zipimport skipped -- No module named zlib

On EC2, when python2.6 distribute_setup.py is run with a Pdb trace at line 1658 n tarfile.py ...
(Pdb) print func(name, "r", fileobj, **kwargs)
*** CompressionError: gzip module is not available

Yet /usr/local/build/Python-2.6.6/Modules/zlib exists for the EC2 build.

Please let me know a work-around. Is there anything I should suggest to AWS?

With thanks,
ned


From regebro at gmail.com  Sun Mar 13 14:52:27 2011
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 13 Mar 2011 09:52:27 -0400
Subject: [Distutils] Distributes from pkg_resources.resource_string returns
 bytes under Python 3
Message-ID: <AANLkTikzqtfHa8dobfWbqMdCpRZAcBk=DVV9AKYPf3hC@mail.gmail.com>

Distributes from pkg_resources.resource_string returns bytes under
Python 3, which is pretty surprising. :-)
Should we fix this? That would mean that we need to introduce a
pkg_resources.resource_bytes that returns bytes under Python 3 and str
(again) in Python 2. And probably we also need a
pkg_resources.resource_unicode that returns Unicode under Python 2 and
str under Python 3...


Opinions on that?

From jim at zope.com  Sun Mar 13 16:44:04 2011
From: jim at zope.com (Jim Fulton)
Date: Sun, 13 Mar 2011 11:44:04 -0400
Subject: [Distutils] Buildout documentation sprint at PyCon
In-Reply-To: <AANLkTikD+A2jDLKD7tD0aB=bxmfJOCkkWnHBpYhi7Od+@mail.gmail.com>
References: <AANLkTikD+A2jDLKD7tD0aB=bxmfJOCkkWnHBpYhi7Od+@mail.gmail.com>
Message-ID: <AANLkTikFUxHqS9-OX8xuu-WqSRcz28zC1JkvM5ds=Wuz@mail.gmail.com>

On Thu, Mar 10, 2011 at 8:58 AM, Jim Fulton <jim at zope.com> wrote:
> I've started working on a buildout documentation project to provide
> decent sphinx documentation for buildout. This documentation will have
> the following structure:
>
> Introduction
> How-tos
> ? ...
> Special Topics
> ? ...
> Reference
> ? ...
>
> This is planned as a collection of relatively small relatively
> independent documents. ?Each document will be first and foremost
> **documentation**, however manuel will be used to try to assure that
> examples work.
>
> This effort lends itself to small contributions and seems well suited
> to sprint work if anyone is interested. ?This is a good opportunity to
> contribute to buildout and to learn about sphinx and manuel.
>
> I'll have the subversion project in place by the time the sprints start.

http://svn.zope.org/zc.buildout/branches/documentation/doc

and especially:

http://svn.zope.org/zc.buildout/branches/documentation/doc/how-tos.rest?view=auto

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From regebro at gmail.com  Sun Mar 13 16:48:03 2011
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 13 Mar 2011 11:48:03 -0400
Subject: [Distutils] Distributes from pkg_resources.resource_string
 returns bytes under Python 3
In-Reply-To: <C789650E-F65D-4200-B34E-82531B45A387@mac.com>
References: <AANLkTikzqtfHa8dobfWbqMdCpRZAcBk=DVV9AKYPf3hC@mail.gmail.com>
	<C789650E-F65D-4200-B34E-82531B45A387@mac.com>
Message-ID: <AANLkTi=x-sEWabMv5JMQwL+mSnDMYJRoY9-Oy0Ono_FE@mail.gmail.com>

On Sun, Mar 13, 2011 at 11:03, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>
> On 13 Mar, 2011, at 9:52, Lennart Regebro wrote:
>
>> Distributes from pkg_resources.resource_string returns bytes under
>> Python 3, which is pretty surprising. :-)
>
> That's exactly what the documentation says it will return (the file is opened in binary mode). ?This is just one of many legacy APIs in Python libraries where the naming is pretty lax w.r.t. bytes and text.

Sure, still surprising. I did however just now, when mentioning in
real life, realize that we could have a mode flag instead. Simpler and
backwards compatible.

//Lennart

From ronaldoussoren at mac.com  Sun Mar 13 16:03:36 2011
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Sun, 13 Mar 2011 11:03:36 -0400
Subject: [Distutils] Distributes from pkg_resources.resource_string
 returns bytes under Python 3
In-Reply-To: <AANLkTikzqtfHa8dobfWbqMdCpRZAcBk=DVV9AKYPf3hC@mail.gmail.com>
References: <AANLkTikzqtfHa8dobfWbqMdCpRZAcBk=DVV9AKYPf3hC@mail.gmail.com>
Message-ID: <C789650E-F65D-4200-B34E-82531B45A387@mac.com>


On 13 Mar, 2011, at 9:52, Lennart Regebro wrote:

> Distributes from pkg_resources.resource_string returns bytes under
> Python 3, which is pretty surprising. :-)

That's exactly what the documentation says it will return (the file is opened in binary mode).  This is just one of many legacy APIs in Python libraries where the naming is pretty lax w.r.t. bytes and text.

> Should we fix this? That would mean that we need to introduce a
> pkg_resources.resource_bytes that returns bytes under Python 3 and str
> (again) in Python 2. And probably we also need a
> pkg_resources.resource_unicode that returns Unicode under Python 2 and
> str under Python 3...

I wouldn't change this at this point in time, it is pretty likely that the change would break existing code. Making sure that distutils2 has the correct naming for their corresponding API would be fine though.

Ronald

> 
> 
> Opinions on that?
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig


From agroszer at gmail.com  Mon Mar 14 15:40:00 2011
From: agroszer at gmail.com (Adam GROSZER)
Date: Mon, 14 Mar 2011 15:40:00 +0100
Subject: [Distutils] window 64bit madness
Message-ID: <4D7E28C0.3080204@gmail.com>

Hello,

Having problems here installing 
http://pypi.python.org/packages/2.6/r/reportlab/reportlab-2.5.win-amd64-py2.6.exe 
. Neither setuptools 0.6c11 nor distribute 0.6.14 gets it right.

We don't have and don't want to have a compiler on the server, so we 
need rely on binary packages.

- setuptools just fails miserably even with easy_install: 
http://paste.lisp.org/+2KZD
- distribute installs it (tho in a ...-win32.egg folder): 
http://paste.lisp.org/+2KZE
- zc.buildout cannot install it with distribute, because it wants to use 
the source tarball (and we don't have a compiler) or when I make it not 
to be able to see the tarball, it does not find the 
reportlab-2.5.win-amd64-py2.6.exe. I guess it does not get the platform 
right.

Any hints?

-- 
Best regards,
  Adam GROSZER
--
Quote of the day:
Age before beauty and pearls before swine. -- Dorothy Parker


From pje at telecommunity.com  Mon Mar 14 19:50:48 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 14 Mar 2011 14:50:48 -0400
Subject: [Distutils] window 64bit madness
In-Reply-To: <4D7E28C0.3080204@gmail.com>
References: <4D7E28C0.3080204@gmail.com>
Message-ID: <20110314185106.CF9863A4077@sparrow.telecommunity.com>

At 03:40 PM 3/14/2011 +0100, Adam GROSZER wrote:
>Hello,
>
>Having problems here installing 
>http://pypi.python.org/packages/2.6/r/reportlab/reportlab-2.5.win-amd64-py2.6.exe 
>. Neither setuptools 0.6c11 nor distribute 0.6.14 gets it right.
>
>We don't have and don't want to have a compiler on the server, so we 
>need rely on binary packages.
>
>- setuptools just fails miserably even with easy_install: 
>http://paste.lisp.org/+2KZD
>- distribute installs it (tho in a ...-win32.egg folder): 
>http://paste.lisp.org/+2KZE
>- zc.buildout cannot install it with distribute, because it wants to 
>use the source tarball (and we don't have a compiler) or when I make 
>it not to be able to see the tarball, it does not find the 
>reportlab-2.5.win-amd64-py2.6.exe. I guess it does not get the platform right.
>
>Any hints?

Run "python -c 'import pkg_resources;print 
pkg_resources.get_build_platform()'" (with the Python interpreter you're using.

(By the way, I think from the log you gave, that reportlab may in 
fact be installed and usable on your machine, despite the attempt to 
download it a second time and build it from source.  But I could be wrong.)


From carl at oddbird.net  Wed Mar 16 09:31:01 2011
From: carl at oddbird.net (Carl Meyer)
Date: Wed, 16 Mar 2011 04:31:01 -0400
Subject: [Distutils] early preview of pythonv
Message-ID: <4D807545.5020309@oddbird.net>

Hello all,

Here at PyCon we've had some discussion about building a
virtualenv-alike into Python core for Python 3.3. The goal is to improve
on virtualenv by providing something that does what virtualenv does
without requiring a copied Python binary, symlinked/copied parts of the
standard library, or a forked site.py.

(I'm not entirely sure that distutils-sig is the right venue for
discussing this, but it's the closest I know of and was recommended by
others in our conversations here; if there's a better place please let
me know; maybe python-ideas? The patch as is stands does affect
sysconfig.py, which is used more by distutils than anything else.)

The idea we discussed is to add to Python's built-in site.py the ability
to set paths up for a virtual environment, triggered by certain
environment variables. Then, for convenience, there'll be a small
executable which can be placed in the "bin/" directory of a virtual
environment and knows how to set up these environment variables and then
exec() the system Python binary.

Larry Hastings had already created this wrapper executable at last
year's PyCon, and tonight I made the necessary modifications to site.py
and sysconfig.py to support it on the Python side. The early prototype
is now working well (at least on Linux; I think it ought to work on OS
X, and should work partially on Windows as well), and I'd welcome review
and comment: https://bitbucket.org/carljm/cpythonv  Look in
Tools/pythonv/README.rst for instructions.

This is an early prototype and will certainly require refinement (not to
mention most likely a PEP, at some point). Please try it out and let me
know if it works for you!

Thanks,

Carl

From carl at oddbird.net  Wed Mar 16 09:37:29 2011
From: carl at oddbird.net (Carl Meyer)
Date: Wed, 16 Mar 2011 04:37:29 -0400
Subject: [Distutils] early preview of pythonv
In-Reply-To: <4D807545.5020309@oddbird.net>
References: <4D807545.5020309@oddbird.net>
Message-ID: <4D8076C9.6010403@oddbird.net>

I forgot to mention - the code for this is in the "pythonv" named branch
in the mercurial repo I linked (https://bitbucket.org/carljm/cpythonv),
so you'll need to "hg update pythonv" after cloning to see it.

Carl

On 03/16/2011 04:31 AM, Carl Meyer wrote:
> This is an early prototype and will certainly require refinement (not to
> mention most likely a PEP, at some point). Please try it out and let me
> know if it works for you!

From agroszer at gmail.com  Wed Mar 16 11:28:32 2011
From: agroszer at gmail.com (Adam GROSZER)
Date: Wed, 16 Mar 2011 11:28:32 +0100
Subject: [Distutils] window 64bit madness
In-Reply-To: <20110314185106.CF9863A4077@sparrow.telecommunity.com>
References: <4D7E28C0.3080204@gmail.com>
	<20110314185106.CF9863A4077@sparrow.telecommunity.com>
Message-ID: <4D8090D0.5050902@gmail.com>

On 03/14/2011 07:50 PM, P.J. Eby wrote:
> At 03:40 PM 3/14/2011 +0100, Adam GROSZER wrote:
>> Hello,
>>
>> Having problems here installing
>> http://pypi.python.org/packages/2.6/r/reportlab/reportlab-2.5.win-amd64-py2.6.exe
>> . Neither setuptools 0.6c11 nor distribute 0.6.14 gets it right.
>>
>> We don't have and don't want to have a compiler on the server, so we
>> need rely on binary packages.
>>
>> - setuptools just fails miserably even with easy_install:
>> http://paste.lisp.org/+2KZD
>> - distribute installs it (tho in a ...-win32.egg folder):
>> http://paste.lisp.org/+2KZE
>> - zc.buildout cannot install it with distribute, because it wants to
>> use the source tarball (and we don't have a compiler) or when I make
>> it not to be able to see the tarball, it does not find the
>> reportlab-2.5.win-amd64-py2.6.exe. I guess it does not get the
>> platform right.
>>
>> Any hints?
>
> Run "python -c 'import pkg_resources;print
> pkg_resources.get_build_platform()'" (with the Python interpreter you're
> using.

D:\install>c:\Python26_64\python.exe
Python 2.6.6 (r266:84297, Aug 24 2010, 18:13:38) [MSC v.1500 64 bit 
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
 >>> import pkg_resources;print pkg_resources.get_build_platform()
win-amd64
 >>>

> (By the way, I think from the log you gave, that reportlab may in fact
> be installed and usable on your machine, despite the attempt to download
> it a second time and build it from source. But I could be wrong.)
>

Might be, but zc.buildout errs out if it detects an error by the 
underlying installer. And I *want* to have reportlab installed by 
zc.buildout.
-- 
Best regards,
  Adam GROSZER
--
Quote of the day:
Nothing is so good as it seems beforehand.
- George Eliot

From regebro at gmail.com  Wed Mar 16 12:09:10 2011
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 16 Mar 2011 07:09:10 -0400
Subject: [Distutils] window 64bit madness
In-Reply-To: <4D8090D0.5050902@gmail.com>
References: <4D7E28C0.3080204@gmail.com>
	<20110314185106.CF9863A4077@sparrow.telecommunity.com>
	<4D8090D0.5050902@gmail.com>
Message-ID: <AANLkTi=8Ckz-aUfh3GFon-tiAH=vyKVMBKXD3qbPh1N4@mail.gmail.com>

Out of curiosity, how does pip handle this?

On Wed, Mar 16, 2011 at 06:28, Adam GROSZER <agroszer at gmail.com> wrote:
> On 03/14/2011 07:50 PM, P.J. Eby wrote:
>>
>> At 03:40 PM 3/14/2011 +0100, Adam GROSZER wrote:
>>>
>>> Hello,
>>>
>>> Having problems here installing
>>>
>>> http://pypi.python.org/packages/2.6/r/reportlab/reportlab-2.5.win-amd64-py2.6.exe
>>> . Neither setuptools 0.6c11 nor distribute 0.6.14 gets it right.
>>>
>>> We don't have and don't want to have a compiler on the server, so we
>>> need rely on binary packages.
>>>
>>> - setuptools just fails miserably even with easy_install:
>>> http://paste.lisp.org/+2KZD
>>> - distribute installs it (tho in a ...-win32.egg folder):
>>> http://paste.lisp.org/+2KZE
>>> - zc.buildout cannot install it with distribute, because it wants to
>>> use the source tarball (and we don't have a compiler) or when I make
>>> it not to be able to see the tarball, it does not find the
>>> reportlab-2.5.win-amd64-py2.6.exe. I guess it does not get the
>>> platform right.
>>>
>>> Any hints?
>>
>> Run "python -c 'import pkg_resources;print
>> pkg_resources.get_build_platform()'" (with the Python interpreter you're
>> using.
>
> D:\install>c:\Python26_64\python.exe
> Python 2.6.6 (r266:84297, Aug 24 2010, 18:13:38) [MSC v.1500 64 bit (AMD64)]
> on win32
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import pkg_resources;print pkg_resources.get_build_platform()
> win-amd64
>>>>
>
>> (By the way, I think from the log you gave, that reportlab may in fact
>> be installed and usable on your machine, despite the attempt to download
>> it a second time and build it from source. But I could be wrong.)
>>
>
> Might be, but zc.buildout errs out if it detects an error by the underlying
> installer. And I *want* to have reportlab installed by zc.buildout.
> --
> Best regards,
> ?Adam GROSZER
> --
> Quote of the day:
> Nothing is so good as it seems beforehand.
> - George Eliot
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>

From jim at zope.com  Wed Mar 16 13:00:21 2011
From: jim at zope.com (Jim Fulton)
Date: Wed, 16 Mar 2011 08:00:21 -0400
Subject: [Distutils] early preview of pythonv
In-Reply-To: <4D807545.5020309@oddbird.net>
References: <4D807545.5020309@oddbird.net>
Message-ID: <AANLkTikfgQNZ5taon8o8QG_NEfQqc=rXWejJvZiA4jKZ@mail.gmail.com>

On Wed, Mar 16, 2011 at 4:31 AM, Carl Meyer <carl at oddbird.net> wrote:
> Hello all,
>
> Here at PyCon we've had some discussion about building a
> virtualenv-alike into Python core for Python 3.3. The goal is to improve
> on virtualenv by providing something that does what virtualenv does
> without requiring a copied Python binary, symlinked/copied parts of the
> standard library, or a forked site.py.
>
> (I'm not entirely sure that distutils-sig is the right venue for
> discussing this, but it's the closest I know of and was recommended by
> others in our conversations here; if there's a better place please let
> me know; maybe python-ideas? The patch as is stands does affect
> sysconfig.py, which is used more by distutils than anything else.)
>
> The idea we discussed is to add to Python's built-in site.py the ability
> to set paths up for a virtual environment, triggered by certain
> environment variables. Then, for convenience, there'll be a small
> executable which can be placed in the "bin/" directory of a virtual
> environment and knows how to set up these environment variables and then
> exec() the system Python binary.
>
> Larry Hastings had already created this wrapper executable at last
> year's PyCon, and tonight I made the necessary modifications to site.py
> and sysconfig.py to support it on the Python side. The early prototype
> is now working well (at least on Linux; I think it ought to work on OS
> X, and should work partially on Windows as well), and I'd welcome review
> and comment: https://bitbucket.org/carljm/cpythonv ?Look in
> Tools/pythonv/README.rst for instructions.
>
> This is an early prototype and will certainly require refinement (not to
> mention most likely a PEP, at some point). Please try it out and let me
> know if it works for you!

Carl,

Thanks for posting this.  I'm very hopeful that buildout can use this
same mechanism to get the isolation it needs.  I would really
appreciate it if buildout users who care about this would test it with
buildout.  In particular, I know of 2 basic use cases:

- Get complete isolation from local additions relative to the standard
  Python distribution.

- Have the ability to cherry pick some local additions while having
  isolation from the rest. (Gary and Ubuntu, I'm looking at you.)

Please, let's make sure this mechanism is enough. To raise the stakes
a bit, when this mechanism is available, presumably in Python 3.3, I
plan to use it exclusively to provide isolation in buildout.

Jim

--
Jim Fulton
http://www.linkedin.com/in/jimfulton

From agroszer at gmail.com  Wed Mar 16 13:38:15 2011
From: agroszer at gmail.com (Adam GROSZER)
Date: Wed, 16 Mar 2011 13:38:15 +0100
Subject: [Distutils] window 64bit madness
In-Reply-To: <AANLkTi=8Ckz-aUfh3GFon-tiAH=vyKVMBKXD3qbPh1N4@mail.gmail.com>
References: <4D7E28C0.3080204@gmail.com>	<20110314185106.CF9863A4077@sparrow.telecommunity.com>	<4D8090D0.5050902@gmail.com>
	<AANLkTi=8Ckz-aUfh3GFon-tiAH=vyKVMBKXD3qbPh1N4@mail.gmail.com>
Message-ID: <4D80AF37.3020305@gmail.com>

On 03/16/2011 12:09 PM, Lennart Regebro wrote:
> Out of curiosity, how does pip handle this?

Not much better.

http://paste.lisp.org/+2KZE/2

and "pip install reportlab==2.5" fails with the usual "no vcvarsall" 
found -- no C compiler.

>
> On Wed, Mar 16, 2011 at 06:28, Adam GROSZER<agroszer at gmail.com>  wrote:
>> On 03/14/2011 07:50 PM, P.J. Eby wrote:
>>>
>>> At 03:40 PM 3/14/2011 +0100, Adam GROSZER wrote:
>>>>
>>>> Hello,
>>>>
>>>> Having problems here installing
>>>>
>>>> http://pypi.python.org/packages/2.6/r/reportlab/reportlab-2.5.win-amd64-py2.6.exe
>>>> . Neither setuptools 0.6c11 nor distribute 0.6.14 gets it right.
>>>>
>>>> We don't have and don't want to have a compiler on the server, so we
>>>> need rely on binary packages.
>>>>
>>>> - setuptools just fails miserably even with easy_install:
>>>> http://paste.lisp.org/+2KZD
>>>> - distribute installs it (tho in a ...-win32.egg folder):
>>>> http://paste.lisp.org/+2KZE
>>>> - zc.buildout cannot install it with distribute, because it wants to
>>>> use the source tarball (and we don't have a compiler) or when I make
>>>> it not to be able to see the tarball, it does not find the
>>>> reportlab-2.5.win-amd64-py2.6.exe. I guess it does not get the
>>>> platform right.
>>>>
>>>> Any hints?
>>>
>>> Run "python -c 'import pkg_resources;print
>>> pkg_resources.get_build_platform()'" (with the Python interpreter you're
>>> using.
>>
>> D:\install>c:\Python26_64\python.exe
>> Python 2.6.6 (r266:84297, Aug 24 2010, 18:13:38) [MSC v.1500 64 bit (AMD64)]
>> on win32
>> Type "help", "copyright", "credits" or "license" for more information.
>>>>> import pkg_resources;print pkg_resources.get_build_platform()
>> win-amd64
>>>>>
>>
>>> (By the way, I think from the log you gave, that reportlab may in fact
>>> be installed and usable on your machine, despite the attempt to download
>>> it a second time and build it from source. But I could be wrong.)
>>>
>>
>> Might be, but zc.buildout errs out if it detects an error by the underlying
>> installer. And I *want* to have reportlab installed by zc.buildout.
>> --
>> Best regards,
>>   Adam GROSZER
>> --
>> Quote of the day:
>> Nothing is so good as it seems beforehand.
>> - George Eliot
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> http://mail.python.org/mailman/listinfo/distutils-sig
>>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>


From a.cavallo at cavallinux.eu  Wed Mar 16 13:38:17 2011
From: a.cavallo at cavallinux.eu (a.cavallo at cavallinux.eu)
Date: Wed, 16 Mar 2011 13:38:17 +0100
Subject: [Distutils] early preview of pythonv
Message-ID: <35142.1300279097@cavallinux.eu>

Hi,

please have a look into my little project:

http://pyvm.sf.net

It does exactly this integration (and build integration and
run time testing) on several different linux distros.
It works stright out of the svn py27 trunk but the work flow
can be implemented for 3.x.

Regards,
Antonio

 On Wed 16/03/11 13:00, "Jim Fulton" jim at zope.com wrote:
> On Wed, Mar 16, 2011 at 4:31 AM, Carl Meyer <carl at oddbird
> .net> wrote:
> Hello all,
> >
> > Here at PyCon we've had some discussion about
> building a
> virtualenv-alike into Python core for Python 3.3.
> The goal is to improve
> on virtualenv by providing something that does what
> virtualenv does
> without requiring a copied Python binary,
> symlinked/copied parts of the
> standard library, or a forked site.py.
> >
> > (I'm not entirely sure that distutils-sig is the
> right venue for
> discussing this, but it's the closest I know of and
> was recommended by
> others in our conversations here; if there's a
> better place please let
> me know; maybe python-ideas? The patch as is stands
> does affect
> sysconfig.py, which is used more by distutils than
> anything else.)
>
> > The idea we discussed is to add to Python's built-in
> site.py the ability
> to set paths up for a virtual environment, triggered
> by certain
> environment variables. Then, for convenience,
> there'll be a small
> executable which can be placed in the "bin/"
> directory of a virtual
> environment and knows how to set up these
> environment variables and then
> exec() the system Python binary.
> >
> > Larry Hastings had already created this wrapper
> executable at last
> year's PyCon, and tonight I made the necessary
> modifications to site.py
> and sysconfig.py to support it on the Python side.
> The early prototype
> is now working well (at least on Linux; I think it
> ought to work on OS
> X, and should work partially on Windows as well),
> and I'd welcome review
> and comment: https://bitbucket.org/carljm/cpythonv ?Look
> in
> Tools/pythonv/README.rst for
> instructions.
>
> > This is an early prototype and will certainly
> require refinement (not to
> mention most likely a PEP, at some point). Please
> try it out and let me
> know if it works for you!
> 
> Carl,
> 
> Thanks for posting this.  I'm very hopeful that buildout can use this
> same mechanism to get the isolation it needs.  I would really
> appreciate it if buildout users who care about this would test it with
> buildout.  In particular, I know of 2 basic use cases:
> 
> - Get complete isolation from local additions relative to the standard
> Python distribution.
> 
> - Have the ability to cherry pick some local additions while having
> isolation from the rest. (Gary and Ubuntu, I'm looking at you.)
> 
> Please, let's make sure this mechanism is enough. To raise the stakes
> a bit, when this mechanism is available, presumably in Python 3.3, I
> plan to use it exclusively to provide isolation in buildout.
> 
> Jim
> 
> --
> Jim Fulton
> http://www.linkedin.com/in/jimfulton
_______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
> 
> 


From martin at v.loewis.de  Wed Mar 16 13:47:57 2011
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 16 Mar 2011 08:47:57 -0400
Subject: [Distutils] early preview of pythonv
In-Reply-To: <35142.1300279097@cavallinux.eu>
References: <35142.1300279097@cavallinux.eu>
Message-ID: <4D80B17D.5090305@v.loewis.de>

Am 16.03.11 08:38, schrieb a.cavallo at cavallinux.eu:
> Hi,
>
> please have a look into my little project:
>
> http://pyvm.sf.net

This redirects me to

http://cclimited.webfactional.com/

Not sure whether it's the right page, if it is,
I'm completely lost as to what this is all about.
On the "Main" page, it apparently says nowhere
what this actually *does*, only that multiple RPM
files of it are available which pass their tests.
The tutorial page offers me to learn "OBS", which
apparently is the OpenSuSE build server, which appears
to be unrelated to Python.

Just my 0.02?.

Regards,
Martin

From vinay_sajip at yahoo.co.uk  Wed Mar 16 15:41:23 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 16 Mar 2011 14:41:23 +0000 (UTC)
Subject: [Distutils] Distribute and Python 3.2
References: <loom.20110312T120426-797@post.gmane.org>
	<AANLkTi=ZP1bM_XYJbQMiEYw3u0h_bbu4O-rn0izUzAWu@mail.gmail.com>
	<AANLkTi=KgV=Nzph1zDpr1RrOHt1R6vvFOPJnapTKCQoX@mail.gmail.com>
	<AANLkTik0+2zaG1A7gY9vBHionGKV9kwn5B50Pa0gsup3@mail.gmail.com>
Message-ID: <loom.20110316T153456-476@post.gmane.org>

Tarek Ziad? <ziade.tarek <at> gmail.com> writes:

> I will cut a release during the sprint, unless someone involved in the
> project would like to volunteer to do a release.
> 

It's great that a 0.6.15 has been uploaded to PyPI, but there are still a couple
of problems with it:

1. distribute_setup.py expects a .tar.gz, and fails with a 404 error because
only the .zip is available.
2. Although the distribute_setup.py in the .zip refers to 0.6.15, the
http://python-distribute.org/distribute_setup.py is still referring to 0.6.14.

I've logged this as an issue on your Bitbucket repo (#193).

Regards,

Vinay Sajip


From vinay_sajip at yahoo.co.uk  Wed Mar 16 15:59:55 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 16 Mar 2011 14:59:55 +0000 (UTC)
Subject: [Distutils] early preview of pythonv
References: <4D807545.5020309@oddbird.net>
Message-ID: <loom.20110316T155623-521@post.gmane.org>

Carl Meyer <carl <at> oddbird.net> writes:

> This is an early prototype and will certainly require refinement (not to
> mention most likely a PEP, at some point). Please try it out and let me
> know if it works for you!

I haven't tried it yet, but I'm curious: was an approach using PYTHONHOME
considered? Or is PYTHONHOME off-limits for an application like this?

Regards,

Vinay Sajip


From ziade.tarek at gmail.com  Wed Mar 16 16:33:09 2011
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 16 Mar 2011 11:33:09 -0400
Subject: [Distutils] Distribute and Python 3.2
In-Reply-To: <loom.20110316T153456-476@post.gmane.org>
References: <loom.20110312T120426-797@post.gmane.org>
	<AANLkTi=ZP1bM_XYJbQMiEYw3u0h_bbu4O-rn0izUzAWu@mail.gmail.com>
	<AANLkTi=KgV=Nzph1zDpr1RrOHt1R6vvFOPJnapTKCQoX@mail.gmail.com>
	<AANLkTik0+2zaG1A7gY9vBHionGKV9kwn5B50Pa0gsup3@mail.gmail.com>
	<loom.20110316T153456-476@post.gmane.org>
Message-ID: <AANLkTi=d8h-S5Yko9Xv1srs7AF6aTabmJdD4ojz_hmbp@mail.gmail.com>

On Wed, Mar 16, 2011 at 10:41 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Tarek Ziad? <ziade.tarek <at> gmail.com> writes:
>
>> I will cut a release during the sprint, unless someone involved in the
>> project would like to volunteer to do a release.
>>
>
> It's great that a 0.6.15 has been uploaded to PyPI, but there are still a couple
> of problems with it:
>
> 1. distribute_setup.py expects a .tar.gz, and fails with a 404 error because
> only the .zip is available.
> 2. Although the distribute_setup.py in the .zip refers to 0.6.15, the
> http://python-distribute.org/distribute_setup.py is still referring to 0.6.14.
>
> I've logged this as an issue on your Bitbucket repo (#193).

Should be fixed now --
>
> Regards,
>
> Vinay Sajip
>
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | http://ziade.org

From carl at meyerloewen.net  Wed Mar 16 17:16:57 2011
From: carl at meyerloewen.net (Carl Meyer)
Date: Wed, 16 Mar 2011 12:16:57 -0400
Subject: [Distutils] window 64bit madness
In-Reply-To: <4D80AF37.3020305@gmail.com>
References: <4D7E28C0.3080204@gmail.com>	<20110314185106.CF9863A4077@sparrow.telecommunity.com>	<4D8090D0.5050902@gmail.com>	<AANLkTi=8Ckz-aUfh3GFon-tiAH=vyKVMBKXD3qbPh1N4@mail.gmail.com>
	<4D80AF37.3020305@gmail.com>
Message-ID: <4D80E279.5030005@meyerloewen.net>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 03/16/2011 08:38 AM, Adam GROSZER wrote:
> On 03/16/2011 12:09 PM, Lennart Regebro wrote:
>> Out of curiosity, how does pip handle this?
> 
> Not much better.
> 
> http://paste.lisp.org/+2KZE/2
> 
> and "pip install reportlab==2.5" fails with the usual "no vcvarsall"
> found -- no C compiler.

FWIW, the "we don't want to install a compiler" requirement means pip is
not a viable option. At this point it doesn't know how to install any
variety of binary distribution, only sdists. (At the very least, the
capability to install bdist_msi is on the roadmap.)

Carl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk2A4nkACgkQFRcxmeyPUXLMSwCdHAJ5TAlIAcXsX21be29xI4RR
DnYAn21IJCoHOeL0cYGIaG/hTu0t6YI8
=G5+H
-----END PGP SIGNATURE-----

From brandon at rhodesmill.org  Wed Mar 16 17:21:36 2011
From: brandon at rhodesmill.org (Brandon Craig Rhodes)
Date: Wed, 16 Mar 2011 12:21:36 -0400
Subject: [Distutils] reservations about pythonv
Message-ID: <874o73ghnz.fsf@asaph.rhodesmill.org>

I suspect that the current "pythonv" proposal should not become how
virtualenv is implemented in the future.

The proposal would add another exec() to every invocation of Python in
the virtualenv.  Not only is exec() among the most costly system calls
on Unix systems, but it is even slower on Windows.

Furthermore, the re-exec() will generate significant noise and annoyance
- if not outright breakage - when people use binary tools like gdb and
valgrind, because MYENV/bin/python will become a binary that looks
nothing like the Python binary (none of the usual symbols, binary
dependencies, and so forth) and that disappears almost immediately after
being run and is replaced with a completely different binary.  Do all
popular binary debugging tools handle an exec() cleanly?  At the very
least everyone's debugging practices and scripts would have to be re-
worked to ignore the stats and results that came back from that first
ephemeral "pythonv" process image.

I certainly agree with everyone that Python 3.3 should natively support
a virtualenv mechanism, but I think that it should be the full "python"
that gets copied.  I realize that this means that the "python" binary
will not get updates when the system Python does; but isn't the whole
idea of a virtualenv that it insulates me from change in a way that
using the system Python does not?  I would be quite happy for either the
"activate" script (or Python itself?) to issue a warning if the binary
in the virtualenv has fallen behind the image in the system Python.

-- 
Brandon Craig Rhodes   brandon at rhodesmill.org   http://rhodesmill.org/brandon

From barry at python.org  Wed Mar 16 17:47:22 2011
From: barry at python.org (Barry Warsaw)
Date: Wed, 16 Mar 2011 12:47:22 -0400
Subject: [Distutils] reservations about pythonv
In-Reply-To: <874o73ghnz.fsf@asaph.rhodesmill.org>
References: <874o73ghnz.fsf@asaph.rhodesmill.org>
Message-ID: <20110316124722.7e1563b1@neurotica>

On Mar 16, 2011, at 12:21 PM, Brandon Craig Rhodes wrote:

>I certainly agree with everyone that Python 3.3 should natively support
>a virtualenv mechanism, but I think that it should be the full "python"
>that gets copied.  I realize that this means that the "python" binary
>will not get updates when the system Python does; but isn't the whole
>idea of a virtualenv that it insulates me from change in a way that
>using the system Python does not?  I would be quite happy for either the
>"activate" script (or Python itself?) to issue a warning if the binary
>in the virtualenv has fallen behind the image in the system Python.

I agree that a separate pythonv binary that execs the real Python binary is
suboptimal.  I understand that to set environment variables such as
$LD_LIBRARY_PATH (on *nix), it might be inevitable that a re-exec is
necessary, but if so, I think it should be the python binary itself that does
it.

Here at the sprints we talked about several possible options, the details of
which of course will have to be hashed out.  There will also be cross platform
considerations.  I think on *nix at least, it would be nice if a symlink and
configuration file were enough to trigger the virtualenv behavior.

For example, if I have a local 'python' symlinked to the real executable, with
a pythonv.conf file next to it, the virtual environment would be enabled.  The
real Python binary would adjust its behavior in that case to know where the
standard library was, but also use the locally installed packages.  Anyway,
that's how I'd *like* it to work on *nix, but it may have to work differently
on Windows, and it may have to work differently if environment variables have
to be set.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110316/4d1b6f26/attachment.pgp>

From carl at oddbird.net  Wed Mar 16 17:47:31 2011
From: carl at oddbird.net (Carl Meyer)
Date: Wed, 16 Mar 2011 12:47:31 -0400
Subject: [Distutils] early preview of pythonv
In-Reply-To: <AANLkTikfgQNZ5taon8o8QG_NEfQqc=rXWejJvZiA4jKZ@mail.gmail.com>
References: <4D807545.5020309@oddbird.net>
	<AANLkTikfgQNZ5taon8o8QG_NEfQqc=rXWejJvZiA4jKZ@mail.gmail.com>
Message-ID: <4D80E9A3.2000304@oddbird.net>

Jim,

On 03/16/2011 08:00 AM, Jim Fulton wrote:
[snip]
> buildout.  In particular, I know of 2 basic use cases:
> 
> - Get complete isolation from local additions relative to the standard
>   Python distribution.

This is the primary design goal, and I believe it's currently achieved.
To be clearer, we have isolation from the system Python's site-packages
and site-python (site directories added to sys.path in site.py), where
all non-stdlib modules should go. We are not isolated from any changes
to the system Python's standard library (which should never be changed
anyway); this is intentional and matches the current behavior of
virtualenv, and avoids the necessity of copying the entire stdlib.

> - Have the ability to cherry pick some local additions while having
>   isolation from the rest. (Gary and Ubuntu, I'm looking at you.)

We don't currently have this. There's a question whether this behavior
should be built in to pythonv, or provided by a tool like distutils2 or
pip or buildout that could be virtual-aware. The latter seems like
perhaps a more natural fit, since it's a persistent modification of the
virtualenv's working set, parallel to installing or uninstalling
packages within the virtualenv. I'm open to further thoughts on this
question.

Carl

From carl at oddbird.net  Wed Mar 16 22:50:06 2011
From: carl at oddbird.net (=?utf-8?B?Q2FybCBNZXllcg==?=)
Date: Wed, 16 Mar 2011 16:50:06 -0500
Subject: [Distutils] =?utf-8?q?early_preview_of_pythonv?=
Message-ID: <E1PzybH-0004CD-Bz@lettuce.threepines.org>

Hi Antonio,

Thanks for the pointer to this. It does look interesting, but I think it hits a rather different set of use cases than virtualenv or pythonv. In particular, virtualenv/pythonv is useful for situations where:

- I may not have administrator access to the system,
- I want to run many projects on exactly the same version of Python, but with different working sets of packages,
- I'd like a cross-platform solution (including Windows and OS X) that maintains a similar workflow on all platforms.

I may not be fully understanding pyvm, but it appears to me that it would not fit those use cases. So while I think it looks interesting and useful, I don't think its a viable alternative for the goals of pythonv.

Carl

Sent from my Android phone

----- Reply message -----
From: a.cavallo at cavallinux.eu
Date: Wed, Mar 16, 2011 7:38 am
Subject: [Distutils] early preview of pythonv
To: "distutils-sig" <distutils-sig at python.org>

Hi,

please have a look into my little project:

http://pyvm.sf.net

It does exactly this integration (and build integration and
run time testing) on several different linux distros.
It works stright out of the svn py27 trunk but the work flow
can be implemented for 3.x.

Regards,
Antonio

 On Wed 16/03/11 13:00, "Jim Fulton" jim at zope.com wrote:
> On Wed, Mar 16, 2011 at 4:31 AM, Carl Meyer <carl at oddbird
> .net> wrote:
> Hello all,
> >
> > Here at PyCon we've had some discussion about
> building a
> virtualenv-alike into Python core for Python 3.3.
> The goal is to improve
> on virtualenv by providing something that does what
> virtualenv does
> without requiring a copied Python binary,
> symlinked/copied parts of the
> standard library, or a forked site.py.
> >
> > (I'm not entirely sure that distutils-sig is the
> right venue for
> discussing this, but it's the closest I know of and
> was recommended by
> others in our conversations here; if there's a
> better place please let
> me know; maybe python-ideas? The patch as is stands
> does affect
> sysconfig.py, which is used more by distutils than
> anything else.)
>
> > The idea we discussed is to add to Python's built-in
> site.py the ability
> to set paths up for a virtual environment, triggered
> by certain
> environment variables. Then, for convenience,
> there'll be a small
> executable which can be placed in the "bin/"
> directory of a virtual
> environment and knows how to set up these
> environment variables and then
> exec() the system Python binary.
> >
> > Larry Hastings had already created this wrapper
> executable at last
> year's PyCon, and tonight I made the necessary
> modifications to site.py
> and sysconfig.py to support it on the Python side.
> The early prototype
> is now working well (at least on Linux; I think it
> ought to work on OS
> X, and should work partially on Windows as well),
> and I'd welcome review
> and comment: https://bitbucket.org/carljm/cpythonv ?Look
> in
> Tools/pythonv/README.rst for
> instructions.
>
> > This is an early prototype and will certainly
> require refinement (not to
> mention most likely a PEP, at some point). Please
> try it out and let me
> know if it works for you!
> 
> Carl,
> 
> Thanks for posting this.  I'm very hopeful that buildout can use this
> same mechanism to get the isolation it needs.  I would really
> appreciate it if buildout users who care about this would test it with
> buildout.  In particular, I know of 2 basic use cases:
> 
> - Get complete isolation from local additions relative to the standard
> Python distribution.
> 
> - Have the ability to cherry pick some local additions while having
> isolation from the rest. (Gary and Ubuntu, I'm looking at you.)
> 
> Please, let's make sure this mechanism is enough. To raise the stakes
> a bit, when this mechanism is available, presumably in Python 3.3, I
> plan to use it exclusively to provide isolation in buildout.
> 
> Jim
> 
> --
> Jim Fulton
> http://www.linkedin.com/in/jimfulton
_______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
> 
> 

_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG at python.org
http://mail.python.org/mailman/listinfo/distutils-sig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110316/a6015c3b/attachment.html>

From a.cavallo at cavallinux.eu  Wed Mar 16 22:52:08 2011
From: a.cavallo at cavallinux.eu (Antonio Cavallo)
Date: Wed, 16 Mar 2011 21:52:08 +0000
Subject: [Distutils] early preview of pythonv
In-Reply-To: <4D80B17D.5090305@v.loewis.de>
References: <35142.1300279097@cavallinux.eu> <4D80B17D.5090305@v.loewis.de>
Message-ID: <5BAFE978-EAC9-4A7B-BC33-F8530F510331@cavallinux.eu>


On 16 Mar 2011, at 12:47, Martin v. L?wis wrote:

> Am 16.03.11 08:38, schrieb a.cavallo at cavallinux.eu:
>> Hi,
>> 
>> please have a look into my little project:
>> 
>> http://pyvm.sf.net
> 
> This redirects me to
> 
> http://cclimited.webfactional.com/
> 
> Not sure whether it's the right page, if it is,
> I'm completely lost as to what this is all about.

I do this in my spare time, I put some documentation from time to time.

> On the "Main" page, it apparently says nowhere
> what this actually *does*, only that multiple RPM
> files of it are available which pass their tests.

The files are python interpreters build from subversion. 

There's a small runtime tests providing a minimal check the interpreter is not missing libraries,
it was build and installed properly etc....


> The tutorial page offers me to learn "OBS", which
> apparently is the OpenSuSE build server, which appears
> to be unrelated to Python.

It's a build server a generic service to build packages automatically.

> Just my 0.02?.

Thank you,

I'll see if I can put some are time into writing a better description,
Regards,
Antonio

From pje at telecommunity.com  Wed Mar 16 23:47:10 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 16 Mar 2011 18:47:10 -0400
Subject: [Distutils] window 64bit madness
In-Reply-To: <4D8090D0.5050902@gmail.com>
References: <4D7E28C0.3080204@gmail.com>
	<20110314185106.CF9863A4077@sparrow.telecommunity.com>
	<4D8090D0.5050902@gmail.com>
Message-ID: <20110316224724.29CC53A4062@sparrow.telecommunity.com>

At 11:28 AM 3/16/2011 +0100, Adam GROSZER wrote:
>On 03/14/2011 07:50 PM, P.J. Eby wrote:
>>Run "python -c 'import pkg_resources;print
>>pkg_resources.get_build_platform()'" (with the Python interpreter you're
>>using.
>
>D:\install>c:\Python26_64\python.exe
>Python 2.6.6 (r266:84297, Aug 24 2010, 18:13:38) [MSC v.1500 64 bit 
>(AMD64)] on win32
>Type "help", "copyright", "credits" or "license" for more information.
> >>> import pkg_resources;print pkg_resources.get_build_platform()
>win-amd64
> >>>

Hm.  What's sys.platform?  Is it win32?

I suspect the problem has to do with all the win32-specificness 
scattered through setuptools; I think I know what I need to do to fix 
it all, but it'll be a fairly substantial patch; can you help with 
the testing if I email you such a patch?


From vinay_sajip at yahoo.co.uk  Thu Mar 17 00:13:09 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 16 Mar 2011 23:13:09 +0000 (UTC)
Subject: [Distutils] reservations about pythonv
References: <874o73ghnz.fsf@asaph.rhodesmill.org>
	<20110316124722.7e1563b1@neurotica>
Message-ID: <loom.20110317T000709-583@post.gmane.org>

Barry Warsaw <barry <at> python.org> writes:
 
> Here at the sprints we talked about several possible options, the details of
> which of course will have to be hashed out.  There will also be cross platform
> considerations.  I think on *nix at least, it would be nice if a symlink and
> configuration file were enough to trigger the virtualenv behavior.

I feel that this approach definitely has potential to be the most useful and
practical. There's no reason for a separate executable if Python itself
incorporates the virtual environment functionality, which is mostly about
setting paths and environment variables.

> For example, if I have a local 'python' symlinked to the real executable, with
> a pythonv.conf file next to it, the virtual environment would be enabled.  The
> real Python binary would adjust its behavior in that case to know where the
> standard library was, but also use the locally installed packages.  Anyway,
> that's how I'd *like* it to work on *nix, but it may have to work differently
> on Windows, and it may have to work differently if environment variables have
> to be set.

Even if on Windows you have to copy rather than symlink, that could just be the
main Python executable, which is not prohibitively large since the core of
Python is in pythonX.Y.dll.

Regards,

Vinay Sajip


From carl at oddbird.net  Thu Mar 17 01:01:09 2011
From: carl at oddbird.net (=?utf-8?B?Q2FybCBNZXllcg==?=)
Date: Wed, 16 Mar 2011 19:01:09 -0500
Subject: [Distutils] =?utf-8?q?early_preview_of_pythonv?=
Message-ID: <E1Q00e7-0004X0-6a@lettuce.threepines.org>

Hi Vinay,

I've certainly considered using PYTHONHOME. I know of at least two problems with the approach:

1 Having a dedicated virtualenv binary of some kind is more amenable to installing scripts with a shebang line that ensures they always run in the virtualenv.

2. Unless we change the semantics of PYTHONHOME, it would require the same kind of stdlib bootstrapping hacks that virtualenv currently uses. Avoiding this is a design goal.

Carl

Sent from my Android phone

----- Reply message -----
From: "Vinay Sajip" <vinay_sajip at yahoo.co.uk>
Date: Wed, Mar 16, 2011 9:59 am
Subject: [Distutils] early preview of pythonv
To: <Distutils-Sig at Python.Org>

Carl Meyer <carl <at> oddbird.net> writes:

> This is an early prototype and will certainly require refinement (not to
> mention most likely a PEP, at some point). Please try it out and let me
> know if it works for you!

I haven't tried it yet, but I'm curious: was an approach using PYTHONHOME
considered? Or is PYTHONHOME off-limits for an application like this?

Regards,

Vinay Sajip

_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG at python.org
http://mail.python.org/mailman/listinfo/distutils-sig

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110316/1a694109/attachment.html>

From carl at oddbird.net  Thu Mar 17 01:19:46 2011
From: carl at oddbird.net (=?utf-8?B?Q2FybCBNZXllcg==?=)
Date: Wed, 16 Mar 2011 19:19:46 -0500
Subject: [Distutils] =?utf-8?q?reservations_about_pythonv?=
Message-ID: <E1Q00w8-0004a6-3l@lettuce.threepines.org>

This alternative approach (with a symlink and config file) sounds pretty good to me as well after our discussion at the sprints. The downsides I see:

1. Cross-platform inconsistency. Windows virtualenvs would be isolated from system python binary upgrades, *nix would not (some, like Brandon, consider this isolation a feature, so maybe *nix should allow that option too?) On the whole this doesn't seem like a big issue.

2. There would be no way to set LD_LIBRARY_PATH to include the virtualenv prior to execution of python. To be honest, I don't understand why this is needed, as current virtualenv doesn't do it and I haven't seen it break anything (compiled modules work fine in a venv). But people with more experience seemed to think it was needed at our open space discussion last Saturday. Perhaps someone can clarify what specifically breaks without this?

I'll dive into this approach and see what I can achieve.

I'd also appreciate feedback from Tarek or others familiar with the new sysconfig module about my changes there. Something along those lines will I think be needed with either approach, but I'm almost wondering if a new "virtual" install scheme would be a better approach?

Carl

Sent from my Android phone

----- Reply message -----
From: "Vinay Sajip" <vinay_sajip at yahoo.co.uk>
Date: Wed, Mar 16, 2011 6:13 pm
Subject: [Distutils] reservations about pythonv
To: <Distutils-Sig at Python.Org>

Barry Warsaw <barry <at> python.org> writes:
 
> Here at the sprints we talked about several possible options, the details of
> which of course will have to be hashed out.  There will also be cross platform
> considerations.  I think on *nix at least, it would be nice if a symlink and
> configuration file were enough to trigger the virtualenv behavior.

I feel that this approach definitely has potential to be the most useful and
practical. There's no reason for a separate executable if Python itself
incorporates the virtual environment functionality, which is mostly about
setting paths and environment variables.

> For example, if I have a local 'python' symlinked to the real executable, with
> a pythonv.conf file next to it, the virtual environment would be enabled.  The
> real Python binary would adjust its behavior in that case to know where the
> standard library was, but also use the locally installed packages.  Anyway,
> that's how I'd *like* it to work on *nix, but it may have to work differently
> on Windows, and it may have to work differently if environment variables have
> to be set.

Even if on Windows you have to copy rather than symlink, that could just be the
main Python executable, which is not prohibitively large since the core of
Python is in pythonX.Y.dll.

Regards,

Vinay Sajip

_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG at python.org
http://mail.python.org/mailman/listinfo/distutils-sig

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110316/06900c85/attachment.html>

From carl at oddbird.net  Thu Mar 17 03:33:31 2011
From: carl at oddbird.net (Carl Meyer)
Date: Wed, 16 Mar 2011 22:33:31 -0400
Subject: [Distutils] pythonv, take two
In-Reply-To: <E1Q00w8-0004a6-3l@lettuce.threepines.org>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>
Message-ID: <4D8172FB.3020109@oddbird.net>

Hi all,

I've pushed this symlink/copy binary approach to the "pythonv2" branch
at http://bitbucket.org/carljm/cpythonv. I was a bit shocked at how easy
it turned out to be: just a few lines in site.py and the same changes to
sysconfig.py as previously. No changes in C code needed at all. It was
so easy I'm wondering what I must have missed, but everything seems to
work well. Please try it and tell me what I missed!

To try it out, create a directory somewhere with a bin/ subdirectory,
symlink or copy (it works either way) the python3 binary from an install
of the pythonv2 branch into bin/, and create lib/python3.3/site-packages
in the virtualenv. Create "pythonv.conf" in the root of the virtualenv -
it can be empty if you want full isolation; if you want the system
site-packages too you can add a [pythonv] section with
"include-system-site=True" (it's a ConfigParser-style file). Start up
your binary and check sys.path. You can download distribute and "python
setup.py install" that into your virtualenv and then easy_install more
stuff into your env.

At this point, unless the lack of LD_LIBRARY_PATH is actually a blocker,
I can't see any reason not to go with this approach instead.

Carl

From barry at python.org  Thu Mar 17 03:39:04 2011
From: barry at python.org (Barry Warsaw)
Date: Wed, 16 Mar 2011 22:39:04 -0400
Subject: [Distutils] reservations about pythonv
In-Reply-To: <E1Q00w8-0004a6-3l@lettuce.threepines.org>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>
Message-ID: <20110316223904.288080ba@neurotica>

On Mar 16, 2011, at 07:19 PM, Carl Meyer wrote:

>This alternative approach (with a symlink and config file) sounds pretty good
>to me as well after our discussion at the sprints. The downsides I see:
>
>1. Cross-platform inconsistency. Windows virtualenvs would be isolated from
>system python binary upgrades, *nix would not (some, like Brandon, consider
>this isolation a feature, so maybe *nix should allow that option too?) On the
>whole this doesn't seem like a big issue.

Agreed.  Windows users are used to doing Python upgrades themselves since
their operating system doesn't help them.  So if they have to copy binaries,
they'll have to be aware that when they upgrade Python, their virtualenvs
won't get automatically updated.

For *nix users, I agree that it might be a useful option to either copy
binaries or use a symlink, though I think the later should be the default.

>2. There would be no way to set LD_LIBRARY_PATH to include the virtualenv
>prior to execution of python. To be honest, I don't understand why this is
>needed, as current virtualenv doesn't do it and I haven't seen it break
>anything (compiled modules work fine in a venv). But people with more
>experience seemed to think it was needed at our open space discussion last
>Saturday. Perhaps someone can clarify what specifically breaks without this?

I was thinking about this too.  I'm trying to recall a similar situation I had
in a long-previous job where we had to re-exec to pick up $LD_LIBRARY_PATH (on
Solaris IIRC).  The issue there was that the Python binary *itself* needed
$LD_LIBRARY_PATH set before it could find some shared libraries, so that envar
had to essentially be set before `python` could run.

We've got a different situation here though.  We're assuming `python` itself
runs just fine, but that some shared library in the virtualenv is needed to
import an extension module.  So, why doesn't Python set $LD_LIBRARY_PATH in
its own environment once it knows its running in a virtualenv?  That way, any
subsequent searches for shared libraries would find them in the virtualenv.

OTOH, I vaguely recall that while this *should* work, it actually doesn't
because once the executable has started, $LD_LIBRARY_PATH isn't consulted
again.

The above just means my memory is too faulty to of much use ;).  I'll just
echo Carl's request for specific cases where $LD_LIBRARY_PATH needs to be set.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110316/5880f1e3/attachment-0001.pgp>

From barry at python.org  Thu Mar 17 04:04:46 2011
From: barry at python.org (Barry Warsaw)
Date: Wed, 16 Mar 2011 23:04:46 -0400
Subject: [Distutils] pythonv, take two
In-Reply-To: <4D8172FB.3020109@oddbird.net>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>
	<4D8172FB.3020109@oddbird.net>
Message-ID: <20110316230446.23fae504@neurotica>

On Mar 16, 2011, at 10:33 PM, Carl Meyer wrote:

>I've pushed this symlink/copy binary approach to the "pythonv2" branch
>at http://bitbucket.org/carljm/cpythonv. I was a bit shocked at how easy
>it turned out to be: just a few lines in site.py and the same changes to
>sysconfig.py as previously. No changes in C code needed at all. It was
>so easy I'm wondering what I must have missed, but everything seems to
>work well. Please try it and tell me what I missed!

Seems to work for me (on Linux at least :)!  Really brilliant work, Carl.

>To try it out, create a directory somewhere with a bin/ subdirectory,
>symlink or copy (it works either way) the python3 binary from an install
>of the pythonv2 branch into bin/, and create lib/python3.3/site-packages
>in the virtualenv. Create "pythonv.conf" in the root of the virtualenv -
>it can be empty if you want full isolation; if you want the system
>site-packages too you can add a [pythonv] section with
>"include-system-site=True" (it's a ConfigParser-style file). Start up
>your binary and check sys.path. You can download distribute and "python
>setup.py install" that into your virtualenv and then easy_install more
>stuff into your env.
>
>At this point, unless the lack of LD_LIBRARY_PATH is actually a blocker,
>I can't see any reason not to go with this approach instead.

+1.  Time for a PEP?

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110316/40ce0632/attachment.pgp>

From agroszer.ll at gmail.com  Thu Mar 17 08:43:17 2011
From: agroszer.ll at gmail.com (Adam GROSZER)
Date: Thu, 17 Mar 2011 08:43:17 +0100
Subject: [Distutils] window 64bit madness
In-Reply-To: <20110316224724.29CC53A4062@sparrow.telecommunity.com>
References: <4D7E28C0.3080204@gmail.com>	<20110314185106.CF9863A4077@sparrow.telecommunity.com>	<4D8090D0.5050902@gmail.com>
	<20110316224724.29CC53A4062@sparrow.telecommunity.com>
Message-ID: <4D81BB95.3040603@gmail.com>

On 03/16/2011 11:47 PM, P.J. Eby wrote:
> At 11:28 AM 3/16/2011 +0100, Adam GROSZER wrote:
>> On 03/14/2011 07:50 PM, P.J. Eby wrote:
>>> Run "python -c 'import pkg_resources;print
>>> pkg_resources.get_build_platform()'" (with the Python interpreter you're
>>> using.
>>
>> D:\install>c:\Python26_64\python.exe
>> Python 2.6.6 (r266:84297, Aug 24 2010, 18:13:38) [MSC v.1500 64 bit
>> (AMD64)] on win32
>> Type "help", "copyright", "credits" or "license" for more information.
>> >>> import pkg_resources;print pkg_resources.get_build_platform()
>> win-amd64
>> >>>
>
> Hm. What's sys.platform? Is it win32?

Exactly.

> I suspect the problem has to do with all the win32-specificness
> scattered through setuptools; I think I know what I need to do to fix it
> all, but it'll be a fairly substantial patch; can you help with the
> testing if I email you such a patch?

Sure thing.

-- 
Best regards,
  Adam GROSZER
--
Quote of the day:
Don't knock President Fillmore.  He kept us out of Vietnam.

From vinay_sajip at yahoo.co.uk  Thu Mar 17 11:25:31 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 17 Mar 2011 10:25:31 +0000 (UTC)
Subject: [Distutils] pythonv, take two
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>
	<4D8172FB.3020109@oddbird.net>
Message-ID: <loom.20110317T112246-771@post.gmane.org>

Carl Meyer <carl <at> oddbird.net> writes:

> work well. Please try it and tell me what I missed!
> 
> To try it out, create a directory somewhere with a bin/ subdirectory,
> symlink or copy (it works either way) the python3 binary from an install
> of the pythonv2 branch into bin/, and create lib/python3.3/site-packages

It seems to work with a symlink but not with a copy. With an empty pythonv.conf
file in ~/projects/vptest:

vinay at eta-natty:~/projects/vptest/bin$ ln -s ~/tools/cpythonv/python
vinay at eta-natty:~/projects/vptest/bin$ ./python
Python 3.3a0 (pythonv2:e56f05883ceb, Mar 17 2011, 09:07:29) 
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/local/lib/python33.zip', '/home/vinay/tools/cpythonv/Lib',
'/home/vinay/tools/cpythonv/Lib/plat-linux2',
'/home/vinay/tools/cpythonv/build/lib.linux-i686-3.3',
'/home/vinay/projects/vptest/lib/python3.3/site-packages']
>>> 
vinay at eta-natty:~/projects/vptest/bin$ rm python
vinay at eta-natty:~/projects/vptest/bin$ cp ~/tools/cpythonv/python .
vinay at eta-natty:~/projects/vptest/bin$ ./python
Python 3.3a0 (pythonv2:e56f05883ceb, Mar 17 2011, 09:07:29) 
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/local/lib/python33.zip', '/usr/local/lib/python3.3',
'/usr/local/lib/python3.3/plat-linux2', '/usr/local/lib/python3.3/lib-dynload',
'/usr/local/lib/python3.3/site-packages']
>>> 

Still, very promising! Nice work.

Regards,

Vinay Sajip


From jim at zope.com  Thu Mar 17 13:07:02 2011
From: jim at zope.com (Jim Fulton)
Date: Thu, 17 Mar 2011 08:07:02 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
Message-ID: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>

Whatever mechanism we end up with, I suggest that a standard python
install include an isolated configuration. This is a common use case
and should be available without having to create a virtualenv (or
whatever) for each project or working directory.

I'm very happy to see this work taking place.

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From fdrake at acm.org  Thu Mar 17 13:36:58 2011
From: fdrake at acm.org (Fred Drake)
Date: Thu, 17 Mar 2011 08:36:58 -0400
Subject: [Distutils] reservations about pythonv
In-Reply-To: <20110316223904.288080ba@neurotica>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>
	<20110316223904.288080ba@neurotica>
Message-ID: <AANLkTinF+ypU2JJy-NQO6ZdpGt8UbvLJoaNSRNtX5E0e@mail.gmail.com>

On Wed, Mar 16, 2011 at 10:39 PM, Barry Warsaw <barry at python.org> wrote:
> I vaguely recall that while this *should* work, it actually doesn't
> because once the executable has started, $LD_LIBRARY_PATH isn't consulted
> again.

I recall less vaguely, since we've had to deal with this problem more
recently than your Solaris antics.  :-)  The loader consults
LD_LIBRARY_PATH before  main()  is called, initializing it's own
state, and doesn't pick up changes.  This makes a re-exec necessary if
the new value is different from the original (worth checking to avoid
an exec).


? -Fred

--
Fred L. Drake, Jr.? ? <fdrake at acm.org>
"A storm broke loose in my mind."? --Albert Einstein

From barry at python.org  Thu Mar 17 15:56:49 2011
From: barry at python.org (Barry Warsaw)
Date: Thu, 17 Mar 2011 10:56:49 -0400
Subject: [Distutils] reservations about pythonv
In-Reply-To: <AANLkTinF+ypU2JJy-NQO6ZdpGt8UbvLJoaNSRNtX5E0e@mail.gmail.com>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>
	<20110316223904.288080ba@neurotica>
	<AANLkTinF+ypU2JJy-NQO6ZdpGt8UbvLJoaNSRNtX5E0e@mail.gmail.com>
Message-ID: <20110317105649.777cf713@neurotica>

On Mar 17, 2011, at 08:36 AM, Fred Drake wrote:

>On Wed, Mar 16, 2011 at 10:39 PM, Barry Warsaw <barry at python.org> wrote:
>> I vaguely recall that while this *should* work, it actually doesn't
>> because once the executable has started, $LD_LIBRARY_PATH isn't consulted
>> again.
>
>I recall less vaguely, since we've had to deal with this problem more
>recently than your Solaris antics.  :-)  The loader consults
>LD_LIBRARY_PATH before  main()  is called, initializing it's own
>state, and doesn't pick up changes.  This makes a re-exec necessary if
>the new value is different from the original (worth checking to avoid
>an exec).

Thanks for helping bolster my memory!  I still think setting $LD_LIBRARY_PATH
won't be necessary in the majority of cases, so generally no re-exec should
happen.  I can imagine that if it *were* necessary, an appropriate section in
the pythonv.conf file would trigger it.  E.g.:

[env]
LD_LIBRARY_PATH: /path/to/add

[pythonv]
re-exec: true

Cheers,
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110317/77ddb067/attachment.pgp>

From barry at python.org  Thu Mar 17 15:59:48 2011
From: barry at python.org (Barry Warsaw)
Date: Thu, 17 Mar 2011 10:59:48 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
Message-ID: <20110317105948.4557667c@neurotica>

On Mar 17, 2011, at 08:07 AM, Jim Fulton wrote:

>Whatever mechanism we end up with, I suggest that a standard python
>install include an isolated configuration. This is a common use case
>and should be available without having to create a virtualenv (or
>whatever) for each project or working directory.

Could you elaborate on what this means?  I don't quite understand what you
mean by "include an isolated configuration".

I do think that Python should include a script to create the small amount of
set up needed to trigger a (built-in) virtual environment.  E.g. create the
bin and lib/... directories, populate bin, and drop a minimal pythonv.conf
file.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110317/bf4cec96/attachment.pgp>

From erik.m.bray at gmail.com  Thu Mar 17 16:42:35 2011
From: erik.m.bray at gmail.com (Erik Bray)
Date: Thu, 17 Mar 2011 11:42:35 -0400
Subject: [Distutils] distutils2 Forward Compatibility module
Message-ID: <AANLkTi=xBT3Gba9JS261boSRjiBzLAkTThhqUS81zpqD@mail.gmail.com>

Hi all,
This is my first post specifically to distutils-sig, though I've had
an interest in packaging for a while (having come up with some fairly
arcane schemes in the past, where no better alternatives were
apparent).

At any rate: I'm currently working on a plan to overhaul how a number
of my company's projects are packaged and distributed.  Right now they
all rely on a monkey-patched, hacked up distutils that needs to go
away.  I'm already on top of that.

But seeing as how distutils2 is going to be the "new hotness" I want
to plan for at as part of my overhaul.  I realize that distutils2 is
still in flux, and anything I do now will have to be tweaked as
development on it continues.  I am fine with this, as I still intend
to use Distribute as the primary installation mechanism.  But I really
like how distutils2 keeps all metadata in the setup.cfg file, and want
to start doing that now, so that I don't have to keep two copies of
everything.

It should be no problem to just have my setup.py read everything it
needs out of setup.cfg, but what I'm wondering is if there is already
an extension to do this, or will I have to roll my own?  It just seems
like an obvious thing to have for transitioning to distutils2, and if
it doesn't already exist it should (I will of course be happy to
contribute).  I should note that I don't want distutils2 itself to be
a dependency for installing my packages, as it is too unstable, so
directly using any machinery built into it is out of the question.

Thanks for any comments,
Erik

From thomas at thomas-lotze.de  Thu Mar 17 16:55:05 2011
From: thomas at thomas-lotze.de (Thomas Lotze)
Date: Thu, 17 Mar 2011 16:55:05 +0100
Subject: [Distutils] New buildout options: checksums and
	allow-omitted-checksums
Message-ID: <pan.2011.03.17.15.55.03.959118@ID-174572.user.uni-berlin.de>

Hi,

two weeks ago I asked about your opinions on a buildout option
that enforces specifying (MD5) checksums for all files downloaded
through buildout's download utility API. I've been discussing the
subject with Christian Theune in the meantime and would like to
describe a more concrete proposal now that deals with the
questions raised in my previous post:

In analogy with version pinning for eggs, two new options could
be introduced to the buildout section:

- "checksums": This option would name a config section that
  contains checksums for any number of resources by URL. I
  suggest a default value of "checksum" for it. Listing some URL
  with an empty checksum would explicitly express that the
  checksum for this resource should never be checked. I'm not
  sure how to structure the contents of the checksums section:
  URLs are not valid config keys in general (they can contain "="
  characters) but I'd still like to be able to rely on the
  existing mechanism for overriding single options to override
  single checksums. Mapping arbitrary keys to
  whitespace-separated pairs of URL and checksum would work but
  sounds inelegant.

- "allow-omitted-checksums": This option would specify whether
  resources should be downloaded that are not listed in the
  checksums section. I'd like to use False as this option's
  default value, meaning that buildout should raise a UserError
  if a resource is omitted from the checksums section.
  (Intentionally empty checksums would still be allowed.) In
  fact, I'm not completely happy about inventing an option with
  negative semantics but doing it this way is at least consistent
  with "allow-picked-versions".

I'd like to hear other people's opinion on both the general idea
and the details. Unless the whole thing gets shot down, I plan to
start implementing it on a branch of zc.buildout next week.

Thank you.

-- 
Thomas




From ziade.tarek at gmail.com  Thu Mar 17 16:55:23 2011
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 17 Mar 2011 08:55:23 -0700
Subject: [Distutils] distutils2 Forward Compatibility module
In-Reply-To: <AANLkTi=xBT3Gba9JS261boSRjiBzLAkTThhqUS81zpqD@mail.gmail.com>
References: <AANLkTi=xBT3Gba9JS261boSRjiBzLAkTThhqUS81zpqD@mail.gmail.com>
Message-ID: <AANLkTik0UzxjtcmQq3g7SDHOSB_o4hTVGzbfXY8EwN-O@mail.gmail.com>

Hi Erik,

On Thu, Mar 17, 2011 at 8:42 AM, Erik Bray <erik.m.bray at gmail.com> wrote:
> Hi all,
> This is my first post specifically to distutils-sig, though I've had
> an interest in packaging for a while (having come up with some fairly
> arcane schemes in the past, where no better alternatives were
> apparent).
>
> At any rate: I'm currently working on a plan to overhaul how a number
> of my company's projects are packaged and distributed. ?Right now they
> all rely on a monkey-patched, hacked up distutils that needs to go
> away. ?I'm already on top of that.
>
> But seeing as how distutils2 is going to be the "new hotness" I want
> to plan for at as part of my overhaul. ?I realize that distutils2 is
> still in flux, and anything I do now will have to be tweaked as
> development on it continues. ?I am fine with this, as I still intend
> to use Distribute as the primary installation mechanism. ?But I really
> like how distutils2 keeps all metadata in the setup.cfg file, and want
> to start doing that now, so that I don't have to keep two copies of
> everything.
>
> It should be no problem to just have my setup.py read everything it
> needs out of setup.cfg, but what I'm wondering is if there is already
> an extension to do this, or will I have to roll my own? ?It just seems
> like an obvious thing to have for transitioning to distutils2, and if
> it doesn't already exist it should (I will of course be happy to
> contribute). ?I should note that I don't want distutils2 itself to be
> a dependency for installing my packages, as it is too unstable, so
> directly using any machinery built into it is out of the question.

There's such a thing, look at this function here:

http://hg.python.org/distutils2/file/6fca65ff60ad/distutils2/util.py#l1098

It's not hooked yet to a command-line tool like mkcfg, but should be soon.

Let us know how it works out for you.


>
> Thanks for any comments,
> Erik
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | http://ziade.org

From carl at oddbird.net  Thu Mar 17 17:16:47 2011
From: carl at oddbird.net (Carl Meyer)
Date: Thu, 17 Mar 2011 12:16:47 -0400
Subject: [Distutils] pythonv, take two
In-Reply-To: <loom.20110317T112246-771@post.gmane.org>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<4D8172FB.3020109@oddbird.net>
	<loom.20110317T112246-771@post.gmane.org>
Message-ID: <4D8233EF.7070205@oddbird.net>

Hi Vinay,

On 03/17/2011 06:25 AM, Vinay Sajip wrote:
> It seems to work with a symlink but not with a copy. With an empty pythonv.conf
> file in ~/projects/vptest:
> 
> vinay at eta-natty:~/projects/vptest/bin$ ln -s ~/tools/cpythonv/python
> vinay at eta-natty:~/projects/vptest/bin$ ./python
> Python 3.3a0 (pythonv2:e56f05883ceb, Mar 17 2011, 09:07:29) 
> [GCC 4.5.2] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import sys
>>>> sys.path
> ['', '/usr/local/lib/python33.zip', '/home/vinay/tools/cpythonv/Lib',
> '/home/vinay/tools/cpythonv/Lib/plat-linux2',
> '/home/vinay/tools/cpythonv/build/lib.linux-i686-3.3',
> '/home/vinay/projects/vptest/lib/python3.3/site-packages']
>>>>
> vinay at eta-natty:~/projects/vptest/bin$ rm python
> vinay at eta-natty:~/projects/vptest/bin$ cp ~/tools/cpythonv/python .
> vinay at eta-natty:~/projects/vptest/bin$ ./python
> Python 3.3a0 (pythonv2:e56f05883ceb, Mar 17 2011, 09:07:29) 
> [GCC 4.5.2] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import sys
>>>> sys.path
> ['', '/usr/local/lib/python33.zip', '/usr/local/lib/python3.3',
> '/usr/local/lib/python3.3/plat-linux2', '/usr/local/lib/python3.3/lib-dynload',
> '/usr/local/lib/python3.3/site-packages']

This is odd, as the same setup works fine for me with a copied binary
(I'm on Ubuntu 10.10). If you're willing to do any more debugging on
this, here's what would be helpful:

1. Check the values of sys.executable, sys.prefix, and sys.exec_prefix.
2. Stick a pdb.set_trace() into site.py near the top of the virtualize()
method and step through it, observing where something goes awry. In
particular: does it find pythonv.conf? Does it modify PREFIXES?

Thanks for testing it out!

Carl

From carl at oddbird.net  Thu Mar 17 17:32:15 2011
From: carl at oddbird.net (Carl Meyer)
Date: Thu, 17 Mar 2011 12:32:15 -0400
Subject: [Distutils] reservations about pythonv
In-Reply-To: <20110317105649.777cf713@neurotica>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<20110316223904.288080ba@neurotica>	<AANLkTinF+ypU2JJy-NQO6ZdpGt8UbvLJoaNSRNtX5E0e@mail.gmail.com>
	<20110317105649.777cf713@neurotica>
Message-ID: <4D82378F.6000802@oddbird.net>



On 03/17/2011 10:56 AM, Barry Warsaw wrote:
> Thanks for helping bolster my memory!  I still think setting $LD_LIBRARY_PATH
> won't be necessary in the majority of cases, so generally no re-exec should
> happen.  I can imagine that if it *were* necessary, an appropriate section in
> the pythonv.conf file would trigger it.  E.g.:
> 
> [env]
> LD_LIBRARY_PATH: /path/to/add
> 
> [pythonv]
> re-exec: true

I also realized last night that if the need for LD_LIBRARY_PATH is as
rare as it seems to be, people could just as well set it themselves
before running stuff in their virtualenv. We could even have our shell
"activate" script set it, so you'd only have to set it yourself if using
the virtualenv's binary directly without "activate." I'm a bit tempted
in this direction, as the whole auto-re-exec business feels like it
introduces an entirely different level of magic. But if it would make
life easier, it shouldn't be hard to do.

Since I'm finding and parsing pythonv.conf in site.py, would it be
acceptable to do the exec there? Or are there good reasons for it to
happen earlier in C code? (I'm a bit loath to have to find and parse
pythonv.conf twice).

If we do this, I'd also appreciate (for the PEP) a more concrete
real-life example of what you might try do in a a virtualenv that would
require this.

Carl

From carl at oddbird.net  Thu Mar 17 17:44:49 2011
From: carl at oddbird.net (Carl Meyer)
Date: Thu, 17 Mar 2011 12:44:49 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <20110317105948.4557667c@neurotica>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica>
Message-ID: <4D823A81.2060406@oddbird.net>



On 03/17/2011 10:59 AM, Barry Warsaw wrote:
> On Mar 17, 2011, at 08:07 AM, Jim Fulton wrote:
>> Whatever mechanism we end up with, I suggest that a standard python
>> install include an isolated configuration. This is a common use case
>> and should be available without having to create a virtualenv (or
>> whatever) for each project or working directory.
> 
> Could you elaborate on what this means?  I don't quite understand what you
> mean by "include an isolated configuration".

I'm also not entirely clear what this means, but I think perhaps "python
-S" already covers it? That will start up a python interpreter without
importing site.py, so it will have no site-packages at all; nothing but
the stdlib. Of course, then you'd have to take care of fixing up
sys.path yourself to include your project and its dependencies: this may
be reasonable for buildout.

> I do think that Python should include a script to create the small amount of
> set up needed to trigger a (built-in) virtual environment.  E.g. create the
> bin and lib/... directories, populate bin, and drop a minimal pythonv.conf
> file.

Definitely. It looks like this'll end up being the bulk of the added
code; I'm planning to do it (borrowing liberally from virtualenv.py) as
soon as I'm confident of the basic approach.

Any opinions on the commandline UI for this? I was thinking of just
adding a "pythonv.py" to the stdlib that you could execute with "python
-m pythonv path/to/new/env" (and would also export appropriate API to
create environments programmatically).

Carl

From barry at python.org  Thu Mar 17 17:48:02 2011
From: barry at python.org (Barry Warsaw)
Date: Thu, 17 Mar 2011 12:48:02 -0400
Subject: [Distutils] reservations about pythonv
In-Reply-To: <4D82378F.6000802@oddbird.net>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>
	<20110316223904.288080ba@neurotica>
	<AANLkTinF+ypU2JJy-NQO6ZdpGt8UbvLJoaNSRNtX5E0e@mail.gmail.com>
	<20110317105649.777cf713@neurotica> <4D82378F.6000802@oddbird.net>
Message-ID: <20110317124802.63a39572@neurotica>

On Mar 17, 2011, at 12:32 PM, Carl Meyer wrote:

>I also realized last night that if the need for LD_LIBRARY_PATH is as
>rare as it seems to be, people could just as well set it themselves
>before running stuff in their virtualenv. We could even have our shell
>"activate" script set it, so you'd only have to set it yourself if using
>the virtualenv's binary directly without "activate."

+1.  Keeps Python nice and simple and AFAICT, it should work because it'll set
the environment variable before Python starts executing.

OTOH, if no actual use case for LD_LIBRARY_PATH gets raised, then even this
won't be necessary.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110317/9f18be64/attachment.pgp>

From leorochael at gmail.com  Thu Mar 17 18:10:16 2011
From: leorochael at gmail.com (Leonardo Rochael Almeida)
Date: Thu, 17 Mar 2011 18:10:16 +0100
Subject: [Distutils] New buildout options: checksums and
	allow-omitted-checksums
In-Reply-To: <pan.2011.03.17.15.55.03.959118@ID-174572.user.uni-berlin.de>
References: <pan.2011.03.17.15.55.03.959118@ID-174572.user.uni-berlin.de>
Message-ID: <AANLkTinXwjn8-K=nu9KY37fsf54qZR1ypJtmB8L_tszg@mail.gmail.com>

Hi Thomas,

I like your idea in general. I'd like to point to the following
suggestion with patch+test (though it might need some cleanup) that is
not exactly related to what you're proposing, but has to do with the
same thing (relationship between files and checksums):

https://bugs.launchpad.net/zc.buildout/+bug/692600

It suggests (and implements) automatically redownloading files when
checksums don't match (which could happen when you update your config
file with a new checksum for a file that has changed upstream).

Other comments below:

On Thu, Mar 17, 2011 at 16:55, Thomas Lotze <thomas at thomas-lotze.de> wrote:
> Hi,
>
> [...]
>
> - "checksums": This option would name a config section that
> ?contains checksums for any number of resources by URL. I
> ?suggest a default value of "checksum" for it.

Won't 'checksums' (plural) as the value be better? It would keep with
the tradition of matching the name of the buildout option and the name
of the section by default.

> Listing some URL
> ?with an empty checksum would explicitly express that the
> ?checksum for this resource should never be checked. I'm not
> ?sure how to structure the contents of the checksums section:
> ?URLs are not valid config keys in general (they can contain "="
> ?characters) but I'd still like to be able to rely on the
> ?existing mechanism for overriding single options to override
> ?single checksums. Mapping arbitrary keys to
> ?whitespace-separated pairs of URL and checksum would work but
> ?sounds inelegant.

I suggest mapping the checksums (which are valid keys) to URLs, and
having a special key with line-break separated values for explicitly
not checking:

  [checksums]
  080d2c6a849ebd6b7fd49049c21b910a = http://example.com/foo/bar.tgz
  no-check =
      http://example.com/foo/baz.tgz
      http://example.com/foo/fred.tgz

This will not be so elegant when you want to extend another
configuration and override some decisions, but it works somewhat:

  [buildout]
  extends = config-file-above.cfg

  [checksums]
  080d2c6a849ebd6b7fd49049c21b910a =
  no-check += http://example.com/foo/bar.tgz

> [...]

Cheers,
Leo

From jim at zope.com  Thu Mar 17 18:23:17 2011
From: jim at zope.com (Jim Fulton)
Date: Thu, 17 Mar 2011 13:23:17 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <4D823A81.2060406@oddbird.net>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
Message-ID: <AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>

On Thu, Mar 17, 2011 at 12:44 PM, Carl Meyer <carl at oddbird.net> wrote:
>
>
> On 03/17/2011 10:59 AM, Barry Warsaw wrote:
>> On Mar 17, 2011, at 08:07 AM, Jim Fulton wrote:
>>> Whatever mechanism we end up with, I suggest that a standard python
>>> install include an isolated configuration. This is a common use case
>>> and should be available without having to create a virtualenv (or
>>> whatever) for each project or working directory.
>>
>> Could you elaborate on what this means? ?I don't quite understand what you
>> mean by "include an isolated configuration".

As I mentioned in an earlier thread, I want to get complete isolation
from local additions relative to the standard Python distribution.
I want python executable I can use to bootstrap a buildout that
doesn't include site packages or it's equivalent.

My understanding of how this will work was that I could created this
myself by creating some sort of configuration file, say clean.cfg and
then link a Python executable to the name "clean".  Reading "pythonv,
take two" more carefully, I see that it is a lot more
virtualenv-oriented than I'd hoped.  <shrug>

> I'm also not entirely clear what this means, but I think perhaps "python
> -S" already covers it?

I thought so too, but people keep telling me that's not enough. One
issue was that maybe site.py might be imported accidentally, for
example to get at one of the helper functions it contains.  I don't
know if this is a significant danger.

I'd like to have some defined way to express my need for isolation
that's more rubust that simply not importing site.py.

> That will start up a python interpreter without
> importing site.py, so it will have no site-packages at all; nothing but
> the stdlib. Of course, then you'd have to take care of fixing up
> sys.path yourself to include your project and its dependencies: this may
> be reasonable for buildout.

Yup.

Jim

--
Jim Fulton
http://www.linkedin.com/in/jimfulton

From carl at oddbird.net  Thu Mar 17 18:41:11 2011
From: carl at oddbird.net (Carl Meyer)
Date: Thu, 17 Mar 2011 13:41:11 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
Message-ID: <4D8247B7.30102@oddbird.net>



On 03/17/2011 01:23 PM, Jim Fulton wrote:
> My understanding of how this will work was that I could created this
> myself by creating some sort of configuration file, say clean.cfg and
> then link a Python executable to the name "clean".  Reading "pythonv,
> take two" more carefully, I see that it is a lot more
> virtualenv-oriented than I'd hoped.  <shrug>
> 
>> I'm also not entirely clear what this means, but I think perhaps "python
>> -S" already covers it?
> 
> I thought so too, but people keep telling me that's not enough. One
> issue was that maybe site.py might be imported accidentally, for
> example to get at one of the helper functions it contains.  I don't
> know if this is a significant danger.
> 
> I'd like to have some defined way to express my need for isolation
> that's more rubust that simply not importing site.py.

Actually, now that I come to think of it, pythonv (take two) does
already cover your requirement. If you have a symlinked or copied python
binary, and an empty pythonv.conf one directory up, and you simply
_don't_ create any lib/python3.3/site-packages relative to pythonv.conf,
what you'll end up with is identical to "python -S" but robust against
import of site.py (it will already have been imported, but it won't have
found any site-packages directories).

If the possibility of someone accidentally creating
lib/python3.3/site-packages is an issue, we could easily add a
"no-site=True" option to pythonv.conf that would prevent it from even
checking for the existence of that directory.

And in either case, we could add a --no-site option to "python -m
pythonv" that would create a virtualenv without the site-packages
directory (and with no-site=True, if we decide that's worth having).

Carl

From jim at zope.com  Thu Mar 17 18:53:43 2011
From: jim at zope.com (Jim Fulton)
Date: Thu, 17 Mar 2011 13:53:43 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <4D8247B7.30102@oddbird.net>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
	<4D8247B7.30102@oddbird.net>
Message-ID: <AANLkTimh2Kw3LuRk2V1hVgQ3J5OtMXXx+sU1OBOHtPPa@mail.gmail.com>

On Thu, Mar 17, 2011 at 1:41 PM, Carl Meyer <carl at oddbird.net> wrote:
> Actually, now that I come to think of it, pythonv (take two) does
> already cover your requirement. If you have a symlinked or copied python
> binary, and an empty pythonv.conf one directory up, and you simply
> _don't_ create any lib/python3.3/site-packages relative to pythonv.conf,
> what you'll end up with is identical to "python -S" but robust against
> import of site.py (it will already have been imported, but it won't have
> found any site-packages directories).

Cool.

> If the possibility of someone accidentally creating
> lib/python3.3/site-packages is an issue, we could easily add a
> "no-site=True" option to pythonv.conf that would prevent it from even
> checking for the existence of that directory.

I don't think this is an issue.

> And in either case, we could add a --no-site option to "python -m
> pythonv" that would create a virtualenv without the site-packages
> directory (and with no-site=True, if we decide that's worth having).

Cool (for virtualenv).

It occurs to me that it would be nice if site.py could grow knowledge
of whether -S was used and not automatically mutate the path if -S was
used.  That would allow -S to work robustly without having to link
anything or create a config file.

Jim

--
Jim Fulton
http://www.linkedin.com/in/jimfulton

From carl at oddbird.net  Thu Mar 17 19:50:34 2011
From: carl at oddbird.net (Carl Meyer)
Date: Thu, 17 Mar 2011 14:50:34 -0400
Subject: [Distutils] pythonv, take two
In-Reply-To: <20110316230446.23fae504@neurotica>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<4D8172FB.3020109@oddbird.net>
	<20110316230446.23fae504@neurotica>
Message-ID: <4D8257FA.1060700@oddbird.net>

On 03/16/2011 11:04 PM, Barry Warsaw wrote:
> +1.  Time for a PEP?

Working on a draft PEP. I'll push it to bitbucket to make collaboration
easier - then the next step would be to present the draft on
python-ideas, if I'm reading PEP 2 correctly?

Carl

From pje at telecommunity.com  Thu Mar 17 20:00:59 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 17 Mar 2011 15:00:59 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <4D8247B7.30102@oddbird.net>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
	<4D8247B7.30102@oddbird.net>
Message-ID: <20110317190105.2A6C03A4062@sparrow.telecommunity.com>

At 01:41 PM 3/17/2011 -0400, Carl Meyer wrote:
>Actually, now that I come to think of it, pythonv (take two) does
>already cover your requirement. If you have a symlinked or copied python
>binary, and an empty pythonv.conf one directory up,

Is there any reason why the configuration file has to be one 
directory up, intead of adjacent to the executable? 


From carl at oddbird.net  Thu Mar 17 20:04:56 2011
From: carl at oddbird.net (Carl Meyer)
Date: Thu, 17 Mar 2011 15:04:56 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <20110317190105.2A6C03A4062@sparrow.telecommunity.com>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
	<4D8247B7.30102@oddbird.net>
	<20110317190105.2A6C03A4062@sparrow.telecommunity.com>
Message-ID: <4D825B58.502@oddbird.net>

On 03/17/2011 03:00 PM, P.J. Eby wrote:
> At 01:41 PM 3/17/2011 -0400, Carl Meyer wrote:
>> Actually, now that I come to think of it, pythonv (take two) does
>> already cover your requirement. If you have a symlinked or copied python
>> binary, and an empty pythonv.conf one directory up,
> 
> Is there any reason why the configuration file has to be one directory
> up, intead of adjacent to the executable?

Not a particularly good one. I inherited this from the standard layout
of virtualenv, which includes a bin/ directory for the python binary and
scripts, and it made more sense to me in that layout for pythonv.conf to
live at the root of the "virtualenv" rather than in bin/.

I'd like to continue to support it being one level up, but I can't think
of any reason not to first check adjacent, and then one level up. I
suppose it could just keep checking up the tree, but that introduces a
bunch more filesystem checks and could give surprising results. I won't
do that unless someone steps up to give convincing arguments for it.

Carl

From tseaver at palladion.com  Thu Mar 17 20:13:29 2011
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 17 Mar 2011 15:13:29 -0400
Subject: [Distutils] pythonv, take two
In-Reply-To: <4D8257FA.1060700@oddbird.net>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<4D8172FB.3020109@oddbird.net>	<20110316230446.23fae504@neurotica>
	<4D8257FA.1060700@oddbird.net>
Message-ID: <iltmgo$46q$1@dough.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 03/17/2011 02:50 PM, Carl Meyer wrote:
> On 03/16/2011 11:04 PM, Barry Warsaw wrote:
>> +1.  Time for a PEP?
> 
> Working on a draft PEP. I'll push it to bitbucket to make collaboration
> easier - then the next step would be to present the draft on
> python-ideas, if I'm reading PEP 2 correctly?

Correct.


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk2CXVgACgkQ+gerLs4ltQ7UhwCdHAa2pk5ia3Ls9w4O6SADdeJ7
ehgAnibKo+/zwJ7aydUAYYbuk/xkmvxk
=K0bJ
-----END PGP SIGNATURE-----


From barry at python.org  Thu Mar 17 20:16:43 2011
From: barry at python.org (Barry Warsaw)
Date: Thu, 17 Mar 2011 15:16:43 -0400
Subject: [Distutils] pythonv, take two
In-Reply-To: <4D8257FA.1060700@oddbird.net>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>
	<4D8172FB.3020109@oddbird.net> <20110316230446.23fae504@neurotica>
	<4D8257FA.1060700@oddbird.net>
Message-ID: <20110317151643.5295a1ca@neurotica>

On Mar 17, 2011, at 02:50 PM, Carl Meyer wrote:

>On 03/16/2011 11:04 PM, Barry Warsaw wrote:
>> +1.  Time for a PEP?
>
>Working on a draft PEP. I'll push it to bitbucket to make collaboration
>easier - then the next step would be to present the draft on
>python-ideas, if I'm reading PEP 2 correctly?

I dunno.  I tend to think python-ideas are where ideas go to die. ;)  I think
we have general consensus that this is a good idea, and we just need to hash
out the details.  So I personally think python-dev is fine at this point.

(Has nothing to do with me not actually being subscribed to python-ideas.
Nope, not at all. :)

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110317/0966f4ff/attachment-0001.pgp>

From carl at oddbird.net  Thu Mar 17 20:46:52 2011
From: carl at oddbird.net (Carl Meyer)
Date: Thu, 17 Mar 2011 15:46:52 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <AANLkTimh2Kw3LuRk2V1hVgQ3J5OtMXXx+sU1OBOHtPPa@mail.gmail.com>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
	<4D8247B7.30102@oddbird.net>
	<AANLkTimh2Kw3LuRk2V1hVgQ3J5OtMXXx+sU1OBOHtPPa@mail.gmail.com>
Message-ID: <4D82652C.1000305@oddbird.net>

On 03/17/2011 01:53 PM, Jim Fulton wrote:
> It occurs to me that it would be nice if site.py could grow knowledge
> of whether -S was used and not automatically mutate the path if -S was
> used.  That would allow -S to work robustly without having to link
> anything or create a config file.

This seems entirely sensible to me (and quite easy to implement). I
can't imagine why anyone would be relying on being able to do "python
-S" and later import site and get the path modifications automatically,
so it seems pretty much like a straightforward bugfix. And the fix is a
one-liner. Issue filed with patch: http://bugs.python.org/issue11591

Carl

From carl at oddbird.net  Thu Mar 17 20:51:36 2011
From: carl at oddbird.net (Carl Meyer)
Date: Thu, 17 Mar 2011 15:51:36 -0400
Subject: [Distutils] pythonv, take two
In-Reply-To: <20110317151643.5295a1ca@neurotica>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<4D8172FB.3020109@oddbird.net>
	<20110316230446.23fae504@neurotica>	<4D8257FA.1060700@oddbird.net>
	<20110317151643.5295a1ca@neurotica>
Message-ID: <4D826648.8020605@oddbird.net>



On 03/17/2011 03:16 PM, Barry Warsaw wrote:
> I dunno.  I tend to think python-ideas are where ideas go to die. ;)  I think
> we have general consensus that this is a good idea, and we just need to hash
> out the details.  So I personally think python-dev is fine at this point.

Seems like it can't hurt to at least ping python-ideas about it (and
follow the documented process). I can be pretty clear that distutils-sig
already thinks it's a good idea in principle, there's no reason we have
to let it die in python-ideas. And who knows, someone there may have
some good ideas...

Carl

From jim at zope.com  Thu Mar 17 22:10:27 2011
From: jim at zope.com (Jim Fulton)
Date: Thu, 17 Mar 2011 17:10:27 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <4D82652C.1000305@oddbird.net>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
	<4D8247B7.30102@oddbird.net>
	<AANLkTimh2Kw3LuRk2V1hVgQ3J5OtMXXx+sU1OBOHtPPa@mail.gmail.com>
	<4D82652C.1000305@oddbird.net>
Message-ID: <AANLkTik_VaxSjXNO2ORAFLqS_Q05MifY8_z08O4EE5Kv@mail.gmail.com>

On Thu, Mar 17, 2011 at 3:46 PM, Carl Meyer <carl at oddbird.net> wrote:
> On 03/17/2011 01:53 PM, Jim Fulton wrote:
>> It occurs to me that it would be nice if site.py could grow knowledge
>> of whether -S was used and not automatically mutate the path if -S was
>> used. ?That would allow -S to work robustly without having to link
>> anything or create a config file.
>
> This seems entirely sensible to me (and quite easy to implement). I
> can't imagine why anyone would be relying on being able to do "python
> -S" and later import site and get the path modifications automatically,
> so it seems pretty much like a straightforward bugfix. And the fix is a
> one-liner. Issue filed with patch: http://bugs.python.org/issue11591

Thanks!

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From jim at zope.com  Thu Mar 17 22:13:56 2011
From: jim at zope.com (Jim Fulton)
Date: Thu, 17 Mar 2011 17:13:56 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <4D825B58.502@oddbird.net>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
	<4D8247B7.30102@oddbird.net>
	<20110317190105.2A6C03A4062@sparrow.telecommunity.com>
	<4D825B58.502@oddbird.net>
Message-ID: <AANLkTikXVgwEr0ueKdKY7iHtgwO5Fj89kNCO1_jMLcnm@mail.gmail.com>

On Thu, Mar 17, 2011 at 3:04 PM, Carl Meyer <carl at oddbird.net> wrote:
> On 03/17/2011 03:00 PM, P.J. Eby wrote:
>> At 01:41 PM 3/17/2011 -0400, Carl Meyer wrote:
>>> Actually, now that I come to think of it, pythonv (take two) does
>>> already cover your requirement. If you have a symlinked or copied python
>>> binary, and an empty pythonv.conf one directory up,
>>
>> Is there any reason why the configuration file has to be one directory
>> up, intead of adjacent to the executable?
>
> Not a particularly good one. I inherited this from the standard layout
> of virtualenv, which includes a bin/ directory for the python binary and
> scripts, and it made more sense to me in that layout for pythonv.conf to
> live at the root of the "virtualenv" rather than in bin/.
>
> I'd like to continue to support it being one level up, but I can't think
> of any reason not to first check adjacent, and then one level up. I
> suppose it could just keep checking up the tree, but that introduces a
> bunch more filesystem checks and could give surprising results. I won't
> do that unless someone steps up to give convincing arguments for it.

I suggest the following:

Look for argv[0]+'.pythonv' and then for '../pythonv.cfg'.

So if I've linked the Python executable to ./bin/clean, look for
./bin/clean.pythonv and ./pythonv.cfg.

Jim


-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From carl at oddbird.net  Thu Mar 17 22:33:54 2011
From: carl at oddbird.net (Carl Meyer)
Date: Thu, 17 Mar 2011 17:33:54 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <AANLkTikXVgwEr0ueKdKY7iHtgwO5Fj89kNCO1_jMLcnm@mail.gmail.com>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
	<4D8247B7.30102@oddbird.net>
	<20110317190105.2A6C03A4062@sparrow.telecommunity.com>
	<4D825B58.502@oddbird.net>
	<AANLkTikXVgwEr0ueKdKY7iHtgwO5Fj89kNCO1_jMLcnm@mail.gmail.com>
Message-ID: <4D827E42.2030004@oddbird.net>



On 03/17/2011 05:13 PM, Jim Fulton wrote:
> I suggest the following:
> 
> Look for argv[0]+'.pythonv' and then for '../pythonv.cfg'.
> 
> So if I've linked the Python executable to ./bin/clean, look for
> ./bin/clean.pythonv and ./pythonv.cfg.

Nice - I like the ability to have multiple interpreters side-by-side
with different pythonv configurations.

Is ".cfg" generally preferred to ".conf" for some good reason? I don't
personally care too much; the former is shorter but the latter looks
less ugly to me ;-)

And I kind of dislike the inconsistency in extension; would
"clean.pythonv.cfg" be acceptable?

To simplify documentation and allow more flexibility, I might just check
for all four: first the executable-specific one in both directories,
then the general one in both directories.

Carl

From benji at benjiyork.com  Thu Mar 17 22:45:02 2011
From: benji at benjiyork.com (Benji York)
Date: Thu, 17 Mar 2011 17:45:02 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <4D827E42.2030004@oddbird.net>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
	<4D8247B7.30102@oddbird.net>
	<20110317190105.2A6C03A4062@sparrow.telecommunity.com>
	<4D825B58.502@oddbird.net>
	<AANLkTikXVgwEr0ueKdKY7iHtgwO5Fj89kNCO1_jMLcnm@mail.gmail.com>
	<4D827E42.2030004@oddbird.net>
Message-ID: <AANLkTikq5B8YYRP-4mfhSos6Byw1mycUMA6x=3skPOJh@mail.gmail.com>

On Thu, Mar 17, 2011 at 5:33 PM, Carl Meyer <carl at oddbird.net> wrote:
>
>
> On 03/17/2011 05:13 PM, Jim Fulton wrote:
>> I suggest the following:
>>
>> Look for argv[0]+'.pythonv' and then for '../pythonv.cfg'.
>>
>> So if I've linked the Python executable to ./bin/clean, look for
>> ./bin/clean.pythonv and ./pythonv.cfg.
>
> Nice - I like the ability to have multiple interpreters side-by-side
> with different pythonv configurations.
>
> Is ".cfg" generally preferred to ".conf" for some good reason? I don't
> personally care too much; the former is shorter but the latter looks
> less ugly to me ;-)
>
> And I kind of dislike the inconsistency in extension; would
> "clean.pythonv.cfg" be acceptable?
>
> To simplify documentation and allow more flexibility, I might just check
> for all four: first the executable-specific one in both directories,
> then the general one in both directories.

Too many variants will get annoying.  Along those lines, if it can be in
multiple places please make having more than one an error.
-- 
Benji York

From fdrake at acm.org  Thu Mar 17 22:48:17 2011
From: fdrake at acm.org (Fred Drake)
Date: Thu, 17 Mar 2011 17:48:17 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <4D827E42.2030004@oddbird.net>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
	<4D8247B7.30102@oddbird.net>
	<20110317190105.2A6C03A4062@sparrow.telecommunity.com>
	<4D825B58.502@oddbird.net>
	<AANLkTikXVgwEr0ueKdKY7iHtgwO5Fj89kNCO1_jMLcnm@mail.gmail.com>
	<4D827E42.2030004@oddbird.net>
Message-ID: <AANLkTi=AXqu9davToVMDaUFZrvDgRj3NMmfMx4pJdxP3@mail.gmail.com>

On Thu, Mar 17, 2011 at 5:33 PM, Carl Meyer <carl at oddbird.net> wrote:
> Is ".cfg" generally preferred to ".conf" for some good reason? I don't
> personally care too much; the former is shorter but the latter looks
> less ugly to me ;-)

That all depends on who you ask; I tend to prefer .conf myself (but
only by a hair).  Jim likely has his own opinion.

> And I kind of dislike the inconsistency in extension; would
> "clean.pythonv.cfg" be acceptable?

I'd accept that, so long as we pick only one of .conf or .cfg.


? -Fred

--
Fred L. Drake, Jr.? ? <fdrake at acm.org>
"A storm broke loose in my mind."? --Albert Einstein

From pje at telecommunity.com  Thu Mar 17 23:08:11 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 17 Mar 2011 18:08:11 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <AANLkTikXVgwEr0ueKdKY7iHtgwO5Fj89kNCO1_jMLcnm@mail.gmail.c
 om>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
	<4D8247B7.30102@oddbird.net>
	<20110317190105.2A6C03A4062@sparrow.telecommunity.com>
	<4D825B58.502@oddbird.net>
	<AANLkTikXVgwEr0ueKdKY7iHtgwO5Fj89kNCO1_jMLcnm@mail.gmail.com>
Message-ID: <20110317220822.E838D3A4062@sparrow.telecommunity.com>

At 05:13 PM 3/17/2011 -0400, Jim Fulton wrote:
>I suggest the following:
>
>Look for argv[0]+'.pythonv' and then for '../pythonv.cfg'.
>
>So if I've linked the Python executable to ./bin/clean, look for
>./bin/clean.pythonv and ./pythonv.cfg.

And on Windows, presumably remove the .exe part?  or are you saying 
'python.exe.pythonv'?


From barry at python.org  Thu Mar 17 21:39:28 2011
From: barry at python.org (Barry Warsaw)
Date: Thu, 17 Mar 2011 16:39:28 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <4D823A81.2060406@oddbird.net>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
Message-ID: <20110317163928.73384de3@neurotica>

On Mar 17, 2011, at 12:44 PM, Carl Meyer wrote:

>Any opinions on the commandline UI for this? I was thinking of just
>adding a "pythonv.py" to the stdlib that you could execute with "python
>-m pythonv path/to/new/env" (and would also export appropriate API to
>create environments programmatically).

Bikeshedding time: how about something a little more descriptive?

$ python -m virtualize path/to/new/env

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110317/e71f34e7/attachment.pgp>

From carl at oddbird.net  Thu Mar 17 23:16:28 2011
From: carl at oddbird.net (Carl Meyer)
Date: Thu, 17 Mar 2011 18:16:28 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <20110317163928.73384de3@neurotica>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>	<20110317105948.4557667c@neurotica>
	<4D823A81.2060406@oddbird.net> <20110317163928.73384de3@neurotica>
Message-ID: <4D82883C.3080404@oddbird.net>

On 03/17/2011 04:39 PM, Barry Warsaw wrote:
> Bikeshedding time: how about something a little more descriptive?
> 
> $ python -m virtualize path/to/new/env

Ok, that color will do nicely.

Carl

From barry at python.org  Thu Mar 17 21:48:38 2011
From: barry at python.org (Barry Warsaw)
Date: Thu, 17 Mar 2011 16:48:38 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <4D827E42.2030004@oddbird.net>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
	<4D8247B7.30102@oddbird.net>
	<20110317190105.2A6C03A4062@sparrow.telecommunity.com>
	<4D825B58.502@oddbird.net>
	<AANLkTikXVgwEr0ueKdKY7iHtgwO5Fj89kNCO1_jMLcnm@mail.gmail.com>
	<4D827E42.2030004@oddbird.net>
Message-ID: <20110317164838.3297aa92@neurotica>

On Mar 17, 2011, at 05:33 PM, Carl Meyer wrote:

>On 03/17/2011 05:13 PM, Jim Fulton wrote:
>> I suggest the following:
>> 
>> Look for argv[0]+'.pythonv' and then for '../pythonv.cfg'.
>> 
>> So if I've linked the Python executable to ./bin/clean, look for
>> ./bin/clean.pythonv and ./pythonv.cfg.
>
>Nice - I like the ability to have multiple interpreters side-by-side
>with different pythonv configurations.

Indeed.  In that case though, wouldn't ./bin/clean.cfg be fine?  Even if the
executable were named ./bin/python a sibling ./bin/python.cfg would be fine
too I think.  I agree with Carl that I dislike the inconsistency in the
extension, but I think argv[0] + '.cfg' would be fine.

Having a non-.cfg or -.conf extension makes it harder to do stuff like
automatically set the major mode in Emacs (although I guess it wouldn't be
that hard to add a mapping for *.pythonv).

>Is ".cfg" generally preferred to ".conf" for some good reason? I don't
>personally care too much; the former is shorter but the latter looks
>less ugly to me ;-)

I guess I'm on the .cfg side, but wouldn't care too much if .conf is the
consensus.

>And I kind of dislike the inconsistency in extension; would
>"clean.pythonv.cfg" be acceptable?

I'm not sure you need the .pythonv. part.

>To simplify documentation and allow more flexibility, I might just check
>for all four: first the executable-specific one in both directories,
>then the general one in both directories.

Definitely add a --verbose option to print out exactly which one got
chosen. :)

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110317/0186c9fe/attachment.pgp>

From carl at oddbird.net  Fri Mar 18 00:53:37 2011
From: carl at oddbird.net (Carl Meyer)
Date: Thu, 17 Mar 2011 19:53:37 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <20110317220822.E838D3A4062@sparrow.telecommunity.com>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
	<4D8247B7.30102@oddbird.net>
	<20110317190105.2A6C03A4062@sparrow.telecommunity.com>
	<4D825B58.502@oddbird.net>
	<AANLkTikXVgwEr0ueKdKY7iHtgwO5Fj89kNCO1_jMLcnm@mail.gmail.com>
	<20110317220822.E838D3A4062@sparrow.telecommunity.com>
Message-ID: <4D829F01.7070204@oddbird.net>

FWIW, I've pushed a reworking of the config-file-finding, with the
following shed-paint color choices:

* File is named <executable-name>.virtual.conf (I kept the .virtual,
despite it being longer and not strictly necessary, because I think it
more clearly expresses the function of the file. The existence or
absence of this file changes Python's behavior significantly, so I think
its name should be quite explicit.)

* <executable-name> has the extension stripped on Windows, but not
otherwise.

* Config file can be located adjacent to the binary or one level up. The
default sys.prefix is always the directory in which the configuration
file is found, but can be overridden with a new "prefix" setting in the
config file (which can be a path relative to the config file's location).

* I eliminated any form of generically-named config file, as I think the
extra possible names (and thus possible sources of confusion) is not
really worth the benefit. The only use case I can think of is if you
have multiple python binaries or symlinks next to each other and want
them all to use the same virtual config - and I can't really think why
you'd want multiple binaries in that case.

Substantive comments on these choices is fine, especially in the form of
presenting real use-cases I'm not currently allowing for.

Testing that reveals a flaw in the basic operation of this style of
virtual environment is much more useful. The bikeshed colors are easy to
change, but I want to be really sure this thing actually does what it's
supposed to reliably before I go presenting a PEP for it.

Thanks!

Carl

From vinay_sajip at yahoo.co.uk  Fri Mar 18 01:03:06 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 18 Mar 2011 00:03:06 +0000 (UTC)
Subject: [Distutils] pythonv, take two
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<4D8172FB.3020109@oddbird.net>
	<loom.20110317T112246-771@post.gmane.org>
	<4D8233EF.7070205@oddbird.net>
Message-ID: <loom.20110318T004329-895@post.gmane.org>

Carl Meyer <carl <at> oddbird.net> writes:

> This is odd, as the same setup works fine for me with a copied binary
> (I'm on Ubuntu 10.10). If you're willing to do any more debugging on
> this, here's what would be helpful:

I think I know what the problem is: the python executable checks to see where it
was run from. If it looks as if it was run from a source build, it looks for
site.py in the Lib folder relative to the executable; otherwise it looks for
site.py in sys.prefix/lib/pythonX.Y. I've copied ~/tools/cpythonv/python to
~/projects/vptest/bin, and here's what happens:

vinay at eta-natty:~/projects/vptest/bin$ ~/tools/cpythonv/python
Python 3.3a0 (pythonv2:e56f05883ceb, Mar 17 2011, 09:07:29) 
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys; sys.prefix
'/usr/local'
>>> sys.path
['', '/usr/local/lib/python33.zip', '/home/vinay/tools/cpythonv/Lib',
'/home/vinay/tools/cpythonv/Lib/plat-linux2',
'/home/vinay/tools/cpythonv/build/lib.linux-i686-3.3',
'/usr/local/lib/python3.3/site-packages']
>>> 
vinay at eta-natty:~/projects/vptest/bin$ ./python
Python 3.3a0 (pythonv2:e56f05883ceb, Mar 17 2011, 09:07:29) 
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys; sys.prefix
'/usr/local'
>>> sys.path
['', '/usr/local/lib/python33.zip', '/usr/local/lib/python3.3',
'/usr/local/lib/python3.3/plat-linux2', '/usr/local/lib/python3.3/lib-dynload',
'/usr/local/lib/python3.3/site-packages']
>>> 

Notice how the sys.prefix is the same in both cases, but the path is different.
Your site.py is never run with the copy, because it gets the site.py from
/usr/local/lib/python3.3:

vinay at eta-natty:~/projects/vptest/bin$ ~/tools/cpythonv/python
Python 3.3a0 (pythonv2:e56f05883ceb, Mar 17 2011, 09:07:29) 
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import site; site.__file__
'/home/vinay/tools/cpythonv/Lib/site.py'
>>> 
vinay at eta-natty:~/projects/vptest/bin$ ./python
Python 3.3a0 (pythonv2:e56f05883ceb, Mar 17 2011, 09:07:29) 
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import site; site.__file__
'/usr/local/lib/python3.3/site.py'
>>> 

I suppose this won't be a problem when the actual Python 3.3 is installed - the
virtualizing site.py will be in /usr/lib/python3.3 or /usr/local/lib/python3.3.
On this system I have a local build of Python made from the official default
branch, which is installed to /usr/local/lib since the system Python is 3.2
(Ubuntu 11.04 - Natty).

Regards,

Vinay Sajip



From vinay_sajip at yahoo.co.uk  Fri Mar 18 01:09:18 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 18 Mar 2011 00:09:18 +0000 (UTC)
Subject: [Distutils] pythonv, take two
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<4D8172FB.3020109@oddbird.net>
	<loom.20110317T112246-771@post.gmane.org>
	<4D8233EF.7070205@oddbird.net>
	<loom.20110318T004329-895@post.gmane.org>
Message-ID: <loom.20110318T010738-292@post.gmane.org>

Vinay Sajip <vinay_sajip <at> yahoo.co.uk> writes:

> On this system I have a local build of Python made from the official default
> branch, which is installed to /usr/local/lib since the system Python is 3.2
> (Ubuntu 11.04 - Natty).

Sorry, I meant to say that the system Python is 2.7 (in /usr). I also have an
official Ubuntu build of 3.2 in /usr, but the source build installs by default
to /usr/local.

Regards,

Vinay Sajip


From pje at telecommunity.com  Fri Mar 18 01:35:26 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 17 Mar 2011 20:35:26 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <4D829F01.7070204@oddbird.net>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
	<4D8247B7.30102@oddbird.net>
	<20110317190105.2A6C03A4062@sparrow.telecommunity.com>
	<4D825B58.502@oddbird.net>
	<AANLkTikXVgwEr0ueKdKY7iHtgwO5Fj89kNCO1_jMLcnm@mail.gmail.com>
	<20110317220822.E838D3A4062@sparrow.telecommunity.com>
	<4D829F01.7070204@oddbird.net>
Message-ID: <20110318003534.73D3F3A4062@sparrow.telecommunity.com>

At 07:53 PM 3/17/2011 -0400, Carl Meyer wrote:
>FWIW, I've pushed a reworking of the config-file-finding, with the
>following shed-paint color choices:
>
>* File is named <executable-name>.virtual.conf (I kept the .virtual,
>despite it being longer and not strictly necessary, because I think it
>more clearly expresses the function of the file. The existence or
>absence of this file changes Python's behavior significantly, so I think
>its name should be quite explicit.)
>
>* <executable-name> has the extension stripped on Windows, but not
>otherwise.

This combination should work well on Windows, where the default 
explorer settings will show e.g. 'python.virtual' next to 'python' in 
the directory.  (Windows strips suffixes from display by default, and 
moves them to a "type" column.)


>* I eliminated any form of generically-named config file, as I think the
>extra possible names (and thus possible sources of confusion) is not
>really worth the benefit. The only use case I can think of is if you
>have multiple python binaries or symlinks next to each other and want
>them all to use the same virtual config - and I can't really think why
>you'd want multiple binaries in that case.

The main reason I'd use differently-named binaries would be if I were 
shipping multiple runnable applications that I wanted to look to 
users like true .exe's.  I don't see a reason why I wouldn't use 
separate .virtual.conf files, though, especially if their contents are minimal.

(Awesomeness bonus: if the executable put *itself* on sys.path, and 
ran __main__, you could simply tack a zipfile on the end of the .exe 
and have a ready-to-run application.)



From barry at python.org  Fri Mar 18 02:33:12 2011
From: barry at python.org (Barry Warsaw)
Date: Thu, 17 Mar 2011 21:33:12 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <4D829F01.7070204@oddbird.net>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
	<4D8247B7.30102@oddbird.net>
	<20110317190105.2A6C03A4062@sparrow.telecommunity.com>
	<4D825B58.502@oddbird.net>
	<AANLkTikXVgwEr0ueKdKY7iHtgwO5Fj89kNCO1_jMLcnm@mail.gmail.com>
	<20110317220822.E838D3A4062@sparrow.telecommunity.com>
	<4D829F01.7070204@oddbird.net>
Message-ID: <20110317213312.1ed4b8d9@neurotica>

Sounds good to me.

On Mar 17, 2011, at 07:53 PM, Carl Meyer wrote:

>* <executable-name> has the extension stripped on Windows, but not
>otherwise.

It should probably also have the extension stripped on OS X too.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110317/21d22dfe/attachment.pgp>

From thomas at thomas-lotze.de  Fri Mar 18 13:39:26 2011
From: thomas at thomas-lotze.de (Thomas Lotze)
Date: Fri, 18 Mar 2011 13:39:26 +0100
Subject: [Distutils] New buildout options: checksums
	and	allow-omitted-checksums
References: <pan.2011.03.17.15.55.03.959118@ID-174572.user.uni-berlin.de>
	<AANLkTinXwjn8-K=nu9KY37fsf54qZR1ypJtmB8L_tszg@mail.gmail.com>
Message-ID: <pan.2011.03.18.12.39.25.813092@ID-174572.user.uni-berlin.de>

Leonardo Rochael Almeida wrote:

> I like your idea in general. I'd like to point to the following suggestion
> with patch+test (though it might need some cleanup) that is not exactly
> related to what you're proposing, but has to do with the same thing
> (relationship between files and checksums):
> 
> https://bugs.launchpad.net/zc.buildout/+bug/692600
> 
> It suggests (and implements) automatically redownloading files when
> checksums don't match (which could happen when you update your config file
> with a new checksum for a file that has changed upstream).

Thank you for the pointer, I'll look into it. Fixing this along with other
download-related things before the next buildout release would be a good
thing, I think.

>> - "checksums": This option would name a config section that ?contains
>> checksums for any number of resources by URL. I ?suggest a default
>> value of "checksum" for it.
> 
> Won't 'checksums' (plural) as the value be better? It would keep with the
> tradition of matching the name of the buildout option and the name of the
> section by default.

Sure, I meant the section name to read "checksums" but a typo crept in.

> I suggest mapping the checksums (which are valid keys) to URLs, and having
> a special key with line-break separated values for explicitly not
> checking:

This would be ugly in my humble opinion because it reverses the meaning of
keys and values and makes the configuration look backwards for a purely
technical reason. Also, the following:

> This will not be so elegant when you want to extend another configuration
> and override some decisions, but it works somewhat:
> 
>   [buildout]
>   extends = config-file-above.cfg
> 
>   [checksums]
>   080d2c6a849ebd6b7fd49049c21b910a =
>   no-check += http://example.com/foo/bar.tgz

becomes even worse when you want to update a checksum specified in a
configuration file that is being extended: you'd have to keep track of two
checksums for each resource, once to invalidate the old one and once to
specify the new one.

Thank you for your input, though!

-- 
Thomas




From marius at pov.lt  Fri Mar 18 14:07:10 2011
From: marius at pov.lt (Marius Gedminas)
Date: Fri, 18 Mar 2011 15:07:10 +0200
Subject: [Distutils] New buildout options: checksums and
 allow-omitted-checksums
In-Reply-To: <pan.2011.03.17.15.55.03.959118@ID-174572.user.uni-berlin.de>
References: <pan.2011.03.17.15.55.03.959118@ID-174572.user.uni-berlin.de>
Message-ID: <20110318130710.GA28064@fridge.pov.lt>

On Thu, Mar 17, 2011 at 04:55:05PM +0100, Thomas Lotze wrote:
> two weeks ago I asked about your opinions on a buildout option
> that enforces specifying (MD5) checksums for all files downloaded
> through buildout's download utility API.

Please don't hardcode the checksum algorithm to MD5.  Security
researchers have been telling everyone to stop using MD5 (and SHA1)
for a while now.

Marius Gedminas
-- 
How much net work could a network work, if a network could net work?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 190 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110318/77592b95/attachment.pgp>

From thomas at thomas-lotze.de  Fri Mar 18 14:38:31 2011
From: thomas at thomas-lotze.de (Thomas Lotze)
Date: Fri, 18 Mar 2011 14:38:31 +0100
Subject: [Distutils] New buildout options: checksums
	and	allow-omitted-checksums
References: <pan.2011.03.17.15.55.03.959118@ID-174572.user.uni-berlin.de>
Message-ID: <pan.2011.03.18.13.38.30.371976@ID-174572.user.uni-berlin.de>

Thomas Lotze wrote:

> In analogy with version pinning for eggs, two new options could be
> introduced to the buildout section:
> 
> - "checksums": This option would name a config section that
>   contains checksums for any number of resources by URL. I suggest a
>   default value of "checksum" for it. Listing some URL with an empty
>   checksum would explicitly express that the checksum for this resource
>   should never be checked. I'm not sure how to structure the contents of
>   the checksums section: URLs are not valid config keys in general (they
>   can contain "=" characters) but I'd still like to be able to rely on the
>   existing mechanism for overriding single options to override single
>   checksums. Mapping arbitrary keys to whitespace-separated pairs of URL
>   and checksum would work but sounds inelegant.

OTOH, thinking further about a line format like "name = url md5sum", I
find more advantages than just the fact that it would be syntactically
valid: Recipes such as zc.recipe.cmmi or gocept.download could reference
the resource name instead of the url, and in analogy to zc.recipe.egg,
they might even infer the resource name from the section name:


[checksums]
foo = http://example.org/foo.tgz 96772abbcb3331f63d05ffba40b7b523
bar = http://example.org/bar.tgz 64d714a998cab0c45c48307698317a0f
baz = http://example.org/baz.tgz 22bfb8c1dd94b5f3813a2b25da67463f

[install-foo-by-url]
recipe = zc.recipe.cmmi
url = http://example.org/foo.tgz

[install-bar-by-name]
recipe = zc.recipe.cmmi
source = bar

[baz]
recipe = zc.recipe.cmmi


It's possibly worth some amount of bike-shedding what the concept of the
resource should really be called: For a cmmi recipe, "source" seems to
work best as a key while a more general download recipe might indeed use
the word "resource", and there's also the question of whether and how to
reference extended configuration files by resource name. But my first
question is whether this whole idea makes sense to other people at all or
whether it adds more abstraction than it is worth.

-- 
Thomas




From thomas at thomas-lotze.de  Fri Mar 18 14:43:08 2011
From: thomas at thomas-lotze.de (Thomas Lotze)
Date: Fri, 18 Mar 2011 14:43:08 +0100
Subject: [Distutils] New buildout options: checksums and
	allow-omitted-checksums
References: <pan.2011.03.17.15.55.03.959118@ID-174572.user.uni-berlin.de>
	<20110318130710.GA28064@fridge.pov.lt>
Message-ID: <pan.2011.03.18.13.43.04.450362@ID-174572.user.uni-berlin.de>

Marius Gedminas wrote:

> Please don't hardcode the checksum algorithm to MD5.  Security researchers
> have been telling everyone to stop using MD5 (and SHA1) for a while now.

Good point. All this talking about MD5 specifically has been due to the
fact that this is what used to be used by the download API and the
gocep.download recipe so far. To take up the idea I posted a few minutes
ago, one might specify checksums like this:

[checksums]
foo = http://example.org/foo.tgz algorithm:checksum-value

Since the checksum would be evaluated by the download API itself, many
checksum algorithms could be used since adding another algorithm in this
one place would add it consistently to all pieces of buildout and recipes
that download things.

-- 
Thomas




From benji at benjiyork.com  Fri Mar 18 14:47:50 2011
From: benji at benjiyork.com (Benji York)
Date: Fri, 18 Mar 2011 09:47:50 -0400
Subject: [Distutils] New buildout options: checksums and
	allow-omitted-checksums
In-Reply-To: <pan.2011.03.18.13.43.04.450362@ID-174572.user.uni-berlin.de>
References: <pan.2011.03.17.15.55.03.959118@ID-174572.user.uni-berlin.de>
	<20110318130710.GA28064@fridge.pov.lt>
	<pan.2011.03.18.13.43.04.450362@ID-174572.user.uni-berlin.de>
Message-ID: <AANLkTikLBmYji8mA=F9mBXBt+i5CzAsT73VwW6ejkM9v@mail.gmail.com>

On Fri, Mar 18, 2011 at 9:43 AM, Thomas Lotze <thomas at thomas-lotze.de> wrote:
> Marius Gedminas wrote:
>
>> Please don't hardcode the checksum algorithm to MD5. ?Security researchers
>> have been telling everyone to stop using MD5 (and SHA1) for a while now.
>
> Good point. All this talking about MD5 specifically has been due to the
> fact that this is what used to be used by the download API and the
> gocep.download recipe so far. To take up the idea I posted a few minutes
> ago, one might specify checksums like this:
>
> [checksums]
> foo = http://example.org/foo.tgz algorithm:checksum-value

+1

-- 
Benji York

From carl at oddbird.net  Fri Mar 18 15:37:22 2011
From: carl at oddbird.net (Carl Meyer)
Date: Fri, 18 Mar 2011 10:37:22 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <20110317213312.1ed4b8d9@neurotica>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>	<20110317105948.4557667c@neurotica>
	<4D823A81.2060406@oddbird.net>	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>	<4D8247B7.30102@oddbird.net>	<20110317190105.2A6C03A4062@sparrow.telecommunity.com>	<4D825B58.502@oddbird.net>	<AANLkTikXVgwEr0ueKdKY7iHtgwO5Fj89kNCO1_jMLcnm@mail.gmail.com>	<20110317220822.E838D3A4062@sparrow.telecommunity.com>	<4D829F01.7070204@oddbird.net>
	<20110317213312.1ed4b8d9@neurotica>
Message-ID: <4D836E22.3030306@oddbird.net>

On 03/17/2011 09:33 PM, Barry Warsaw wrote:
> On Mar 17, 2011, at 07:53 PM, Carl Meyer wrote:
>> * <executable-name> has the extension stripped on Windows, but not
>> otherwise.
> 
> It should probably also have the extension stripped on OS X too.

Hmm, really? What extension does the executable typically have on OS X
that ought to be stripped? I don't use OS X or have access to it, so
happy to defer to those who know better, but when I've worked on others'
OS X machines I think the binary has usually been just "python". I
figured OS X was in the same boat as Linux, that if you've named your
symlink python.foo you probably did it intentionally to distinguish it
from another binary/symlink, and that shouldn't be stripped.

Carl

From jim at zope.com  Fri Mar 18 15:38:56 2011
From: jim at zope.com (Jim Fulton)
Date: Fri, 18 Mar 2011 10:38:56 -0400
Subject: [Distutils] Buildout Sprint Report
Message-ID: <AANLkTinMD8T98uTSxGwmuuZnScFY2eFHFQ=iCLYL=oAv@mail.gmail.com>

We did some sprinting on buildout at PyCon.

Alex Plugaru, fixed serveral bugs and clarified the development
process a little.

We had some good discussions about how buildout achieves isolation
that I expect will lead to simplification in the future.

I researched some test failures and ultimately had to disable some
tests except on windows and newer linux kernels.  The tests rely on
being able to name a script on a shebang line.

I worked on porting buildout to Python 3. My plan:

0. My goal is to have one code base that works in both Python 2 and
   Python 3.

1. Run 2to3 and get tests passing.  I spent the last 4+ days working
   on this.  There's a lot that 2to3 doesn't take care of.  I think
   I'm getting pretty close and will be able to finish the work this
   weekend.

   Lennart Regebro provided valuable advice and moral support through
   this process. :)

2. Review the diffs and adjust to make buildout work under Python 2&3.
   Also look for stupidities committed in the heat of hacking.  In
   particular, I used encode/decode calls in places as a band-aid to
   overcome mistakes in doing IO properly.

3. Make sure tests pass under both Python 2&3 and make a beta release.

Note that I'm reliying on distribute because:

- distutils2 aka packaging isn't ready (and when it is, it won't
  provide all of the features I need),

- setuptools isn't ported to Python 3 yet.

I'm sadly, but sorely, tempted to drop explicit support for
setuptools.  Maintaining support for both implementations is feeling
like a waste of time.

Jim

--
Jim Fulton
http://www.linkedin.com/in/jimfulton

From carl at oddbird.net  Fri Mar 18 15:52:31 2011
From: carl at oddbird.net (Carl Meyer)
Date: Fri, 18 Mar 2011 10:52:31 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <20110318003534.73D3F3A4062@sparrow.telecommunity.com>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
	<4D8247B7.30102@oddbird.net>
	<20110317190105.2A6C03A4062@sparrow.telecommunity.com>
	<4D825B58.502@oddbird.net>
	<AANLkTikXVgwEr0ueKdKY7iHtgwO5Fj89kNCO1_jMLcnm@mail.gmail.com>
	<20110317220822.E838D3A4062@sparrow.telecommunity.com>
	<4D829F01.7070204@oddbird.net>
	<20110318003534.73D3F3A4062@sparrow.telecommunity.com>
Message-ID: <4D8371AF.1070904@oddbird.net>



On 03/17/2011 08:35 PM, P.J. Eby wrote:
>> * I eliminated any form of generically-named config file, as I think the
>> extra possible names (and thus possible sources of confusion) is not
>> really worth the benefit. The only use case I can think of is if you
>> have multiple python binaries or symlinks next to each other and want
>> them all to use the same virtual config - and I can't really think why
>> you'd want multiple binaries in that case.
> 
> The main reason I'd use differently-named binaries would be if I were
> shipping multiple runnable applications that I wanted to look to users
> like true .exe's.  I don't see a reason why I wouldn't use separate
> .virtual.conf files, though, especially if their contents are minimal.

I've actually already backpedaled on this one as I considered it
overnight. For the virtualenv-style use case, you could easily end up
with e.g. "python" and "python-3.3" in your bin/ dir, and want them both
to reliably run in the virtualenv. So I think a fallback to
"python.virtual.conf" as a catchall is necessary after all.

> (Awesomeness bonus: if the executable put *itself* on sys.path, and ran
> __main__, you could simply tack a zipfile on the end of the .exe and
> have a ready-to-run application.)

Brandon Craig Rhodes was talking up that same idea (or something quite
similar) at PyCon. Scope creep for this PEP, I think, but interesting.

Carl

From jim at zope.com  Fri Mar 18 16:04:34 2011
From: jim at zope.com (Jim Fulton)
Date: Fri, 18 Mar 2011 11:04:34 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <4D827E42.2030004@oddbird.net>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
	<4D8247B7.30102@oddbird.net>
	<20110317190105.2A6C03A4062@sparrow.telecommunity.com>
	<4D825B58.502@oddbird.net>
	<AANLkTikXVgwEr0ueKdKY7iHtgwO5Fj89kNCO1_jMLcnm@mail.gmail.com>
	<4D827E42.2030004@oddbird.net>
Message-ID: <AANLkTim8=JTdDUS92JAeK16a80rxmDove9tcr4MzQy2i@mail.gmail.com>

On Thu, Mar 17, 2011 at 5:33 PM, Carl Meyer <carl at oddbird.net> wrote:
>
>
> On 03/17/2011 05:13 PM, Jim Fulton wrote:
>> I suggest the following:
>>
>> Look for argv[0]+'.pythonv' and then for '../pythonv.cfg'.
>>
>> So if I've linked the Python executable to ./bin/clean, look for
>> ./bin/clean.pythonv and ./pythonv.cfg.
>
> Nice - I like the ability to have multiple interpreters side-by-side
> with different pythonv configurations.
>
> Is ".cfg" generally preferred to ".conf" for some good reason?

I dunno.

> I don't
> personally care too much; the former is shorter but the latter looks
> less ugly to me ;-)

I don't care either.

> And I kind of dislike the inconsistency in extension; would
> "clean.pythonv.cfg" be acceptable?

Anything's acceptable. :)  The main idea is to base the config file name
on the executable name.

> To simplify documentation and allow more flexibility, I might just check
> for all four: first the executable-specific one in both directories,
> then the general one in both directories.

<shrug>  I don't think that's necessary.

Jim


-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From jim at zope.com  Fri Mar 18 16:05:57 2011
From: jim at zope.com (Jim Fulton)
Date: Fri, 18 Mar 2011 11:05:57 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <20110317220822.E838D3A4062@sparrow.telecommunity.com>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
	<4D8247B7.30102@oddbird.net>
	<20110317190105.2A6C03A4062@sparrow.telecommunity.com>
	<4D825B58.502@oddbird.net>
	<AANLkTikXVgwEr0ueKdKY7iHtgwO5Fj89kNCO1_jMLcnm@mail.gmail.com>
	<20110317220822.E838D3A4062@sparrow.telecommunity.com>
Message-ID: <AANLkTimT7_SwwEyWBu_Ssp0qLKd9yv0-SXH02sZKX525@mail.gmail.com>

On Thu, Mar 17, 2011 at 6:08 PM, P.J. Eby <pje at telecommunity.com> wrote:
> At 05:13 PM 3/17/2011 -0400, Jim Fulton wrote:
>>
>> I suggest the following:
>>
>> Look for argv[0]+'.pythonv' and then for '../pythonv.cfg'.
>>
>> So if I've linked the Python executable to ./bin/clean, look for
>> ./bin/clean.pythonv and ./pythonv.cfg.
>
> And on Windows, presumably remove the .exe part? ?or are you saying
> 'python.exe.pythonv'?

I wasn't thinking that hard. Yes, remove the .exe part.

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From jim at zope.com  Fri Mar 18 16:09:38 2011
From: jim at zope.com (Jim Fulton)
Date: Fri, 18 Mar 2011 11:09:38 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <4D836E22.3030306@oddbird.net>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
	<4D8247B7.30102@oddbird.net>
	<20110317190105.2A6C03A4062@sparrow.telecommunity.com>
	<4D825B58.502@oddbird.net>
	<AANLkTikXVgwEr0ueKdKY7iHtgwO5Fj89kNCO1_jMLcnm@mail.gmail.com>
	<20110317220822.E838D3A4062@sparrow.telecommunity.com>
	<4D829F01.7070204@oddbird.net>
	<20110317213312.1ed4b8d9@neurotica> <4D836E22.3030306@oddbird.net>
Message-ID: <AANLkTikCB2=Xg18qyBXOegHhMD_oQ8zj92ts=xUc4A39@mail.gmail.com>

On Fri, Mar 18, 2011 at 10:37 AM, Carl Meyer <carl at oddbird.net> wrote:
> On 03/17/2011 09:33 PM, Barry Warsaw wrote:
>> On Mar 17, 2011, at 07:53 PM, Carl Meyer wrote:
>>> * <executable-name> has the extension stripped on Windows, but not
>>> otherwise.
>>
>> It should probably also have the extension stripped on OS X too.
>
> Hmm, really? What extension does the executable typically have on OS X
> that ought to be stripped?

None. Barry's been using Ubuntu too long and has forgotten that Mac OS
X is Unix. :)

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From fdrake at acm.org  Fri Mar 18 16:18:02 2011
From: fdrake at acm.org (Fred Drake)
Date: Fri, 18 Mar 2011 11:18:02 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <AANLkTikCB2=Xg18qyBXOegHhMD_oQ8zj92ts=xUc4A39@mail.gmail.com>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
	<4D8247B7.30102@oddbird.net>
	<20110317190105.2A6C03A4062@sparrow.telecommunity.com>
	<4D825B58.502@oddbird.net>
	<AANLkTikXVgwEr0ueKdKY7iHtgwO5Fj89kNCO1_jMLcnm@mail.gmail.com>
	<20110317220822.E838D3A4062@sparrow.telecommunity.com>
	<4D829F01.7070204@oddbird.net>
	<20110317213312.1ed4b8d9@neurotica> <4D836E22.3030306@oddbird.net>
	<AANLkTikCB2=Xg18qyBXOegHhMD_oQ8zj92ts=xUc4A39@mail.gmail.com>
Message-ID: <AANLkTimgJCEPobx=NYcUuK2Dm8OfWYaTiutiFKZZJQrx@mail.gmail.com>

On Fri, Mar 18, 2011 at 11:09 AM, Jim Fulton <jim at zope.com> wrote:
> None. Barry's been using Ubuntu too long and has forgotten that Mac OS
> X is Unix. :)

There's also the fact that Python on Mac OS X builds with a .exe
extension; I don't recall it getting installed that way, though.  On a
case-senseless filesystem, distinguishing the "python" executable from
the "Python" directory in the sources requires that unfortunate
extension.


? -Fred

--
Fred L. Drake, Jr.? ? <fdrake at acm.org>
"A storm broke loose in my mind."? --Albert Einstein

From jim at zope.com  Fri Mar 18 16:35:58 2011
From: jim at zope.com (Jim Fulton)
Date: Fri, 18 Mar 2011 11:35:58 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <AANLkTimgJCEPobx=NYcUuK2Dm8OfWYaTiutiFKZZJQrx@mail.gmail.com>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
	<4D8247B7.30102@oddbird.net>
	<20110317190105.2A6C03A4062@sparrow.telecommunity.com>
	<4D825B58.502@oddbird.net>
	<AANLkTikXVgwEr0ueKdKY7iHtgwO5Fj89kNCO1_jMLcnm@mail.gmail.com>
	<20110317220822.E838D3A4062@sparrow.telecommunity.com>
	<4D829F01.7070204@oddbird.net>
	<20110317213312.1ed4b8d9@neurotica> <4D836E22.3030306@oddbird.net>
	<AANLkTikCB2=Xg18qyBXOegHhMD_oQ8zj92ts=xUc4A39@mail.gmail.com>
	<AANLkTimgJCEPobx=NYcUuK2Dm8OfWYaTiutiFKZZJQrx@mail.gmail.com>
Message-ID: <AANLkTimdj-hZHOC196Ny+YCrxTROGFdWn7W-hX53+V6W@mail.gmail.com>

On Fri, Mar 18, 2011 at 11:18 AM, Fred Drake <fdrake at acm.org> wrote:
> On Fri, Mar 18, 2011 at 11:09 AM, Jim Fulton <jim at zope.com> wrote:
>> None. Barry's been using Ubuntu too long and has forgotten that Mac OS
>> X is Unix. :)
>
> There's also the fact that Python on Mac OS X builds with a .exe
> extension;

Oh wow. In the build directory, it does have a ".exe" extension.

> I don't recall it getting installed that way, though.

Right. It's installed without the extension.

> ?On a
> case-senseless filesystem, distinguishing the "python" executable from
> the "Python" directory in the sources requires that unfortunate
> extension.

So sad.

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From sienkiew at stsci.edu  Fri Mar 18 18:46:26 2011
From: sienkiew at stsci.edu (Mark Sienkiewicz)
Date: Fri, 18 Mar 2011 13:46:26 -0400
Subject: [Distutils] reservations about pythonv
In-Reply-To: <20110316223904.288080ba@neurotica>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>
	<20110316223904.288080ba@neurotica>
Message-ID: <4D839A72.1030705@stsci.edu>


> The above just means my memory is too faulty to of much use ;).  I'll just
> echo Carl's request for specific cases where $LD_LIBRARY_PATH needs to be set.
>   

Here is a case that might resemble what you are talking about:

Compile a C extension that requires a shared library that is not in the 
standard system path.  To import it, LD_LIBRARY_PATH needs to be right.

This is not really different from what happens in a compiled language, 
except in one way:  In C, I can compile it -static or I can give the 
full path to the .so file.  Either results in a thing that works without 
LD_LIBRARY_PATH.

With distutils, you can't.  It goes to great lengths to ensure that you 
can only compile a C extension with "cc ... -L/some/directory -lname" -- 
I can't find any way to make it do "cc ... /some/directory/libname.so"

So, the real problem here is that distutils uses "cc -L", but it 
demonstrates a case where LD_LIBRARY_PATH can be important to a python 
program even when Python itself can run without it.



From carl at oddbird.net  Fri Mar 18 19:44:23 2011
From: carl at oddbird.net (Carl Meyer)
Date: Fri, 18 Mar 2011 14:44:23 -0400
Subject: [Distutils] pythonv, take two
In-Reply-To: <loom.20110318T004329-895@post.gmane.org>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<4D8172FB.3020109@oddbird.net>	<loom.20110317T112246-771@post.gmane.org>	<4D8233EF.7070205@oddbird.net>
	<loom.20110318T004329-895@post.gmane.org>
Message-ID: <4D83A807.30003@oddbird.net>

Hi Vinay,

Thanks for the additional digging in here. I think your analysis is
right - it actually occurred to me yesterday that this could be the
problem, and I filed a bug to track it here:
https://bitbucket.org/carljm/cpythonv/issue/6/if-binary-is-copied-prefix-finding-could

The issue is finding the initial sys.prefix and thus the standard
library, which happens in the search_for_prefix function
Modules/getpath.c. The algorithm used is this (presuming no PYTHONHOME):

1. Step up the tree from the (symlinks-dereferenced) binary location
looking for anything that appears to be the standard library.

2. Fall back to the location hardcoded in the binary from the --prefix
option to ./configure.

With a symlink it's fully reliable, because the symlink is dereferenced
so the stdlib is found just as it normally would be.

With a copy of the binary, there are two situations where it can break:

A. The Python installation we are copying from has an incorrect
hardcoded prefix in the binary, and is relying on the tree-stepping to
find its standard library. In this case, the hardcoded prefix will be
used because the tree-stepping won't find anything. I think this is the
situation you are seeing: I think your binary is compiling with a prefix
of "/usr/local", and so it falls back on that prefix when it can't find
anything by tree-stepping up through /home. If you didn't actually have
a Python 3.3 installation in /usr/local, it would break more loudly due
to not finding any stdlib at all. I think you'll find that it works for
you if you compile with an explicit --prefix and then actually install
to that prefix, then copy that binary and make an env with it.

B. There is _another_ Python standard library (from the same version of
Python, on *nix - the Windows stdlib isn't in a version-specific
directory) up the hierarchy from the location of our virtualenv. In this
case, tree-stepping might find the wrong standard library and use it. I
think this case should be quite rare. It's most problematic on Windows,
but that's also where it shouldn't really happen, since Python is
installed self-contained and I can't imagine why you'd make a virtualenv
for one Python version _inside_ the installation directory of another
Python version. In Linux I guess somebody might try to make a
virtualenv, with a copied binary, somewhere in /usr/local, using some
Python _other_ than the one installed in /usr, but with the same
major/minor version. A very edge case, but possible.

So, possibilities I see for addressing this:

1. Decide that in real cases of real Python installations, it's so
unlikely to happen that we won't worry about it. I think it's possible
that this is acceptable; the biggest practical problem is likely to be
people trying to test this out during PEP review from a not-installed
checkout, just like you did. We'd have to be careful to instruct people
that it doesn't work that way, and might also want to add a check in the
env-creation script to verify that the created env works properly, and
if it doesn't give them some clue why not.

2. Decide that we just don't support copied binaries, only symlinked
ones. Apparently (I am Windows-ignorant) recent Windows versions do
support symlinks? So this might only involve dropping support for old
Windows'? How important is it for a new feature like this to fully
support all operating systems that Python supports? We could also not
expose the copy-binary option to the user, but fall back to it if we
have no symlinks; which ends up being option (1) but trying to narrow
even more the potential breakage cases.

3. The fully-reliable fix would be to somehow give the copied binary a
hint where to find the right standard library, and this would involve
adding something to the algorithm in getpath.c. The hint could take the
form of a key in the config file, but I'd really like to avoid fully
parsing the config-file in C and then again in Python later on. The hint
could also be some kind of specially-formatted comment line at the top
of the config file, which would require less C code to find and parse?

Any thoughts on this (or alternative solutions I haven't thought of) are
most welcome.

Carl

From carl at meyerloewen.net  Fri Mar 18 20:00:51 2011
From: carl at meyerloewen.net (Carl Meyer)
Date: Fri, 18 Mar 2011 15:00:51 -0400
Subject: [Distutils] reservations about pythonv
In-Reply-To: <4D839A72.1030705@stsci.edu>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<20110316223904.288080ba@neurotica>
	<4D839A72.1030705@stsci.edu>
Message-ID: <4D83ABE3.1090705@meyerloewen.net>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Mark,

On 03/18/2011 01:46 PM, Mark Sienkiewicz wrote:
> Here is a case that might resemble what you are talking about:
> 
> Compile a C extension that requires a shared library that is not in the
> standard system path.  To import it, LD_LIBRARY_PATH needs to be right.

But this required shared library that is not on the standard system path
- - is it something that might have been installed, via Python
(distutils/setuptools/d2/packaging), in the virtualenv, where if it had
been installed in that same way in the system Python installation it
would be on the standard system path?

If so, do you have any specific examples of installable Python projects
that install a shared library that other ones might need in order to
compile, so that I can test this case and make it work?

If not, then it seems this is a general Python packaging issue, not
anything specific to this virtual-python proposal. In which case I don't
have to care, at least not just now ;-)

> This is not really different from what happens in a compiled language,
> except in one way:  In C, I can compile it -static or I can give the
> full path to the .so file.  Either results in a thing that works without
> LD_LIBRARY_PATH.
> 
> With distutils, you can't.  It goes to great lengths to ensure that you
> can only compile a C extension with "cc ... -L/some/directory -lname" --
> I can't find any way to make it do "cc ... /some/directory/libname.so"
> 
> So, the real problem here is that distutils uses "cc -L", but it
> demonstrates a case where LD_LIBRARY_PATH can be important to a python
> program even when Python itself can run without it.

Thanks!

Carl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk2Dq+MACgkQFRcxmeyPUXJbHwCfa1FpWif7nRJK90EARxxs8yGq
bTMAnAxNj8eU3ohfRiCZyHOlNGIiL33T
=Zmep
-----END PGP SIGNATURE-----

From pje at telecommunity.com  Fri Mar 18 20:20:14 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 18 Mar 2011 15:20:14 -0400
Subject: [Distutils] pythonv, take two
In-Reply-To: <4D83A807.30003@oddbird.net>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>
	<4D8172FB.3020109@oddbird.net>
	<loom.20110317T112246-771@post.gmane.org>
	<4D8233EF.7070205@oddbird.net>
	<loom.20110318T004329-895@post.gmane.org>
	<4D83A807.30003@oddbird.net>
Message-ID: <20110318192023.60E233A4062@sparrow.telecommunity.com>

At 02:44 PM 3/18/2011 -0400, Carl Meyer wrote:
>Apparently (I am Windows-ignorant) recent Windows versions do
>support symlinks?

Technically, some Windows filesystems can support this.  In practice, 
no user-visible tools actually support making or using them sanely, AFAIK.

So, I suggest promoting symlinks as the standard way, with 
binary-copying being a Windows-only workaround. 


From erik.m.bray at gmail.com  Fri Mar 18 21:05:33 2011
From: erik.m.bray at gmail.com (Erik Bray)
Date: Fri, 18 Mar 2011 16:05:33 -0400
Subject: [Distutils] distutils2 Forward Compatibility module
In-Reply-To: <AANLkTik0UzxjtcmQq3g7SDHOSB_o4hTVGzbfXY8EwN-O@mail.gmail.com>
References: <AANLkTi=xBT3Gba9JS261boSRjiBzLAkTThhqUS81zpqD@mail.gmail.com>
	<AANLkTik0UzxjtcmQq3g7SDHOSB_o4hTVGzbfXY8EwN-O@mail.gmail.com>
Message-ID: <AANLkTinNPDuc4yudCA1L_-AkOuC8Bcwu58_PFFg+rbUV@mail.gmail.com>

On Thu, Mar 17, 2011 at 11:55 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> Hi Erik,
>
> On Thu, Mar 17, 2011 at 8:42 AM, Erik Bray <erik.m.bray at gmail.com> wrote:
>> Hi all,
>> This is my first post specifically to distutils-sig, though I've had
>> an interest in packaging for a while (having come up with some fairly
>> arcane schemes in the past, where no better alternatives were
>> apparent).
>>
>> At any rate: I'm currently working on a plan to overhaul how a number
>> of my company's projects are packaged and distributed. ?Right now they
>> all rely on a monkey-patched, hacked up distutils that needs to go
>> away. ?I'm already on top of that.
>>
>> But seeing as how distutils2 is going to be the "new hotness" I want
>> to plan for at as part of my overhaul. ?I realize that distutils2 is
>> still in flux, and anything I do now will have to be tweaked as
>> development on it continues. ?I am fine with this, as I still intend
>> to use Distribute as the primary installation mechanism. ?But I really
>> like how distutils2 keeps all metadata in the setup.cfg file, and want
>> to start doing that now, so that I don't have to keep two copies of
>> everything.
>>
>> It should be no problem to just have my setup.py read everything it
>> needs out of setup.cfg, but what I'm wondering is if there is already
>> an extension to do this, or will I have to roll my own? ?It just seems
>> like an obvious thing to have for transitioning to distutils2, and if
>> it doesn't already exist it should (I will of course be happy to
>> contribute). ?I should note that I don't want distutils2 itself to be
>> a dependency for installing my packages, as it is too unstable, so
>> directly using any machinery built into it is out of the question.
>
> There's such a thing, look at this function here:
>
> http://hg.python.org/distutils2/file/6fca65ff60ad/distutils2/util.py#l1098
>
> It's not hooked yet to a command-line tool like mkcfg, but should be soon.
>
> Let us know how it works out for you.

I accidentally replied to this the other day directly to Tarek instead
of to the list.  At any rate, the cfg_to_args() function there is
mostly working out, after a couple bug fixes (see
http://bugs.python.org/issue11595).  I also have another patch in my
local repository that adds support for the package_data options.

But now I'm stuck wondering what the plan is for extension modules.
Different versiions of the documentation, such that it exists, say
different things.  Looking at the source code it looks like right now
each extension module is configured in a section called
[extension=<module_name>], but I wonder how likely that is to change.

Furthermore, I have many extension modules that use NumPy.  Typically,
when building an extension with NumPy, one gets the path to the NumPy
includes using numpy.get_includes().  But of course, there's no way to
do this in setup.cfg.  Right now the only solution I can think of is
to add a setup_hook.

Anyways, worse comes to worse I'll build my extension modules in
setup.py only for now, but I'm just wondering if there's a decided
upon plan for extension modules in distutils2 yet.

Thanks,
Erik

From tseaver at palladion.com  Fri Mar 18 22:24:37 2011
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 18 Mar 2011 17:24:37 -0400
Subject: [Distutils] pythonv, take two
In-Reply-To: <20110318192023.60E233A4062@sparrow.telecommunity.com>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<4D8172FB.3020109@oddbird.net>	<loom.20110317T112246-771@post.gmane.org>	<4D8233EF.7070205@oddbird.net>	<loom.20110318T004329-895@post.gmane.org>	<4D83A807.30003@oddbird.net>
	<20110318192023.60E233A4062@sparrow.telecommunity.com>
Message-ID: <im0iil$i1e$1@dough.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 03/18/2011 03:20 PM, P.J. Eby wrote:
> At 02:44 PM 3/18/2011 -0400, Carl Meyer wrote:
>> Apparently (I am Windows-ignorant) recent Windows versions do
>> support symlinks?
> 
> Technically, some Windows filesystems can support this.  In practice, 
> no user-visible tools actually support making or using them sanely, AFAIK.
> 
> So, I suggest promoting symlinks as the standard way, with 
> binary-copying being a Windows-only workaround. 

Could the config file contain an optional hint for finding the "right"
stdlib in cases where the binary copy had been made?  I realize that
parsing a config file *without* the stdlib is painful:  perhaps looking
for a line starting with 'stdlib =' would be enough?


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk2DzZUACgkQ+gerLs4ltQ6YtgCfcXlvVYLfwaLLfbAkF9YzOzog
9ekAnAhVygL2APAWR+Cd5h5ePEYblbBN
=wazt
-----END PGP SIGNATURE-----


From carl at oddbird.net  Fri Mar 18 23:35:01 2011
From: carl at oddbird.net (Carl Meyer)
Date: Fri, 18 Mar 2011 18:35:01 -0400
Subject: [Distutils] pythonv, take two
In-Reply-To: <im0iil$i1e$1@dough.gmane.org>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<4D8172FB.3020109@oddbird.net>	<loom.20110317T112246-771@post.gmane.org>	<4D8233EF.7070205@oddbird.net>	<loom.20110318T004329-895@post.gmane.org>	<4D83A807.30003@oddbird.net>	<20110318192023.60E233A4062@sparrow.telecommunity.com>
	<im0iil$i1e$1@dough.gmane.org>
Message-ID: <4D83DE15.7070709@oddbird.net>

On 03/18/2011 05:24 PM, Tres Seaver wrote:
> Could the config file contain an optional hint for finding the "right"
> stdlib in cases where the binary copy had been made?  I realize that
> parsing a config file *without* the stdlib is painful:  perhaps looking
> for a line starting with 'stdlib =' would be enough?

Yes, this was one possible solution I was considering. Two things bother
me about it:

1. It seems... tempting but ill-advised to make it a "fake" ConfigParser
option if in the place where I use it, I'm not actually parsing the file
with full ConfigParser semantics. It might work, but then someone tries
to get clever with a [DEFAULT] section, or interpolation, or who knows
what else, and it breaks mysteriously. That's why I was leaning towards
a specially-tagged comment at the top of the file. Ugly, but at least
not pretending to be something it isn't.

2. I still have to duplicate in C and Python code the logic for finding
the config file in the first place. Or I guess just move it to C and
stash the location in sys or somewhere the Python code can get to it
later. I'd been hoping to avoid this, but oh well...

Carl

From merwok at netwok.org  Sat Mar 19 00:22:49 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Sat, 19 Mar 2011 00:22:49 +0100
Subject: [Distutils] pythonv, take two
In-Reply-To: <4D83DE15.7070709@oddbird.net>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<4D8172FB.3020109@oddbird.net>	<loom.20110317T112246-771@post.gmane.org>	<4D8233EF.7070205@oddbird.net>	<loom.20110318T004329-895@post.gmane.org>	<4D83A807.30003@oddbird.net>	<20110318192023.60E233A4062@sparrow.telecommunity.com>	<im0iil$i1e$1@dough.gmane.org>
	<4D83DE15.7070709@oddbird.net>
Message-ID: <4D83E949.8020701@netwok.org>

> 1. It seems... tempting but ill-advised to make it a "fake" ConfigParser
> option if in the place where I use it, I'm not actually parsing the file
> with full ConfigParser semantics. It might work, but then someone tries
> to get clever with a [DEFAULT] section, or interpolation, or who knows
> what else, and it breaks mysteriously.

It?s still time to define the file format as RawConfigParser-style,
without interpolation.

> 2. I still have to duplicate in C and Python code the logic for finding
> the config file in the first place. Or I guess just move it to C and
> stash the location in sys or somewhere the Python code can get to it
> later. I'd been hoping to avoid this, but oh well...

*hands a courage cookie* I have to say the proof of concept in pure
Python was impressive.  Too bad it?s not enough :)

Cheers

From carl at oddbird.net  Sat Mar 19 00:33:44 2011
From: carl at oddbird.net (Carl Meyer)
Date: Fri, 18 Mar 2011 19:33:44 -0400
Subject: [Distutils] pythonv, take two
In-Reply-To: <4D83E949.8020701@netwok.org>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<4D8172FB.3020109@oddbird.net>	<loom.20110317T112246-771@post.gmane.org>	<4D8233EF.7070205@oddbird.net>	<loom.20110318T004329-895@post.gmane.org>	<4D83A807.30003@oddbird.net>	<20110318192023.60E233A4062@sparrow.telecommunity.com>	<im0iil$i1e$1@dough.gmane.org>
	<4D83DE15.7070709@oddbird.net> <4D83E949.8020701@netwok.org>
Message-ID: <4D83EBD8.7060208@oddbird.net>



On 03/18/2011 07:22 PM, ?ric Araujo wrote:
>> 1. It seems... tempting but ill-advised to make it a "fake" ConfigParser
>> option if in the place where I use it, I'm not actually parsing the file
>> with full ConfigParser semantics. It might work, but then someone tries
>> to get clever with a [DEFAULT] section, or interpolation, or who knows
>> what else, and it breaks mysteriously.
> 
> It?s still time to define the file format as RawConfigParser-style,
> without interpolation.

Indeed. And I guess there's little to no reason why someone would try to
define a default section, or use multiline syntax, or anything else like
that. Perhaps this would be robust enough.

>> 2. I still have to duplicate in C and Python code the logic for finding
>> the config file in the first place. Or I guess just move it to C and
>> stash the location in sys or somewhere the Python code can get to it
>> later. I'd been hoping to avoid this, but oh well...
> 
> *hands a courage cookie* I have to say the proof of concept in pure
> Python was impressive.  Too bad it?s not enough :)

Courage cookie accepted ;)

Carl

From vinay_sajip at yahoo.co.uk  Sat Mar 19 01:01:48 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 19 Mar 2011 00:01:48 +0000 (UTC)
Subject: [Distutils] pythonv, take two
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<4D8172FB.3020109@oddbird.net>	<loom.20110317T112246-771@post.gmane.org>	<4D8233EF.7070205@oddbird.net>
	<loom.20110318T004329-895@post.gmane.org>
	<4D83A807.30003@oddbird.net>
Message-ID: <loom.20110319T005135-109@post.gmane.org>

Hi Carl,

> So, possibilities I see for addressing this:
> 
> 1. Decide that in real cases of real Python installations, it's so
> unlikely to happen that we won't worry about it. I think it's possible
> that this is acceptable; the biggest practical problem is likely to be
> people trying to test this out during PEP review from a not-installed
> checkout, just like you did. We'd have to be careful to instruct people
> that it doesn't work that way, and might also want to add a check in the
> env-creation script to verify that the created env works properly, and
> if it doesn't give them some clue why not.

Yes, I think this is important to do, or at least to have -v output show which
stdlib is being used.

> 2. Decide that we just don't support copied binaries, only symlinked
> ones. Apparently (I am Windows-ignorant) recent Windows versions do
> support symlinks? So this might only involve dropping support for old
> Windows'? How important is it for a new feature like this to fully
> support all operating systems that Python supports? We could also not
> expose the copy-binary option to the user, but fall back to it if we
> have no symlinks; which ends up being option (1) but trying to narrow
> even more the potential breakage cases.

Well, Windows XP is still pretty common and likely to remain so for some time.
On Windows XP, symlinks don't work, but confusingly os.symlink exists on recent
Pythons (certainly 3.2). I had to make a patch in my virtualenv fork to handle
this in copyfile: on XP, os.symlink raises NotImplementedError because the
underlying Windows API function is missing.

So, even if it's just for Windows, I think copying will need to be supported,
but perhaps only from Pythons whose sys.prefix is correctly set.

> 3. The fully-reliable fix would be to somehow give the copied binary a
> hint where to find the right standard library, and this would involve
> adding something to the algorithm in getpath.c. The hint could take the
> form of a key in the config file, but I'd really like to avoid fully
> parsing the config-file in C and then again in Python later on. The hint
> could also be some kind of specially-formatted comment line at the top
> of the config file, which would require less C code to find and parse?
> 
> Any thoughts on this (or alternative solutions I haven't thought of) are
> most welcome.

I know that ConfigParser format files are the first thing one thinks of, but
perhaps a simpler C-parseable format deserves further consideration.

Regards,

Vinay Sajip



From tseaver at palladion.com  Sat Mar 19 01:15:52 2011
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 18 Mar 2011 20:15:52 -0400
Subject: [Distutils] pythonv, take two
In-Reply-To: <loom.20110319T005135-109@post.gmane.org>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<4D8172FB.3020109@oddbird.net>	<loom.20110317T112246-771@post.gmane.org>	<4D8233EF.7070205@oddbird.net>	<loom.20110318T004329-895@post.gmane.org>	<4D83A807.30003@oddbird.net>
	<loom.20110319T005135-109@post.gmane.org>
Message-ID: <im0sjr$6jc$1@dough.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 03/18/2011 08:01 PM, Vinay Sajip wrote:
> Hi Carl,
> 
>> So, possibilities I see for addressing this:
>>
>> 1. Decide that in real cases of real Python installations, it's so
>> unlikely to happen that we won't worry about it. I think it's possible
>> that this is acceptable; the biggest practical problem is likely to be
>> people trying to test this out during PEP review from a not-installed
>> checkout, just like you did. We'd have to be careful to instruct people
>> that it doesn't work that way, and might also want to add a check in the
>> env-creation script to verify that the created env works properly, and
>> if it doesn't give them some clue why not.
> 
> Yes, I think this is important to do, or at least to have -v output show which
> stdlib is being used.
> 
>> 2. Decide that we just don't support copied binaries, only symlinked
>> ones. Apparently (I am Windows-ignorant) recent Windows versions do
>> support symlinks? So this might only involve dropping support for old
>> Windows'? How important is it for a new feature like this to fully
>> support all operating systems that Python supports? We could also not
>> expose the copy-binary option to the user, but fall back to it if we
>> have no symlinks; which ends up being option (1) but trying to narrow
>> even more the potential breakage cases.
> 
> Well, Windows XP is still pretty common and likely to remain so for some time.
> On Windows XP, symlinks don't work, but confusingly os.symlink exists on recent
> Pythons (certainly 3.2). I had to make a patch in my virtualenv fork to handle
> this in copyfile: on XP, os.symlink raises NotImplementedError because the
> underlying Windows API function is missing.
> 
> So, even if it's just for Windows, I think copying will need to be supported,
> but perhaps only from Pythons whose sys.prefix is correctly set.
> 
>> 3. The fully-reliable fix would be to somehow give the copied binary a
>> hint where to find the right standard library, and this would involve
>> adding something to the algorithm in getpath.c. The hint could take the
>> form of a key in the config file, but I'd really like to avoid fully
>> parsing the config-file in C and then again in Python later on. The hint
>> could also be some kind of specially-formatted comment line at the top
>> of the config file, which would require less C code to find and parse?
>>
>> Any thoughts on this (or alternative solutions I haven't thought of) are
>> most welcome.
> 
> I know that ConfigParser format files are the first thing one thinks of, but
> perhaps a simpler C-parseable format deserves further consideration.

One possibility is to make the config file format just a sequence of
long-form command-line options, one per line, with the leading '--'
stripped.  This option would involve inventing a '--use-prefix' option
(bikeshedders, start your sprayguns ;), but would make parsing it a
mostly trivial exercise.


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk2D9bgACgkQ+gerLs4ltQ7VwwCfQku+E1AnGzAYTG6blRgiq6IF
n8MAoMGYbjS/E8+2l2LsvhYO2BhLGtEb
=mA1G
-----END PGP SIGNATURE-----


From greg.ewing at canterbury.ac.nz  Sat Mar 19 02:47:41 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 19 Mar 2011 14:47:41 +1300
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <AANLkTikCB2=Xg18qyBXOegHhMD_oQ8zj92ts=xUc4A39@mail.gmail.com>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
	<4D8247B7.30102@oddbird.net>
	<20110317190105.2A6C03A4062@sparrow.telecommunity.com>
	<4D825B58.502@oddbird.net>
	<AANLkTikXVgwEr0ueKdKY7iHtgwO5Fj89kNCO1_jMLcnm@mail.gmail.com>
	<20110317220822.E838D3A4062@sparrow.telecommunity.com>
	<4D829F01.7070204@oddbird.net> <20110317213312.1ed4b8d9@neurotica>
	<4D836E22.3030306@oddbird.net>
	<AANLkTikCB2=Xg18qyBXOegHhMD_oQ8zj92ts=xUc4A39@mail.gmail.com>
Message-ID: <4D840B3D.7030608@canterbury.ac.nz>

Jim Fulton wrote:

> None. Barry's been using Ubuntu too long and has forgotten that Mac OS
> X is Unix. :)

Or possibly he's thinking of the .app extension that application
bundles have on MacOSX. But that's *not* the executable that
gets run -- the actual executable is buried inside the Contents/MacOS
directory of the bundle and usually has no extension.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Sat Mar 19 02:50:43 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 19 Mar 2011 14:50:43 +1300
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <AANLkTimdj-hZHOC196Ny+YCrxTROGFdWn7W-hX53+V6W@mail.gmail.com>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
	<4D8247B7.30102@oddbird.net>
	<20110317190105.2A6C03A4062@sparrow.telecommunity.com>
	<4D825B58.502@oddbird.net>
	<AANLkTikXVgwEr0ueKdKY7iHtgwO5Fj89kNCO1_jMLcnm@mail.gmail.com>
	<20110317220822.E838D3A4062@sparrow.telecommunity.com>
	<4D829F01.7070204@oddbird.net> <20110317213312.1ed4b8d9@neurotica>
	<4D836E22.3030306@oddbird.net>
	<AANLkTikCB2=Xg18qyBXOegHhMD_oQ8zj92ts=xUc4A39@mail.gmail.com>
	<AANLkTimgJCEPobx=NYcUuK2Dm8OfWYaTiutiFKZZJQrx@mail.gmail.com>
	<AANLkTimdj-hZHOC196Ny+YCrxTROGFdWn7W-hX53+V6W@mail.gmail.com>
Message-ID: <4D840BF3.3000004@canterbury.ac.nz>

Jim Fulton wrote:
> On Fri, Mar 18, 2011 at 11:18 AM, Fred Drake <fdrake at acm.org> wrote:
> 
>> On a
>>case-senseless filesystem, distinguishing the "python" executable from
>>the "Python" directory in the sources requires that unfortunate
>>extension.
> 
> So sad.

We don't have to make it look so Windows-like, though. We could
use something more cheerful such as 'python.nothisisnotthedirectory'.
:-)

-- 
Greg

From greg.ewing at canterbury.ac.nz  Sat Mar 19 02:58:36 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 19 Mar 2011 14:58:36 +1300
Subject: [Distutils] reservations about pythonv
In-Reply-To: <4D839A72.1030705@stsci.edu>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>
	<20110316223904.288080ba@neurotica> <4D839A72.1030705@stsci.edu>
Message-ID: <4D840DCC.6010709@canterbury.ac.nz>

Mark Sienkiewicz wrote:

> With distutils, you can't.  It goes to great lengths to ensure that you 
> can only compile a C extension with "cc ... -L/some/directory -lname" -- 
> I can't find any way to make it do "cc ... /some/directory/libname.so"

Are you sure you can't use extra_link_args for this?

-- 
Greg

From fdrake at acm.org  Sat Mar 19 05:25:08 2011
From: fdrake at acm.org (Fred Drake)
Date: Sat, 19 Mar 2011 00:25:08 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <4D840BF3.3000004@canterbury.ac.nz>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
	<4D8247B7.30102@oddbird.net>
	<20110317190105.2A6C03A4062@sparrow.telecommunity.com>
	<4D825B58.502@oddbird.net>
	<AANLkTikXVgwEr0ueKdKY7iHtgwO5Fj89kNCO1_jMLcnm@mail.gmail.com>
	<20110317220822.E838D3A4062@sparrow.telecommunity.com>
	<4D829F01.7070204@oddbird.net>
	<20110317213312.1ed4b8d9@neurotica> <4D836E22.3030306@oddbird.net>
	<AANLkTikCB2=Xg18qyBXOegHhMD_oQ8zj92ts=xUc4A39@mail.gmail.com>
	<AANLkTimgJCEPobx=NYcUuK2Dm8OfWYaTiutiFKZZJQrx@mail.gmail.com>
	<AANLkTimdj-hZHOC196Ny+YCrxTROGFdWn7W-hX53+V6W@mail.gmail.com>
	<4D840BF3.3000004@canterbury.ac.nz>
Message-ID: <AANLkTikSAZpfZaX=DqwOjk4qCTsaFykjkRVcOGzEYEKK@mail.gmail.com>

On Fri, Mar 18, 2011 at 9:50 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> We don't have to make it look so Windows-like, though. We could
> use something more cheerful such as 'python.nothisisnotthedirectory'.

Yes, but... the sad part is that a turd has to be added, or the name
changed in even less satisfactory ways.  Case-senselessness is still
senseless.

/me mourns the loss of case-sense in "pop" operating systems.


? -Fred

--
Fred L. Drake, Jr.? ? <fdrake at acm.org>
"A storm broke loose in my mind."? --Albert Einstein

From agroszer at gmail.com  Sat Mar 19 08:45:31 2011
From: agroszer at gmail.com (Adam GROSZER)
Date: Sat, 19 Mar 2011 08:45:31 +0100
Subject: [Distutils] Fwd: Re:  window 64bit madness
Message-ID: <4D845F1B.6060205@gmail.com>

Hello,

As Jim wants to dump setuptools support from buildout, would you mind 
taking a look at this issue for distribute please?

-------- Original Message --------
Subject: Re: [Distutils] window 64bit madness
Date: Thu, 17 Mar 2011 08:43:17 +0100
From: Adam GROSZER <agroszer.ll at gmail.com>
To: distutils-sig at python.org

On 03/16/2011 11:47 PM, P.J. Eby wrote:
> At 11:28 AM 3/16/2011 +0100, Adam GROSZER wrote:
>> On 03/14/2011 07:50 PM, P.J. Eby wrote:
>>> Run "python -c 'import pkg_resources;print
>>> pkg_resources.get_build_platform()'" (with the Python interpreter you're
>>> using.
>>
>> D:\install>c:\Python26_64\python.exe
>> Python 2.6.6 (r266:84297, Aug 24 2010, 18:13:38) [MSC v.1500 64 bit
>> (AMD64)] on win32
>> Type "help", "copyright", "credits" or "license" for more information.
>> >>> import pkg_resources;print pkg_resources.get_build_platform()
>> win-amd64
>> >>>
>
> Hm. What's sys.platform? Is it win32?

Exactly.

> I suspect the problem has to do with all the win32-specificness
> scattered through setuptools; I think I know what I need to do to fix it
> all, but it'll be a fairly substantial patch; can you help with the
> testing if I email you such a patch?

Sure thing.

-- 
Best regards,
  Adam GROSZER
--
Quote of the day:
Nothing ever built arose to touch the skies unless some man dreamed that 
it should, some man believed that it could, and some man willed that it 
must.
- Charles F. Kettering

From merwok at netwok.org  Sat Mar 19 13:54:09 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Sat, 19 Mar 2011 13:54:09 +0100
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <4D827E42.2030004@oddbird.net>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>	<20110317105948.4557667c@neurotica>
	<4D823A81.2060406@oddbird.net>	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>	<4D8247B7.30102@oddbird.net>	<20110317190105.2A6C03A4062@sparrow.telecommunity.com>	<4D825B58.502@oddbird.net>	<AANLkTikXVgwEr0ueKdKY7iHtgwO5Fj89kNCO1_jMLcnm@mail.gmail.com>
	<4D827E42.2030004@oddbird.net>
Message-ID: <4D84A771.8050508@netwok.org>

> Is ".cfg" generally preferred to ".conf" for some good reason?

FTR, it?s the form used by modules in the standard library: distutils,
idle, unittest in a near future.  (One exception is .pypirc.)  I find
?.conf? ugly :)

de-gustibus-non-disputandum-ly yours

From jim at zope.com  Sat Mar 19 15:23:17 2011
From: jim at zope.com (Jim Fulton)
Date: Sat, 19 Mar 2011 10:23:17 -0400
Subject: [Distutils] Stop using setuptools or distribute in buildout (was
 Re: Fwd: Re: window 64bit madness)
Message-ID: <AANLkTimmkQN8Uzi0GCfFMDBM_RqSOF26LrfdMEPTzsne@mail.gmail.com>

2011/3/19 Adam GROSZER <agroszer at gmail.com>:
> Hello,
>
> As Jim wants to dump setuptools support from buildout, ...

I don't want to, and won't until it gets too painful to support
both. I just suspect that time is drawing near.

I can also foresee a time at which buildout stops using either.  That
might be the best path for buildout....wow, the more I think of that,
the more appealing it seems.  Replacing use of setuptools/distribute
could be a really nice project for a certain upcoming German buildout
sprint. :)

Jim

--
Jim Fulton
http://www.linkedin.com/in/jimfulton

From fdrake at acm.org  Sat Mar 19 15:44:27 2011
From: fdrake at acm.org (Fred Drake)
Date: Sat, 19 Mar 2011 10:44:27 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <4D84A771.8050508@netwok.org>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
	<4D8247B7.30102@oddbird.net>
	<20110317190105.2A6C03A4062@sparrow.telecommunity.com>
	<4D825B58.502@oddbird.net>
	<AANLkTikXVgwEr0ueKdKY7iHtgwO5Fj89kNCO1_jMLcnm@mail.gmail.com>
	<4D827E42.2030004@oddbird.net> <4D84A771.8050508@netwok.org>
Message-ID: <AANLkTi=jBZiyzX2NtEbketYuFbGh10Kb-hpAB-iCWVVr@mail.gmail.com>

On Sat, Mar 19, 2011 at 8:54 AM, ?ric Araujo <merwok at netwok.org> wrote:
> it?s the form used by modules in the standard library: distutils,
> idle, unittest in a near future.

Which is a good reason to continue using the less-readable .cfg extension.

>?(One exception is .pypirc.)

Traditional dotfiles don't have extensions, and Greg was definitely a
traditionalist.


? -Fred

--
Fred L. Drake, Jr.? ? <fdrake at acm.org>
"A storm broke loose in my mind."? --Albert Einstein

From merwok at netwok.org  Sat Mar 19 16:20:03 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Sat, 19 Mar 2011 16:20:03 +0100
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <AANLkTi=jBZiyzX2NtEbketYuFbGh10Kb-hpAB-iCWVVr@mail.gmail.com>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
	<4D8247B7.30102@oddbird.net>
	<20110317190105.2A6C03A4062@sparrow.telecommunity.com>
	<4D825B58.502@oddbird.net>
	<AANLkTikXVgwEr0ueKdKY7iHtgwO5Fj89kNCO1_jMLcnm@mail.gmail.com>
	<4D827E42.2030004@oddbird.net> <4D84A771.8050508@netwok.org>
	<AANLkTi=jBZiyzX2NtEbketYuFbGh10Kb-hpAB-iCWVVr@mail.gmail.com>
Message-ID: <4D84C9A3.60300@netwok.org>

>>  (One exception is .pypirc.)
> Traditional dotfiles don't have extensions, and Greg was definitely a
> traditionalist.
Except that Greg?s code used .pydistutils.cfg and pydistutils.cfg;
.pypirc was a later addition by amk (PEP 301).

whippersnapperly yours

From fdrake at acm.org  Sat Mar 19 16:25:43 2011
From: fdrake at acm.org (Fred Drake)
Date: Sat, 19 Mar 2011 11:25:43 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <4D84C9A3.60300@netwok.org>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
	<4D8247B7.30102@oddbird.net>
	<20110317190105.2A6C03A4062@sparrow.telecommunity.com>
	<4D825B58.502@oddbird.net>
	<AANLkTikXVgwEr0ueKdKY7iHtgwO5Fj89kNCO1_jMLcnm@mail.gmail.com>
	<4D827E42.2030004@oddbird.net> <4D84A771.8050508@netwok.org>
	<AANLkTi=jBZiyzX2NtEbketYuFbGh10Kb-hpAB-iCWVVr@mail.gmail.com>
	<4D84C9A3.60300@netwok.org>
Message-ID: <AANLkTimFbhSNNT7HfmJ068vcdAgf+JB_btAz8+f+7iFW@mail.gmail.com>

On Sat, Mar 19, 2011 at 11:20 AM, ?ric Araujo <merwok at netwok.org> wrote:
> Except that Greg?s code used .pydistutils.cfg and pydistutils.cfg;
> .pypirc was a later addition by amk (PEP 301).

--sigh--

Getting up on Saturday is always a mistake.

> whippersnapperly yours

Yes, indeed.


? -Fred

--
Fred L. Drake, Jr.? ? <fdrake at acm.org>
"A storm broke loose in my mind."? --Albert Einstein

From ziade.tarek at gmail.com  Sat Mar 19 16:36:27 2011
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 19 Mar 2011 08:36:27 -0700
Subject: [Distutils] Stop using setuptools or distribute in buildout
 (was Re: Fwd: Re: window 64bit madness)
In-Reply-To: <AANLkTimmkQN8Uzi0GCfFMDBM_RqSOF26LrfdMEPTzsne@mail.gmail.com>
References: <AANLkTimmkQN8Uzi0GCfFMDBM_RqSOF26LrfdMEPTzsne@mail.gmail.com>
Message-ID: <AANLkTimWrM=4Tt62gZSTmzu5NTr61vdtUL172-+rhzpw@mail.gmail.com>

What are the features that you need ? I'd be interested to know wrt
packaging package we're adding in Python to see if it can be useful there

Cheers
Le 19 mars 2011 07:23, "Jim Fulton" <jim at zope.com> a ?crit :
> 2011/3/19 Adam GROSZER <agroszer at gmail.com>:
>> Hello,
>>
>> As Jim wants to dump setuptools support from buildout, ...
>
> I don't want to, and won't until it gets too painful to support
> both. I just suspect that time is drawing near.
>
> I can also foresee a time at which buildout stops using either. That
> might be the best path for buildout....wow, the more I think of that,
> the more appealing it seems. Replacing use of setuptools/distribute
> could be a really nice project for a certain upcoming German buildout
> sprint. :)
>
> Jim
>
> --
> Jim Fulton
> http://www.linkedin.com/in/jimfulton
> _______________________________________________
> Distutils-SIG maillist - Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110319/c5b2ebb5/attachment.html>

From jim at zope.com  Sat Mar 19 16:49:06 2011
From: jim at zope.com (Jim Fulton)
Date: Sat, 19 Mar 2011 11:49:06 -0400
Subject: [Distutils] pythonv, take two
In-Reply-To: <4D83DE15.7070709@oddbird.net>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>
	<4D8172FB.3020109@oddbird.net>
	<loom.20110317T112246-771@post.gmane.org>
	<4D8233EF.7070205@oddbird.net>
	<loom.20110318T004329-895@post.gmane.org> <4D83A807.30003@oddbird.net>
	<20110318192023.60E233A4062@sparrow.telecommunity.com>
	<im0iil$i1e$1@dough.gmane.org> <4D83DE15.7070709@oddbird.net>
Message-ID: <AANLkTikEuxzFGWnGBfhzWHdWF6pKAW-9WHyptDFuV80i@mail.gmail.com>

On Fri, Mar 18, 2011 at 6:35 PM, Carl Meyer <carl at oddbird.net> wrote:
> On 03/18/2011 05:24 PM, Tres Seaver wrote:
>> Could the config file contain an optional hint for finding the "right"
>> stdlib in cases where the binary copy had been made? ?I realize that
>> parsing a config file *without* the stdlib is painful: ?perhaps looking
>> for a line starting with 'stdlib =' would be enough?
>
> Yes, this was one possible solution I was considering. Two things bother
> me about it:
>
> 1. It seems... tempting but ill-advised to make it a "fake" ConfigParser
> option if in the place where I use it, I'm not actually parsing the file
> with full ConfigParser semantics. It might work, but then someone tries
> to get clever with a [DEFAULT] section, or interpolation, or who knows
> what else, and it breaks mysteriously. That's why I was leaning towards
> a specially-tagged comment at the top of the file. Ugly, but at least
> not pretending to be something it isn't.

Don't call it ConfigParser format. Call it "ini" format, or whatever.

http://bit.ly/hNVmhz

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From jim at zope.com  Sat Mar 19 17:04:20 2011
From: jim at zope.com (Jim Fulton)
Date: Sat, 19 Mar 2011 12:04:20 -0400
Subject: [Distutils] Stop using setuptools or distribute in buildout
 (was Re: Fwd: Re: window 64bit madness)
In-Reply-To: <AANLkTimWrM=4Tt62gZSTmzu5NTr61vdtUL172-+rhzpw@mail.gmail.com>
References: <AANLkTimmkQN8Uzi0GCfFMDBM_RqSOF26LrfdMEPTzsne@mail.gmail.com>
	<AANLkTimWrM=4Tt62gZSTmzu5NTr61vdtUL172-+rhzpw@mail.gmail.com>
Message-ID: <AANLkTinjDh=x7=1pCx+sShfBD12B0BtNO=ZU50SDMyP8@mail.gmail.com>

On Sat, Mar 19, 2011 at 11:36 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> What are the features that you need ? I'd be interested to know wrt
> packaging package we're adding in Python to see if it can be useful there

For the most part, I use the package-index and package-resource
features.  In many ways I'd do better doing a lot of this myself.  For
example, people want to allow multiple indexes and I'd like buildout
to do more network IO in parallel, using either an async IO library or
threads.

Buildout uses the easy_install command to build, but it would do
better to just invoke a project's setup script, possibly after
arranging that certain other required packages are accessible to the
setup script.

I could go into more detail if I remembered the details. :) If a
sprint attacks this, as I hope, then a more detailed report would be a
good product of the sprint.

Jim

--
Jim Fulton
http://www.linkedin.com/in/jimfulton

From barry at python.org  Sat Mar 19 23:21:44 2011
From: barry at python.org (Barry Warsaw)
Date: Sat, 19 Mar 2011 18:21:44 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <4D836E22.3030306@oddbird.net>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
	<4D8247B7.30102@oddbird.net>
	<20110317190105.2A6C03A4062@sparrow.telecommunity.com>
	<4D825B58.502@oddbird.net>
	<AANLkTikXVgwEr0ueKdKY7iHtgwO5Fj89kNCO1_jMLcnm@mail.gmail.com>
	<20110317220822.E838D3A4062@sparrow.telecommunity.com>
	<4D829F01.7070204@oddbird.net> <20110317213312.1ed4b8d9@neurotica>
	<4D836E22.3030306@oddbird.net>
Message-ID: <20110319182144.0271515a@neurotica.wooz.org>

On Mar 18, 2011, at 10:37 AM, Carl Meyer wrote:

>Hmm, really? What extension does the executable typically have on OS X
>that ought to be stripped?

As others have pointed out, the OS X build leaves a python.exe in the
development tree, but installs without the .exe.  I think it's useful to be
able to create virtual environments with a build-tree version of Python.

(I'd love to be able to verify this, but my OS X builds are failing atm.)

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110319/432cbd8a/attachment.pgp>

From ronaldoussoren at mac.com  Sat Mar 19 23:19:11 2011
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Sat, 19 Mar 2011 18:19:11 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
	install includes an "isolated" python
In-Reply-To: <20110317213312.1ed4b8d9@neurotica>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>
	<20110317105948.4557667c@neurotica> <4D823A81.2060406@oddbird.net>
	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>
	<4D8247B7.30102@oddbird.net>
	<20110317190105.2A6C03A4062@sparrow.telecommunity.com>
	<4D825B58.502@oddbird.net>
	<AANLkTikXVgwEr0ueKdKY7iHtgwO5Fj89kNCO1_jMLcnm@mail.gmail.com>
	<20110317220822.E838D3A4062@sparrow.telecommunity.com>
	<4D829F01.7070204@oddbird.net> <20110317213312.1ed4b8d9@neurotica>
Message-ID: <FD0B02EB-5841-4C9B-BF16-87062A76AAA5@mac.com>


On 17 Mar, 2011, at 21:33, Barry Warsaw wrote:

> Sounds good to me.
> 
> On Mar 17, 2011, at 07:53 PM, Carl Meyer wrote:
> 
>> * <executable-name> has the extension stripped on Windows, but not
>> otherwise.
> 
> It should probably also have the extension stripped on OS X too.

There is no extension on OSX. There is one in the build tree, but that's because the default filesystem on OSX is case insensive and hence the name 'python' is already taken during a build.

Ronald


From leorochael at gmail.com  Sun Mar 20 01:18:53 2011
From: leorochael at gmail.com (Leonardo Rochael Almeida)
Date: Sun, 20 Mar 2011 01:18:53 +0100
Subject: [Distutils] reservations about pythonv
In-Reply-To: <4D83ABE3.1090705@meyerloewen.net>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>
	<20110316223904.288080ba@neurotica>
	<4D839A72.1030705@stsci.edu> <4D83ABE3.1090705@meyerloewen.net>
Message-ID: <AANLkTinbW=AqTb-p51Xv1BRzx7dWofbqpLD5QFQbHkeR@mail.gmail.com>

Another variant of this case, one we're actually facing here at our
company (Nexedi) right now, is when you need to compile extension
modules with libraries that are newer than the ones in the system, and
you don't have root access. The absense of LD_LIBRARY_PATH means a
segfault on an arbitrary moment during execution, not just an
unloadable library.

On Fri, Mar 18, 2011 at 20:00, Carl Meyer <carl at meyerloewen.net> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi Mark,
>
> On 03/18/2011 01:46 PM, Mark Sienkiewicz wrote:
>> Here is a case that might resemble what you are talking about:
>>
>> Compile a C extension that requires a shared library that is not in the
>> standard system path. ?To import it, LD_LIBRARY_PATH needs to be right.
>
> But this required shared library that is not on the standard system path
> - - is it something that might have been installed, via Python
> (distutils/setuptools/d2/packaging), in the virtualenv, where if it had
> been installed in that same way in the system Python installation it
> would be on the standard system path?
>
> If so, do you have any specific examples of installable Python projects
> that install a shared library that other ones might need in order to
> compile, so that I can test this case and make it work?
>
> If not, then it seems this is a general Python packaging issue, not
> anything specific to this virtual-python proposal. In which case I don't
> have to care, at least not just now ;-)
>
>> This is not really different from what happens in a compiled language,
>> except in one way: ?In C, I can compile it -static or I can give the
>> full path to the .so file. ?Either results in a thing that works without
>> LD_LIBRARY_PATH.
>>
>> With distutils, you can't. ?It goes to great lengths to ensure that you
>> can only compile a C extension with "cc ... -L/some/directory -lname" --
>> I can't find any way to make it do "cc ... /some/directory/libname.so"
>>
>> So, the real problem here is that distutils uses "cc -L", but it
>> demonstrates a case where LD_LIBRARY_PATH can be important to a python
>> program even when Python itself can run without it.
>
> Thanks!
>
> Carl
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAk2Dq+MACgkQFRcxmeyPUXJbHwCfa1FpWif7nRJK90EARxxs8yGq
> bTMAnAxNj8eU3ohfRiCZyHOlNGIiL33T
> =Zmep
> -----END PGP SIGNATURE-----
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>

From carl at oddbird.net  Sun Mar 20 04:06:40 2011
From: carl at oddbird.net (Carl Meyer)
Date: Sat, 19 Mar 2011 23:06:40 -0400
Subject: [Distutils] reservations about pythonv
In-Reply-To: <AANLkTinbW=AqTb-p51Xv1BRzx7dWofbqpLD5QFQbHkeR@mail.gmail.com>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<20110316223904.288080ba@neurotica>	<4D839A72.1030705@stsci.edu>
	<4D83ABE3.1090705@meyerloewen.net>
	<AANLkTinbW=AqTb-p51Xv1BRzx7dWofbqpLD5QFQbHkeR@mail.gmail.com>
Message-ID: <4D856F40.2050907@oddbird.net>

Hi Leonardo,

On 03/19/2011 08:18 PM, Leonardo Rochael Almeida wrote:
> Another variant of this case, one we're actually facing here at our
> company (Nexedi) right now, is when you need to compile extension
> modules with libraries that are newer than the ones in the system, and
> you don't have root access. The absense of LD_LIBRARY_PATH means a
> segfault on an arbitrary moment during execution, not just an
> unloadable library.

Hmm. So my design goal for pythonv is that it "just work" to the same
extent a real installation of Python would. IIUC in this case even if
you were compiling your own full installation of Python, you would still
need to manually set LD_LIBRARY_PATH to ensure that your newer shared
libraries would be found before the system ones. So I don't think
pythonv would change anything here.

To be clear, pythonv is not going to _prevent_ anyone from setting
LD_LIBRARY_PATH before running the virtualized python. The open question
is whether pythonv needs to automatically and transparently set
LD_LIBRARY_PATH to point to some location in the virtual environment
every time you use it, in order to work as well as a normal installation
of Python.

Actually, the comparison to installing a custom-compiled Python to a
non-standard location increases my sense that pythonv should not
automatically add to LD_LIBRARY_PATH. If you "make install" Python to a
non-standard location (on my Ubuntu system I have various versions of
Python that I've compiled and installed in /opt), it doesn't take it
upon itself to modify your system's library path accordingly, does it?
So why should pythonv do any differently?

Carl

From leorochael at gmail.com  Sun Mar 20 18:48:49 2011
From: leorochael at gmail.com (Leonardo Rochael Almeida)
Date: Sun, 20 Mar 2011 18:48:49 +0100
Subject: [Distutils] reservations about pythonv
In-Reply-To: <4D856F40.2050907@oddbird.net>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>
	<20110316223904.288080ba@neurotica>
	<4D839A72.1030705@stsci.edu> <4D83ABE3.1090705@meyerloewen.net>
	<AANLkTinbW=AqTb-p51Xv1BRzx7dWofbqpLD5QFQbHkeR@mail.gmail.com>
	<4D856F40.2050907@oddbird.net>
Message-ID: <AANLkTi=A63hf45yENXFZB8FyOspLSEHSGjEVaG3ifDxN@mail.gmail.com>

Hi Carl,

On Sun, Mar 20, 2011 at 04:06, Carl Meyer <carl at oddbird.net> wrote:
> Hi Leonardo,
>
> On 03/19/2011 08:18 PM, Leonardo Rochael Almeida wrote:
>> Another variant of this case, one we're actually facing here at our
>> company (Nexedi) right now, is when you need to compile extension
>> modules with libraries that are newer than the ones in the system, and
>> you don't have root access. The absense of LD_LIBRARY_PATH means a
>> segfault on an arbitrary moment during execution, not just an
>> unloadable library.
>
> [...]
>
> Actually, the comparison to installing a custom-compiled Python to a
> non-standard location increases my sense that pythonv should not
> automatically add to LD_LIBRARY_PATH. If you "make install" Python to a
> non-standard location (on my Ubuntu system I have various versions of
> Python that I've compiled and installed in /opt), it doesn't take it
> upon itself to modify your system's library path accordingly, does it?
> So why should pythonv do any differently?

It's likely that pythonv shoudn't do anything differently, at least
from the POV of the Python binary. I just wanted to mention a use-case
for needing LD_LIBRARY_PATH set while running Python.

This being distutils-sig, though, maybe we could talk about whether
supporting scripts or generated scripts could do something about it.
For example, the 'activate' script, or, in buildout, pip, etc. case,
shebang lines that included an 'env' with the correct LD_LIBRARY_PATH,
like:

#!/usr/bin/env LD_LIBRARY_PATH=/my/custom/lib /my/custom/bin/python

Cheers,

Leo

From setuptools at bugs.python.org  Sun Mar 20 22:27:48 2011
From: setuptools at bugs.python.org (Johannes Lindenbaum)
Date: Sun, 20 Mar 2011 21:27:48 +0000
Subject: [Distutils] [issue122] Xcode 4 removes PPC compiler OS X 10.6,
	setup.py fails installs
In-Reply-To: <1300656468.36.0.658710630296.issue122@psf.upfronthosting.co.za>
Message-ID: <1300656468.36.0.658710630296.issue122@psf.upfronthosting.co.za>


New submission from Johannes Lindenbaum <johanneslindenbaum at gmail.com>:

Summary:
OSX 10.6.x with Xcode 4 installed. Xcode 4 removes the PPC assembler from GCC. I attempted to install Fabric-1.0.0 which depended on pycrypto, during the pycrypto build there was a failure regarding "Broken Pipe".

I downloaded the package separately and attempted to do the install there. Same error.

I was pointed in the right direction that the broken pipe error was coming from the '-arch ppc' flag during build.


Expected Result
10.6.x should not report itself as "universal" when it's no longer possible to compile for PPC.


Actual Result
Attempted PPC compilation fails.


More detail:
http://superuser.com/questions/259278/python-2-6-1-pycrypto-2-3-pypi-package-broken-pipe-during-build/260106#260106

Let me know if you require more information.

Kind Regards,
Johannes

----------
messages: 594
nosy: jlindenbaum
priority: bug
status: unread
title: Xcode 4 removes PPC compiler OS X 10.6, setup.py fails installs

_______________________________________________
Setuptools tracker <setuptools at bugs.python.org>
<http://bugs.python.org/setuptools/issue122>
_______________________________________________

From fdrake at acm.org  Mon Mar 21 14:40:20 2011
From: fdrake at acm.org (Fred Drake)
Date: Mon, 21 Mar 2011 09:40:20 -0400
Subject: [Distutils] reservations about pythonv
In-Reply-To: <AANLkTi=A63hf45yENXFZB8FyOspLSEHSGjEVaG3ifDxN@mail.gmail.com>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>
	<20110316223904.288080ba@neurotica>
	<4D839A72.1030705@stsci.edu> <4D83ABE3.1090705@meyerloewen.net>
	<AANLkTinbW=AqTb-p51Xv1BRzx7dWofbqpLD5QFQbHkeR@mail.gmail.com>
	<4D856F40.2050907@oddbird.net>
	<AANLkTi=A63hf45yENXFZB8FyOspLSEHSGjEVaG3ifDxN@mail.gmail.com>
Message-ID: <AANLkTinOSXWzpsniGMJwBrnA09ooyp0jfFY4UUFepfBj@mail.gmail.com>

On Sun, Mar 20, 2011 at 1:48 PM, Leonardo Rochael Almeida
<leorochael at gmail.com> wrote:
> It's likely that pythonv shoudn't do anything differently, at least
> from the POV of the Python binary. I just wanted to mention a use-case
> for needing LD_LIBRARY_PATH set while running Python.

I'd agree that for the use-cases I've dealt with, it's not clear that
pythonv in particular should be responsible for handling
LD_LIBRARY_PATH; it would at times be convenient, but not essential.
Existing workarounds would still apply, and in some cases would still
be necessary.

There are two primary changes we've wanted to make when the Python
process is started:

- setting environment variables that have to be set for the dynamic
loaded (LD_LIBRARY_PATH is the most common), and

- running as a particular user.

Both of these would require restarting Python to get set up, and
neither is part of getting the basic "virtual" environment set up.
Getting environment changes made & restarting the interpreter wouldn't
solve both of these, and the run-as-user behavior is definitely
inappropriate for a basic pythonv behavior.


? -Fred

--
Fred L. Drake, Jr.? ? <fdrake at acm.org>
"A storm broke loose in my mind."? --Albert Einstein

From barry at python.org  Mon Mar 21 19:32:26 2011
From: barry at python.org (Barry Warsaw)
Date: Mon, 21 Mar 2011 14:32:26 -0400
Subject: [Distutils] reservations about pythonv
In-Reply-To: <AANLkTinbW=AqTb-p51Xv1BRzx7dWofbqpLD5QFQbHkeR@mail.gmail.com>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>
	<20110316223904.288080ba@neurotica> <4D839A72.1030705@stsci.edu>
	<4D83ABE3.1090705@meyerloewen.net>
	<AANLkTinbW=AqTb-p51Xv1BRzx7dWofbqpLD5QFQbHkeR@mail.gmail.com>
Message-ID: <20110321143226.319fb268@limelight.wooz.org>

On Mar 20, 2011, at 01:18 AM, Leonardo Rochael Almeida wrote:

>Another variant of this case, one we're actually facing here at our
>company (Nexedi) right now, is when you need to compile extension
>modules with libraries that are newer than the ones in the system, and
>you don't have root access. The absense of LD_LIBRARY_PATH means a
>segfault on an arbitrary moment during execution, not just an
>unloadable library.

The question is whether pythonv makes any difference here.  IOW, if you used
plain ol' virtualenv with some custom-location libraries, you'd have the same
problem right?

If so, then in both cases you'd have to arrange for LD_LIBRARY_PATH to be set
correctly for things to work.  So pythonv wouldn't need to do anything
special.

Cheers,
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110321/b5c1e7a4/attachment.pgp>

From thomas at thomas-lotze.de  Mon Mar 21 21:24:02 2011
From: thomas at thomas-lotze.de (Thomas Lotze)
Date: Mon, 21 Mar 2011 21:24:02 +0100
Subject: [Distutils] New buildout options: checksums
	and	allow-omitted-checksums
References: <pan.2011.03.17.15.55.03.959118@ID-174572.user.uni-berlin.de>
	<20110318130710.GA28064@fridge.pov.lt>
	<pan.2011.03.18.13.43.04.450362@ID-174572.user.uni-berlin.de>
Message-ID: <pan.2011.03.21.20.24.02.48043@ID-174572.user.uni-berlin.de>

Thomas Lotze wrote:

> Good point. All this talking about MD5 specifically has been due to the
> fact that this is what used to be used by the download API and the
> gocep.download recipe so far. To take up the idea I posted a few minutes
> ago, one might specify checksums like this:
> 
> [checksums]
> foo = http://example.org/foo.tgz algorithm:checksum-value

After some further offline discussion, I'd like to suggest using MD5 as
the default algorithm, though. It has been the algorithm of choice in
buildout and recipes and allowing to omit the md5: prefix in what's very
likely the majority of cases sounds like a good bargain. I agree that the
cleanest solution would be not having a default algorithm at all but this
may just be one instance where practicality beats purity.

-- 
Thomas




From sienkiew at stsci.edu  Mon Mar 21 22:38:47 2011
From: sienkiew at stsci.edu (Mark Sienkiewicz)
Date: Mon, 21 Mar 2011 17:38:47 -0400
Subject: [Distutils] reservations about pythonv
In-Reply-To: <4D83ABE3.1090705@meyerloewen.net>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<20110316223904.288080ba@neurotica>
	<4D839A72.1030705@stsci.edu> <4D83ABE3.1090705@meyerloewen.net>
Message-ID: <4D87C567.2090803@stsci.edu>

Carl Meyer wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi Mark,
>
> On 03/18/2011 01:46 PM, Mark Sienkiewicz wrote:
>   
>> Here is a case that might resemble what you are talking about:
>>
>> Compile a C extension that requires a shared library that is not in the
>> standard system path.  To import it, LD_LIBRARY_PATH needs to be right.
>>     
>
> But this required shared library that is not on the standard system path
> - - is it something that might have been installed, via Python
> (distutils/setuptools/d2/packaging), in the virtualenv, where if it had
> been installed in that same way in the system Python installation it
> would be on the standard system path?
>   

No.  I'm thinking of examples like this:

- In some of my Macs and Solaris machines, the BLAS library does not 
contain all of the functions that numpy wants.  I download the FORTRAN 
sources for BLAS, compile them, and put the libraries in 
$HOME/I_wish_I_had_root

- I need a newer version of the freetype library, so I ./configure 
--prefix $HOME/newer_freetype

python is never involved, except as an application that uses the shared 
libraries.

I have several different linux releases to support, so I have a lot of 
libraries that I install in order to have the same version on each machine.

If I had a way for python to automatically augment LD_LIBRARY_PATH when 
it starts up, I would use it, whether virtualenv is involved or not.

If I had a way to make distutils link shared libraries by a full path 
name instead of just libfoo.a, I would be happy to have that too.

In practice, I usually try to ensure that I have only static libraries 
available.  Everything gets statically linked, so I don't need 
LD_LIBRARY_PATH.


> If so, do you have any specific examples of installable Python projects
> that install a shared library that other ones might need in order to
> compile, so that I can test this case and make it work?
>   

I know of an example, but I don't have much to show you that I think 
would be useful.

We have two packages:

pywcs does coordinate transformations on astronomical images; it is a 
python wrapper for wcslib (or libwcs or something), written in C.

betadrizzle does some complex manipulations of multiple images.  In 
doing that, it sometimes needs to do huge numbers of coordinate 
transformations.

The betadrizzle C extension would like to call directly into the pywcs 
copy of wcslib, but without calling into python and back to C because of 
the overhead of doing it potentially millions of times.

I'm not sure what we are doing with that right now.  I remember 
suggesting that pywcs offer some C entry points through a jump table, 
but I haven't caught up with the guy doing the actual work to find out 
what he ended up doing.

Unfortunately, betadrizzle is still an experimental project and I'm not 
sure I could tell you how to compile it.  I am sure that I could not 
tell you how to run it.  But you see the idea.


> If not, then it seems this is a general Python packaging issue, not
> anything specific to this virtual-python proposal. In which case I don't
> have to care, at least not just now ;-)
>   

No, you don't have to care.  Shared libraries cause me way more problems 
than they solve, and nothing you do or don't do is going to change that. 
:( 

Mark S.


From greg.ewing at canterbury.ac.nz  Tue Mar 22 00:51:35 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 22 Mar 2011 12:51:35 +1300
Subject: [Distutils] New buildout options: checksums	and
 allow-omitted-checksums
In-Reply-To: <pan.2011.03.21.20.24.02.48043@ID-174572.user.uni-berlin.de>
References: <pan.2011.03.17.15.55.03.959118@ID-174572.user.uni-berlin.de>
	<20110318130710.GA28064@fridge.pov.lt>
	<pan.2011.03.18.13.43.04.450362@ID-174572.user.uni-berlin.de>
	<pan.2011.03.21.20.24.02.48043@ID-174572.user.uni-berlin.de>
Message-ID: <4D87E487.2000000@canterbury.ac.nz>

Thomas Lotze wrote:

> After some further offline discussion, I'd like to suggest using MD5 as
> the default algorithm, though.

Warnings against using md5 are mainly about cryptographic
security, aren't they? For just detecting accidental
corruption it should still be good enough.

-- 
Greg

From agroszer.ll at gmail.com  Tue Mar 22 08:56:05 2011
From: agroszer.ll at gmail.com (Adam GROSZER)
Date: Tue, 22 Mar 2011 08:56:05 +0100
Subject: [Distutils] broken buildout tests on windows was: [Zope-dev] Zope
 Tests: 68 OK, 34 Failed, 6 Unknown
In-Reply-To: <im7s8p$b8u$1@dough.gmane.org>
References: <20110321115756.08B627505B@epy.co.at>
	<im7s8p$b8u$1@dough.gmane.org>
Message-ID: <4D885615.7090209@gmail.com>

Hello,

>> Subject: FAILED : winbot / zc_buildout_dev py_254_win32
>> From: buildbot at winbot.zope.org
>> Date: Sun Mar 20 17:07:28 EDT 2011
>> URL: http://mail.zope.org/pipermail/zope-tests/2011-March/035690.html
>>
>> Subject: FAILED : winbot / zc_buildout_dev py_265_win32
>> From: buildbot at winbot.zope.org
>> Date: Sun Mar 20 17:07:39 EDT 2011
>> URL: http://mail.zope.org/pipermail/zope-tests/2011-March/035691.html
>>
>> Subject: FAILED : winbot / zc_buildout_dev py_265_win64
>> From: buildbot at winbot.zope.org
>> Date: Sun Mar 20 17:07:50 EDT 2011
>> URL: http://mail.zope.org/pipermail/zope-tests/2011-March/035692.html
>>
>> Subject: FAILED : winbot / zc_buildout_dev py_270_win32
>> From: buildbot at winbot.zope.org
>> Date: Sun Mar 20 17:08:00 EDT 2011
>> URL: http://mail.zope.org/pipermail/zope-tests/2011-March/035693.html
>>
>> Subject: FAILED : winbot / zc_buildout_dev py_270_win64
>> From: buildbot at winbot.zope.org
>> Date: Sun Mar 20 17:08:14 EDT 2011
>> URL: http://mail.zope.org/pipermail/zope-tests/2011-March/035694.html
>
> More weird Windows build failures::
>
>
>   c:\buildslave\zc_buildout_dev_py_254_win32\build>bin\test.exe
> - --exit-with-status -1"
>   'bin\test.exe' is not recognized as an internal or external command,
> operable program or batch file.

Bootstrap failed, it just does not exit with the right exitcode.
See:
http://winbot.zope.org/builders/zc_buildout_dev%20py_270_win32/builds/239/steps/bootstrap/logs/stdio

Anyone feeling guilty?

-- 
Best regards,
  Adam GROSZER
--
Quote of the day:
Zimmerman's Law of Complaints: Nobody notices when things go right.

From jim at zope.com  Tue Mar 22 11:19:16 2011
From: jim at zope.com (Jim Fulton)
Date: Tue, 22 Mar 2011 06:19:16 -0400
Subject: [Distutils] broken buildout tests on windows was: [Zope-dev]
 Zope Tests: 68 OK, 34 Failed, 6 Unknown
In-Reply-To: <4D885615.7090209@gmail.com>
References: <20110321115756.08B627505B@epy.co.at>
	<im7s8p$b8u$1@dough.gmane.org> <4D885615.7090209@gmail.com>
Message-ID: <AANLkTinmyATkfOMDC54Zw0VX7eCqcSVbQtoXVJnAkAG4@mail.gmail.com>

On Tue, Mar 22, 2011 at 3:56 AM, Adam GROSZER <agroszer.ll at gmail.com> wrote:
> Hello,
>
>>> Subject: FAILED : winbot / zc_buildout_dev py_254_win32
>>> From: buildbot at winbot.zope.org
>>> Date: Sun Mar 20 17:07:28 EDT 2011
>>> URL: http://mail.zope.org/pipermail/zope-tests/2011-March/035690.html
>>>
>>> Subject: FAILED : winbot / zc_buildout_dev py_265_win32
>>> From: buildbot at winbot.zope.org
>>> Date: Sun Mar 20 17:07:39 EDT 2011
>>> URL: http://mail.zope.org/pipermail/zope-tests/2011-March/035691.html
>>>
>>> Subject: FAILED : winbot / zc_buildout_dev py_265_win64
>>> From: buildbot at winbot.zope.org
>>> Date: Sun Mar 20 17:07:50 EDT 2011
>>> URL: http://mail.zope.org/pipermail/zope-tests/2011-March/035692.html
>>>
>>> Subject: FAILED : winbot / zc_buildout_dev py_270_win32
>>> From: buildbot at winbot.zope.org
>>> Date: Sun Mar 20 17:08:00 EDT 2011
>>> URL: http://mail.zope.org/pipermail/zope-tests/2011-March/035693.html
>>>
>>> Subject: FAILED : winbot / zc_buildout_dev py_270_win64
>>> From: buildbot at winbot.zope.org
>>> Date: Sun Mar 20 17:08:14 EDT 2011
>>> URL: http://mail.zope.org/pipermail/zope-tests/2011-March/035694.html
>>
>> More weird Windows build failures::
>>
>>
>> ?c:\buildslave\zc_buildout_dev_py_254_win32\build>bin\test.exe
>> - --exit-with-status -1"
>> ?'bin\test.exe' is not recognized as an internal or external command,
>> operable program or batch file.
>
> Bootstrap failed, it just does not exit with the right exitcode.
> See:
> http://winbot.zope.org/builders/zc_buildout_dev%20py_270_win32/builds/239/steps/bootstrap/logs/stdio
>
> Anyone feeling guilty?

There were some changes made in a sprint recently. I'll investigate.

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From reinout at vanrees.org  Tue Mar 22 11:50:54 2011
From: reinout at vanrees.org (Reinout van Rees)
Date: Tue, 22 Mar 2011 11:50:54 +0100
Subject: [Distutils] issues with namespace packages on Debian Squeeze
	and Python 2.6
In-Reply-To: <20110311082730.20677686@neurotica>
References: <4D7010B8.4070408@gmail.com> <20110311082730.20677686@neurotica>
Message-ID: <im9uue$bsk$1@dough.gmane.org>

On 11-03-11 14:27, Barry Warsaw wrote:
>> I have tried with system default python 2.5, and namespace packages
>> >works. The same when I create a virtual environment with python 2.6.
>> >The problem seems to just be with system default python 2.6.
>> >
>> >I'm using the original setuptools (installed from sources), and not the
>> >one provided by Debian.
> Yes, I think this is going to give you problems because it doesn't know about
> dist-packages, and site-packages is not on sys.path by default in the system
> Python.  Please try again with the Ubuntu setuptools package (which is really
> distribute).  It's been modified to know about dist-packages.

Note that this debian-specific behaviour also "ruins" other things: 
buildout 1.5+ can detect system-wide installed eggs and use them if you 
tell it to. Handy if you want to use stuff like mapnik and so with lots 
of dependencies.

Only: buildout cannot detect the eggs in their non-standard locations as 
buildout obviously doesn't have the ubuntu setuptools' fix...

Do you know any way around this? I'm stuck with buildouts below 1.5 for 
the time being!


Reinout


From carl at oddbird.net  Tue Mar 22 18:57:07 2011
From: carl at oddbird.net (Carl Meyer)
Date: Tue, 22 Mar 2011 13:57:07 -0400
Subject: [Distutils] issues with namespace packages on Debian Squeeze
 and Python 2.6
In-Reply-To: <im9uue$bsk$1@dough.gmane.org>
References: <4D7010B8.4070408@gmail.com> <20110311082730.20677686@neurotica>
	<im9uue$bsk$1@dough.gmane.org>
Message-ID: <4D88E2F3.7070906@oddbird.net>

On 03/22/2011 06:50 AM, Reinout van Rees wrote:
> On 11-03-11 14:27, Barry Warsaw wrote:
>>> I have tried with system default python 2.5, and namespace packages
>>> >works. The same when I create a virtual environment with python 2.6.
>>> >The problem seems to just be with system default python 2.6.
>>> >
>>> >I'm using the original setuptools (installed from sources), and not the
>>> >one provided by Debian.
>> Yes, I think this is going to give you problems because it doesn't
>> know about
>> dist-packages, and site-packages is not on sys.path by default in the
>> system
>> Python.  Please try again with the Ubuntu setuptools package (which is
>> really
>> distribute).  It's been modified to know about dist-packages.

Debian/Ubuntu's setuptools package has been modified to _install into_
dist-packages in /usr/local, but it has no control over sys.path.
Putting dist-packages onto sys.path is handled by Debian/Ubuntu
modifications to the site.py in the Python standard library; it isn't
affected by which setuptools you use.

That said, I haven't looked into the specific problem with namespace
packages - it's still possible that using the Debian/Ubuntu setuptools
package to install the namespace packages might solve that.

> Note that this debian-specific behaviour also "ruins" other things:
> buildout 1.5+ can detect system-wide installed eggs and use them if you
> tell it to. Handy if you want to use stuff like mapnik and so with lots
> of dependencies.
> 
> Only: buildout cannot detect the eggs in their non-standard locations as
> buildout obviously doesn't have the ubuntu setuptools' fix...

This sounds more likely to be caused by not having the site.py changes I
mentioned above. IIRC buildout 1.5 uses "python -S" to avoid the
automatic import of site.py, so it would never see the Debian/Ubuntu
modifications in site.py. As long as buildout uses "-S", I think the
only fix for that would be for buildout's "detect system-wide eggs"
feature to add in explicit workaround/support for dist-packages. This is
what virtualenv does.

Carl

From d.camata at gmail.com  Wed Mar 23 02:56:57 2011
From: d.camata at gmail.com (d.camata at gmail.com)
Date: Tue, 22 Mar 2011 22:56:57 -0300
Subject: [Distutils] VirtualPython proposal
Message-ID: <AANLkTinQu7ByiF-y_jZrxwLcRSWgXxukDH6xDsJJK8Nf@mail.gmail.com>

Hello all,

I've found virtualpython idea in the PSF GSoC page very interesting and I'm
looking for Martin v. Lowis to talk about the proposal. I didn't knew where
to contact him, so I went to IRC and somebody told me to try here.

Regards,

-- 
Douglas Camata
Graduando em Ci?ncia da Computa??o (UENF)

Blog: http://blog.douglascamata.net
Github: http://github.com/douglasamata
Twitter: @douglascamata <http://twitter.com/douglascamata>
<http://twitter.com/douglascamata>Skype: douglas_camata
-----------------------------------
Linux User #509211
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110322/30a43b7f/attachment.html>

From d.camata at gmail.com  Wed Mar 23 04:12:26 2011
From: d.camata at gmail.com (d.camata at gmail.com)
Date: Wed, 23 Mar 2011 00:12:26 -0300
Subject: [Distutils] VirtualPython proposal
In-Reply-To: <AANLkTinQu7ByiF-y_jZrxwLcRSWgXxukDH6xDsJJK8Nf@mail.gmail.com>
References: <AANLkTinQu7ByiF-y_jZrxwLcRSWgXxukDH6xDsJJK8Nf@mail.gmail.com>
Message-ID: <AANLkTi=dQK9_8c+GAcCeZPukKEGCPnVndp6TRHyXWqXq@mail.gmail.com>

Sorry for the lack of information.

I'm a student from Northern University of Rio de Janeiro, Brazil. I met
Python like 1 year ago, when I was applying for a project in the Information
Systems Research Group, at Rio de Janeiro's Federal Institute. My team's
work was to port the main project of the group to a services architecture
using Cyclone, Redis and zc.buildout. Later we moved on from xmlrpc to a
RESTful service.

I built a zc.buildout recipe that bootstraps a clean web2py environment,
installs an app and make it the default app. Contributed to a few
open-source projects, like restfulie-py, specloude, RestMQ and Cyclone, the
last only reporting bugs.

I liked virtualpython idea because I saw in it a very good oportunity to
learn how things like virtualenv and virtualenvwrapper works depthly and
contribute to the community at same time.

2011/3/22 d.camata at gmail.com <d.camata at gmail.com>

> Hello all,
>
> I've found virtualpython idea in the PSF GSoC page very interesting and I'm
> looking for Martin v. Lowis to talk about the proposal. I didn't knew where
> to contact him, so I went to IRC and somebody told me to try here.
>
> Regards,
>
> --
> Douglas Camata
> Graduando em Ci?ncia da Computa??o (UENF)
>
> Blog: http://blog.douglascamata.net
> Github: http://github.com/douglasamata
> Twitter: @douglascamata <http://twitter.com/douglascamata>
> <http://twitter.com/douglascamata>Skype: douglas_camata
> -----------------------------------
> Linux User #509211
>
>


-- 
Douglas Camata
Graduando em Ci?ncia da Computa??o (UENF)

Blog: http://blog.douglascamata.net
Github: http://github.com/douglascamata <http://github.com/douglasamata>
Twitter: @douglascamata <http://twitter.com/douglascamata>
<http://twitter.com/douglascamata>Skype: douglas_camata
-----------------------------------
Linux User #509211
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110323/08337c33/attachment.html>

From reinout at vanrees.org  Wed Mar 23 14:42:10 2011
From: reinout at vanrees.org (Reinout van Rees)
Date: Wed, 23 Mar 2011 14:42:10 +0100
Subject: [Distutils] issues with namespace packages on Debian Squeeze
	and Python 2.6
In-Reply-To: <4D88E2F3.7070906@oddbird.net>
References: <4D7010B8.4070408@gmail.com>
	<20110311082730.20677686@neurotica>	<im9uue$bsk$1@dough.gmane.org>
	<4D88E2F3.7070906@oddbird.net>
Message-ID: <imctbj$tka$1@dough.gmane.org>

On 22-03-11 18:57, Carl Meyer wrote:
>> Only: buildout cannot detect the eggs in their non-standard locations as
>> >  buildout obviously doesn't have the ubuntu setuptools' fix...
> This sounds more likely to be caused by not having the site.py changes I
> mentioned above. IIRC buildout 1.5 uses "python -S" to avoid the
> automatic import of site.py, so it would never see the Debian/Ubuntu
> modifications in site.py. As long as buildout uses "-S", I think the
> only fix for that would be for buildout's "detect system-wide eggs"
> feature to add in explicit workaround/support for dist-packages. This is
> what virtualenv does.

Just making sure: you're suggesting a debian-specific workaround in 
buildout?

Buildout maintainers: would adding such a workaround be OK with you?


Reinout


From leorochael at gmail.com  Wed Mar 23 14:51:10 2011
From: leorochael at gmail.com (Leonardo Rochael Almeida)
Date: Wed, 23 Mar 2011 14:51:10 +0100
Subject: [Distutils] issues with namespace packages on Debian Squeeze
 and Python 2.6
In-Reply-To: <imctbj$tka$1@dough.gmane.org>
References: <4D7010B8.4070408@gmail.com> <20110311082730.20677686@neurotica>
	<im9uue$bsk$1@dough.gmane.org> <4D88E2F3.7070906@oddbird.net>
	<imctbj$tka$1@dough.gmane.org>
Message-ID: <AANLkTimcN+AtB-_EkZQETSEABqwTQkU4sE1+n7of9p9W@mail.gmail.com>

Maybe not a debian-specific workaround, but a setting like:

  [buildout]
  additional-site-packages =
      /usr/lib/python2.6/dist-packages
      /usr/local/lib/python2.6/dist-packages

On Wed, Mar 23, 2011 at 14:42, Reinout van Rees <reinout at vanrees.org> wrote:
> On 22-03-11 18:57, Carl Meyer wrote:
>>>
>>> Only: buildout cannot detect the eggs in their non-standard locations as
>>> > ?buildout obviously doesn't have the ubuntu setuptools' fix...
>>
>> This sounds more likely to be caused by not having the site.py changes I
>> mentioned above. IIRC buildout 1.5 uses "python -S" to avoid the
>> automatic import of site.py, so it would never see the Debian/Ubuntu
>> modifications in site.py. As long as buildout uses "-S", I think the
>> only fix for that would be for buildout's "detect system-wide eggs"
>> feature to add in explicit workaround/support for dist-packages. This is
>> what virtualenv does.
>
> Just making sure: you're suggesting a debian-specific workaround in
> buildout?
>
> Buildout maintainers: would adding such a workaround be OK with you?
>
>
> Reinout
>
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>

From setuptools at bugs.python.org  Wed Mar 23 15:12:52 2011
From: setuptools at bugs.python.org (Tres Seaver)
Date: Wed, 23 Mar 2011 14:12:52 +0000
Subject: [Distutils] [issue123] [PATCH] Tolerate responses with multiple
	Content-Length	headers
In-Reply-To: <1300889572.25.0.438921081513.issue123@psf.upfronthosting.co.za>
Message-ID: <1300889572.25.0.438921081513.issue123@psf.upfronthosting.co.za>


New submission from Tres Seaver <tseaver at palladion.com>:

Some servers (e.g., wwwsearch.sourceforge.net) apparently send multiple
'Content-Length' headers, which causes setuptools to barf trying to convert
a '<length>, <length>' string to an integer.

This bug breaks installing 'mechanize', which lists wwwsearch.sourceforge.net
as its Download-URL, and therefore causes a bunch of Zope-related tests to fail
(e.g., https://mail.zope.org/pipermail/cmf-tests/2011-March/014576.html ).

The attached patch uses 'headers.getheaders('Content-Length')[0] to use only
the first value found.

----------
assignedto: pje
files: setuptools-multi_content_length.patch
messages: 598
nosy: pje, tseaver
priority: urgent
status: unread
title: [PATCH] Tolerate responses with multiple Content-Length headers
Added file: http://bugs.python.org/setuptools/file75/setuptools-multi_content_length.patch

_______________________________________________
Setuptools tracker <setuptools at bugs.python.org>
<http://bugs.python.org/setuptools/issue123>
_______________________________________________
-------------- next part --------------
A non-text attachment was scrubbed...
Name: setuptools-multi_content_length.patch
Type: text/x-diff
Size: 669 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110323/2ace421e/attachment.patch>

From jim at zope.com  Wed Mar 23 16:19:51 2011
From: jim at zope.com (Jim Fulton)
Date: Wed, 23 Mar 2011 11:19:51 -0400
Subject: [Distutils] issues with namespace packages on Debian Squeeze
 and Python 2.6
In-Reply-To: <imctbj$tka$1@dough.gmane.org>
References: <4D7010B8.4070408@gmail.com> <20110311082730.20677686@neurotica>
	<im9uue$bsk$1@dough.gmane.org> <4D88E2F3.7070906@oddbird.net>
	<imctbj$tka$1@dough.gmane.org>
Message-ID: <AANLkTim0pQHpS35RqxsuwfOUNT4Yoexfr1aHV8-Xe51i@mail.gmail.com>

On Wed, Mar 23, 2011 at 9:42 AM, Reinout van Rees <reinout at vanrees.org> wrote:
> On 22-03-11 18:57, Carl Meyer wrote:
>>>
>>> Only: buildout cannot detect the eggs in their non-standard locations as
>>> > ?buildout obviously doesn't have the ubuntu setuptools' fix...
>>
>> This sounds more likely to be caused by not having the site.py changes I
>> mentioned above. IIRC buildout 1.5 uses "python -S" to avoid the
>> automatic import of site.py, so it would never see the Debian/Ubuntu
>> modifications in site.py. As long as buildout uses "-S", I think the
>> only fix for that would be for buildout's "detect system-wide eggs"
>> feature to add in explicit workaround/support for dist-packages. This is
>> what virtualenv does.
>
> Just making sure: you're suggesting a debian-specific workaround in
> buildout?
>
> Buildout maintainers: would adding such a workaround be OK with you?

I abhor screwing w site.py, but buildout is already doing so. I
believe it copies site.py so it *should* work with debian
modifications.  I don't really understand this code and honestly don't
want to.  It was contributed by someone at and for Ubuntu so I'd
expect it to work with debian.  I would submit a bug report and
hopefully, they'll look at it.

In the long term, I'd really like to move this (site.py copying) code
out of buildout into recipes, as I don't want to maintain it.  In the
longer term, I'd love this to get addresses by the pythonv work
somehow. :)

Jim

--
Jim Fulton
http://www.linkedin.com/in/jimfulton

From marius at pov.lt  Wed Mar 23 18:43:47 2011
From: marius at pov.lt (Marius Gedminas)
Date: Wed, 23 Mar 2011 19:43:47 +0200
Subject: [Distutils] New buildout options: checksums	and
 allow-omitted-checksums
In-Reply-To: <4D87E487.2000000@canterbury.ac.nz>
References: <pan.2011.03.17.15.55.03.959118@ID-174572.user.uni-berlin.de>
	<20110318130710.GA28064@fridge.pov.lt>
	<pan.2011.03.18.13.43.04.450362@ID-174572.user.uni-berlin.de>
	<pan.2011.03.21.20.24.02.48043@ID-174572.user.uni-berlin.de>
	<4D87E487.2000000@canterbury.ac.nz>
Message-ID: <20110323174346.GA13648@fridge.pov.lt>

On Tue, Mar 22, 2011 at 12:51:35PM +1300, Greg Ewing wrote:
> Thomas Lotze wrote:
> 
> >After some further offline discussion, I'd like to suggest using MD5 as
> >the default algorithm, though.
> 
> Warnings against using md5 are mainly about cryptographic
> security, aren't they? For just detecting accidental
> corruption it should still be good enough.

Yes, that's my understanding too.

(My only point for raising this was to consider a future-proof syntax for
specifying the checksums, so that we're not locked in the past when the
world moves on.)

Marius Gedminas
-- 
At most companies, programmers aren't trusted with words that a user might
actually see (and for good reason, much of the time).
		-- Joel Spolski
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 190 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110323/efadfd33/attachment.pgp>

From Ronny.Pfannschmidt at gmx.de  Wed Mar 23 20:20:33 2011
From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt)
Date: Wed, 23 Mar 2011 20:20:33 +0100
Subject: [Distutils] the usecase of continious integration based
 release-management (which could use more support on the ds2 side)
Message-ID: <1300908033.3254.14.camel@Klappe2>

Hi,

i'd like to propose making the work-flow of creating releases by just
pushing vcs tags to a ci system as easy as possible

personally all i ever want to do to release a new version is::

  $vcs tag $version
  $vcs push

currently that would require various nasty hacks to get the version
meta-data static on sdist and to grab the version number in a setup hook


there are various ways to ease this, the ones i imagined are

a) provide a hook to get the version, make the version static on
sdist/upload
 pro: straightforward, simple
 cons: kind of a hack

b) provide support for multiple setup-hooks and support for command
mixin's
 
 pro: generic way to do it
 cons: needs more code, more stuff in setup.cfg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110323/c0152bc1/attachment-0001.pgp>

From sienkiew at stsci.edu  Wed Mar 23 21:49:42 2011
From: sienkiew at stsci.edu (Mark Sienkiewicz)
Date: Wed, 23 Mar 2011 16:49:42 -0400
Subject: [Distutils] pythonv: let's also make sure the standard Python
 install includes an "isolated" python
In-Reply-To: <AANLkTikSAZpfZaX=DqwOjk4qCTsaFykjkRVcOGzEYEKK@mail.gmail.com>
References: <AANLkTimrSY=GaOf37PjNDfL4R59vUO96z0n-8fUKq9gQ@mail.gmail.com>	<20110317105948.4557667c@neurotica>
	<4D823A81.2060406@oddbird.net>	<AANLkTinsHDHg8VEsqFNd5hNK5Cwtrf2gFeVp1jptBNP+@mail.gmail.com>	<4D8247B7.30102@oddbird.net>	<20110317190105.2A6C03A4062@sparrow.telecommunity.com>	<4D825B58.502@oddbird.net>	<AANLkTikXVgwEr0ueKdKY7iHtgwO5Fj89kNCO1_jMLcnm@mail.gmail.com>	<20110317220822.E838D3A4062@sparrow.telecommunity.com>	<4D829F01.7070204@oddbird.net>	<20110317213312.1ed4b8d9@neurotica>
	<4D836E22.3030306@oddbird.net>	<AANLkTikCB2=Xg18qyBXOegHhMD_oQ8zj92ts=xUc4A39@mail.gmail.com>	<AANLkTimgJCEPobx=NYcUuK2Dm8OfWYaTiutiFKZZJQrx@mail.gmail.com>	<AANLkTimdj-hZHOC196Ny+YCrxTROGFdWn7W-hX53+V6W@mail.gmail.com>	<4D840BF3.3000004@canterbury.ac.nz>
	<AANLkTikSAZpfZaX=DqwOjk4qCTsaFykjkRVcOGzEYEKK@mail.gmail.com>
Message-ID: <4D8A5CE6.50804@stsci.edu>

Fred Drake wrote:
> On Fri, Mar 18, 2011 at 9:50 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>   
>> We don't have to make it look so Windows-like, though. We could
>> use something more cheerful such as 'python.nothisisnotthedirectory'.
>>     
>
> Yes, but... the sad part is that a turd has to be added, or the name
> changed in even less satisfactory ways.  Case-senselessness is still
> senseless.
>
> /me mourns the loss of case-sense in "pop" operating systems.
>   

Actually, "loss" is not quite what happened here.  Unix-derived systems 
(Unix, Linux, *BSD) are somewhat unusual in having case-sensitive file 
systems.  Of the alphabet soup of operating systems that I've used, I 
can remember case sensitive files systems in the Unix-like systems, Plan 
9, and Apple II DOS -- but the Apple II doesn't have lower case letters 
on the keyboard/display, so I'm not sure it counts.

I used to think that case-sensitive identifiers are a good idea, but I 
now think that it is just too error-prone to use identifiers that differ 
only in case.  Also, it's not portable... :)

b.t.w.  The extension is ".cfg" because RT-11 stored file names in a 
character set called "Radix 50" that could pack 3 characters into a 16 
bit word.  CP/M pretty obviously copied RT-11, MS-DOS copied CP/M, and 
MS Windows was originally a GUI framework for MS-DOS.  Meanwhile, Unix 
initially ran on other DEC systems, so a lot of early Unix users had 
contact with RT-11 or its descendants, and re-used some of the same 
conventions.

So, "We use .cfg because we have calling it that since 1970..."  
Interesting, but perhaps not a particularly compelling reason. :)

 >sniff< - what's that smell?  Oh yeah -- old farts...


From sienkiew at stsci.edu  Wed Mar 23 21:59:35 2011
From: sienkiew at stsci.edu (Mark Sienkiewicz)
Date: Wed, 23 Mar 2011 16:59:35 -0400
Subject: [Distutils] the usecase of continious integration based
 release-management (which could use more support on the ds2 side)
In-Reply-To: <1300908033.3254.14.camel@Klappe2>
References: <1300908033.3254.14.camel@Klappe2>
Message-ID: <4D8A5F37.80009@stsci.edu>

Ronny Pfannschmidt wrote:
> Hi,
>
> i'd like to propose making the work-flow of creating releases by just
> pushing vcs tags to a ci system as easy as possible
>
> personally all i ever want to do to release a new version is::
>
>   $vcs tag $version
>   $vcs push
>
> currently that would require various nasty hacks to get the version
> meta-data static on sdist and to grab the version number in a setup hook
>   

Maybe you should go the other way:  Put the version number in your 
source code.  Make a short script that picks out the version number and 
constructs a tag name for the vcs.  Raise an error if the tag already 
exists.

This method is easy to implement for your project alone, it is easy to 
distribute to others who want to do the same thing, and there are no 
nasty hacks involved.  For example,
 
% python tagdist.py
You didn't change the version number!
% emacs setup.cfg
% python tagdist.py
Tagging release 0.0.2
%

If you find it works well in practice, you might then propose to move 
the script into "python setup.py tagdist".


From greg.ewing at canterbury.ac.nz  Wed Mar 23 22:56:08 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 24 Mar 2011 10:56:08 +1300
Subject: [Distutils] the usecase of continious integration based
 release-management (which could use more support on the ds2 side)
In-Reply-To: <4D8A5F37.80009@stsci.edu>
References: <1300908033.3254.14.camel@Klappe2> <4D8A5F37.80009@stsci.edu>
Message-ID: <4D8A6C78.101@canterbury.ac.nz>

Mark Sienkiewicz wrote:

> Maybe you should go the other way:  Put the version number in your 
> source code.  Make a short script that picks out the version number and 
> constructs a tag name for the vcs.

In my projects I tend to keep the definitive version number
in the Makefile, and have a make target that generates a
version.py file from it. This is convenient because the
Makefile often needs the version number for other things
like creating release tarballs. Tagging vcs commits would
be another use.

-- 
Greg

From alexis at notmyidea.org  Wed Mar 23 23:47:05 2011
From: alexis at notmyidea.org (=?UTF-8?B?QWxleGlzIE3DqXRhaXJlYXU=?=)
Date: Wed, 23 Mar 2011 22:47:05 +0000
Subject: [Distutils] the usecase of continious integration based
 release-management (which could use more support on the ds2 side)
In-Reply-To: <1300908033.3254.14.camel@Klappe2>
References: <1300908033.3254.14.camel@Klappe2>
Message-ID: <4D8A7869.50209@notmyidea.org>

Le 23/03/2011 19:20, Ronny Pfannschmidt a ?crit :
> Hi,

Hey ronny !

> i'd like to propose making the work-flow of creating releases by just
> pushing vcs tags to a ci system as easy as possible

That's an usage a lot of people have, +1 to make it easy to do in a few
steps.

> personally all i ever want to do to release a new version is::
> 
>   $vcs tag $version
>   $vcs push

Do you mean "pysetup upload" (or whatever similar), or are you thinking
about a vcs hook that generates the setup.cfg and upload for you ?

What tools are we talking about ? distutils? distutils2?
setuptools/distribute?

The way to do it with distutils2 would be to register a hook able to get
information from the vcs and put it in the setup.cfg.


> a) provide a hook to get the version, make the version static on
> sdist/upload
>  pro: straightforward, simple
>  cons: kind of a hack

why are you considering it a hack ? seems the right way to me.

> b) provide support for multiple setup-hooks and support for command
> mixin's
>  
>  pro: generic way to do it
>  cons: needs more code, more stuff in setup.cfg

multiple setup-hooks ? Doesnt seems clear to me, can you clarify ?

-- 
Alexis ? http://notmyidea.org

From Ronny.Pfannschmidt at gmx.de  Wed Mar 23 23:47:05 2011
From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt)
Date: Wed, 23 Mar 2011 23:47:05 +0100
Subject: [Distutils] the usecase of continious integration based
 release-management (which could use more support on the ds2 side)
In-Reply-To: <4D8A5F37.80009@stsci.edu>
References: <1300908033.3254.14.camel@Klappe2>  <4D8A5F37.80009@stsci.edu>
Message-ID: <1300920425.2951.2.camel@Klappe2>

On Wed, 2011-03-23 at 16:59 -0400, Mark Sienkiewicz wrote:
> Ronny Pfannschmidt wrote:
> > Hi,
> >
> > i'd like to propose making the work-flow of creating releases by just
> > pushing vcs tags to a ci system as easy as possible
> >
> > personally all i ever want to do to release a new version is::
> >
> >   $vcs tag $version
> >   $vcs push
> >
> > currently that would require various nasty hacks to get the version
> > meta-data static on sdist and to grab the version number in a setup hook
> >   
> 
> Maybe you should go the other way:  Put the version number in your 
> source code.  Make a short script that picks out the version number and 
> constructs a tag name for the vcs.  Raise an error if the tag already 
> exists.
> 

i never ever want to have a version number of the project outside of a
vcs tag inside the vcs repo

also its convient to automatically imply different version numbers for
each commit (which includes a dev marker + a revision id)

> This method is easy to implement for your project alone, it is easy to 
> distribute to others who want to do the same thing, and there are no 
> nasty hacks involved.  For example,
>  
> % python tagdist.py
> You didn't change the version number!
> % emacs setup.cfg
> % python tagdist.py
> Tagging release 0.0.2
> %
> 
> If you find it works well in practice, you might then propose to move 
> the script into "python setup.py tagdist".

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110323/8723de74/attachment.pgp>

From ben+python at benfinney.id.au  Wed Mar 23 23:57:57 2011
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 24 Mar 2011 09:57:57 +1100
Subject: [Distutils] the usecase of continious integration based
	release-management (which could use more support on the ds2 side)
References: <1300908033.3254.14.camel@Klappe2> <4D8A5F37.80009@stsci.edu>
	<4D8A6C78.101@canterbury.ac.nz>
Message-ID: <87ei5x30ne.fsf@benfinney.id.au>

Greg Ewing <greg.ewing at canterbury.ac.nz> writes:

> In my projects I tend to keep the definitive version number in the
> Makefile, and have a make target that generates a version.py file from
> it. This is convenient because the Makefile often needs the version
> number for other things like creating release tarballs. Tagging vcs
> commits would be another use.

I often do something similar, but IMO simpler: the definitive version
string (note: version strings are rarely single numbers!) is kept as the
sole content of a file at the top of the project tree, named ?version?.

That way, it's available equally to anything that can read text content
from a file ? the Makefile, any program code, even many configuration
files.

-- 
 \       ?I went to a general store. They wouldn't let me buy anything |
  `\                                     specifically.? ?Steven Wright |
_o__)                                                                  |
Ben Finney


From Ronny.Pfannschmidt at gmx.de  Thu Mar 24 00:02:24 2011
From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt)
Date: Thu, 24 Mar 2011 00:02:24 +0100
Subject: [Distutils] the usecase of continious integration based
 release-management (which could use more support on the ds2 side)
In-Reply-To: <4D8A7869.50209@notmyidea.org>
References: <1300908033.3254.14.camel@Klappe2> <4D8A7869.50209@notmyidea.org>
Message-ID: <1300921344.2951.7.camel@Klappe2>

On Wed, 2011-03-23 at 22:47 +0000, Alexis M?taireau wrote:
> Le 23/03/2011 19:20, Ronny Pfannschmidt a ?crit :
> > Hi,
> 
> Hey ronny !
> 
> > i'd like to propose making the work-flow of creating releases by just
> > pushing vcs tags to a ci system as easy as possible
> 
> That's an usage a lot of people have, +1 to make it easy to do in a few
> steps.
> 
> > personally all i ever want to do to release a new version is::
> > 
> >   $vcs tag $version
> >   $vcs push
> 
> Do you mean "pysetup upload" (or whatever similar), or are you thinking
> about a vcs hook that generates the setup.cfg and upload for you ?

"pysetup upload" would happen somewhere else, most likely a continious
build/integration server

> 
> What tools are we talking about ? distutils? distutils2?
> setuptools/distribute?
> 
mostly distutils2, the rest is in deprecation/better not change limbo
anyway

> The way to do it with distutils2 would be to register a hook able to get
> information from the vcs and put it in the setup.cfg.
> 
i tried hacking that up, it also needs a hook in the sdist command to
statically put the data there
> 
> > a) provide a hook to get the version, make the version static on
> > sdist/upload
> >  pro: straightforward, simple
> >  cons: kind of a hack
> 
> why are you considering it a hack ? seems the right way to me.

its the implementation of a single dynamic metadata item as kind of
special case

> > b) provide support for multiple setup-hooks and support for command
> > mixin's
> >  
> >  pro: generic way to do it
> >  cons: needs more code, more stuff in setup.cfg
> 
> multiple setup-hooks ? Doesnt seems clear to me, can you clarify ?
> 
people might want to do more than just the hook for the version
it shouldn't be necessary to manually write functions that combine those
hooks, if all that's necessary is call in order
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110324/82976cfd/attachment.pgp>

From reinout at vanrees.org  Thu Mar 24 03:30:22 2011
From: reinout at vanrees.org (Reinout van Rees)
Date: Thu, 24 Mar 2011 03:30:22 +0100
Subject: [Distutils] the usecase of continious integration based
 release-management (which could use more support on the ds2 side)
In-Reply-To: <1300908033.3254.14.camel@Klappe2>
References: <1300908033.3254.14.camel@Klappe2>
Message-ID: <imeabv$89l$1@dough.gmane.org>

On 23-03-11 20:20, Ronny Pfannschmidt wrote:
> personally all i ever want to do to release a new version is::
>
>    $vcs tag $version
>    $vcs push
>
> currently that would require various nasty hacks to get the version
> meta-data static on sdist and to grab the version number in a setup hook

This means a bit of machinery on the server side, basically.

I'm doing this on my laptop with zest.releaser (see pypi).
I just enter "fullrelease" and press enter a few times and it is ready.
It grabs the version number from setup.py.
It proposes that version number as the new tag.
It then increases the version number and commits that one (with a dev 
marker).

It does a bit more, like updating your changelog with a new header after 
releasing and by recording your release date in that very same changelog.

=> I think this is best handled on the client's side: the tag you're 
making is tied to quite a lot of things and I think setup.py's version 
is the best source of that.


Reinout



From barry at python.org  Thu Mar 24 19:17:47 2011
From: barry at python.org (Barry Warsaw)
Date: Thu, 24 Mar 2011 14:17:47 -0400
Subject: [Distutils] DRAFT PEP 396 - module version number
Message-ID: <20110324141747.6d90382e@neurotica.wooz.org>

Hi distutilizers!

At Pycon, I signed up to write an informational PEP on module version numbers.
There have been some email discussion since then, and while I haven't
completely worked my way through all those messages yet, I think I have
covered the important top-level issues.

I'm posting my current draft (as yet uncommitted to the new hg repository)
here before taking it to python-dev.  I'd love to get your feedback about any
aspect of the PEP, but especially on deriving module version numbers from
setup.cfg files, and on including them in metadata at build time (as opposed
to installation time).   Also, I'd appreciate any corrections based on my
misunderstanding of distutils2/packaging and the relevant PEPs.

Current draft attached below.  Remember, this is an informational PEP, so its
purpose is to codify best practices.  Module authors will continue to have the
freedom to do whatever they want.

Cheers,
-Barry

PEP: 396
Title: Module Version Numbers
Version: $Revision: 65628 $
Last-Modified: $Date: 2008-08-10 09:59:20 -0400 (Sun, 10 Aug 2008) $
Author: Barry Warsaw <barry at python.org>
Status: Draft
Type: Informational
Content-Type: text/x-rst
Created: 2011-03-16
Post-History:


Abstract
========

Given that it is useful and common to specify version numbers for
Python modules, and given that different ways of doing this have grown
organically within the Python community, it is useful to establish
standard conventions for module authors to adhere to and reference.
This informational PEP describes best practices for Python module
authors who want to define the version number of their Python module.

Conformance with this PEP is optional, however other Python tools (such as
`distutils2` [1]_) may be adapted to use the conventions defined here.


User Stories
============

Alice is writing a new module, called ``alice.py``, which she wants to
share with other Python developers.  ``alice.py`` is a simple module
and lives in one file.  Alice wants to specify a version number so
that her users can tell which version they are using.  Because her
module lives entirely in one file, she wants to add the version number
to that file.

Bob has written a module called ``bob.py`` which he has shared with
many users.  ``bob.py`` contains a version number for the convenience
of his users.  Bob learns about the Cheeseshop [2]_, and adds some
simple packaging using classic distutils so that he can upload *The
Bob Package* to the Cheeseshop.  Because ``bob.py`` already specifies
a version number which his users can access programmatically, he wants
the same API to continue to work even though his users now get it from
the Cheeseshop.

Carole maintains several namespace packages, each of which are
independently developed and distributed.  In order for her users to
properly specify dependencies on the right versions of her packages,
she specifies the version numbers in the namespace package's
``setup.py`` file.  Because Carol wants to have to update one version
number per package, she specifies the version number in her module and
has the ``setup.py`` extract the module version number at *sdist*
tarball build time.

David maintains a package in the standard library, and also produces
standalone versions for other versions of Python.  The standard
library copy defines the version number in the module, and this same
version number is used for the standalone distributions as well.


Rationale
=========

Python modules, both in the standard library and available from third
parties, have long included version numbers.  There are established
de-facto standards for describing version numbers, and many ad-hoc
ways have grown organically over the years.  Often, version numbers
can be retrieved from a module programmatically, by importing the
module and inspecting an attribute.  Classic Python distutils
``setup()`` functions [3]_ describe a ``version`` argument where the
package's version number can be specified.  PEP 8 [4]_ describes the
use of a module attribute called ``__version__`` for recording
"Subversion, CVS, or RCS" version strings using keyword expansion.  In
the PEP author's own email archives, the earliest example of the use
of an ``__version__`` module attribute by independent module
developers dates back to 1995.

Another example of version information is the sqlite3 [5]_ library
with its ``sqlite_version_info``, ``version``, and ``version_info``
attributes.  It may not be immediately obvious which attribute
contains a version number for the module, and which contains a version
number for the underlying SQLite3 library.

This informational PEP codifies established practice, and recommends
standard ways of describing module version numbers, along with some
use cases for when -- and when *not* -- to include them.  Its adoption
is purely voluntary, but other tools in the Python universe may
support the standards defined herein.


Specification
=============

#. In general, modules in the standard library SHOULD NOT have version
   numbers.  They implicitly carry the version number of the Python
   release they are included in.

#. On a case-by-case basis, standard library modules which are also
   distributed as standalone packages for other Python versions MAY
   include a module version number when included in the standard
   library, and SHOULD include a version number when packaged
   separately.

#. When a module includes a version number, it SHOULD be available in
   the ``__version__`` attribute on that module.

#. For modules which are also packages, the top level module namespace
   SHOULD include the ``__version__`` attribute.

#. For modules which live inside a namespace package, the sub-package
   name SHOULD include the ``__version__`` attribute.  The namespace
   module itself SHOULD NOT include its own ``__version__`` attribute.

#. The ``__version__`` attribute's value SHOULD be a simple string.

#. Module version numbers SHOULD conform to the normalized version
   format specified in PEP 386 [6]_.

#. Module version numbers SHOULD NOT contain version control system
   supplied revision numbers, or any other semantically different
   version numbers (e.g. underlying library version number).

#. Wherever a ``__version__`` attribute exists, a module MAY also
   include a ``__version_info__`` attribute, containing a tuple
   representation of the module version number, for easy comparisons.

#. ``__version_info__`` SHOULD be of the format returned by PEP 386's
   ``parse_version()`` function.

#. The ``version`` attribute in a classic distutils ``setup.py`` file,
   or the ``Version`` metadata field in a PEP 345 [7]_ SHOULD be
   derived from the ``__version__`` field, or vice versa.


Examples
========

Retrieving the version number from a third party package::

    >>> import bzrlib
    >>> bzrlib.__version__
    '2.3.0'

Retrieving the version number from a standard library package that is
also distributed as a standalone module::

    >>> import email
    >>> email.__version__
    '5.1.0'

Version numbers for namespace packages::

    >>> import flufl.i18n
    >>> import flufl.enum
    >>> import flufl.lock

    >>> print flufl.i18n.__version__
    1.0.4
    >>> print flufl.enum.__version__
    3.1
    >>> print flufl.lock.__version__
    2.1

    >>> import flufl
    >>> flufl.__version__
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    AttributeError: 'module' object has no attribute '__version__'
    >>>


Deriving
========

Module version numbers can appear in at least two places, and
sometimes more.  For example, in accordance with this PEP, they are
available programmatically on the module's ``__version__`` attribute.
They also need to be specified in either the classic distutils
``setup()`` function, as the ``version`` attribute in a ``setup.py``,
or in the distutils2 ``setup.cfg`` file's ``version`` keyword, and
they need to get into the PEP 345 package metadata, preferably at
*sdist* package build-time.  It's also desirable for module authors to
only have to specify the version number once, and have all the other
uses derive from this single definition.

While there are any number of ways this could be done, this section
describes one possible approach, for each scenario.

Let's say Elle adds this attribute to her module, ``elle.py``::

    __version__ = '3.1.1'


Classic distutils
-----------------

In classic distutils, the simplest way to add this version to the
``setup()`` function in ``setup.py`` is to do something like this::

    from elle import __version__
    setup(name='elle', version=__version__)

In the PEP author's experience however, this can fail in some cases,
such as when the module uses automatic Python 3 conversion via the
``2to3`` program.  In that case, it's not much more difficult to write
a little code to parse the ``__version__`` from the file rather than
importing it::

    import re
    DEFAULT_VERSION_RE = re.compile(r'(?P<version>\d+\.\d(?:\.\d+)?)')

    def get_version(filename, pattern=None):
        if pattern is None:
            cre = DEFAULT_VERSION_RE
        else:
            cre = re.compile(pattern)
        with open(filename) as fp:
            for line in fp:
                if line.startswith('__version__'):
                    mo = cre.search(line)
                    assert mo, 'No valid __version__ string found'
                    return mo.group('version')
        raise AssertionError('No __version__ assignment found')

    setup(name='elle', version=get_version('elle.py'))


Distutils2
----------

Because the distutils2 style ``setup.cfg`` is declarative, we can't
run any code to extract the ``__version__`` attribute, either via
import or via parsing.  This PEP suggests a special key be allowed for
the ``version`` field in a ``setup.cfg`` file to indicate "get the
version from this file".  Something like this might work::

    [metadata]
    version: parse:elle.py

where ``parse`` means to use a parsing method similar to the above, on
the file named after the colon.  The exact recipe for doing this will
be discussed in the appropriate distutils2 development forum.

An alternative is to only define the version number in setup.cfg and
use the ``pkgutil`` module [8]_ to make it available
programmatically.  E.g. in ``elle.py``::

    from distutils2._backport import pkgutil
    __version__ = pkgutil.get_distribution('elle').metadata['version']


Static egg info
===============

For distutils2, it is highly desirable to place the version
information into the egg metadata at build-time rather than
install-time.  This way, the metadata will be available for
introspection even when the code is not installed.  For example, a
package's metadata can be made available via a simple HTTP GET
operation on a URL within the package's presence on the Cheeseshop.


References
==========

.. [1] Distutils2 documentation
   (http://distutils2.notmyidea.org/)

.. [2] The Cheeseshop (Python Package Index)
   (http://pypi.python.org)

.. [3] http://docs.python.org/distutils/setupscript.html

.. [4] PEP 8, Style Guide for Python Code
   (http://www.python.org/dev/peps/pep-0008)

.. [5] sqlite3 module documentation
   (http://docs.python.org/library/sqlite3.html)

.. [6] PEP 386, Changing the version comparison module in Distutils
   (http://www.python.org/dev/peps/pep-0386/)

.. [7] PEP 345, Metadata for Python Software Packages 1.2
   (http://www.python.org/dev/peps/pep-0345/#version)

.. [8] pkgutil - Package utilities
   (http://distutils2.notmyidea.org/library/pkgutil.html)


Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End:

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110324/708aa768/attachment.pgp>

From merwok at netwok.org  Fri Mar 25 03:52:14 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Fri, 25 Mar 2011 03:52:14 +0100
Subject: [Distutils] DRAFT PEP 396 - module version number
In-Reply-To: <20110324141747.6d90382e@neurotica.wooz.org>
References: <20110324141747.6d90382e@neurotica.wooz.org>
Message-ID: <4D8C035E.4050306@netwok.org>

Hi Barry,

Thanks for writing this PEP.  I?m +1 on the general idea, but not on all
sections.  I agree with all text that I haven?t quoted; quoted parts are
commented with my reservations, suggested amendments and typo fixes.

Martin and Alexis are directly cc?d because I have asked them to confirm
or reply to two things that I wasn?t sure about.

> `distutils2` [1]_) may be adapted to use the conventions defined here.
Nit: I never rely on docutils/sphinx default role (using a bare `text`
without explicit :role:), cause I never remember what it is, and the
produced text may not be what you thought it would.

> Alice is writing a new module, called ``alice.py``, which she wants to
> share with other Python developers.  ``alice.py`` is a simple module
> and lives in one file.
You should call the module alice and the file alice.py to eliminate
weirdness.  (Ditto for other examples throughout the PEP.)

> Bob learns about the Cheeseshop [2]_, and adds some
> simple packaging using classic distutils so that he can upload *The
> Bob Package* to the Cheeseshop.
I know that the referenced PEP 345 call these things ?software
packages?, but let?s try to use consistent terminology.  See threads
starting at
http://mail.python.org/pipermail/distutils-sig/2011-March/017438.html
and http://mail.python.org/pipermail/distutils-sig/2011-March/017450.html .

> Carole maintains several namespace packages, each of which are
> independently developed and distributed.
I am not sure we should advertise setuptools namespace packages, given
that standardization is under way (PEP 382).  On one hand it would be
childish not to acknowledge that setuptools is widely used, on the other
hand in this particular time and place I think we should wait for
official namespace packages to be implemented and talk about those.
That said, even if the mechanism changes, the packaging-related parts
won?t, so the story is useful.  Make that +0 after all.

> at *sdist* tarball build time.
Do non-UNIX people understand ?tarball??  You could remove it or replace
it with ?archive?.

> David maintains a package in the standard library, and also produces
> standalone versions for other versions of Python.
Thoughtful hat tip :)

> Often, version numbers
> can be retrieved from a module programmatically, by importing the
> module and inspecting an attribute.  Classic Python distutils
> ``setup()`` functions [3]_ describe a ``version`` argument where the
> package's version number can be specified.
See above for package vs. distribution terminology.

This makes me think: It is obvious to us that Python packages are a type
of Python modules, but the PEP could mention that near the beginning, to
make clear that your PEP is relevant to any kind of module, whatever its
physical incarnation.

> of an ``__version__`` module attribute by independent module
s/an/a/

> Another example of version information is the sqlite3 [5]_ library
the sqlite3 module

> This informational PEP codifies established practice, and recommends
> standard ways of describing module version numbers, along with some
> use cases for when -- and when *not* -- to include them.  Its adoption
> is purely voluntary, but other tools in the Python universe may
> support the standards defined herein.
You say ?other tools? without having referred to tools before, I find it
confusing (I?m not a native speaker, however).  The text could mention
here that official tools will have opt-in support for the PEP:

  Its adoption is purely voluntary; the packaging tools in the standard
  library will provide optional support for the standards defined herein
  (see section XX), and other tools in the Python universe may comply
  too.

> #. On a case-by-case basis, standard library modules which are also
>    distributed as standalone packages for other Python versions MAY
Problematic ?packages? again.  What about this: ?which are also released
in standalone form for other Python versions??

> #. For modules which are also packages, the top level module namespace
>    SHOULD include the ``__version__`` attribute.
Just a remark: I don?t remember ever reading the term ?top-level module
namespace?.  It?s not hard to understand, but it might be helpful to
some readers if you add ?(i.e. the :file:`{somepackage}/__init__.py`
file)?.  (The brackets will cause the enclosed text to be marked up as
replaceable text, just a nicety.)

> #. For modules which live inside a namespace package, the sub-package
>    name SHOULD include the ``__version__`` attribute.
I think this works with both setuptools and PEP 382 namespace packages,
which is nice (see above questioning).

> The namespace module itself SHOULD NOT include its own
> ``__version__`` attribute.
I guess this makes sense for setuptools namespace packages, but from my
understanding of PEP 382, it is possible to have a Python package that
is a namespace package and has submodules.  (I hope Martin will correct
me if needed.)  This thing (?portion? in PEP 382 lingo) should be
allowed to declare a version IMO.

> #. The ``__version__`` attribute's value SHOULD be a simple string.
What?s a non-simple string?  This may be only me, but here I think you
tried to be simple and had the opposite effect.  I?d just use the simple
?a string? or the precise ?a string (str object in 3.x)?.

> #. The ``version`` attribute in a classic distutils ``setup.py`` file,
>    or the ``Version`` metadata field in a PEP 345 [7]_ SHOULD be
>    derived from the ``__version__`` field, or vice versa.
?In a PEP 345? what? :) file, I guess.

I think this part should either take a position, that is choose one way
instead of vaguely telling about two ways, or be removed, given that the
section Deriving has the same info.

> They also need to be specified in either the classic distutils
> ``setup()`` function, as the ``version`` attribute in a ``setup.py``,
> or in the distutils2 ``setup.cfg`` file's ``version`` keyword,
I?d say the setup.py has a version argument and the setup.cfg has a
version key.

> they need to get into the PEP 345 package metadata, preferably at
> *sdist* package build-time.
s/package metadata/metadata/; s/*sdist* package/*sdist*/

> In the PEP author's experience however, this can fail in some cases,
> such as when the module uses automatic Python 3 conversion via the
> ``2to3`` program.
Just to make this 100% explicit, you could add ?because the setup.py
script is executed by Python 3 before the elle module has been converted?.

> In that case, it's not much more difficult to write a little code
> to parse the ``__version__`` from the file rather than
The function could be simplified (use re.search, hard-code the re), but
that?s very minor.

> Because the distutils2 style ``setup.cfg`` is declarative, we can't
> run any code to extract the ``__version__`` attribute, either via
> import or via parsing.  This PEP suggests a special key be allowed for
> the ``version`` field in a ``setup.cfg`` file to indicate "get the
> version from this file".
With my distutils2 hat on, I recommend a KISS approach similar to what?s
done for the long description: just define another key in the setup.cfg
specification:

  [metadata]
  version-from-file: elle.py

> An alternative is to only define the version number in setup.cfg and
> use the ``pkgutil`` module [8]_ to make it available
> programmatically.  E.g. in ``elle.py``::
> 
>     from distutils2._backport import pkgutil
>     __version__ = pkgutil.get_distribution('elle').metadata['version']
One of the main complaint against setuptools is that having to change
your application code because of the packaging tool used was not a good
idea.  Having to use require instead of import or resource_whatever
instead of open (or get_data, the most sadly underused function in the
stdlib) because you use setuptools instead of distutils was a bad thing.

As stated in the PEP, having a __version__ attribute in the module is
common, so my opinion is that making the packaging tool use that info is
the Right Thing?, and having the dependency in the reverse sense is
wrong.  I don?t see a problem with having harmless duplication in the
*installed* system, once in  elle.__version__ and once in the pkgutil
metadata database.

(FWIW, I personally think that it?s not crazy to have the version number
duplicated in the module, setup script, changelog, Sphinx conf.py file
and so on; other people like Ronny don?t even want to have the version
number in one file, but make their build process use the VCS tag.  I
think this PEP addresses a useful middle ground.)

> Static egg info
Is that the distutils 2.5+ generated metadata?  You may want to read PEP
376 and update the terminology to ?dist-info metadata? <0.2 wink>.

> For example, a package's metadata can be made available via a simple
> HTTP GET operation on a URL within the package's presence on the
> Cheeseshop.
First, insert usual remark about ?package? here; second, I think
discussing PyPI is out of scope here.  The rationale that ?the metadata
will be available for introspection even when the code is not installed?
is already strong enough.

> .. [1] Distutils2 documentation
>    (http://distutils2.notmyidea.org/)
Alexis, is this still the good link to use?

Regards

From alexis at notmyidea.org  Fri Mar 25 17:43:41 2011
From: alexis at notmyidea.org (=?UTF-8?B?QWxleGlzIE3DqXRhaXJlYXU=?=)
Date: Fri, 25 Mar 2011 16:43:41 +0000
Subject: [Distutils] DRAFT PEP 396 - module version number
In-Reply-To: <4D8C035E.4050306@netwok.org>
References: <20110324141747.6d90382e@neurotica.wooz.org>
	<4D8C035E.4050306@netwok.org>
Message-ID: <4D8CC63D.2020401@notmyidea.org>

On 25/03/2011 02:52, ?ric Araujo wrote:
>> >  .. [1] Distutils2 documentation
>> >      (http://distutils2.notmyidea.org/)
> Alexis, is this still the good link to use?

Yes, at least for now. Waiting for docs.python.org ;)

-- 
Alexis ? http://notmyidea.org

From chris at simplistix.co.uk  Fri Mar 25 20:24:05 2011
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 25 Mar 2011 19:24:05 +0000
Subject: [Distutils] [buildout] "private" releases
In-Reply-To: <imih2a$uig$1@dough.gmane.org>
References: <imih2a$uig$1@dough.gmane.org>
Message-ID: <4D8CEBD5.7010903@simplistix.co.uk>

Hi Christian,

On 25/03/2011 16:49, Christian Theune wrote:
> the German speaking Zope Users Group (DZUG e.V.) organizes a series of 4
> sprints this year to support feature development within the proximity of
> the ZTK and solve problems encountered by Zope, Plone and Python developers.
>
<snip>
> Topics
> ======
>
> * Discussing how to deal with "private" releases

FWIW, I've had no problems with this, here's a sample buildout.cfg:

[buildout]
extensions = lovely.buildouthttp
find-links = https://example.com/password/protected/folder

...and just dump the .tgz sdists in that folder.

Of course, if you don't need password protection such as when you have 
your "egg server" on a private network, you just need the find-links.

I'm not really sure why people have written a complicated array of "egg 
servers" and the like when a simple http or file system served folder is 
just fine ;-)

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From ruslan.spivak at gmail.com  Sat Mar 26 03:35:44 2011
From: ruslan.spivak at gmail.com (Ruslan Spivak)
Date: Fri, 25 Mar 2011 22:35:44 -0400
Subject: [Distutils] What happened to Buildout documentation sprint?
Message-ID: <m3oc4yfw1r.fsf@gmail.com>

Hi,

I just saw
http://svn.zope.org/zc.buildout/branches/documentation/doc/how-tos.rest?view=auto
and the topic list looks very interesting.
Was there any activity around Buildout documentation during the Buildout
sprint?

Ruslan
-- 
Genius may have its limitations, but stupidity is not thus handicapped.
   - Elbert Hubbard

From jim at zope.com  Sat Mar 26 22:20:34 2011
From: jim at zope.com (Jim Fulton)
Date: Sat, 26 Mar 2011 17:20:34 -0400
Subject: [Distutils] What happened to Buildout documentation sprint?
In-Reply-To: <m3oc4yfw1r.fsf@gmail.com>
References: <m3oc4yfw1r.fsf@gmail.com>
Message-ID: <AANLkTimPpn13+8Mn9ysqUgVgCpDXc6+W8dpk11nVOxtw@mail.gmail.com>

On Fri, Mar 25, 2011 at 10:35 PM, Ruslan Spivak <ruslan.spivak at gmail.com> wrote:
> Hi,
>
> I just saw
> http://svn.zope.org/zc.buildout/branches/documentation/doc/how-tos.rest?view=auto
> and the topic list looks very interesting.
> Was there any activity around Buildout documentation during the Buildout
> sprint?

Nada.

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From ygingras at ygingras.net  Sun Mar 27 21:36:10 2011
From: ygingras at ygingras.net (Yannick Gingras)
Date: Sun, 27 Mar 2011 15:36:10 -0400
Subject: [Distutils] DRAFT PEP 396 - module version number
In-Reply-To: <4D8C035E.4050306@netwok.org>
References: <20110324141747.6d90382e@neurotica.wooz.org>
	<4D8C035E.4050306@netwok.org>
Message-ID: <201103271536.10593.ygingras@ygingras.net>

On March 24, 2011, ?ric Araujo wrote:
> > Bob learns about the Cheeseshop [2]_, and adds some
> > simple packaging using classic distutils so that he can upload *The
> > Bob Package* to the Cheeseshop.
> I know that the referenced PEP 345 call these things ?software
> packages?, but let?s try to use consistent terminology.  See threads
> starting at
> http://mail.python.org/pipermail/distutils-sig/2011-March/017438.html
> and http://mail.python.org/pipermail/distutils-sig/2011-March/017450.html .

?ric, can you summarize the discussion that took place on these
threads?  As far as I can tell, no consensus was reached for a new
terminology and so the old definition still prevails:

  http://docs.python.org/dev/distutils/introduction#concepts-terminology

In that case, a "package" in PEP 396 would refer to:

  a module that contains other modules; typically contained in a
  directory in the filesystem and distinguished from other directories
  by the presence of a file __init__.py.

I fell like the usage proposed for *The Bob Package* is pretty good,
except for the choice of "upload".  Of course, you don't upload the
package, you upload "distributions" that contain that package.

-- 
Yannick Gingras
http://ygingras.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110327/692363a9/attachment.pgp>

From setuptools at bugs.python.org  Mon Mar 28 16:50:55 2011
From: setuptools at bugs.python.org (Tres Seaver)
Date: Mon, 28 Mar 2011 14:50:55 +0000
Subject: [Distutils] [issue124] Add support for installing test requirements
	to 'develop'	command
In-Reply-To: <1301323854.99.0.448458193747.issue124@psf.upfronthosting.co.za>
Message-ID: <1301323854.99.0.448458193747.issue124@psf.upfronthosting.co.za>


New submission from Tres Seaver <tseaver at palladion.com>:

The attached patch adds a '--with-test-requirements' / '-t' option to the 'develop' command:  the rationale is that when developing a package, one should
be able to run the tests without polluting the local checkout directory with
the eggs specified by its 'tests_require':  those eggs should be installed just
as the other ones mandated by 'install_requires'.

I would actually prefer to make this behavior the default, perhaps with a
command-line-switch to disable it, but think that backward compatibility
trumps that itch.

----------
assignedto: pje
files: setuptools-0.6-develop-with_test_requirements.patch
messages: 606
nosy: pje, tseaver
priority: feature
status: unread
title: Add support for installing test requirements to 'develop' command
Added file: http://bugs.python.org/setuptools/file76/setuptools-0.6-develop-with_test_requirements.patch

_______________________________________________
Setuptools tracker <setuptools at bugs.python.org>
<http://bugs.python.org/setuptools/issue124>
_______________________________________________
-------------- next part --------------
A non-text attachment was scrubbed...
Name: setuptools-0.6-develop-with_test_requirements.patch
Type: text/x-diff
Size: 1728 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110328/f423f20c/attachment.patch>

From phunehehe at gmail.com  Mon Mar 28 17:08:53 2011
From: phunehehe at gmail.com (Hoang Xuan Phu)
Date: Mon, 28 Mar 2011 22:08:53 +0700
Subject: [Distutils] easy_install doesn't work when there are multiple
	Content-Length headers
Message-ID: <AANLkTik5YRcSYPhH6_28vPC0scwpO_T2Ctnu125qqE+Q@mail.gmail.com>

Hi all,

Just today I'm using easy_install to install mechanize and it is failing
with the error "ValueError: invalid literal for int() with base 10: '382727,
382727'". By reading the source code and looking at the headers I see that
the server is returning 2 Content-Length headers (same value, 382727), which
is turned into '382727, 382727'. Fixing this should be very easy and I can
do it then submit a patch. I'm just wondering, as distutils seem to be in a
forking process, what's the best way to solve this?

Ho?ng Xu?n Ph?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110328/846f284b/attachment.html>

From fdrake at acm.org  Mon Mar 28 18:29:31 2011
From: fdrake at acm.org (Fred Drake)
Date: Mon, 28 Mar 2011 12:29:31 -0400
Subject: [Distutils] easy_install doesn't work when there are multiple
 Content-Length headers
In-Reply-To: <AANLkTik5YRcSYPhH6_28vPC0scwpO_T2Ctnu125qqE+Q@mail.gmail.com>
References: <AANLkTik5YRcSYPhH6_28vPC0scwpO_T2Ctnu125qqE+Q@mail.gmail.com>
Message-ID: <AANLkTim40ggPE5KutdOjJjiN1XMqbpx+=rLjOw6qqpnf@mail.gmail.com>

On Mon, Mar 28, 2011 at 11:08 AM, Hoang Xuan Phu <phunehehe at gmail.com> wrote:
> Just today I'm using easy_install to install mechanize and it is failing
> with the error "ValueError: invalid literal for int() with base 10: '382727,
> 382727'". By reading the source code and looking at the headers I see that
> the server is returning 2 Content-Length headers (same value, 382727), which
> is turned into?'382727, 382727'. Fixing this should be very easy and I can
> do it then submit a patch. I'm just wondering, as distutils seem to be in a
> forking process, what's the best way to solve this?

This is a problem with httplib, which should be resilient to this
situation.  I've already filed a bug against SourceForge, for
generating an extra (useless) header, but it's only vaguely a bug.


? -Fred

--
Fred L. Drake, Jr.? ? <fdrake at acm.org>
"A storm broke loose in my mind."? --Albert Einstein

From tseaver at palladion.com  Mon Mar 28 18:50:11 2011
From: tseaver at palladion.com (Tres Seaver)
Date: Mon, 28 Mar 2011 12:50:11 -0400
Subject: [Distutils] easy_install doesn't work when there are multiple
 Content-Length headers
In-Reply-To: <AANLkTik5YRcSYPhH6_28vPC0scwpO_T2Ctnu125qqE+Q@mail.gmail.com>
References: <AANLkTik5YRcSYPhH6_28vPC0scwpO_T2Ctnu125qqE+Q@mail.gmail.com>
Message-ID: <imqe83$7ho$1@dough.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 03/28/2011 11:08 AM, Hoang Xuan Phu wrote:
> Hi all,
> 
> Just today I'm using easy_install to install mechanize and it is failing
> with the error "ValueError: invalid literal for int() with base 10: '382727,
> 382727'". By reading the source code and looking at the headers I see that
> the server is returning 2 Content-Length headers (same value, 382727), which
> is turned into '382727, 382727'. Fixing this should be very easy and I can
> do it then submit a patch. I'm just wondering, as distutils seem to be in a
> forking process, what's the best way to solve this?

I reported this SF bug:

 http://sourceforge.net/apps/trac/sourceforge/ticket/18486

PJE has checked in my patch working around this bug.  See issue 123:

 http://bugs.python.org/setuptools/issue123


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk2QvEMACgkQ+gerLs4ltQ6M8ACdEjNaxg1w3M2jCc0JuqfAJeLa
SG8AoLOfZdbIpvDwrgvidOk4g72i61zU
=ZoDQ
-----END PGP SIGNATURE-----


From pje at telecommunity.com  Mon Mar 28 19:58:23 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 28 Mar 2011 13:58:23 -0400
Subject: [Distutils] easy_install doesn't work when there are multiple
 Content-Length headers
In-Reply-To: <AANLkTik5YRcSYPhH6_28vPC0scwpO_T2Ctnu125qqE+Q@mail.gmail.c
 om>
References: <AANLkTik5YRcSYPhH6_28vPC0scwpO_T2Ctnu125qqE+Q@mail.gmail.com>
Message-ID: <20110328175848.54CA53A4077@sparrow.telecommunity.com>

At 10:08 PM 3/28/2011 +0700, Hoang Xuan Phu wrote:
>Hi all,
>
>Just today I'm using easy_install to install mechanize and it is 
>failing with the error "ValueError: invalid literal for int() with 
>base 10: '382727, 382727'". By reading the source code and looking 
>at the headers I see that the server is returning 2 Content-Length 
>headers (same value, 382727), which is turned into? '382727, 
>382727'. Fixing this should be very easy and I can do it then submit 
>a patch. I'm just wondering, as distutils seem to be in a forking 
>process, what's the best way to solve this?

easy_install -U setuptools will update you to a version that fixes 
this, as I've already checked in a fix.


From phunehehe at gmail.com  Tue Mar 29 03:48:23 2011
From: phunehehe at gmail.com (Hoang Xuan Phu)
Date: Tue, 29 Mar 2011 08:48:23 +0700
Subject: [Distutils] easy_install doesn't work when there are multiple
 Content-Length headers
In-Reply-To: <20110328175848.54CA53A4077@sparrow.telecommunity.com>
References: <AANLkTik5YRcSYPhH6_28vPC0scwpO_T2Ctnu125qqE+Q@mail.gmail.com>
	<20110328175848.54CA53A4077@sparrow.telecommunity.com>
Message-ID: <AANLkTikkJG=ATr_pYa8qTYFpbOwu1KsQQ=fyoGrx7VDA@mail.gmail.com>

Thanks a lot P.J. Eby

It's great that things get fixed!

Ho?ng Xu?n Ph?


On Tue, Mar 29, 2011 at 12:58 AM, P.J. Eby <pje at telecommunity.com> wrote:

> At 10:08 PM 3/28/2011 +0700, Hoang Xuan Phu wrote:
>
>> Hi all,
>>
>> Just today I'm using easy_install to install mechanize and it is failing
>> with the error "ValueError: invalid literal for int() with base 10: '382727,
>> 382727'". By reading the source code and looking at the headers I see that
>> the server is returning 2 Content-Length headers (same value, 382727), which
>> is turned into? '382727, 382727'. Fixing this should be very easy and I can
>> do it then submit a patch. I'm just wondering, as distutils seem to be in a
>> forking process, what's the best way to solve this?
>>
>
> easy_install -U setuptools will update you to a version that fixes this, as
> I've already checked in a fix.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110329/6b00989e/attachment-0001.html>

From vinay_sajip at yahoo.co.uk  Tue Mar 29 15:45:50 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Tue, 29 Mar 2011 13:45:50 +0000 (UTC)
Subject: [Distutils] pythonv, take two
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<4D8172FB.3020109@oddbird.net>	<loom.20110317T112246-771@post.gmane.org>	<4D8233EF.7070205@oddbird.net>
	<loom.20110318T004329-895@post.gmane.org>
	<4D83A807.30003@oddbird.net>
Message-ID: <loom.20110329T154252-221@post.gmane.org>

Carl Meyer <carl <at> oddbird.net> writes:

> Thanks for the additional digging in here. I think your analysis is
> right - it actually occurred to me yesterday that this could be the
> problem, and I filed a bug to track it here:

> 3. The fully-reliable fix would be to somehow give the copied binary a
> hint where to find the right standard library, and this would involve
> adding something to the algorithm in getpath.c. The hint could take the
> form of a key in the config file, but I'd really like to avoid fully
> parsing the config-file in C and then again in Python later on. The hint
> could also be some kind of specially-formatted comment line at the top
> of the config file, which would require less C code to find and parse?
> 
> Any thoughts on this (or alternative solutions I haven't thought of) are
> most welcome.
> 

Hi Carl,

I've been thinking about the C-level configuration some more (cpythonv issue
#6). Suppose we have a virtual env at absolute path <testenv>, and in
<testenv>/bin we have a copied, non-installed Python, obtained from absolute
path <pythonsrc>. Say we provide the location of the source in the
configuration, and that this is read in getpath.c. Logically, it should just
replace the argv0_path value, and if the subsequent logic is unchanged, at the
point that site.py's main() runs,  you would have (on Linux i686):

sys.prefix, sys.exec_prefix would have values set from the configure script
('/usr/local' by default).
sys.path would be ['/usr/local/lib/python33.zip', '<pythonsrc>/Lib',
'<pythonsrc>/Lib/plat-linux2', '<pythonsrc>/build/lib.linux-i686-3.3']

which seems OK.

If a virtual environment is in absolute path <testenv>, then after the call to
virtualize() in site.py, we would have sys.prefix and sys.exec_prefix both set
to <testenv>. Since sys.executable still points to <testenv>/bin, the code in
sysconfig.py appears not to be doing the right thing, e.g.
sysconfig._PROJECT_BASE would be <testenv>/bin, which seems wrong.

So it seems as if site.py and/or sysconfig.py might need further changes, as
well as changes to getpath.c. What do you think?

Regards,

Vinay Sajip


From carl at oddbird.net  Tue Mar 29 18:36:13 2011
From: carl at oddbird.net (Carl Meyer)
Date: Tue, 29 Mar 2011 12:36:13 -0400
Subject: [Distutils] pythonv, take two
In-Reply-To: <loom.20110329T154252-221@post.gmane.org>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<4D8172FB.3020109@oddbird.net>	<loom.20110317T112246-771@post.gmane.org>	<4D8233EF.7070205@oddbird.net>	<loom.20110318T004329-895@post.gmane.org>	<4D83A807.30003@oddbird.net>
	<loom.20110329T154252-221@post.gmane.org>
Message-ID: <4D920A7D.9040109@oddbird.net>

Hi Vinay,

On 03/29/2011 09:45 AM, Vinay Sajip wrote:
> I've been thinking about the C-level configuration some more (cpythonv issue
> #6). Suppose we have a virtual env at absolute path <testenv>, and in
> <testenv>/bin we have a copied, non-installed Python, obtained from absolute
> path <pythonsrc>. Say we provide the location of the source in the
> configuration, and that this is read in getpath.c. Logically, it should just
> replace the argv0_path value, and if the subsequent logic is unchanged, at the
> point that site.py's main() runs,  you would have (on Linux i686):
> 
> sys.prefix, sys.exec_prefix would have values set from the configure script
> ('/usr/local' by default).
> sys.path would be ['/usr/local/lib/python33.zip', '<pythonsrc>/Lib',
> '<pythonsrc>/Lib/plat-linux2', '<pythonsrc>/build/lib.linux-i686-3.3']
> 
> which seems OK.
> 
> If a virtual environment is in absolute path <testenv>, then after the call to
> virtualize() in site.py, we would have sys.prefix and sys.exec_prefix both set
> to <testenv>. Since sys.executable still points to <testenv>/bin, the code in
> sysconfig.py appears not to be doing the right thing, e.g.
> sysconfig._PROJECT_BASE would be <testenv>/bin, which seems wrong.
> 
> So it seems as if site.py and/or sysconfig.py might need further changes, as
> well as changes to getpath.c. What do you think?

In general, I think the "copied binary" case should be as similar as
possible to the "symlinked binary" case. Python doesn't dereference
symlinks in setting sys.executable, so in either case sys.executable
will point to the virtual environment's binary, which I think is clearly
what we want (think code using sys.executable to decide how to re-exec
itself).

The changes I made to sysconfig thus far were the minimum necessary to
be able to successfully install stuff in a pythonv virtualenv. On a
brief glance, it looks to me like you are right; we'll need to modify
the setting of _PROJECT_BASE in sysconfig.py to be virtual-aware (this
is part -- maybe all -- of what's needed to fix issue #1).

Also, see issue #10 - I'm wavering on whether changing sys.prefix and
sys.exec_prefix to point to the virtualenv is actually the right
approach. We're already forced to modify sysconfig.py to be
virtual-aware in order to avoid having to hackily copy/symlink bits of
the stdlib and include directories and the Makefile and config.h and
whatnot into the virtualenv. Once we've taken the step of making
sysconfig virtual-aware, it seems to me that perhaps sys.prefix is
better left pointing to the actual Python installation, and we add a
sys.virtual_prefix or some such to tell site.py and sysconfig.py where
"virtualized" stuff belongs.

There are a variety of possible meanings of sys.prefix, and the only one
we really want to modify is "where should I install third-party
packages?" Looking at sys.prefix references in the stdlib, I see them in
trace, tkinter, and idlelib. It appears to me that in all of those
cases, having sys.prefix pointing to the virtualenv would be wrong.

If we switch to sys.virtual_prefix and leave sys.prefix alone, the
danger would be that some installer that's using sys.prefix directly,
and ignoring sysconfig.py, would try to install stuff to the global
Python installation. But I'm not aware of any such installer, and I
think we could safely call any installer of Python code that ignores the
layout schemes in sysconfig thoroughly broken anyway.

Your thoughts? (Or anyone else?)

Carl

P.S. The elephant in the room here is the fact that if we want pythonv
to be backwards-compatible with distutils/distribute, which we do, we
can't get away with just modifying sysconfig.py, we also have to modify
the old distutils/sysconfig.py. In general distutils is frozen; I'm
hoping that Tarek will let me get away with a few changes for pythonv if
they aren't changing APIs at all, just modifying some install-scheme
paths to be virtual-aware.

From jjkunce at gmail.com  Tue Mar 29 22:25:51 2011
From: jjkunce at gmail.com (Jeff Kunce)
Date: Tue, 29 Mar 2011 16:25:51 -0400
Subject: [Distutils] buildout, setuptools, sourceforge problems
Message-ID: <AANLkTi=dvcYVTTVv8caMMVkb1m-58_9mMgZtn9ySQJL+@mail.gmail.com>

There seem to be some problems with packages on sourceforge not working with
buildout. Something about sourceforge is sending two content-length
headers.  Anything the buildout gurus here can add?

Description of problem. Suggests just eliminating sourceforge from allowed
hosts:
http://plone.293351.n2.nabble.com/buildout-dies-due-quot-invalid-literal-for-int-with-base-10-quot-td6200410.html

Ticket at sourceforge:
http://sourceforge.net/apps/trac/sourceforge/ticket/18486

Thanks.

  -- Jeff
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110329/edb8a69f/attachment.html>

From pje at telecommunity.com  Tue Mar 29 22:54:27 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 29 Mar 2011 16:54:27 -0400
Subject: [Distutils] buildout, setuptools, sourceforge problems
In-Reply-To: <AANLkTi=dvcYVTTVv8caMMVkb1m-58_9mMgZtn9ySQJL+@mail.gmail.c
 om>
References: <AANLkTi=dvcYVTTVv8caMMVkb1m-58_9mMgZtn9ySQJL+@mail.gmail.com>
Message-ID: <20110329205459.1A5933A40AF@sparrow.telecommunity.com>

At 04:25 PM 3/29/2011 -0400, Jeff Kunce wrote:
>There seem to be some problems with packages on sourceforge not 
>working with buildout. Something about sourceforge is sending two 
>content-length headers.

Update your setuptools to 0.6c12dev-r88975 for the fix.

(If you're using a forked setuptoools, you'll need to uninstall it 
first so it doesn't block your update.)


From vinay_sajip at yahoo.co.uk  Wed Mar 30 01:00:27 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Tue, 29 Mar 2011 23:00:27 +0000 (UTC)
Subject: [Distutils] pythonv, take two
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<4D8172FB.3020109@oddbird.net>	<loom.20110317T112246-771@post.gmane.org>	<4D8233EF.7070205@oddbird.net>	<loom.20110318T004329-895@post.gmane.org>	<4D83A807.30003@oddbird.net>
	<loom.20110329T154252-221@post.gmane.org>
	<4D920A7D.9040109@oddbird.net>
Message-ID: <loom.20110330T005934-410@post.gmane.org>

Carl Meyer <carl <at> oddbird.net> writes:

> In general, I think the "copied binary" case should be as similar as
> possible to the "symlinked binary" case. Python doesn't dereference
> symlinks in setting sys.executable, so in either case sys.executable
> will point to the virtual environment's binary, which I think is clearly
> what we want (think code using sys.executable to decide how to re-exec
> itself).

I agree.

> The changes I made to sysconfig thus far were the minimum necessary to
> be able to successfully install stuff in a pythonv virtualenv. On a
> brief glance, it looks to me like you are right; we'll need to modify
> the setting of _PROJECT_BASE in sysconfig.py to be virtual-aware (this
> is part -- maybe all -- of what's needed to fix issue #1).
> 
> Also, see issue #10 - I'm wavering on whether changing sys.prefix and
> sys.exec_prefix to point to the virtualenv is actually the right
> approach. We're already forced to modify sysconfig.py to be
> virtual-aware in order to avoid having to hackily copy/symlink bits of
> the stdlib and include directories and the Makefile and config.h and
> whatnot into the virtualenv. Once we've taken the step of making
> sysconfig virtual-aware, it seems to me that perhaps sys.prefix is
> better left pointing to the actual Python installation, and we add a
> sys.virtual_prefix or some such to tell site.py and sysconfig.py where
> "virtualized" stuff belongs.

This sounds better, too.

> If we switch to sys.virtual_prefix and leave sys.prefix alone, the
> danger would be that some installer that's using sys.prefix directly,
> and ignoring sysconfig.py, would try to install stuff to the global
> Python installation. But I'm not aware of any such installer, and I
> think we could safely call any installer of Python code that ignores the
> layout schemes in sysconfig thoroughly broken anyway.
> 
> Your thoughts? (Or anyone else?)

What about the possibility of third-party code which uses
sys.prefix/sys.exec_prefix, though? (I mean apart from the installer code.)

> P.S. The elephant in the room here is the fact that if we want pythonv
> to be backwards-compatible with distutils/distribute, which we do, we
> can't get away with just modifying sysconfig.py, we also have to modify
> the old distutils/sysconfig.py. In general distutils is frozen; I'm
> hoping that Tarek will let me get away with a few changes for pythonv if
> they aren't changing APIs at all, just modifying some install-scheme
> paths to be virtual-aware.

Here's hoping :-)

On the question of a C-readable configuration file, I've started work on this
on the assumption that it's a UTF8-encoded file which contains lines of the
form key = value. Having thought about it, since this file is only for
bootstrapping the virtualenv anyway, there's no strong case for it to be
ConfigParser-compatible.

I've modified calculate_path in getpath.c to look for the file adjacent to the
executable or one level up. If found, we look for a line "home =
/path/to/source/of/copied/python" and, if found, replace argv0_path with the
value, then let the existing code take its course. I've modified your
virtualize() in site.py to cater for the new config file format: we default the
prefix to the parent of the directory containing the executable (hence, the
virtualenv root), then set sys.virtual_prefix to this prefix, and optionally
disallow the system site-packages from the system path. For now (since
[distutils.]sysconfig is not sys.virtual_prefix aware) I continue to set
sys.prefix and sys.exec_prefix to sys.virtual_prefix.

So as not to cause confusion with existing config files you might have, I
changed the config file name to 'env.cfg'. This can be changed to the
appropriate bikeshed colour in two places - site.py and getpath.c.

Progress seems good. To test:

1. Clone the repo at https://bitbucket.org/vinay.sajip/pythonv
2. In that repo, run ./configure and make
3. Download the "pmv.py" script at https://gist.github.com/893507
4. Run pmv.py with the python you just built - see the header comment in that
script.

Note that the pmv.py (Poor Man's virtualize.py) script deliberately runs with a
copied rather than a symlinked Python.

Regards,

Vinay Sajip



From fdrake at acm.org  Wed Mar 30 05:49:07 2011
From: fdrake at acm.org (Fred Drake)
Date: Tue, 29 Mar 2011 23:49:07 -0400
Subject: [Distutils] buildout, setuptools, sourceforge problems
In-Reply-To: <20110329205459.1A5933A40AF@sparrow.telecommunity.com>
References: <AANLkTi=dvcYVTTVv8caMMVkb1m-58_9mMgZtn9ySQJL+@mail.gmail.com>
	<20110329205459.1A5933A40AF@sparrow.telecommunity.com>
Message-ID: <AANLkTi=nSP-+1To+aO76bHQsWrbwTQMUpzuMQfXWaU-r@mail.gmail.com>

On Tue, Mar 29, 2011 at 4:54 PM, P.J. Eby <pje at telecommunity.com> wrote:
> Update your setuptools to 0.6c12dev-r88975 for the fix.

Egads, Philip!  Do regular version numbers cost more in your neighborhood?


? -Fred

--
Fred L. Drake, Jr.? ? <fdrake at acm.org>
"A storm broke loose in my mind."? --Albert Einstein

From wichert at wiggy.net  Wed Mar 30 08:32:12 2011
From: wichert at wiggy.net (Wichert Akkerman)
Date: Wed, 30 Mar 2011 08:32:12 +0200
Subject: [Distutils] buildout, setuptools, sourceforge problems
In-Reply-To: <20110329205459.1A5933A40AF@sparrow.telecommunity.com>
References: <AANLkTi=dvcYVTTVv8caMMVkb1m-58_9mMgZtn9ySQJL+@mail.gmail.com>
	<20110329205459.1A5933A40AF@sparrow.telecommunity.com>
Message-ID: <4D92CE6C.7090809@wiggy.net>

On 2011-3-29 22:54, P.J. Eby wrote:
> At 04:25 PM 3/29/2011 -0400, Jeff Kunce wrote:
>> There seem to be some problems with packages on sourceforge not
>> working with buildout. Something about sourceforge is sending two
>> content-length headers.
>
> Update your setuptools to 0.6c12dev-r88975 for the fix.

Would you be willing to make a 0.6c12 release as well?

Wichert.

-- 
Wichert Akkerman <wichert at wiggy.net>   It is simple to make things.
http://www.wiggy.net/                  It is hard to make things simple.

From marius at pov.lt  Wed Mar 30 12:58:18 2011
From: marius at pov.lt (Marius Gedminas)
Date: Wed, 30 Mar 2011 13:58:18 +0300
Subject: [Distutils] buildout, setuptools, sourceforge problems
In-Reply-To: <AANLkTi=dvcYVTTVv8caMMVkb1m-58_9mMgZtn9ySQJL+@mail.gmail.com>
References: <AANLkTi=dvcYVTTVv8caMMVkb1m-58_9mMgZtn9ySQJL+@mail.gmail.com>
Message-ID: <20110330105818.GA8264@fridge.pov.lt>

On Tue, Mar 29, 2011 at 04:25:51PM -0400, Jeff Kunce wrote:
> There seem to be some problems with packages on sourceforge not working with
> buildout. Something about sourceforge is sending two content-length
> headers.  Anything the buildout gurus here can add?
> 
> Description of problem. Suggests just eliminating sourceforge from allowed
> hosts:
> http://plone.293351.n2.nabble.com/buildout-dies-due-quot-invalid-literal-for-int-with-base-10-quot-td6200410.html
> 
> Ticket at sourceforge:
> http://sourceforge.net/apps/trac/sourceforge/ticket/18486

One workaround people have suggested is to add

  [buildout]
  allow-hosts =
    *.python.org

  # plus other machines that are *not* sourceforge.net, if you use eggs
  # that aren't uploaded to PyPI

to your buildout.cfg.

Marius Gedminas
-- 
I am a computer. I am dumber than any human and smarter than any administrator.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 190 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110330/447d7415/attachment.pgp>

From jim at zope.com  Wed Mar 30 15:08:28 2011
From: jim at zope.com (Jim Fulton)
Date: Wed, 30 Mar 2011 09:08:28 -0400
Subject: [Distutils] [buildout] "private" releases
In-Reply-To: <4D8CEBD5.7010903@simplistix.co.uk>
References: <imih2a$uig$1@dough.gmane.org> <4D8CEBD5.7010903@simplistix.co.uk>
Message-ID: <AANLkTinkHfT8Rm1B58fve_eME-28r7m02Sd3wtxBdiWX@mail.gmail.com>

On Fri, Mar 25, 2011 at 3:24 PM, Chris Withers <chris at simplistix.co.uk> wrote:
> Hi Christian,
>
> On 25/03/2011 16:49, Christian Theune wrote:
>>
>> the German speaking Zope Users Group (DZUG e.V.) organizes a series of 4
>> sprints this year to support feature development within the proximity of
>> the ZTK and solve problems encountered by Zope, Plone and Python
>> developers.
>>
> <snip>
>>
>> Topics
>> ======
>>
>> * Discussing how to deal with "private" releases
>
> FWIW, I've had no problems with this, here's a sample buildout.cfg:

...

We do something similar with sftp (zc.buildoutsftp).  To publish eggs,
we just use scp.
The advantage of this is that it leverages ssh infrastructure, so *no*
additional password management is needed.  This is wildly better, IMO,
than keeping passwords in clear text in your buildout configuration or
in a dot file.

Jim



-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From carl at oddbird.net  Wed Mar 30 19:35:33 2011
From: carl at oddbird.net (Carl Meyer)
Date: Wed, 30 Mar 2011 13:35:33 -0400
Subject: [Distutils] pythonv, take two
In-Reply-To: <loom.20110330T005934-410@post.gmane.org>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<4D8172FB.3020109@oddbird.net>	<loom.20110317T112246-771@post.gmane.org>	<4D8233EF.7070205@oddbird.net>	<loom.20110318T004329-895@post.gmane.org>	<4D83A807.30003@oddbird.net>	<loom.20110329T154252-221@post.gmane.org>	<4D920A7D.9040109@oddbird.net>
	<loom.20110330T005934-410@post.gmane.org>
Message-ID: <4D9369E5.1050506@oddbird.net>

Hi Vinay,

On 03/29/2011 07:00 PM, Vinay Sajip wrote:
>> If we switch to sys.virtual_prefix and leave sys.prefix alone, the
>> danger would be that some installer that's using sys.prefix directly,
>> and ignoring sysconfig.py, would try to install stuff to the global
>> Python installation. But I'm not aware of any such installer, and I
>> think we could safely call any installer of Python code that ignores the
>> layout schemes in sysconfig thoroughly broken anyway.
>>
>> Your thoughts? (Or anyone else?)
> 
> What about the possibility of third-party code which uses
> sys.prefix/sys.exec_prefix, though? (I mean apart from the installer code.)

Right. The question is, for third-party code using
sys.prefix/sys.exec_prefix, which value would be the right one, the
installation prefix or the virtual prefix? It depends on what they're
using it for. I did a quick Google Code Search for "sys.prefix," and
unfortunately right on the front page I can find examples where either
approach we take would be "wrong."

The majority of uses I found are people copying code from trace.py which
is intended to exclude the standard library and system-wide packages
from coverage tracing. In this case, leaving sys.prefix pointing to the
installation prefix would give the desired result. Also, for instance,
here where matplotlib's setup looks for DLLs on Windows, the system
prefix would be correct: http://bit.ly/eQSVYc

On the other hand, here scons uses sys.prefix to try to find
site-packages: http://bit.ly/ik0PwJ  If we leave sys.prefix pointing to
the installation directory, this would leave scons unable to find its
config file in the virtualenv's site-packages directory. Unless they fix
their code to use the get_python_lib() function in sysconfig, which is
what they ought to be doing instead of duplicating all that logic.

So... it seems to me that we're likely to break _some_ third-party code
using sys.prefix regardless of what we do here. My instinct says adding
sys.virtual_prefix and leaving sys.prefix alone is the better approach,
but I'm not very firmly entrenched in that position.

> On the question of a C-readable configuration file, I've started work on this
> on the assumption that it's a UTF8-encoded file which contains lines of the
> form key = value. Having thought about it, since this file is only for
> bootstrapping the virtualenv anyway, there's no strong case for it to be
> ConfigParser-compatible.

Agreed. I only used ConfigParser initially because that was easiest from
Python code :-)

> I've modified calculate_path in getpath.c to look for the file adjacent to the
> executable or one level up. If found, we look for a line "home =
> /path/to/source/of/copied/python" and, if found, replace argv0_path with the
> value, then let the existing code take its course. I've modified your
> virtualize() in site.py to cater for the new config file format: we default the
> prefix to the parent of the directory containing the executable (hence, the
> virtualenv root), then set sys.virtual_prefix to this prefix, and optionally
> disallow the system site-packages from the system path. For now (since
> [distutils.]sysconfig is not sys.virtual_prefix aware) I continue to set
> sys.prefix and sys.exec_prefix to sys.virtual_prefix.

This sounds good.

> So as not to cause confusion with existing config files you might have, I
> changed the config file name to 'env.cfg'. This can be changed to the
> appropriate bikeshed colour in two places - site.py and getpath.c.
> 
> Progress seems good. To test:
> 
> 1. Clone the repo at https://bitbucket.org/vinay.sajip/pythonv
> 2. In that repo, run ./configure and make
> 3. Download the "pmv.py" script at https://gist.github.com/893507
> 4. Run pmv.py with the python you just built - see the header comment in that
> script.
> 
> Note that the pmv.py (Poor Man's virtualize.py) script deliberately runs with a
> copied rather than a symlinked Python.

Hmm, it doesn't seem to be working for me. sys.prefix and
sys.exec_prefix are still /usr/local using the python binary in my
"pmv", and sys.virtual_prefix is not set at all, despite env.cfg being
present and looking as it should.

Oddly when I run the compiled python binary directly from the checkout,
it does set sys.virtual_prefix to point one level above the checkout,
despite there being no env.cfg in the vicinity at all.

You can see both of these things (in reverse order) here:
http://paste.pocoo.org/show/362798/

I'll dig in a bit more and see if I can figure out what's happening.

Is there a reason you didn't base your changesets on mine (in the
cpythonv repo), and instead apparently copied over a bunch of the
changes manually as a diff? I can transplant or copy your work back, or
try to do a full merge, but in the long run if we'll both be working on
this it seems best if we are easily able to merge our work back and forth.

Carl

From sienkiew at stsci.edu  Wed Mar 30 23:41:45 2011
From: sienkiew at stsci.edu (Mark Sienkiewicz)
Date: Wed, 30 Mar 2011 17:41:45 -0400
Subject: [Distutils] reservations about pythonv
In-Reply-To: <4D939BD2.7080003@cryptelium.net>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<20110316223904.288080ba@neurotica>
	<4D839A72.1030705@stsci.edu> <4D939BD2.7080003@cryptelium.net>
Message-ID: <4D93A399.30108@stsci.edu>

kiorky wrote:
> You have rpath to record search paths for shared libraries at link time.
> This enables you not to set LD_LIBRARY_PATH at runtime.
>
> Something like "cc -Wl,-rpath,/path/to/libdir -L/path/to/libdir -lfoo" is what
> you want.
>   

You left out the hard part:  How do I make distutils do that?



> Le 18/03/2011 18:46, Mark Sienkiewicz a ?crit :
>   
>>> The above just means my memory is too faulty to of much use ;).  I'll just
>>> echo Carl's request for specific cases where $LD_LIBRARY_PATH needs to be set.
>>>   
>>>       
>> Here is a case that might resemble what you are talking about:
>>
>> Compile a C extension that requires a shared library that is not in the standard
>> system path.  To import it, LD_LIBRARY_PATH needs to be right.
>>
>> This is not really different from what happens in a compiled language, except in
>> one way:  In C, I can compile it -static or I can give the full path to the .so
>> file.  Either results in a thing that works without LD_LIBRARY_PATH.
>>
>> With distutils, you can't.  It goes to great lengths to ensure that you can only
>> compile a C extension with "cc ... -L/some/directory -lname" -- I can't find any
>> way to make it do "cc ... /some/directory/libname.so"
>>
>> So, the real problem here is that distutils uses "cc -L", but it demonstrates a
>> case where LD_LIBRARY_PATH can be important to a python program even when Python
>> itself can run without it.
>>
>>
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> http://mail.python.org/mailman/listinfo/distutils-sig
>>     
>
>   


From vinay_sajip at yahoo.co.uk  Thu Mar 31 01:18:39 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 30 Mar 2011 23:18:39 +0000 (UTC)
Subject: [Distutils] pythonv, take two
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<4D8172FB.3020109@oddbird.net>	<loom.20110317T112246-771@post.gmane.org>	<4D8233EF.7070205@oddbird.net>	<loom.20110318T004329-895@post.gmane.org>	<4D83A807.30003@oddbird.net>	<loom.20110329T154252-221@post.gmane.org>	<4D920A7D.9040109@oddbird.net>
	<loom.20110330T005934-410@post.gmane.org>
	<4D9369E5.1050506@oddbird.net>
Message-ID: <loom.20110331T005457-50@post.gmane.org>

Carl Meyer <carl <at> oddbird.net> writes:

> So... it seems to me that we're likely to break _some_ third-party code
> using sys.prefix regardless of what we do here. My instinct says adding
> sys.virtual_prefix and leaving sys.prefix alone is the better approach,
> but I'm not very firmly entrenched in that position.

That's problematic, of course (third-party breakage, I mean). It may be that
despite using virtual_prefix being a cleaner approach, it may be necessary
to alter sys.prefix to ensure compatibility, a la virtualenv.

> Hmm, it doesn't seem to be working for me. sys.prefix and
> sys.exec_prefix are still /usr/local using the python binary in my
> "pmv", and sys.virtual_prefix is not set at all, despite env.cfg being
> present and looking as it should.
> 
> Oddly when I run the compiled python binary directly from the checkout,
> it does set sys.virtual_prefix to point one level above the checkout,
> despite there being no env.cfg in the vicinity at all.
> 
> You can see both of these things (in reverse order) here:
> http://paste.pocoo.org/show/362798/
> 
> I'll dig in a bit more and see if I can figure out what's happening.

It may be that the thing isn't working properly, but it's masked on my
system because I have an installed 3.3 in /usr/local (from when I was doing
my pip/virtualenv tests). This means that there is a Makefile in a location
which would not be there if I'd never installed 3.3 using sudo make install.

However, the getpath.c stuff should work - to test this, insert an
"import pdb; pdb.set_trace()" in site.main, and check sys.path. This should
contain entries from the virtualenv. The problems are more likely to be in
the Python code: the C code always sets sys.prefix and sys.exec_prefix to
/usr/local (or whatever configure was invoked with) and the sys.prefix /
sys.exec_prefix stuff is done in site.virtualize, as before.

I'll try to test this on a machine which never had 3.3 installed - it seems
you can't uninstall Python using sudo make uninstall :-(

> Is there a reason you didn't base your changesets on mine (in the
> cpythonv repo), and instead apparently copied over a bunch of the
> changes manually as a diff? I can transplant or copy your work back, or
> try to do a full merge, but in the long run if we'll both be working on
> this it seems best if we are easily able to merge our work back and forth.

Sorry - I got in a muddle because I've got too many clones around and ran
into free disk space issues, and goofed somewhere. Don't bother to do a full
merge for now, as I'm not sure if my code's working yet. Since I don't expect
to change more than 4 files (getpath.c, site.py, distutils/sysconfig.py and
sysconfig.py) I don't think we'll have too much trouble merging in due
course, and if/when we can agree that my experiment in pythonv is worth
taking further, I'll do the work of merging into cpythonv. For now, though,
my changes might be treading on your toes, making merging needlessly
annoying.

I'll try to test on a Python-clean machine in a day or two, will report back
then.

Regards,

Vinay



From kiorky at cryptelium.net  Wed Mar 30 23:08:34 2011
From: kiorky at cryptelium.net (kiorky)
Date: Wed, 30 Mar 2011 23:08:34 +0200
Subject: [Distutils] reservations about pythonv
In-Reply-To: <4D839A72.1030705@stsci.edu>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<20110316223904.288080ba@neurotica>
	<4D839A72.1030705@stsci.edu>
Message-ID: <4D939BD2.7080003@cryptelium.net>

You have rpath to record search paths for shared libraries at link time.
This enables you not to set LD_LIBRARY_PATH at runtime.

Something like "cc -Wl,-rpath,/path/to/libdir -L/path/to/libdir -lfoo" is what
you want.

Le 18/03/2011 18:46, Mark Sienkiewicz a ?crit :
> 
>> The above just means my memory is too faulty to of much use ;).  I'll just
>> echo Carl's request for specific cases where $LD_LIBRARY_PATH needs to be set.
>>   
> 
> Here is a case that might resemble what you are talking about:
> 
> Compile a C extension that requires a shared library that is not in the standard
> system path.  To import it, LD_LIBRARY_PATH needs to be right.
> 
> This is not really different from what happens in a compiled language, except in
> one way:  In C, I can compile it -static or I can give the full path to the .so
> file.  Either results in a thing that works without LD_LIBRARY_PATH.
> 
> With distutils, you can't.  It goes to great lengths to ensure that you can only
> compile a C extension with "cc ... -L/some/directory -lname" -- I can't find any
> way to make it do "cc ... /some/directory/libname.so"
> 
> So, the real problem here is that distutils uses "cc -L", but it demonstrates a
> case where LD_LIBRARY_PATH can be important to a python program even when Python
> itself can run without it.
> 
> 
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig

-- 
Cordialement,
KiOrKY
GPG Key FingerPrint: 0x1A1194B7681112AF
Pensez ? l?environnement.
N?imprimez ce courriel que si vous en avez vraiment besoin.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 261 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110330/c36226b4/attachment.pgp>

From pje at telecommunity.com  Thu Mar 31 05:23:57 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 30 Mar 2011 23:23:57 -0400
Subject: [Distutils] pythonv, take two
In-Reply-To: <4D9369E5.1050506@oddbird.net>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>
	<4D8172FB.3020109@oddbird.net>
	<loom.20110317T112246-771@post.gmane.org>
	<4D8233EF.7070205@oddbird.net>
	<loom.20110318T004329-895@post.gmane.org>
	<4D83A807.30003@oddbird.net>
	<loom.20110329T154252-221@post.gmane.org>
	<4D920A7D.9040109@oddbird.net>
	<loom.20110330T005934-410@post.gmane.org>
	<4D9369E5.1050506@oddbird.net>
Message-ID: <20110331032429.C3D403A4077@sparrow.telecommunity.com>

At 01:35 PM 3/30/2011 -0400, Carl Meyer wrote:
>So... it seems to me that we're likely to break _some_ third-party code
>using sys.prefix regardless of what we do here. My instinct says adding
>sys.virtual_prefix and leaving sys.prefix alone is the better approach,
>but I'm not very firmly entrenched in that position.

Looking at it from a software distribution POV, I would say that the 
virtual prefix is what it should point to, since that means things 
won't get installed to the wrong place.

(Of course, a configuration option could be used to override it, if 
really necessary.) 


From carl at oddbird.net  Thu Mar 31 07:32:51 2011
From: carl at oddbird.net (Carl Meyer)
Date: Thu, 31 Mar 2011 01:32:51 -0400
Subject: [Distutils] pythonv, take two
In-Reply-To: <loom.20110331T005457-50@post.gmane.org>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<4D8172FB.3020109@oddbird.net>	<loom.20110317T112246-771@post.gmane.org>	<4D8233EF.7070205@oddbird.net>	<loom.20110318T004329-895@post.gmane.org>	<4D83A807.30003@oddbird.net>	<loom.20110329T154252-221@post.gmane.org>	<4D920A7D.9040109@oddbird.net>	<loom.20110330T005934-410@post.gmane.org>	<4D9369E5.1050506@oddbird.net>
	<loom.20110331T005457-50@post.gmane.org>
Message-ID: <4D941203.3020804@oddbird.net>

On 03/30/2011 07:18 PM, Vinay Sajip wrote:
>> So... it seems to me that we're likely to break _some_ third-party code
>> using sys.prefix regardless of what we do here. My instinct says adding
>> sys.virtual_prefix and leaving sys.prefix alone is the better approach,
>> but I'm not very firmly entrenched in that position.
> 
> That's problematic, of course (third-party breakage, I mean). It may be that
> despite using virtual_prefix being a cleaner approach, it may be necessary
> to alter sys.prefix to ensure compatibility, a la virtualenv.

Well, as I said, I think there will be compatibility problems either
way. We are splitting site-packages from the base Python installation,
and when third-party code assumes that site-packages and the rest of the
base Python installation are both found relative to sys.prefix, neither
choice we make can possibly be right.

Virtualenv tries to avoid this problem by copying or symlinking in
enough stuff from the base Python install in order to convince third
parties using sys.prefix that they really are looking at a full Python
installation. But that approach is hacky and far from problem-free, and
requires keeping up with various things that third-party code might try
to look for under sys.prefix (there is at least one current open bug on
virtualenv, Tkinter failing in a Windows virtualenv, that is the result
of this problem). The logical conclusion of that approach, if you want
full compatibility with third-party code, is to just copy the entire
Python installation to the new location.

So three options as I see it:

1) Add sys.virtual_prefix (or perhaps sys.site_prefix would be a better
name), and require third-party code to use that (or sysconfig) to find
site-packages, if it wants to be virtual-compatible.

2) Repoint sys.prefix, and require third-party code to understand some
new sys.system_prefix or sys.installed_prefix or equivalent to find the
rest of the base Python installation.

3) Attempt the virtualenv approach of trying to fool third-party code
into thinking the virtualenv IS a full Python installation, even though
it's not.

I think #1 is far preferable to #2, and in the long run preferable to #3
even if it involves some short-term pain. But maybe I'm being too
idealistic, and #3 is the pragmatic choice even though it's "wrong."

> It may be that the thing isn't working properly, but it's masked on my
> system because I have an installed 3.3 in /usr/local (from when I was doing
> my pip/virtualenv tests). This means that there is a Makefile in a location
> which would not be there if I'd never installed 3.3 using sudo make install.
> 
> However, the getpath.c stuff should work - to test this, insert an
> "import pdb; pdb.set_trace()" in site.main, and check sys.path. This should
> contain entries from the virtualenv. The problems are more likely to be in
> the Python code: the C code always sets sys.prefix and sys.exec_prefix to
> /usr/local (or whatever configure was invoked with) and the sys.prefix /
> sys.exec_prefix stuff is done in site.virtualize, as before.

No, if I break early in site.py (or run python -S), sys.path doesn't
contain any virtualenv paths.

> Sorry - I got in a muddle because I've got too many clones around and ran
> into free disk space issues, and goofed somewhere. Don't bother to do a full
> merge for now, as I'm not sure if my code's working yet. Since I don't expect
> to change more than 4 files (getpath.c, site.py, distutils/sysconfig.py and
> sysconfig.py) I don't think we'll have too much trouble merging in due
> course, and if/when we can agree that my experiment in pythonv is worth
> taking further, I'll do the work of merging into cpythonv. For now, though,
> my changes might be treading on your toes, making merging needlessly
> annoying.
> 
> I'll try to test on a Python-clean machine in a day or two, will report back
> then.

Ok, thanks!

Carl

From kiorky at cryptelium.net  Thu Mar 31 07:56:22 2011
From: kiorky at cryptelium.net (kiorky)
Date: Thu, 31 Mar 2011 07:56:22 +0200
Subject: [Distutils] reservations about pythonv
In-Reply-To: <4D93A399.30108@stsci.edu>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<20110316223904.288080ba@neurotica>
	<4D839A72.1030705@stsci.edu> <4D939BD2.7080003@cryptelium.net>
	<4D93A399.30108@stsci.edu>
Message-ID: <4D941786.70108@cryptelium.net>

export LDFLAGS="what i want to be in linker flags".
You have also play the LD_RUN_PATH to modify the RPATH.
See [1]

You can have a look to minitage.recipe.{common, eggs, scripts} for a good
buildout integration.


[1] - http://goo.gl/oPPj0


Le 30/03/2011 23:41, Mark Sienkiewicz a ?crit :
> kiorky wrote:
>> You have rpath to record search paths for shared libraries at link time.
>> This enables you not to set LD_LIBRARY_PATH at runtime.
>>
>> Something like "cc -Wl,-rpath,/path/to/libdir -L/path/to/libdir -lfoo" is what
>> you want.
>>   
> 
> You left out the hard part:  How do I make distutils do that?
> 
> 
> 
>> Le 18/03/2011 18:46, Mark Sienkiewicz a ?crit :
>>  
>>>> The above just means my memory is too faulty to of much use ;).  I'll just
>>>> echo Carl's request for specific cases where $LD_LIBRARY_PATH needs to be set.
>>>>         
>>> Here is a case that might resemble what you are talking about:
>>>
>>> Compile a C extension that requires a shared library that is not in the standard
>>> system path.  To import it, LD_LIBRARY_PATH needs to be right.
>>>
>>> This is not really different from what happens in a compiled language, except in
>>> one way:  In C, I can compile it -static or I can give the full path to the .so
>>> file.  Either results in a thing that works without LD_LIBRARY_PATH.
>>>
>>> With distutils, you can't.  It goes to great lengths to ensure that you can only
>>> compile a C extension with "cc ... -L/some/directory -lname" -- I can't find any
>>> way to make it do "cc ... /some/directory/libname.so"
>>>
>>> So, the real problem here is that distutils uses "cc -L", but it demonstrates a
>>> case where LD_LIBRARY_PATH can be important to a python program even when Python
>>> itself can run without it.
>>>
>>>
>>> _______________________________________________
>>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>>> http://mail.python.org/mailman/listinfo/distutils-sig
>>>     
>>
>>   
> 

-- 
Cordialement,
KiOrKY
GPG Key FingerPrint: 0x1A1194B7681112AF
Pensez ? l?environnement.
N?imprimez ce courriel que si vous en avez vraiment besoin.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 261 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110331/98cdb9e3/attachment.pgp>

From carl at oddbird.net  Thu Mar 31 07:52:17 2011
From: carl at oddbird.net (Carl Meyer)
Date: Thu, 31 Mar 2011 01:52:17 -0400
Subject: [Distutils] pythonv, take two
In-Reply-To: <20110331032429.C3D403A4077@sparrow.telecommunity.com>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>
	<4D8172FB.3020109@oddbird.net>
	<loom.20110317T112246-771@post.gmane.org>
	<4D8233EF.7070205@oddbird.net>
	<loom.20110318T004329-895@post.gmane.org>
	<4D83A807.30003@oddbird.net>
	<loom.20110329T154252-221@post.gmane.org>
	<4D920A7D.9040109@oddbird.net>
	<loom.20110330T005934-410@post.gmane.org>
	<4D9369E5.1050506@oddbird.net>
	<20110331032429.C3D403A4077@sparrow.telecommunity.com>
Message-ID: <4D941691.3090608@oddbird.net>



On 03/30/2011 11:23 PM, P.J. Eby wrote:
> At 01:35 PM 3/30/2011 -0400, Carl Meyer wrote:
>> So... it seems to me that we're likely to break _some_ third-party code
>> using sys.prefix regardless of what we do here. My instinct says adding
>> sys.virtual_prefix and leaving sys.prefix alone is the better approach,
>> but I'm not very firmly entrenched in that position.
> 
> Looking at it from a software distribution POV, I would say that the
> virtual prefix is what it should point to, since that means things won't
> get installed to the wrong place.

Indeed. The issue is that from every point of view other than software
distribution (code trying to find anything other than site-directories),
sys.prefix pointing to the virtualenv is wrong. Unless, like virtualenv,
we try to make it "just right enough" by copying/symlinking things into
the virtualenv that otherwise wouldn't need to be there.

But this may be moot; I didn't realize until I checked just now that
setuptools (well, easy_install) builds its own list of site-dirs based
on sys.prefix. It doesn't look like that's used for installation, but it
is used for pre-flight checking pth-capability and finding pth files. So
if easy_install directly relies on site-directories always being
sys.prefix-based, that probably forces the issue.

> (Of course, a configuration option could be used to override it, if
> really necessary.)

This seems like the worst option of all - then third-party code really
would have no idea at all what sys.prefix is supposed to mean, or what
can be reliably found there.

Carl

From vinay_sajip at yahoo.co.uk  Thu Mar 31 10:12:58 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 31 Mar 2011 08:12:58 +0000 (UTC)
Subject: [Distutils] pythonv, take two
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<4D8172FB.3020109@oddbird.net>	<loom.20110317T112246-771@post.gmane.org>	<4D8233EF.7070205@oddbird.net>	<loom.20110318T004329-895@post.gmane.org>	<4D83A807.30003@oddbird.net>	<loom.20110329T154252-221@post.gmane.org>	<4D920A7D.9040109@oddbird.net>	<loom.20110330T005934-410@post.gmane.org>	<4D9369E5.1050506@oddbird.net>
	<loom.20110331T005457-50@post.gmane.org>
	<4D941203.3020804@oddbird.net>
Message-ID: <loom.20110331T090727-413@post.gmane.org>

Carl Meyer <carl <at> oddbird.net> writes:

> The logical conclusion of that approach, if you want
> full compatibility with third-party code, is to just copy the entire
> Python installation to the new location.

We do want that compatibility, don't we? Using a virtual environment should be a
no-brainer, as virtualenv currently is (most of the time). Of course, the
downside of the copying would be around 50MB per virtualenv on Windows, and much
less where symlinking or hardlinking is available. Not ideal, of course, but
disk space is cheap ;-)

> So three options as I see it:
> 
> 1) Add sys.virtual_prefix (or perhaps sys.site_prefix would be a better
> name), and require third-party code to use that (or sysconfig) to find
> site-packages, if it wants to be virtual-compatible.

But the success of virtualenv is because you can install just about any package
in an environment, and packages don't have to do anything to be "virtual
compatible".

> 2) Repoint sys.prefix, and require third-party code to understand some
> new sys.system_prefix or sys.installed_prefix or equivalent to find the
> rest of the base Python installation.

I think that the answer is for third party code not to use any prefix directly,
but rely on an API such as  sysconfig.get_path('purelib'). There's a distutils
version of this - distutils.sysconfig.get_python_lib() - but this is brittle as
per issue #6087.

> 3) Attempt the virtualenv approach of trying to fool third-party code
> into thinking the virtualenv IS a full Python installation, even though
> it's not.

> I think #1 is far preferable to #2, and in the long run preferable to #3
> even if it involves some short-term pain. But maybe I'm being too
> idealistic, and #3 is the pragmatic choice even though it's "wrong."

I'm afraid you may be right, at least until people stop using sys.prefix and
sys.exec_prefix directly and call functions instead, which can operate
differently in a virtual environment.
 
Regards,

Vinay Sajip


From vinay_sajip at yahoo.co.uk  Thu Mar 31 13:10:11 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 31 Mar 2011 11:10:11 +0000 (UTC)
Subject: [Distutils] pythonv, take two
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<4D8172FB.3020109@oddbird.net>	<loom.20110317T112246-771@post.gmane.org>	<4D8233EF.7070205@oddbird.net>	<loom.20110318T004329-895@post.gmane.org>	<4D83A807.30003@oddbird.net>	<loom.20110329T154252-221@post.gmane.org>	<4D920A7D.9040109@oddbird.net>
	<loom.20110330T005934-410@post.gmane.org>
	<4D9369E5.1050506@oddbird.net>
Message-ID: <loom.20110331T124928-850@post.gmane.org>

Carl Meyer <carl <at> oddbird.net> writes:

> Hmm, it doesn't seem to be working for me. sys.prefix and
> sys.exec_prefix are still /usr/local using the python binary in my
> "pmv", and sys.virtual_prefix is not set at all, despite env.cfg being
> present and looking as it should.
> 
> Oddly when I run the compiled python binary directly from the checkout,
> it does set sys.virtual_prefix to point one level above the checkout,
> despite there being no env.cfg in the vicinity at all.
> 
> You can see both of these things (in reverse order) here:
> http://paste.pocoo.org/show/362798/

After testing on a machine with no Python 3.3 installed, I get a similar failure
to yours (invalid Python installation due to missing Makefile). However, rhe
getpath.c code seems to be working as expected: see
http://paste.pocoo.org/show/363159/ for example. The path has been set to
reflect the source build where the python was copied from. Of course at this
point virtualize() hasn't been called, so virtual_prefix etc. haven't been set,
and prefixes are pointing to /usr/local.

I think the problem is occurring because there's incomplete support for running
from a source build. I'm not sure what to do about this; I think proper support
will need changes in multiple other places, which I haven't bottomed out yet but
which includes distutils.sysconfig (see issue #6087). Since running from a
source build does not set sys.prefix and sys.exec_prefix, either this needs to
change, or there needs to be some additional concept like "sys.python_home"
which is available to scripts, or perhaps something else. Of course fixing the
source build issue is of use only for core developers, so I'm not sure what
appetite there'd be for regularising things for this case.

Regards,

Vinay Sajip


From jim at zope.com  Thu Mar 31 13:18:58 2011
From: jim at zope.com (Jim Fulton)
Date: Thu, 31 Mar 2011 07:18:58 -0400
Subject: [Distutils] [Zope-dev]  [buildout] "private" releases
In-Reply-To: <AANLkTikB7HqGb-jokJUhEkBTfinL6Uw3s0ACE5nKLzLk@mail.gmail.com>
References: <imih2a$uig$1@dough.gmane.org> <4D8CEBD5.7010903@simplistix.co.uk>
	<AANLkTinkHfT8Rm1B58fve_eME-28r7m02Sd3wtxBdiWX@mail.gmail.com>
	<AANLkTikB7HqGb-jokJUhEkBTfinL6Uw3s0ACE5nKLzLk@mail.gmail.com>
Message-ID: <AANLkTi=76qDZaDpKr5AYdCyq+NdvnDP89Exc0+4aU__Y@mail.gmail.com>

On Thu, Mar 31, 2011 at 5:38 AM, Martijn Pieters <mj at zopatista.com> wrote:
> On Wed, Mar 30, 2011 at 15:08, Jim Fulton <jim at zope.com> wrote:
>> We do something similar with sftp (zc.buildoutsftp). ?To publish eggs,
>> we just use scp.
>> The advantage of this is that it leverages ssh infrastructure, so *no*
>> additional password management is needed. ?This is wildly better, IMO,
>> than keeping passwords in clear text in your buildout configuration or
>> in a dot file.
>
> That depends on your deployment scenarios. We generate separate
> passwords per customer, and give them a dedicated URL to load their
> private eggs from, then put the password in the buildout.cfg. To load
> the buildout.cfg in the first place, the exact same password is used.
>
> Managing SSH accounts and keys for those customers would cost us much
> more overhead, and would complicate our instructions for deployment to
> them.
>
> On the other hand, for deployments of a buildout from a SVN repository
> already served over SSH would make the sftp route the logical choice.

Some customers are too dumb to be secure. OK, makes sense. :)

Seriously, I assume this is a read-only scenario, in which case having
clear-text passwords laying around in prominent places seems less
problematic.  If they could write to the repo, then I would still have
serious problems with this approach.

Another approach would be to integrate with some secure key-management
service (keychain) on the customer's machines, but I expect that would
be as painful as helping them figure out ssh.

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From agroszer at gmail.com  Thu Mar 31 14:12:27 2011
From: agroszer at gmail.com (Adam GROSZER)
Date: Thu, 31 Mar 2011 14:12:27 +0200
Subject: [Distutils] how to test distribute
Message-ID: <4D946FAB.20803@gmail.com>

Hello,

I'd appreciate some hints how to run the tests of
bitbucket.org/tarek/distribute/#0.6-maintenance
and how to "test-drive" it.

I want to check/fix the bug with win-amd64.exe's.
That means I need to be able to somehow run
"easy_install reportlab-2.5.win-amd64-py2.6.exe"

-- 
Best regards,
  Adam GROSZER
--
Quote of the day:
Is this really happening?


From fdrake at acm.org  Thu Mar 31 14:04:51 2011
From: fdrake at acm.org (Fred Drake)
Date: Thu, 31 Mar 2011 08:04:51 -0400
Subject: [Distutils] reservations about pythonv
In-Reply-To: <4D939BD2.7080003@cryptelium.net>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>
	<20110316223904.288080ba@neurotica>
	<4D839A72.1030705@stsci.edu> <4D939BD2.7080003@cryptelium.net>
Message-ID: <AANLkTik2Ez5NMWf9yjfnUNJGsK7t3_n85q_xHOAZ-dpJ@mail.gmail.com>

On Wed, Mar 30, 2011 at 5:08 PM, kiorky <kiorky at cryptelium.net> wrote:
> You have rpath to record search paths for shared libraries at link time.
> This enables you not to set LD_LIBRARY_PATH at runtime.
>
> Something like "cc -Wl,-rpath,/path/to/libdir -L/path/to/libdir -lfoo" is what
> you want.

This fails once you want to relocate the built packages, but if you
know what the installed location is going to be, can be incredibly
helpful.


-- 
Fred L. Drake, Jr.? ? <fdrake at acm.org>
"Give me the luxuries of life and I will willingly do without the necessities."
?? --Frank Lloyd Wright

From mj at zopatista.com  Thu Mar 31 11:38:33 2011
From: mj at zopatista.com (Martijn Pieters)
Date: Thu, 31 Mar 2011 11:38:33 +0200
Subject: [Distutils] [Zope-dev]  [buildout] "private" releases
In-Reply-To: <AANLkTinkHfT8Rm1B58fve_eME-28r7m02Sd3wtxBdiWX@mail.gmail.com>
References: <imih2a$uig$1@dough.gmane.org> <4D8CEBD5.7010903@simplistix.co.uk>
	<AANLkTinkHfT8Rm1B58fve_eME-28r7m02Sd3wtxBdiWX@mail.gmail.com>
Message-ID: <AANLkTikB7HqGb-jokJUhEkBTfinL6Uw3s0ACE5nKLzLk@mail.gmail.com>

On Wed, Mar 30, 2011 at 15:08, Jim Fulton <jim at zope.com> wrote:
> We do something similar with sftp (zc.buildoutsftp). ?To publish eggs,
> we just use scp.
> The advantage of this is that it leverages ssh infrastructure, so *no*
> additional password management is needed. ?This is wildly better, IMO,
> than keeping passwords in clear text in your buildout configuration or
> in a dot file.

That depends on your deployment scenarios. We generate separate
passwords per customer, and give them a dedicated URL to load their
private eggs from, then put the password in the buildout.cfg. To load
the buildout.cfg in the first place, the exact same password is used.

Managing SSH accounts and keys for those customers would cost us much
more overhead, and would complicate our instructions for deployment to
them.

On the other hand, for deployments of a buildout from a SVN repository
already served over SSH would make the sftp route the logical choice.

-- 
Martijn Pieters

From flub at devork.be  Thu Mar 31 14:45:56 2011
From: flub at devork.be (Floris Bruynooghe)
Date: Thu, 31 Mar 2011 13:45:56 +0100
Subject: [Distutils] reservations about pythonv
In-Reply-To: <AANLkTik2Ez5NMWf9yjfnUNJGsK7t3_n85q_xHOAZ-dpJ@mail.gmail.com>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>
	<20110316223904.288080ba@neurotica> <4D839A72.1030705@stsci.edu>
	<4D939BD2.7080003@cryptelium.net>
	<AANLkTik2Ez5NMWf9yjfnUNJGsK7t3_n85q_xHOAZ-dpJ@mail.gmail.com>
Message-ID: <AANLkTinJkt3OLtcGRWu1UC8zta=daKu4cSvGBBGr99i1@mail.gmail.com>

On 31 March 2011 13:04, Fred Drake <fdrake at acm.org> wrote:
> On Wed, Mar 30, 2011 at 5:08 PM, kiorky <kiorky at cryptelium.net> wrote:
>> You have rpath to record search paths for shared libraries at link time.
>> This enables you not to set LD_LIBRARY_PATH at runtime.
>>
>> Something like "cc -Wl,-rpath,/path/to/libdir -L/path/to/libdir -lfoo" is what
>> you want.
>
> This fails once you want to relocate the built packages, but if you
> know what the installed location is going to be, can be incredibly
> helpful.

On SYSV-like systems you can use $ORIGIN inside RUNPATH (and RPATH) to
create relative paths starting from the location of the library.

Regards
Floris


-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From vinay_sajip at yahoo.co.uk  Thu Mar 31 15:21:57 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 31 Mar 2011 13:21:57 +0000 (UTC)
Subject: [Distutils] pythonv, take two
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<4D8172FB.3020109@oddbird.net>	<loom.20110317T112246-771@post.gmane.org>	<4D8233EF.7070205@oddbird.net>	<loom.20110318T004329-895@post.gmane.org>	<4D83A807.30003@oddbird.net>	<loom.20110329T154252-221@post.gmane.org>	<4D920A7D.9040109@oddbird.net>
	<loom.20110330T005934-410@post.gmane.org>
	<4D9369E5.1050506@oddbird.net>
	<loom.20110331T124928-850@post.gmane.org>
Message-ID: <loom.20110331T145603-128@post.gmane.org>

Vinay Sajip <vinay_sajip <at> yahoo.co.uk> writes:

> I think the problem is occurring because there's incomplete support for running
> from a source build.

To confirm this, I installed pythonv - and then running pmv.py produces the
expected result - an env with a copied Python which allows you to install stuff
into the env. I packaged the pythonv I used as a .deb using checkinstall, so
that it's uninstallable:

http://www.red-dove.com/pythonv_3.3-1_i386.deb

and having installed it, I re-ran pmv and did some smoke testing, output is at

http://paste.pocoo.org/show/363186/

I know we're not done yet, but at least this shows that the getpath.c code seems
to be working (because sys.path seems correct when running the env's copy of
Python) and distribute_setup.py installs without errors, followed by
easy_installing sample packages setuptools-git, ply, Pygments, Jinja2,
SQLAlchemy and coverage ... all seemingly without problems.

Of course, this is still using sys.prefix/sys.exec_prefix.

Regards,

Vinay


From brandon at rhodesmill.org  Thu Mar 31 17:21:30 2011
From: brandon at rhodesmill.org (Brandon Craig Rhodes)
Date: Thu, 31 Mar 2011 11:21:30 -0400
Subject: [Distutils] reservations about pythonv
In-Reply-To: <4D839A72.1030705@stsci.edu> (Mark Sienkiewicz's message of "Fri, 
	18 Mar 2011 13:46:26 -0400")
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>
	<20110316223904.288080ba@neurotica> <4D839A72.1030705@stsci.edu>
Message-ID: <87aagbmi2t.fsf@asaph.rhodesmill.org>

Mark Sienkiewicz <sienkiew at stsci.edu> writes:

> Compile a C extension that requires a shared library that is not in the
> standard system path.  To import it, LD_LIBRARY_PATH needs to be right.

I just discovered, in the course of repairing my "pyzmq-static" project
after the "pyzmq" folks went crazy and broke their single extension
module into a dozen modules, that there is another alternative to the
LD_LIBRARY_PATH environment variable.  If an extension wants to build
and use a .so shared library, then the Python package can build and
install the library somewhere beneath the package's main directory, then
the package can preload the library manually as the first thing
__init__.py does, by writing code like this:

import ctypes
import os
p = os.path.join(os.path.dirname(__file__), "mylibrary.so")
_library = ctypes.CDLL(p, mode=ctypes.RTLD_GLOBAL)

Because the RTLD_GLOBAL mode makes all of the symbols in "mylibrary.so"
available to subsequently-loaded shared object files, any subsequent
import of a dynamically linked Python extension should link its symbols
against "mylibrary.so" just fine without a need to set LD_LIBRARY_PATH.

-- 
Brandon Craig Rhodes   brandon at rhodesmill.org   http://rhodesmill.org/brandon

From ziade.tarek at gmail.com  Thu Mar 31 17:54:49 2011
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 31 Mar 2011 17:54:49 +0200
Subject: [Distutils] how to test distribute
In-Reply-To: <4D946FAB.20803@gmail.com>
References: <4D946FAB.20803@gmail.com>
Message-ID: <AANLkTin2-UNDKgbMzeSmh=1zMGKtBizvm5w_n1HcDakY@mail.gmail.com>

There's the test.sh you can use, it basically runs "python setup.py
test" on all supported Python versions

On Thu, Mar 31, h011 at 2:12 PM, Adam GROSZER <agroszer at gmail.com> wrote:
> Hello,
>
> I'd appreciate some hints how to run the tests of
> bitbucket.org/tarek/distribute/#0.6-maintenance
> and how to "test-drive" it.
>
> I want to check/fix the bug with win-amd64.exe's.
> That means I need to be able to somehow run
> "easy_install reportlab-2.5.win-amd64-py2.6.exe"
>
> --
> Best regards,
> ?Adam GROSZER
> --
> Quote of the day:
> Is this really happening?
>
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | http://ziade.org

From mark at shotgunsoftware.com  Thu Mar 31 19:39:07 2011
From: mark at shotgunsoftware.com (Mark Visser)
Date: Thu, 31 Mar 2011 13:39:07 -0400
Subject: [Distutils] passing command-line options to an additional python
	interpreter?
Message-ID: <B769ED2A-56B0-4442-A88C-1088382E299F@shotgunsoftware.com>

I'm using buildout for managing a project that runs on several different python interpreters, some of which are embedded in other applications. As such, I'm at the whim of those packaging python for those applications, and sometimes they don't set things up... well, optimally.

The problem I've run into is that one of the exposed non-standard python interpreters (mayapy) is actually a shell script. Thus, you can't use its direct path with a shebang. Normally, I get around this by using #!/usr/bin/env mayapy.

For the life of me, I can't figure out a way to pass this to buildout. I've tried this:

[mayapy]
executable = /usr/bin/env mayapy

[mayatests]
python = mayapy
... etc

But buildout doesn't like the space. Is there a way to specify executable options? I tried this to no avail:

[mayapy]
executable = /usr/bin/env
options = mayapy

Can anyone suggest a solution?

thanks,
-Mark