From setuptools at bugs.python.org  Fri Oct  1 16:28:14 2010
From: setuptools at bugs.python.org (Wichert Akkerman)
Date: Fri, 01 Oct 2010 14:28:14 +0000
Subject: [Distutils] [issue118] extracting of symlinks looses original
In-Reply-To: <1285943294.69.0.459472289163.issue118@psf.upfronthosting.co.za>
Message-ID: <1285943294.69.0.459472289163.issue118@psf.upfronthosting.co.za>


New submission from Wichert Akkerman <wichert at wiggy.net>:

I ran into something odd related to handling of symlinks today. I have a source which contains a symlink. The relevant directory looks like this:

drwxr-xr-x  5 wichert staff  170 Oct  1 10:55 ./
drwxr-xr-x 40 wichert staff 1360 Oct  1 10:55 ../
lrwxr-xr-x  1 wichert staff   20 Oct  1 10:55 editor_plugin.js -> editor_plugin_src.js
-rw-r--r--  1 wichert staff 1159 Oct  1 10:55 editor_plugin_src.js

when I create an sdist both the original file and the symlink are included in the .tar.gz. When I install the distribution using the sdist as uploaded on pypi the same directory in the generated egg looks like this:

drwxr-xr-x  3 wichert staff  102 Sep 30 14:50 ./
drwxr-xr-x 39 wichert staff 1326 Sep 30 14:50 ../
-rw-r--r--  1 wichert staff 1159 Sep 30 14:50 editor_plugin.js

the symlink has been replaced with a copy, but the original went missing. Unfortunately both filenames are used, so the result is a broken package.

The relevant package is NuPlone 1.0b2 (http://pypi.python.org/packages/source/N/NuPlone/NuPlone-1.0b2.tar.gz), and the directory is plonetheme/nuplone/z3cform/tiny_mce/plugins/linefield .

----------
messages: 555
nosy: wichert
priority: bug
status: unread
title: extracting of symlinks looses original

_______________________________________________
Setuptools tracker <setuptools at bugs.python.org>
<http://bugs.python.org/setuptools/issue118>
_______________________________________________

From greg.allen at nomura.com  Fri Oct  1 17:27:58 2010
From: greg.allen at nomura.com (greg.allen at nomura.com)
Date: Fri, 1 Oct 2010 16:27:58 +0100
Subject: [Distutils] headless py2exe install?
Message-ID: <6658700105A0764799DBC12B942FA9901CFE99F6@LONEV3102.EUROPE.NOM>

I would like to be able to run the binary installer
"coverage-3.4.win32-py2.5.exe" onto my build servers. I'm pretty sure
that it's made with distutils and py2exe.

 

My build server installation is scripted, so I really don't want the
installer's GUI to be used. Is there a way to skip it?

 

Note that since this module has some parts that require C compilation,
installing from source isn't really an option.

 

Cheers,

Greg




This e-mail (including any attachments) is confidential, may contain
proprietary or privileged information and is intended for the named
recipient(s) only. Unintended recipients are prohibited from taking action
on the basis of information in this e-mail and must delete all copies.
Nomura will not accept responsibility or liability for the accuracy or
completeness of, or the presence of any virus or disabling code in, this
e-mail. If verification is sought please request a hard copy. Any reference
to the terms of executed transactions should be treated as preliminary only
and subject to formal written confirmation by Nomura. Nomura reserves the
right to monitor e-mail communications through its networks (in accordance
with applicable laws). No confidentiality or privilege is waived or lost by
Nomura by any mistransmission of this e-mail. Any reference to "Nomura" is
a reference to any entity in the Nomura Holdings, Inc. group. Please read
our Electronic Communications Legal Notice which forms part of this e-mail:
http://www.Nomura.com/email_disclaimer.htm

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20101001/44af9a74/attachment.html>

From jeremy.kloth at gmail.com  Fri Oct  1 17:55:44 2010
From: jeremy.kloth at gmail.com (Jeremy Kloth)
Date: Fri, 1 Oct 2010 09:55:44 -0600
Subject: [Distutils] headless py2exe install?
In-Reply-To: <6658700105A0764799DBC12B942FA9901CFE99F6@LONEV3102.EUROPE.NOM>
References: <6658700105A0764799DBC12B942FA9901CFE99F6@LONEV3102.EUROPE.NOM>
Message-ID: <201010010955.44904.jeremy.kloth@gmail.com>

On Friday, October 01, 2010 09:27:58 am greg.allen at nomura.com wrote:
> I would like to be able to run the binary installer
> "coverage-3.4.win32-py2.5.exe" onto my build servers. I'm pretty sure
> that it's made with distutils and py2exe.

Actually it is make with just distutils. It it the bdist_wininst installer. 
py2exe creates standalone executables, not installers.

> My build server installation is scripted, so I really don't want the
> installer's GUI to be used. Is there a way to skip it?

Sorry, but not as of now. Distutils2 will hopefully included this feature (if 
I can get it done in time).

-- 
Jeremy Kloth

From agroszer at gmail.com  Fri Oct  1 17:55:46 2010
From: agroszer at gmail.com (Adam GROSZER)
Date: Fri, 1 Oct 2010 17:55:46 +0200
Subject: [Distutils] headless py2exe install?
In-Reply-To: <6658700105A0764799DBC12B942FA9901CFE99F6@LONEV3102.EUROPE.NOM>
References: <6658700105A0764799DBC12B942FA9901CFE99F6@LONEV3102.EUROPE.NOM>
Message-ID: <16510736097.20101001175546@gmail.com>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20101001/79924455/attachment-0001.html>

From Ronny.Pfannschmidt at gmx.de  Fri Oct  1 23:09:35 2010
From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt)
Date: Fri, 01 Oct 2010 23:09:35 +0200
Subject: [Distutils] implementing egg-info/pkg_resources source support for
 custom importers
Message-ID: <1285967375.2534.17.camel@Klappe2>

hi,

I'm implementing a custom pep302 importer in order to ship sets of
python packages within a generated python script,

I'd like to support egg-info entry-points, so I can ship extensions as
well as choose the script entry-point based on the executable name.

However I couldn't find simple docs on doing that.

regards Ronny
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20101001/1b19a170/attachment.pgp>

From pje at telecommunity.com  Sat Oct  2 04:05:14 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 01 Oct 2010 22:05:14 -0400
Subject: [Distutils] implementing egg-info/pkg_resources source support
 for custom importers
In-Reply-To: <1285967375.2534.17.camel@Klappe2>
References: <1285967375.2534.17.camel@Klappe2>
Message-ID: <20101002020515.889E93A40B2@sparrow.telecommunity.com>

At 11:09 PM 10/1/2010 +0200, Ronny Pfannschmidt wrote:
>hi,
>
>I'm implementing a custom pep302 importer in order to ship sets of
>python packages within a generated python script,
>
>I'd like to support egg-info entry-points, so I can ship extensions as
>well as choose the script entry-point based on the executable name.
>
>However I couldn't find simple docs on doing that.

You need to register some adapters for your importer type, using the 
functions described here:

http://peak.telecommunity.com/DevCenter/PkgResources#supporting-custom-importers

Essentially, you will need to define two functions and a class:

def my_finder(path_item, importer, only):
     # yield Distribution objects for path

def my_namespace_fixer(importer, path_item, modulename, module):
     # return a __path__ entry to add to module

class MyProvider(EggProvider):

     def _has(self, path):
         # os.path.exists(path)

     def _isdir(self,path):
         # os.path.isdir(path)

     def _listdir(self,path):
         # os.listdir(path)

     def _get(self, path):
         # open(path,'rb').read()

and then register them like so:

     register_finder(MyImporter, my_finder)
     register_namespace_handler(MyImporter, my_namespace_fixer)
     register_loader_type(MyLoaderType, MyProvider)

Without knowing how your PEP 302 importer works, it's hard to get 
more specific than this; it all depends on how your sys.path entry 
strings are formatted.  Are you using a virtual OS path the way .zip 
paths work, or something else?  (If you're using pseudo-OS paths, you 
can reuse pkg_resources.file_ns_handler in place of writing your own 
my_namespace_fixer.)

Note, by the way, that if your system doesn't physically place 
individual files on disk, you probably actually want something that 
emulates .egg rather than .egg-info, as then you can support 
resource_filename() operations.  (In which case, you'll probably also 
subclass ZipProvider rather than EggProvider.)

So, yeah, there are no simple docs.  ;-)

It would probably be good if pkg_resources had better support for 
rolling your own stuff over a virtual file system; it already has 
some, but the "finder" part could be a lot simpler if there was a way 
to reuse the built-in ones with a VFS, and the "provider" class would 
be easier if the parts of ZipProvider that aren't truly 
zipfile-specific were moved to EggProvider or NullProvider.


From roberto.riggio at create-net.org  Tue Oct  5 16:21:49 2010
From: roberto.riggio at create-net.org (Roberto Riggio)
Date: Tue, 05 Oct 2010 16:21:49 +0200
Subject: [Distutils] Copying entire directory
Message-ID: <4CAB347D.3030708@create-net.org>

  Hi,

I'm not sure if I'm doing something wrong, anyway, I want to
copy the content of a directory included in a package generated with
./setup sdist to a certain destination.

What I did is to add something like this to my setup.py:

data_files = [('etc/foo/www/', [ 'dashboard/*' ])]

Just to put things into a context: the directory dashboard contains a
number of html files that are served by my application. I cannot list
all the files one by one because are too many, but also the apprach
that I'm using does not work and produce the following error:

error: can't copy dashboard/*: doesn't not exist or not a regular file

Any suggestions?

Thanks
R.

From pje at telecommunity.com  Tue Oct  5 18:39:05 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 05 Oct 2010 12:39:05 -0400
Subject: [Distutils] Copying entire directory
In-Reply-To: <4CAB347D.3030708@create-net.org>
References: <4CAB347D.3030708@create-net.org>
Message-ID: <20101005163859.39E5E3A4105@sparrow.telecommunity.com>

At 04:21 PM 10/5/2010 +0200, Roberto Riggio wrote:
>  Hi,
>
>I'm not sure if I'm doing something wrong, anyway, I want to
>copy the content of a directory included in a package generated with
>./setup sdist to a certain destination.
>
>What I did is to add something like this to my setup.py:
>
>data_files = [('etc/foo/www/', [ 'dashboard/*' ])]
>
>Just to put things into a context: the directory dashboard contains a
>number of html files that are served by my application. I cannot list
>all the files one by one because are too many, but also the apprach
>that I'm using does not work and produce the following error:
>
>error: can't copy dashboard/*: doesn't not exist or not a regular file
>
>Any suggestions?

data_files = [('etc/foo/www/', ['dashboard/'+f for f in 
os.listdir('dashboard')])]

(I would also question the idea of targeting installation directories 
in this way for a general-use package, but that's a separate issue.)


From roberto.riggio at create-net.org  Tue Oct  5 19:31:47 2010
From: roberto.riggio at create-net.org (Roberto Riggio)
Date: Tue, 05 Oct 2010 19:31:47 +0200
Subject: [Distutils] Copying entire directory
In-Reply-To: <20101005163859.39E5E3A4105@sparrow.telecommunity.com>
References: <4CAB347D.3030708@create-net.org>
	<20101005163859.39E5E3A4105@sparrow.telecommunity.com>
Message-ID: <4CAB6103.1070904@create-net.org>

  On 10/05/2010 06:39 PM, P.J. Eby wrote:
> data_files = [('etc/foo/www/', ['dashboard/'+f for f in 
> os.listdir('dashboard')])]
>
> (I would also question the idea of targeting installation directories 
> in this way for a general-use package, but that's a separate issue.)
>
Why? Which would be a better approach? My software is a daemon
that implements a p2p overlay. The files are an ajax interface to the
daemon that is served by an embedded web server.

R.

From pje at telecommunity.com  Tue Oct  5 19:55:13 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 05 Oct 2010 13:55:13 -0400
Subject: [Distutils] Copying entire directory
In-Reply-To: <4CAB6103.1070904@create-net.org>
References: <4CAB347D.3030708@create-net.org>
	<20101005163859.39E5E3A4105@sparrow.telecommunity.com>
	<4CAB6103.1070904@create-net.org>
Message-ID: <20101005175507.761833A4108@sparrow.telecommunity.com>

At 07:31 PM 10/5/2010 +0200, Roberto Riggio wrote:
>  On 10/05/2010 06:39 PM, P.J. Eby wrote:
>>data_files = [('etc/foo/www/', ['dashboard/'+f for f in 
>>os.listdir('dashboard')])]
>>
>>(I would also question the idea of targeting installation 
>>directories in this way for a general-use package, but that's a 
>>separate issue.)
>Why? Which would be a better approach? My software is a daemon
>that implements a p2p overlay. The files are an ajax interface to the
>daemon that is served by an embedded web server.

That's not a general-use package.  ;-)


From barry at python.org  Tue Oct  5 22:31:06 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 5 Oct 2010 16:31:06 -0400
Subject: [Distutils] Oddities in setuptools/distribute
Message-ID: <20101005163106.5461f827@mission>

I'm working on some patches that allow for multiple builds of Python with
different options to coexist.  This is an extension to PEP 3149 and has been
discussed recently in python-dev:

http://mail.python.org/pipermail/python-dev/2010-October/104382.html

This has led me into the twisty maze of setuptools and distribute:

http://mail.python.org/pipermail/python-dev/2010-October/104430.html

I believe I've figured out a patch that *should* make this work, but doesn't,
and I have a few questions about some things I've noticed:

1) Why does setuptools write a stub loader for the .so in the first place?
   Why not just let the import of the shared library happen directly using the
   normal Python import rules?

   I had thought that build_ext.py was the place to modify but it turns out (I
   think) build_ext's stub writer only gets called when building an extension
   inplace.  Since I'm trying to install an extension, it's actually
   bdist_egg's stub writer that gets called.

2) Why does build_ext.py and bdist_egg.py have similar but slightly different
   stub writers?

   I'm sure there's an important reason for this, but at least in the case of
   shared library loading, it seems like the two stubs ought to be more
   similar than they are.

3) Why does site-packages/<package>.egg/ get deleted when the package is
   re-installed?  Can this be prevented?

   This is actually the show-stopper I'm stuck on because if I install my
   extension with build-A of Python, then try to install it with build-B of
   Python, the build-A version gets wiped.  Because the shared libraries now
   have build-flag discriminators in the file name, the two installs *should*
   be able to coexist, with this diff against distribute-0.6.15dev, but the
   blowing away of the .egg directory prevents this.

   http://pastebin.ubuntu.com/506759/

Thanks for any feedback you can provide.
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20101005/e1d91c49/attachment.pgp>

From pje at telecommunity.com  Wed Oct  6 01:04:10 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 05 Oct 2010 19:04:10 -0400
Subject: [Distutils] Oddities in setuptools/distribute
In-Reply-To: <20101005163106.5461f827@mission>
References: <20101005163106.5461f827@mission>
Message-ID: <20101005230404.98E033A4105@sparrow.telecommunity.com>

At 04:31 PM 10/5/2010 -0400, Barry Warsaw wrote:
>I'm working on some patches that allow for multiple builds of Python with
>different options to coexist.  This is an extension to PEP 3149 and has been
>discussed recently in python-dev:
>
>http://mail.python.org/pipermail/python-dev/2010-October/104382.html
>
>This has led me into the twisty maze of setuptools and distribute:
>
>http://mail.python.org/pipermail/python-dev/2010-October/104430.html
>
>I believe I've figured out a patch that *should* make this work, but doesn't,
>and I have a few questions about some things I've noticed:
>
>1) Why does setuptools write a stub loader for the .so in the first place?
>    Why not just let the import of the shared library happen 
> directly using the
>    normal Python import rules?

It does.  The normal import rules load .so files first, so the stub 
loaders only have an effect if the module is imported from a 
*zipfile* (where .so files won't normally load at all).

When unzipped, the .py file is ignored, and so has no effect.  But 
when zipped, it forces extraction of the .so to a cache directory, 
where it can then be imported.


>2) Why does build_ext.py and bdist_egg.py have similar but slightly different
>    stub writers?
>
>    I'm sure there's an important reason for this, but at least in the case of
>    shared library loading, it seems like the two stubs ought to be more
>    similar than they are.

Dynamic shared library support in setuptools is still officially an 
experimental feature, so there's some possibility that this is a 
bug.  That is, bdist_egg's stubs may be broken for extensions using 
dynamic library loader stubs.


>3) Why does site-packages/<package>.egg/ get deleted when the package is
>    re-installed?

Because if you're reinstalling it, it's presumably because the 
original is fux0red in some manner.  Also, depending on your 
command-line options, you might be installing an .egg file where a 
directory existed before, or vice versa.


>   Can this be prevented?
>
>    This is actually the show-stopper I'm stuck on because if I install my
>    extension with build-A of Python, then try to install it with build-B of
>    Python, the build-A version gets wiped.  Because the shared libraries now
>    have build-flag discriminators in the file name, the two installs *should*
>    be able to coexist,

On a version of Python that does this, it would probably be best if 
the platform string included the build flags -- then you would have 
two separate .eggs, each of which would only be loaded by a compatible runtime.


>Thanks for any feedback you can provide.

Thanks for asking!


From barry at python.org  Wed Oct  6 03:32:48 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 5 Oct 2010 21:32:48 -0400
Subject: [Distutils] Oddities in setuptools/distribute
In-Reply-To: <20101005230404.98E033A4105@sparrow.telecommunity.com>
References: <20101005163106.5461f827@mission>
	<20101005230404.98E033A4105@sparrow.telecommunity.com>
Message-ID: <20101005213248.252ce85f@limelight.wooz.org>

On Oct 05, 2010, at 07:04 PM, P.J. Eby wrote:

>At 04:31 PM 10/5/2010 -0400, Barry Warsaw wrote:
>>1) Why does setuptools write a stub loader for the .so in the first place?
>>    Why not just let the import of the shared library happen 
>> directly using the
>>    normal Python import rules?
>
>It does.  The normal import rules load .so files first, so the stub 
>loaders only have an effect if the module is imported from a 
>*zipfile* (where .so files won't normally load at all).
>
>When unzipped, the .py file is ignored, and so has no effect.  But 
>when zipped, it forces extraction of the .so to a cache directory, 
>where it can then be imported.

Interesting, that makes sense, thanks.

I hit this while debugging interpreter crashes during my multi-build work.  I
think this assumption is problematic in that case.

Let's say I install extension module 'stupid' using a python3.2m build
(i.e. +pymalloc !wide-unicode !pydebug).  This has an _stupid.c extension which
gets built and installed in the egg directory under site-packages, as
_stupid.cpython-32m.so.  A _stupid.py stub also gets installed.  This hard
codes loading _stupid.cpython-32m.so.

Now I come along and run a python3.2dmu interpreter for which _stupid has not
been installed.  In that interpreter I do an import of 'stupid' which imports
_stupid.  Now, because the two builds share site-packages (which *should* be
fine since .pyc's don't care about the build flags and .so's are distinguished
by the build flags), python3.2dmu sees the stupid package.  It imports
stupid.py which imports _stupid.  This triggers the stub, but it tries to load
_stupid.cpython-32m.so (the file name is hardcoded in the stub), and Python
crashes.

I have a patch for distribute (which should be adapted to setuptools) that
essentially substitutes the current interpreter's build flags into
_stupid.{}.so so that the right one will get loaded.  This fixes the crashes,
except then I hit the second problem...

>>2) Why does build_ext.py and bdist_egg.py have similar but slightly different
>>    stub writers?
>>
>>    I'm sure there's an important reason for this, but at least in the case of
>>    shared library loading, it seems like the two stubs ought to be more
>>    similar than they are.
>
>Dynamic shared library support in setuptools is still officially an 
>experimental feature, so there's some possibility that this is a 
>bug.  That is, bdist_egg's stubs may be broken for extensions using 
>dynamic library loader stubs.

'k.  Definitely seems like some refactoring is in order.

>>3) Why does site-packages/<package>.egg/ get deleted when the package is
>>    re-installed?
>
>Because if you're reinstalling it, it's presumably because the 
>original is fux0red in some manner.  Also, depending on your 
>command-line options, you might be installing an .egg file where a 
>directory existed before, or vice versa.

It probably would be better to only blow away the original egg when a
--reinstall option were given.  This might change current semantics too much,
so a --keep option would probably be in order.

>>   Can this be prevented?
>>
>>    This is actually the show-stopper I'm stuck on because if I install my
>>    extension with build-A of Python, then try to install it with build-B of
>>    Python, the build-A version gets wiped.  Because the shared libraries now
>>    have build-flag discriminators in the file name, the two installs *should*
>>    be able to coexist,
>
>On a version of Python that does this, it would probably be best if the
>platform string included the build flags -- then you would have two separate
>.eggs, each of which would only be loaded by a compatible runtime.

For zip'd eggs, you might be right.  For egg directories, it's not necessary.
Everything is sharable, and when not (i.e. the .so's) they are build-flag
discriminated.  I'd like to share as much as possible, but I'm open to any
suggestions.

>>Thanks for any feedback you can provide.
>
>Thanks for asking!

Thanks for the explanations! :)

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20101005/9985ab25/attachment-0001.pgp>

From pje at telecommunity.com  Wed Oct  6 04:06:03 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 05 Oct 2010 22:06:03 -0400
Subject: [Distutils] Oddities in setuptools/distribute
In-Reply-To: <20101005213248.252ce85f@limelight.wooz.org>
References: <20101005163106.5461f827@mission>
	<20101005230404.98E033A4105@sparrow.telecommunity.com>
	<20101005213248.252ce85f@limelight.wooz.org>
Message-ID: <20101006020557.76C573A4105@sparrow.telecommunity.com>

At 09:32 PM 10/5/2010 -0400, Barry Warsaw wrote:
>It probably would be better to only blow away the original egg when a
>--reinstall option were given.  This might change current semantics too much,
>so a --keep option would probably be in order.

My worry here is that this breaks the invariant that .egg files and 
.egg directories are identical/interchangeable.  I'm not sure whether 
there's anything in the ecosystem that depends on that (besides 
perhaps the --always-copy flag to easy_install, which I'd have to 
check to see if it depends on that).


>For zip'd eggs, you might be right.  For egg directories, it's not necessary.
>Everything is sharable, and when not (i.e. the .so's) they are build-flag
>discriminated.  I'd like to share as much as possible, but I'm open to any
>suggestions.

Given that platform string discrimination has been battle-hardened by 
many years of deployment, my personal inclination is to suggest that approach.

The relevant functions are:

* pkg_resources.get_build_platform() -- returns the "minimum" 
platform string that should be used for eggs built by this interpreter

* pkg_resources.get_supported_platform() -- returns the platform 
string that represents the "maximum" platform supported by this interpreter

* pkg_resources.compatible_platforms(provided, required) -- Can code 
for the `provided` platform run on the `required` platform?

In practical terms, though, you could probably just change 
distutils.util.get_platform() (or whatever it's called in Py3) to add 
the debug and width flags to the platform string.  The 
get_*_platform() functions are based on that, and the default 
compatibility check is just string equality. 


From email at lnowak.com  Wed Oct  6 12:15:04 2010
From: email at lnowak.com (=?UTF-8?Q?=C5=81ukasz_Nowak?=)
Date: Wed, 6 Oct 2010 12:15:04 +0200
Subject: [Distutils] using python generated by buildout to run buildout
Message-ID: <AANLkTinFMnZOu2t3aoUZTx4mY4DnKuv2PYWZ9qXCoSPg@mail.gmail.com>

Hello,

I am playing with zc.buildout for some time. I experienced issues with
system provided pythons (even virutalenved), so I switched to build my
own python. Of course as I am lazy, I am using buildout to configure
and compile various python versions on my machine.

Quite often I need to provide "self contained" buildout profile, which
would be able to run using python compiled by itself.

So my example profile looks like this:

[buildout]
parts =
  rebootstrap2.4
  realrun

find-links = http://www.nexedi.org/static/packages/source/

[python2.4]
prefix = ${buildout:parts-directory}/${:_buildout_section_name_}
executable = ${:prefix}/bin/python2.4

# this is just wrapper for hexagonit.recipe.cmmi which makes it not
returning python part
recipe = erp5.recipe.cmmisafe
url =
  http://python.org/ftp/python/2.4.6/Python-2.4.6.tgz
configure-options =
  --enable-unicode=ucs4
  --with-threads
  --with-readline
  --with-dbm
  --with-zlib
  --with-ssl
  --with-bz2

[rebootstrap2.4]
recipe = zc.recipe.egg
eggs = zc.buildout
python = python2.4

[realrun]
recipe = plone.recipe.command
command =
  echo Running with python ${buildout:executable}
update-command = ${:command}

So first step for me is to bootstrap builodut:

curl -s http://svn.zope.org/*checkout*/zc.buildout/trunk/bootstrap/bootstrap.py
| python -S -

So now my bin/buildout is referring to "system python":
head -n1 bin/buildout
#!/home/luke/bin/python -S

Then I need to rebootstrap it:

bin/buildout install python2.4 rebootstrap2.4

python2.4* parts are installing and compiling python, rebootstrap2.4
part is reinstalling bin/buildout using this python.

Cool, my bin/buildout is using python it compiled:

head -n1 bin/buildout
#!/home/luke/pybuildout/parts/python2.4/bin/python2.4

So I can run again, this time usually. python will be compiled, as
buildout signature changed, but this is not an issue (ccache + distcc
makes it really fast). Buildout is using python he provided,
chicken-and-egg issue solved by hand:

bin/buildout
...

realrun: Running '
echo Running with python /home/luke/pybuildout/parts/python2.4/bin/python2.4'
Running with python /home/luke/pybuildout/parts/python2.4/bin/python2.4
Unused options for realrun: 'update-command'.

Re running builodut does not affect python anymore:

bin/buildout
Updating python2.4-dbm-patch.
Updating python2.4.
Updating rebootstrap2.4.
Updating realrun.
realrun: Running
echo Running with python /home/luke/pybuildout/parts/python2.4/bin/python2.4
Running with python /home/luke/pybuildout/parts/python2.4/bin/python2.4

And so far I am happy, I am using correct python version.

What I'd like to develop is to make those steps automatically, so that
after running:

curl -s http://svn.zope.org/*checkout*/zc.buildout/trunk/bootstrap/bootstrap.py
| python -S -
bin/buildout

Such steps would happen automagically:

 1 normal bootstrap with system python
 2 compile python as needed
 3 install new bin/buildout
 4 re-run bin/buildout
 5 compile python as signature change
 6 install new bin/buildout (which is the same)
 7 re-run bin/buildout
 8 do not compile python, as signature had not changed, continue...

Theoretically steps 5,6 and 7 could be avoided, but lets say I am
purist -- I do not trust that python compiled by buildout which is
running python I do not trust is bad :)

Easy question: Is there ability for buildout to do smart rebootstrap
in one run, with selecting part which contains python?

If no I am ready to develop such thing. So I read a bit about
extensions - I can hook before and after buildout is run. Extensions
have access to buildout object, I think they can play with buildout
externals a lot, including re-running buildout (I saw in output that
buildout re-runs itself after upgrading himself).

Are extension the way to go with such thing?

If needed I am even ready to play a bit more with zc.buildout
internals code to extend if required, but according to my
understanding how buildout works there are enough places I can "hook"
to.

Any other comments? References?

Regards,
Luke

From friedrichromstedt at gmail.com  Wed Oct  6 18:54:16 2010
From: friedrichromstedt at gmail.com (Friedrich Romstedt)
Date: Wed, 6 Oct 2010 18:54:16 +0200
Subject: [Distutils] Copying entire directory
In-Reply-To: <4CAB347D.3030708@create-net.org>
References: <4CAB347D.3030708@create-net.org>
Message-ID: <AANLkTikYhL23mK2MWYJb7aaUvD47widnwNwbqAx4Gjtq@mail.gmail.com>

2010/10/5 Roberto Riggio <roberto.riggio at create-net.org>:
> data_files = [('etc/foo/www/', [ 'dashboard/*' ])]

You can also use the `glob` module.  And as a side remark: It's in my
opinion always a good idea to try to avoid hard-coding directory
separators (/), and to use os.path.join() instead.

Friedrich

From fdrake at acm.org  Wed Oct  6 18:57:50 2010
From: fdrake at acm.org (Fred Drake)
Date: Wed, 6 Oct 2010 12:57:50 -0400
Subject: [Distutils] Copying entire directory
In-Reply-To: <AANLkTikYhL23mK2MWYJb7aaUvD47widnwNwbqAx4Gjtq@mail.gmail.com>
References: <4CAB347D.3030708@create-net.org>
	<AANLkTikYhL23mK2MWYJb7aaUvD47widnwNwbqAx4Gjtq@mail.gmail.com>
Message-ID: <AANLkTi=-Y_o+HDxY5botdUA0YDewpkD=ok2xCxGCdgZZ@mail.gmail.com>

On Wed, Oct 6, 2010 at 12:54 PM, Friedrich Romstedt
<friedrichromstedt at gmail.com> wrote:
> It's in my
> opinion always a good idea to try to avoid hard-coding directory
> separators (/), and to use os.path.join() instead.

Distutils specifically wants "/" rather than the local platform's
separator, and takes care of conversion as needed.


? -Fred

--
Fred L. Drake, Jr.? ? <fdrake at acm.org>
"A storm broke loose in my mind."? --Albert Einstein

From friedrichromstedt at gmail.com  Wed Oct  6 19:06:08 2010
From: friedrichromstedt at gmail.com (Friedrich Romstedt)
Date: Wed, 6 Oct 2010 19:06:08 +0200
Subject: [Distutils] Copying entire directory
In-Reply-To: <AANLkTi=-Y_o+HDxY5botdUA0YDewpkD=ok2xCxGCdgZZ@mail.gmail.com>
References: <4CAB347D.3030708@create-net.org>
	<AANLkTikYhL23mK2MWYJb7aaUvD47widnwNwbqAx4Gjtq@mail.gmail.com>
	<AANLkTi=-Y_o+HDxY5botdUA0YDewpkD=ok2xCxGCdgZZ@mail.gmail.com>
Message-ID: <AANLkTik1CpHxOxO+KaSgSA0issW7toxsLcS6QDETKV4t@mail.gmail.com>

2010/10/6 Fred Drake <fdrake at acm.org>:
> On Wed, Oct 6, 2010 at 12:54 PM, Friedrich Romstedt
> <friedrichromstedt at gmail.com> wrote:
>> It's in my
>> opinion always a good idea to try to avoid hard-coding directory
>> separators (/), and to use os.path.join() instead.
>
> Distutils specifically wants "/" rather than the local platform's
> separator, and takes care of conversion as needed.

Sure, thanks for the correction!  Don't know if glob would need
platform-specific separators, though?

Friedrich

From jim at zope.com  Wed Oct  6 19:20:50 2010
From: jim at zope.com (Jim Fulton)
Date: Wed, 6 Oct 2010 13:20:50 -0400
Subject: [Distutils] using python generated by buildout to run buildout
In-Reply-To: <AANLkTinFMnZOu2t3aoUZTx4mY4DnKuv2PYWZ9qXCoSPg@mail.gmail.com>
References: <AANLkTinFMnZOu2t3aoUZTx4mY4DnKuv2PYWZ9qXCoSPg@mail.gmail.com>
Message-ID: <AANLkTim5UW0tjcqbQTdKOmCp6F_w5+zqk045jGOhgdhn@mail.gmail.com>

2010/10/6 ?ukasz Nowak <email at lnowak.com>:
> Hello,
>
> I am playing with zc.buildout for some time. I experienced issues with
> system provided pythons (even virutalenved), so I switched to build my
> own python. Of course as I am lazy, I am using buildout to configure
> and compile various python versions on my machine.
>
> Quite often I need to provide "self contained" buildout profile, which
> would be able to run using python compiled by itself.
>
> So my example profile looks like this:
>
> [buildout]
> parts =
> ?rebootstrap2.4
> ?realrun
>
> find-links = http://www.nexedi.org/static/packages/source/
>
> [python2.4]
> prefix = ${buildout:parts-directory}/${:_buildout_section_name_}
> executable = ${:prefix}/bin/python2.4
>
> # this is just wrapper for hexagonit.recipe.cmmi which makes it not
> returning python part
> recipe = erp5.recipe.cmmisafe
> url =
> ?http://python.org/ftp/python/2.4.6/Python-2.4.6.tgz
> configure-options =
> ?--enable-unicode=ucs4
> ?--with-threads
> ?--with-readline
> ?--with-dbm
> ?--with-zlib
> ?--with-ssl
> ?--with-bz2
>
> [rebootstrap2.4]
> recipe = zc.recipe.egg
> eggs = zc.buildout
> python = python2.4
>
> [realrun]
> recipe = plone.recipe.command
> command =
> ?echo Running with python ${buildout:executable}
> update-command = ${:command}
>
> So first step for me is to bootstrap builodut:
>
> curl -s http://svn.zope.org/*checkout*/zc.buildout/trunk/bootstrap/bootstrap.py
> | python -S -
>
> So now my bin/buildout is referring to "system python":
> head -n1 bin/buildout
> #!/home/luke/bin/python -S
>
> Then I need to rebootstrap it:
>
> bin/buildout install python2.4 rebootstrap2.4
>
> python2.4* parts are installing and compiling python, rebootstrap2.4
> part is reinstalling bin/buildout using this python.
>
> Cool, my bin/buildout is using python it compiled:
>
> head -n1 bin/buildout
> #!/home/luke/pybuildout/parts/python2.4/bin/python2.4
>
> So I can run again, this time usually. python will be compiled, as
> buildout signature changed, but this is not an issue (ccache + distcc
> makes it really fast). Buildout is using python he provided,
> chicken-and-egg issue solved by hand:
>
> bin/buildout
> ...
>
> realrun: Running '
> echo Running with python /home/luke/pybuildout/parts/python2.4/bin/python2.4'
> Running with python /home/luke/pybuildout/parts/python2.4/bin/python2.4
> Unused options for realrun: 'update-command'.
>
> Re running builodut does not affect python anymore:
>
> bin/buildout
> Updating python2.4-dbm-patch.
> Updating python2.4.
> Updating rebootstrap2.4.
> Updating realrun.
> realrun: Running
> echo Running with python /home/luke/pybuildout/parts/python2.4/bin/python2.4
> Running with python /home/luke/pybuildout/parts/python2.4/bin/python2.4

Sounds cool!

> And so far I am happy, I am using correct python version.
>
> What I'd like to develop is to make those steps automatically, so that
> after running:
>
> curl -s http://svn.zope.org/*checkout*/zc.buildout/trunk/bootstrap/bootstrap.py
> | python -S -
> bin/buildout
>
> Such steps would happen automagically:
>
> ?1 normal bootstrap with system python
> ?2 compile python as needed
> ?3 install new bin/buildout
> ?4 re-run bin/buildout
> ?5 compile python as signature change
> ?6 install new bin/buildout (which is the same)
> ?7 re-run bin/buildout
> ?8 do not compile python, as signature had not changed, continue...
>
> Theoretically steps 5,6 and 7 could be avoided, but lets say I am
> purist -- I do not trust that python compiled by buildout which is
> running python I do not trust is bad :)
>
> Easy question: Is there ability for buildout to do smart rebootstrap
> in one run, with selecting part which contains python?

No.  It might be worthwhile to build in support for this use case.

> If no I am ready to develop such thing.

Cool.

> So I read a bit about
> extensions - I can hook before and after buildout is run. Extensions
> have access to buildout object, I think they can play with buildout
> externals a lot, including re-running buildout (I saw in output that
> buildout re-runs itself after upgrading himself).
>
> Are extension the way to go with such thing?

Extensions are a valuable experiment, but they aren't
really fleshed out.

The mechanism is very simple, which is good.  To make it more
valuable would require providing more hook points in buildout itself.

As you point out though, you could convievably use the unload
extension to cause the buildout to be rerun.

> If needed I am even ready to play a bit more with zc.buildout
> internals code to extend if required, but according to my
> understanding how buildout works there are enough places I can "hook"
> to.

You may be right.  It sounds like a good next step.

You could also propose additional apis that might help your
extension.

> Any other comments?

Only that your use case is not that uncommon.  Perhaps there
should be a built in option to have buildout build it's own Python.

One thought is that building Python is expensive enough that you'd like
to share the work accross buildouts.  *Maybe* it would be interesting
to point buildout at a separate Python buildout.  It would look at the
Python buildout and build *it* if necessary and then use the resulting
Python.

Jim

--
Jim Fulton

From pje at telecommunity.com  Wed Oct  6 19:26:22 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 06 Oct 2010 13:26:22 -0400
Subject: [Distutils] Copying entire directory
In-Reply-To: <AANLkTik1CpHxOxO+KaSgSA0issW7toxsLcS6QDETKV4t@mail.gmail.c
 om>
References: <4CAB347D.3030708@create-net.org>
	<AANLkTikYhL23mK2MWYJb7aaUvD47widnwNwbqAx4Gjtq@mail.gmail.com>
	<AANLkTi=-Y_o+HDxY5botdUA0YDewpkD=ok2xCxGCdgZZ@mail.gmail.com>
	<AANLkTik1CpHxOxO+KaSgSA0issW7toxsLcS6QDETKV4t@mail.gmail.com>
Message-ID: <20101006172618.5F2823A4100@sparrow.telecommunity.com>

At 07:06 PM 10/6/2010 +0200, Friedrich Romstedt wrote:
>2010/10/6 Fred Drake <fdrake at acm.org>:
> > On Wed, Oct 6, 2010 at 12:54 PM, Friedrich Romstedt
> > <friedrichromstedt at gmail.com> wrote:
> >> It's in my
> >> opinion always a good idea to try to avoid hard-coding directory
> >> separators (/), and to use os.path.join() instead.
> >
> > Distutils specifically wants "/" rather than the local platform's
> > separator, and takes care of conversion as needed.
>
>Sure, thanks for the correction!  Don't know if glob would need
>platform-specific separators, though?

glob accepts '/' on most platforms, but returns platform-specific 
separators.  For this application, though, it's simpler to just use 
['prefix/'+f for f in os.listdir('prefix')].

Or, if you like obfuscational, er, I mean, functional programming, 
you can user:

     map('prefix/'.__add__, os.listdir('prefix'))

;-)


From fdrake at acm.org  Wed Oct  6 19:30:42 2010
From: fdrake at acm.org (Fred Drake)
Date: Wed, 6 Oct 2010 13:30:42 -0400
Subject: [Distutils] Copying entire directory
In-Reply-To: <AANLkTik1CpHxOxO+KaSgSA0issW7toxsLcS6QDETKV4t@mail.gmail.com>
References: <4CAB347D.3030708@create-net.org>
	<AANLkTikYhL23mK2MWYJb7aaUvD47widnwNwbqAx4Gjtq@mail.gmail.com>
	<AANLkTi=-Y_o+HDxY5botdUA0YDewpkD=ok2xCxGCdgZZ@mail.gmail.com>
	<AANLkTik1CpHxOxO+KaSgSA0issW7toxsLcS6QDETKV4t@mail.gmail.com>
Message-ID: <AANLkTinp-ZyXaiMQKW9J345-jB7hrVjgFR3zq1886G1o@mail.gmail.com>

On Wed, Oct 6, 2010 at 1:06 PM, Friedrich Romstedt
<friedrichromstedt at gmail.com> wrote:
> Sure, thanks for the correction! ?Don't know if glob would need
> platform-specific separators, though?

The glob module does need platform separators.  When used indirectly
via distutils, "/" is always the right thing, though.  distutils is
required to ensure the transformation is performed before passing
paths through to glob.


? -Fred

--
Fred L. Drake, Jr.? ? <fdrake at acm.org>
"A storm broke loose in my mind."? --Albert Einstein

From ktenney at gmail.com  Wed Oct  6 19:58:13 2010
From: ktenney at gmail.com (Kent Tenney)
Date: Wed, 6 Oct 2010 12:58:13 -0500
Subject: [Distutils] using python generated by buildout to run buildout
In-Reply-To: <AANLkTim5UW0tjcqbQTdKOmCp6F_w5+zqk045jGOhgdhn@mail.gmail.com>
References: <AANLkTinFMnZOu2t3aoUZTx4mY4DnKuv2PYWZ9qXCoSPg@mail.gmail.com>
	<AANLkTim5UW0tjcqbQTdKOmCp6F_w5+zqk045jGOhgdhn@mail.gmail.com>
Message-ID: <AANLkTim558dGfYTRVxxin8KOwrGaRz9CzPuW_P1gcR8x@mail.gmail.com>

2010/10/6 Jim Fulton <jim at zope.com>:
> 2010/10/6 ?ukasz Nowak <email at lnowak.com>:
>> Hello,
>>
>> I am playing with zc.buildout for some time. I experienced issues with
>> system provided pythons (even virutalenved), so I switched to build my
>> own python. Of course as I am lazy, I am using buildout to configure
>> and compile various python versions on my machine.
>>
>> Quite often I need to provide "self contained" buildout profile, which
>> would be able to run using python compiled by itself.
>>
>> So my example profile looks like this:
>>
>> [buildout]
>> parts =
>> ?rebootstrap2.4
>> ?realrun
>>
>> find-links = http://www.nexedi.org/static/packages/source/
>>
>> [python2.4]
>> prefix = ${buildout:parts-directory}/${:_buildout_section_name_}
>> executable = ${:prefix}/bin/python2.4
>>
>> # this is just wrapper for hexagonit.recipe.cmmi which makes it not
>> returning python part
>> recipe = erp5.recipe.cmmisafe
>> url =
>> ?http://python.org/ftp/python/2.4.6/Python-2.4.6.tgz
>> configure-options =
>> ?--enable-unicode=ucs4
>> ?--with-threads
>> ?--with-readline
>> ?--with-dbm
>> ?--with-zlib
>> ?--with-ssl
>> ?--with-bz2
>>
>> [rebootstrap2.4]
>> recipe = zc.recipe.egg
>> eggs = zc.buildout
>> python = python2.4
>>
>> [realrun]
>> recipe = plone.recipe.command
>> command =
>> ?echo Running with python ${buildout:executable}
>> update-command = ${:command}
>>
>> So first step for me is to bootstrap builodut:
>>
>> curl -s http://svn.zope.org/*checkout*/zc.buildout/trunk/bootstrap/bootstrap.py
>> | python -S -
>>
>> So now my bin/buildout is referring to "system python":
>> head -n1 bin/buildout
>> #!/home/luke/bin/python -S
>>
>> Then I need to rebootstrap it:
>>
>> bin/buildout install python2.4 rebootstrap2.4
>>
>> python2.4* parts are installing and compiling python, rebootstrap2.4
>> part is reinstalling bin/buildout using this python.
>>
>> Cool, my bin/buildout is using python it compiled:
>>
>> head -n1 bin/buildout
>> #!/home/luke/pybuildout/parts/python2.4/bin/python2.4
>>
>> So I can run again, this time usually. python will be compiled, as
>> buildout signature changed, but this is not an issue (ccache + distcc
>> makes it really fast). Buildout is using python he provided,
>> chicken-and-egg issue solved by hand:
>>
>> bin/buildout
>> ...
>>
>> realrun: Running '
>> echo Running with python /home/luke/pybuildout/parts/python2.4/bin/python2.4'
>> Running with python /home/luke/pybuildout/parts/python2.4/bin/python2.4
>> Unused options for realrun: 'update-command'.
>>
>> Re running builodut does not affect python anymore:
>>
>> bin/buildout
>> Updating python2.4-dbm-patch.
>> Updating python2.4.
>> Updating rebootstrap2.4.
>> Updating realrun.
>> realrun: Running
>> echo Running with python /home/luke/pybuildout/parts/python2.4/bin/python2.4
>> Running with python /home/luke/pybuildout/parts/python2.4/bin/python2.4
>
> Sounds cool!
>
>> And so far I am happy, I am using correct python version.
>>
>> What I'd like to develop is to make those steps automatically, so that
>> after running:
>>
>> curl -s http://svn.zope.org/*checkout*/zc.buildout/trunk/bootstrap/bootstrap.py
>> | python -S -
>> bin/buildout
>>
>> Such steps would happen automagically:
>>
>> ?1 normal bootstrap with system python
>> ?2 compile python as needed
>> ?3 install new bin/buildout
>> ?4 re-run bin/buildout
>> ?5 compile python as signature change
>> ?6 install new bin/buildout (which is the same)
>> ?7 re-run bin/buildout
>> ?8 do not compile python, as signature had not changed, continue...
>>
>> Theoretically steps 5,6 and 7 could be avoided, but lets say I am
>> purist -- I do not trust that python compiled by buildout which is
>> running python I do not trust is bad :)
>>
>> Easy question: Is there ability for buildout to do smart rebootstrap
>> in one run, with selecting part which contains python?
>
> No. ?It might be worthwhile to build in support for this use case.
>
>> If no I am ready to develop such thing.
>
> Cool.
>
>> So I read a bit about
>> extensions - I can hook before and after buildout is run. Extensions
>> have access to buildout object, I think they can play with buildout
>> externals a lot, including re-running buildout (I saw in output that
>> buildout re-runs itself after upgrading himself).
>>
>> Are extension the way to go with such thing?
>
> Extensions are a valuable experiment, but they aren't
> really fleshed out.
>
> The mechanism is very simple, which is good. ?To make it more
> valuable would require providing more hook points in buildout itself.
>
> As you point out though, you could convievably use the unload
> extension to cause the buildout to be rerun.
>
>> If needed I am even ready to play a bit more with zc.buildout
>> internals code to extend if required, but according to my
>> understanding how buildout works there are enough places I can "hook"
>> to.
>
> You may be right. ?It sounds like a good next step.
>
> You could also propose additional apis that might help your
> extension.
>
>> Any other comments?
>
> Only that your use case is not that uncommon.

/me raises hand, that's _exactly_ what I'm futzing with at the moment,
as I try to automate building a Bacula server from tarballs, using a buildout.

Currently I have a companion shell script which builds the private Python.
Feels dirty.

I'm doing it on a vm, reverting between attempts.
I would be tickled to test any work along these lines.

Thanks,
Kent

 ?Perhaps there
> should be a built in option to have buildout build it's own Python.
>
> One thought is that building Python is expensive enough that you'd like
> to share the work accross buildouts. ?*Maybe* it would be interesting
> to point buildout at a separate Python buildout. ?It would look at the
> Python buildout and build *it* if necessary and then use the resulting
> Python.
>
> Jim
>
> --
> Jim Fulton
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>

From asheesh at asheesh.org  Wed Oct  6 21:48:22 2010
From: asheesh at asheesh.org (Asheesh Laroia)
Date: Wed, 6 Oct 2010 15:48:22 -0400 (EDT)
Subject: [Distutils] distribute and setuptools are fighting;
 my application is a casualty
Message-ID: <alpine.DEB.2.00.1010061520350.10500@rose.makesad.us>

Hey all,

Despite the dramatic subject line, I'm willing to be patient, friendly, 
and flexible. I'm just trying to figure out the best way to do something 
that used to work.

I have a (open source) (web) application at http://openhatch.org/ (code at 
"git clone git://code.openhatch.org/milestone-a.git") that uses 
bootstrap.py. Since a year ago, I used bootstrap.py plus zc.buildout to 
provide a mini environment for my application.

Recently, that stopped working properly. A contributor ran "bootstrap.py" 
as a regular user on Ubuntu 10.04, and got this error 
<https://openhatch.org/bugs/msg475>. I'm using ... to skip parts but you 
can see the full message at the link.

 	karen at laesa:~/Code/milestone-a$ python2.6 bootstrap.py
 	...
 	Getting distribution for 'distribute'.
 	...
 	Setuptools installation detected at /usr/lib/python2.6/dist-packages
 	...
 	Already patched.
 	/usr/lib/python2.6/dist-packages/setuptools.egg-info already patched.
 	After install bootstrap.
 	Creating
 	/usr/local/lib/python2.6/dist-packages/setuptools-0.6c11-py2.6.egg-info
 	error:
 	/usr/local/lib/python2.6/dist-packages/setuptools-0.6c11-py2.6.egg-info:
 	Permission denied

At this point it gives up:

 	An error occurred when trying to install distribute 0.6.14. Look above
 	this message for any errors that were output by easy_install.

The buildout is being done by a user who isn't root, so it's 
straightforward that the user cannot modify /usr/local/lib/*. I thought 
the nice thing about buildout was that it didn't require mutating the 
system's state (and therefore doesn't require to be run as root). The only 
way I've found to fix this is to ask people to "sudo easy_install 
distribute". I'm okay putting that in the docs, but I want to make sure 
that people don't run into this situation by accident.

I can reproduce it on my Debian desktop, too, if I do "pip uninstall 
distribute" and then try to run bootstrap.py.

So my question is -- what should I do now? Should I provide a modified 
bootstrap.py in my application that detects that the user needs to "sudo 
easy_install distribute"? If so, what does that code look like?

(I've tried using the distribute.py from 
http://bitbucket.org/tarek/buildout-distribute/src/tip/bootstrap.py and 
get the same error.)

Honestly confused, and seeking help,

-- Asheesh.

P.S. I feel like a child whose parents are fighting, and while the 
argument takes place, I sit hungry waiting for dinner. See also 
http://lyrics.wikia.com/Moxy_Fr%C3%BCvous:The_Kids%27_Song

-- 
The last thing one knows in constructing a work is what to put first.
 		-- Blaise Pascal

From ziade.tarek at gmail.com  Thu Oct  7 10:21:53 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 7 Oct 2010 10:21:53 +0200
Subject: [Distutils] distribute and setuptools are fighting;
 my application is a casualty
In-Reply-To: <alpine.DEB.2.00.1010061520350.10500@rose.makesad.us>
References: <alpine.DEB.2.00.1010061520350.10500@rose.makesad.us>
Message-ID: <AANLkTik3cBrsbsBVNS7EGevJ3rXpwOEfGgLB8cdTw4un@mail.gmail.com>

On Wed, Oct 6, 2010 at 9:48 PM, Asheesh Laroia <asheesh at asheesh.org> wrote:
...
>
> (I've tried using the distribute.py from
> http://bitbucket.org/tarek/buildout-distribute/src/tip/bootstrap.py and get
> the same error.)

distribute.py ?

>
> Honestly confused, and seeking help,

This looks like a bug, the bootstrap.py script should only affect the
buildout environment it was run in, not the global Python
installation.

You can add a bug in the zc.buildout tracker, with all the details (no
need to add funny quotes or song lyrics there) and I can take it from
there. Also please make the issue title explicit.  Something like
"bootstrap.py tries to write in the global python" can work.

Thanks!

Tarek
-- 
Tarek Ziad? | http://ziade.org

From suresh_vv at yahoo.com  Thu Oct  7 10:32:04 2010
From: suresh_vv at yahoo.com (Suresh V.)
Date: Thu, 07 Oct 2010 14:02:04 +0530
Subject: [Distutils] distribute and setuptools are fighting;
	my application is a casualty
In-Reply-To: <alpine.DEB.2.00.1010061520350.10500@rose.makesad.us>
References: <alpine.DEB.2.00.1010061520350.10500@rose.makesad.us>
Message-ID: <4CAD8584.7020009@yahoo.com>

Use virtualenv. Will provide a good foster home for you!


From asheesh at asheesh.org  Wed Oct  6 21:48:22 2010
From: asheesh at asheesh.org (Asheesh Laroia)
Date: Wed, 6 Oct 2010 15:48:22 -0400 (EDT)
Subject: [Distutils] distribute and setuptools are fighting;
 my application is a casualty
Message-ID: <alpine.DEB.2.00.1010061520350.10500@rose.makesad.us>

Hey all,

Despite the dramatic subject line, I'm willing to be patient, friendly, 
and flexible. I'm just trying to figure out the best way to do something 
that used to work.

I have a (open source) (web) application at http://openhatch.org/ (code at 
"git clone git://code.openhatch.org/milestone-a.git") that uses 
bootstrap.py. Since a year ago, I used bootstrap.py plus zc.buildout to 
provide a mini environment for my application.

Recently, that stopped working properly. A contributor ran "bootstrap.py" 
as a regular user on Ubuntu 10.04, and got this error 
<https://openhatch.org/bugs/msg475>. I'm using ... to skip parts but you 
can see the full message at the link.

 	karen at laesa:~/Code/milestone-a$ python2.6 bootstrap.py
 	...
 	Getting distribution for 'distribute'.
 	...
 	Setuptools installation detected at /usr/lib/python2.6/dist-packages
 	...
 	Already patched.
 	/usr/lib/python2.6/dist-packages/setuptools.egg-info already patched.
 	After install bootstrap.
 	Creating
 	/usr/local/lib/python2.6/dist-packages/setuptools-0.6c11-py2.6.egg-info
 	error:
 	/usr/local/lib/python2.6/dist-packages/setuptools-0.6c11-py2.6.egg-info:
 	Permission denied

At this point it gives up:

 	An error occurred when trying to install distribute 0.6.14. Look above
 	this message for any errors that were output by easy_install.

The buildout is being done by a user who isn't root, so it's 
straightforward that the user cannot modify /usr/local/lib/*. I thought 
the nice thing about buildout was that it didn't require mutating the 
system's state (and therefore doesn't require to be run as root). The only 
way I've found to fix this is to ask people to "sudo easy_install 
distribute". I'm okay putting that in the docs, but I want to make sure 
that people don't run into this situation by accident.

I can reproduce it on my Debian desktop, too, if I do "pip uninstall 
distribute" and then try to run bootstrap.py.

So my question is -- what should I do now? Should I provide a modified 
bootstrap.py in my application that detects that the user needs to "sudo 
easy_install distribute"? If so, what does that code look like?

(I've tried using the distribute.py from 
http://bitbucket.org/tarek/buildout-distribute/src/tip/bootstrap.py and 
get the same error.)

Honestly confused, and seeking help,

-- Asheesh.

P.S. I feel like a child whose parents are fighting, and while the 
argument takes place, I sit hungry waiting for dinner. See also 
http://lyrics.wikia.com/Moxy_Fr%C3%BCvous:The_Kids%27_Song

-- 
The last thing one knows in constructing a work is what to put first.
 		-- Blaise Pascal


From asheesh at asheesh.org  Thu Oct  7 15:38:01 2010
From: asheesh at asheesh.org (Asheesh Laroia)
Date: Thu, 7 Oct 2010 09:38:01 -0400 (EDT)
Subject: [Distutils] distribute and setuptools are fighting;
 my application is a casualty
In-Reply-To: <AANLkTik3cBrsbsBVNS7EGevJ3rXpwOEfGgLB8cdTw4un@mail.gmail.com>
References: <alpine.DEB.2.00.1010061520350.10500@rose.makesad.us>
	<AANLkTik3cBrsbsBVNS7EGevJ3rXpwOEfGgLB8cdTw4un@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1010070659120.10500@rose.makesad.us>

On Thu, 7 Oct 2010, Tarek Ziad? wrote:

> On Wed, Oct 6, 2010 at 9:48 PM, Asheesh Laroia <asheesh at asheesh.org> wrote:
> ...
>>
>> (I've tried using the distribute.py from
>> http://bitbucket.org/tarek/buildout-distribute/src/tip/bootstrap.py and get
>> the same error.)
>
> distribute.py ?

Er, I mean I tried using bootstrap.py from there.

>>
>> Honestly confused, and seeking help,
>
> This looks like a bug, the bootstrap.py script should only affect the 
> buildout environment it was run in, not the global Python installation.

Great, okay.

> You can add a bug in the zc.buildout tracker, with all the details (no 
> need to add funny quotes or song lyrics there) and I can take it from 
> there. Also please make the issue title explicit.  Something like 
> "bootstrap.py tries to write in the global python" can work.

Great, just wanted to make sure that was a reasonable thing to do! (Maybe 
this was intended behavior for some reason.)

Filed: https://bugs.launchpad.net/zc.buildout/+bug/656303 !

-- Asheesh.

-- 
Tomorrow, this will be part of the unchangeable past but fortunately,
it can still be changed today.

From doutriaux1 at llnl.gov  Fri Oct  8 01:40:12 2010
From: doutriaux1 at llnl.gov (Charles Doutriaux)
Date: Thu, 07 Oct 2010 16:40:12 -0700
Subject: [Distutils] distutils rebuilds EVERYTHING
Message-ID: <4CAE5A5C.1050108@llnl.gov>

 Hello,

It seems that since python 2.7 the default behaviour of distutils changed.

I have a project with a LOT of .c files used in an extension

It used to be that when rebuilding (python setup.py build install) it
would only "rebuild" the .o for the C files that actually changed.

Since Python 2.7 this seems to have changed. It knows rebuilds EVERY C
file in my extension no matter if they changed or not...

I can see why the "Default" behaviour of distutils has been changed, it
is safer, but is there a way to "revert" to the old way where only the
changed files are recompiled.

I'm running Python 2.7 (framework) on a Mac 64bit. (10.6)

Thank you very much for any help with this,

Charles Doutriaux.



From ziade.tarek at gmail.com  Fri Oct  8 08:59:07 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 8 Oct 2010 08:59:07 +0200
Subject: [Distutils] distutils rebuilds EVERYTHING
In-Reply-To: <4CAE5A5C.1050108@llnl.gov>
References: <4CAE5A5C.1050108@llnl.gov>
Message-ID: <AANLkTinDmVdEyy4Smr06V3p6aENXGYTZeqyp2WS3ksdp@mail.gmail.com>

Hello,

On Fri, Oct 8, 2010 at 1:40 AM, Charles Doutriaux <doutriaux1 at llnl.gov> wrote:
> ?Hello,
>
> It seems that since python 2.7 the default behaviour of distutils changed.
>
> I have a project with a LOT of .c files used in an extension
>
> It used to be that when rebuilding (python setup.py build install) it
> would only "rebuild" the .o for the C files that actually changed.
>
> Since Python 2.7 this seems to have changed. It knows rebuilds EVERY C
> file in my extension no matter if they changed or not...
>
> I can see why the "Default" behaviour of distutils has been changed, it
> is safer, but is there a way to "revert" to the old way where only the
> changed files are recompiled.
>
> I'm running Python 2.7 (framework) on a Mac 64bit. (10.6)
>
> Thank you very much for any help with this,

This is a regression I introduced to fix the fact that the .so are not
rebuilt we you do subtle changes in your project.

We ended up reverting this, and the changes will be done in
Distutils2. see: http://bugs.python.org/issue8688 for the details.


>
> Charles Doutriaux.
>
>
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | http://ziade.org

From barry at python.org  Fri Oct  8 21:48:00 2010
From: barry at python.org (Barry Warsaw)
Date: Fri, 8 Oct 2010 15:48:00 -0400
Subject: [Distutils] Recent buildout failure in Mailman 3
Message-ID: <20101008154800.00366c79@mission>

I just got a new report of a buildout problem with Mailman 3.0a6.

https://bugs.edge.launchpad.net/mailman/+bug/656946

The first part of the problem (corrupt eggs) was easily fixed.  It was caused
by a faulty MANIFEST.in that let eggs/ and parts/ leak into the tarball.
However, even with that fixed, buildout is failing for reasons I can't yet
figure out.  A full build is pastebin'd here:

http://pastebin.ubuntu.com/509012/

I've tried pinning the setuptools version number < 0.6c12 and I've tried
pinning the logilab-common package to < 0.52, but neither workaround helps.  I
actually can't tell where the problem is: is it setuptools, distribute,
logilab-common or something else?  It *feels* like a bug in
setuptools-0.6c12dev.

Any thoughts?

Thanks,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20101008/d663579a/attachment.pgp>

From pje at telecommunity.com  Sat Oct  9 05:30:44 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 08 Oct 2010 23:30:44 -0400
Subject: [Distutils] Recent buildout failure in Mailman 3
In-Reply-To: <20101008154800.00366c79@mission>
References: <20101008154800.00366c79@mission>
Message-ID: <20101009033044.7C4153A4114@sparrow.telecommunity.com>

At 03:48 PM 10/8/2010 -0400, Barry Warsaw wrote:
>I just got a new report of a buildout problem with Mailman 3.0a6.
>
>https://bugs.edge.launchpad.net/mailman/+bug/656946
>
>The first part of the problem (corrupt eggs) was easily fixed.  It was caused
>by a faulty MANIFEST.in that let eggs/ and parts/ leak into the tarball.
>However, even with that fixed, buildout is failing for reasons I can't yet
>figure out.  A full build is pastebin'd here:
>
>http://pastebin.ubuntu.com/509012/
>
>I've tried pinning the setuptools version number < 0.6c12 and I've tried
>pinning the logilab-common package to < 0.52, but neither workaround helps.  I
>actually can't tell where the problem is: is it setuptools, distribute,
>logilab-common or something else?  It *feels* like a bug in
>setuptools-0.6c12dev.

I recently added symlink extraction support to 0.6c12dev; apparently 
it doesn't work with the symlink found in logilab-common.  (I'll have 
to investigate further to find out why.)

Older versions of setuptools simply didn't extract symlinks at all, 
so this problem didn't occur there.  I've put in a workaround ( 
http://peak.telecommunity.com/snapshots/setuptools-0.6c12dev-r85332.tar.gz 
) to fix the immediate issue (which is that an unextractable symlink 
causes an error), and next week I'll take a look at finding out why 
this *particular* symlink isn't extractable.

(Preliminary guess: the tarfile module doesn't support relative 
links, and thus needs some path fixup help from setuptools.)


From pje at telecommunity.com  Sat Oct  9 05:37:12 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 08 Oct 2010 23:37:12 -0400
Subject: [Distutils] Recent buildout failure in Mailman 3
In-Reply-To: <20101009033044.7C4153A4114@sparrow.telecommunity.com>
References: <20101008154800.00366c79@mission>
	<20101009033044.7C4153A4114@sparrow.telecommunity.com>
Message-ID: <20101009033712.C1B6A3A4114@sparrow.telecommunity.com>

At 11:30 PM 10/8/2010 -0400, P.J. Eby wrote:
>(Preliminary guess: the tarfile module doesn't support relative 
>links, and thus needs some path fixup help from setuptools.)

That's the problem; the one use case I had for symlink extraction 
(NuPlone) has a correct linkname with a full path within the tarball; 
logilab-common's tarball just has a relative pathname within the 
subdirectory in the tarfile.

At this point, I'm a bit stumped, as I don't know enough about how 
tarballs are supposed to work internally; should I just whip up a 
patch for the situation where the path has no slashes in it (the 
logilab case), or do I need to do something more sophisticated in the 
general case?


From mailing at franzoni.eu  Mon Oct 11 00:07:53 2010
From: mailing at franzoni.eu (Alan Franzoni)
Date: Mon, 11 Oct 2010 00:07:53 +0200
Subject: [Distutils] easy_install best match issue
Message-ID: <4CB23939.10907@franzoni.eu>

Hello,

I've just run into a strange issue I'm not able to solve without further
info. Maybe it's a problem with Macports or with my config.

I'm trying to install the "byteplay" pypi module on macosx 10.6.4,
python 2.6.6 built by Macports, distribute 0.6.14 installed.

So here I go:

melquiades:/ alan$ sudo /opt/local/bin/easy_install-2.6 byteplay
install_dir
/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/
Searching for byteplay
Reading http://pypi.python.org/simple/byteplay/
Reading http://code.google.com/p/byteplay
Reading http://code.google.com/p/byteplay/downloads/list
Best match: byteplay 0.2.linux-i686
Downloading
http://pypi.python.org/packages/any/b/byteplay/byteplay-0.2.linux-i686.tar.gz#md5=8243c7004826cccd182fd0c58f75fb0e
Processing byteplay-0.2.linux-i686.tar.gz
error: Couldn't find a setup script in
/tmp/easy_install-oYBFVP/byteplay-0.2.linux-i686.tar.gz
melquiades:/ alan$

There're two separate issues I see here:

- why does easy_install pick the dumb binary dist as the best match even
though prebuilt eggs are available?
- why does macosx easy_install pick a LINUX dumb binary dist as the best
match?

This is the download page:
http://code.google.com/p/byteplay/downloads/list





-- 
Alan Franzoni
contact me at public@[mysurname].eu

From pje at telecommunity.com  Mon Oct 11 02:09:35 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Sun, 10 Oct 2010 20:09:35 -0400
Subject: [Distutils] easy_install best match issue
In-Reply-To: <4CB23939.10907@franzoni.eu>
References: <4CB23939.10907@franzoni.eu>
Message-ID: <20101011000931.858583A4108@sparrow.telecommunity.com>

At 12:07 AM 10/11/2010 +0200, Alan Franzoni wrote:
>Hello,
>
>I've just run into a strange issue I'm not able to solve without further
>info. Maybe it's a problem with Macports or with my config.
>
>I'm trying to install the "byteplay" pypi module on macosx 10.6.4,
>python 2.6.6 built by Macports, distribute 0.6.14 installed.
>
>So here I go:
>
>melquiades:/ alan$ sudo /opt/local/bin/easy_install-2.6 byteplay
>install_dir
>/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/
>Searching for byteplay
>Reading http://pypi.python.org/simple/byteplay/
>Reading http://code.google.com/p/byteplay
>Reading http://code.google.com/p/byteplay/downloads/list
>Best match: byteplay 0.2.linux-i686
>Downloading
>http://pypi.python.org/packages/any/b/byteplay/byteplay-0.2.linux-i686.tar.gz#md5=8243c7004826cccd182fd0c58f75fb0e
>Processing byteplay-0.2.linux-i686.tar.gz
>error: Couldn't find a setup script in
>/tmp/easy_install-oYBFVP/byteplay-0.2.linux-i686.tar.gz
>melquiades:/ alan$
>
>There're two separate issues I see here:
>
>- why does easy_install pick the dumb binary dist as the best match even
>though prebuilt eggs are available?
>- why does macosx easy_install pick a LINUX dumb binary dist as the best
>match?

Because it thinks that 0.2.linux-i686 is a version number, and thus a 
higher version than '0.2'.  "easy_install byteplay==0.2" should fix you up. 


From mailing at franzoni.eu  Mon Oct 11 10:08:59 2010
From: mailing at franzoni.eu (Alan Franzoni)
Date: Mon, 11 Oct 2010 10:08:59 +0200
Subject: [Distutils] easy_install best match issue
In-Reply-To: <20101011000931.858583A4108@sparrow.telecommunity.com>
References: <4CB23939.10907@franzoni.eu>
	<20101011000931.858583A4108@sparrow.telecommunity.com>
Message-ID: <AANLkTimce2zDzZ4Q7ZyMVQ+-V2bZ=suvE_ixoaQ40coj@mail.gmail.com>

On Mon, Oct 11, 2010 at 2:09 AM, P.J. Eby <pje at telecommunity.com> wrote:
> Because it thinks that 0.2.linux-i686 is a version number, and thus a higher
> version than '0.2'. ?"easy_install byteplay==0.2" should fix you up.

I had already quick-solved it by easy_installing from the whole url,
but this is a bit of mess when setting dependencies in setup.py and
retrieving them.

The problem around is with the project download page then; I should
just ask the developer to remove the dumb binary dist (which is mostly
useless btw, containing paths to the developer's machine homedir)?


-- 
Alan Franzoni
--
contact me at public@[mysurname].eu

From barry at python.org  Mon Oct 11 17:46:11 2010
From: barry at python.org (Barry Warsaw)
Date: Mon, 11 Oct 2010 11:46:11 -0400
Subject: [Distutils] Recent buildout failure in Mailman 3
In-Reply-To: <20101009033712.C1B6A3A4114@sparrow.telecommunity.com>
References: <20101008154800.00366c79@mission>
	<20101009033044.7C4153A4114@sparrow.telecommunity.com>
	<20101009033712.C1B6A3A4114@sparrow.telecommunity.com>
Message-ID: <20101011114611.24aced8a@mission>

On Oct 08, 2010, at 11:37 PM, P.J. Eby wrote:

>At this point, I'm a bit stumped, as I don't know enough about how tarballs
>are supposed to work internally; should I just whip up a patch for the
>situation where the path has no slashes in it (the logilab case), or do I
>need to do something more sophisticated in the general case?

I don't know, but the fix you did commit fixes the build problem for me.

Thanks!
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20101011/31b458d2/attachment.pgp>

From a.badger at gmail.com  Mon Oct 11 18:33:19 2010
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Mon, 11 Oct 2010 12:33:19 -0400
Subject: [Distutils] Recent buildout failure in Mailman 3
In-Reply-To: <20101011114611.24aced8a@mission>
References: <20101008154800.00366c79@mission>
	<20101009033044.7C4153A4114@sparrow.telecommunity.com>
	<20101009033712.C1B6A3A4114@sparrow.telecommunity.com>
	<20101011114611.24aced8a@mission>
Message-ID: <20101011163319.GI10153@unaka.lan>

On Mon, Oct 11, 2010 at 11:46:11AM -0400, Barry Warsaw wrote:
> On Oct 08, 2010, at 11:37 PM, P.J. Eby wrote:
> 
> >At this point, I'm a bit stumped, as I don't know enough about how tarballs
> >are supposed to work internally; should I just whip up a patch for the
> >situation where the path has no slashes in it (the logilab case), or do I
> >need to do something more sophisticated in the general case?
> 
> I don't know, but the fix you did commit fixes the build problem for me.
> 
tarfile.extract() takes care of symlinks just fine.  If you're going to
reimplement extract()'s functionality by using tarfile's private methods
directly, you probably should copy tarfile.extract()'s symlink handling
routine.

Another option, less efficient but letting the tarfile module handle any
nuances of the file format, would be to restructure the setuptools code to
call tarfile.extract() and then move the file if filter_progress() returned
a changed dst.  Something like this:

            if not name.startswith('/') and '..' not in name:
                prelim_dst = os.path.join(extract_dir, *name.split('/'))

                if member.isfile() or member.isdir() or member.islnk():
                    final_dst = progress_filter(name, prelim_dst)
                    if final_dst:
                        tarfile.extract(member, extract_dir)
                        if final_dst != prelim_dst:
                            shutil.move(prelim_dst, final_dst)

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20101011/82bdf51f/attachment.pgp>

From doutriaux1 at llnl.gov  Mon Oct 11 20:36:38 2010
From: doutriaux1 at llnl.gov (Charles Doutriaux)
Date: Mon, 11 Oct 2010 11:36:38 -0700
Subject: [Distutils] distutils rebuilds EVERYTHING
In-Reply-To: <AANLkTinDmVdEyy4Smr06V3p6aENXGYTZeqyp2WS3ksdp@mail.gmail.com>
References: <4CAE5A5C.1050108@llnl.gov>
	<AANLkTinDmVdEyy4Smr06V3p6aENXGYTZeqyp2WS3ksdp@mail.gmail.com>
Message-ID: <4CB35936.9080406@llnl.gov>

 Hi Tarek,

Is there a patch out there or a manifest I can create or someway so I
can overwrite this behaviour?

If I download distutils2 do you think it would work?

Thanks,

C.

On 10/7/10 11:59 PM, Tarek Ziad? wrote:
> Hello,
>
> On Fri, Oct 8, 2010 at 1:40 AM, Charles Doutriaux <doutriaux1 at llnl.gov> wrote:
>>  Hello,
>>
>> It seems that since python 2.7 the default behaviour of distutils changed.
>>
>> I have a project with a LOT of .c files used in an extension
>>
>> It used to be that when rebuilding (python setup.py build install) it
>> would only "rebuild" the .o for the C files that actually changed.
>>
>> Since Python 2.7 this seems to have changed. It knows rebuilds EVERY C
>> file in my extension no matter if they changed or not...
>>
>> I can see why the "Default" behaviour of distutils has been changed, it
>> is safer, but is there a way to "revert" to the old way where only the
>> changed files are recompiled.
>>
>> I'm running Python 2.7 (framework) on a Mac 64bit. (10.6)
>>
>> Thank you very much for any help with this,
> This is a regression I introduced to fix the fact that the .so are not
> rebuilt we you do subtle changes in your project.
>
> We ended up reverting this, and the changes will be done in
> Distutils2. see: http://BLOCKEDbugs.python.org/issue8688 for the details.
>
>
>> Charles Doutriaux.
>>
>>
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> http://BLOCKEDmail.python.org/mailman/listinfo/distutils-sig
>>
>
>


From ronaldoussoren at mac.com  Tue Oct 12 10:54:04 2010
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 12 Oct 2010 10:54:04 +0200
Subject: [Distutils] distutils rebuilds EVERYTHING
In-Reply-To: <AANLkTinDmVdEyy4Smr06V3p6aENXGYTZeqyp2WS3ksdp@mail.gmail.com>
References: <4CAE5A5C.1050108@llnl.gov>
	<AANLkTinDmVdEyy4Smr06V3p6aENXGYTZeqyp2WS3ksdp@mail.gmail.com>
Message-ID: <25A338DC-51DB-4084-B6FB-7BD521770C40@mac.com>


On 8 Oct, 2010, at 8:59, Tarek Ziad? wrote:

> Hello,
> 
> On Fri, Oct 8, 2010 at 1:40 AM, Charles Doutriaux <doutriaux1 at llnl.gov> wrote:
>>  Hello,
>> 
>> It seems that since python 2.7 the default behaviour of distutils changed.
>> 
>> I have a project with a LOT of .c files used in an extension
>> 
>> It used to be that when rebuilding (python setup.py build install) it
>> would only "rebuild" the .o for the C files that actually changed.
>> 
>> Since Python 2.7 this seems to have changed. It knows rebuilds EVERY C
>> file in my extension no matter if they changed or not...
>> 
>> I can see why the "Default" behaviour of distutils has been changed, it
>> is safer, but is there a way to "revert" to the old way where only the
>> changed files are recompiled.
>> 
>> I'm running Python 2.7 (framework) on a Mac 64bit. (10.6)
>> 
>> Thank you very much for any help with this,
> 
> This is a regression I introduced to fix the fact that the .so are not
> rebuilt we you do subtle changes in your project.
> 
> We ended up reverting this, and the changes will be done in
> Distutils2. see: http://bugs.python.org/issue8688 for the details.

This issue is more relevant: http://bugs.python.org/issue7894

Ronald


From marrakis at gmail.com  Tue Oct 12 15:51:48 2010
From: marrakis at gmail.com (Mathieu Leduc-Hamel)
Date: Tue, 12 Oct 2010 09:51:48 -0400
Subject: [Distutils] Forcing buildout to exclude packages from site-packages
Message-ID: <AANLkTim1f+oNKb22pQTbdKBoX-AE2FiD8yaFetJf+2+6@mail.gmail.com>

Hey All

In a debate between virtualenv and buildout at work, i've started to
search a way to force buildout to exclude packages from any
site-packages. Do you know if there's to make it work ? like the
--no-site-package option of virtualenv ?

ciao

-- 
Mathieu Leduc-Hamel

From benji at benjiyork.com  Tue Oct 12 15:54:27 2010
From: benji at benjiyork.com (Benji York)
Date: Tue, 12 Oct 2010 09:54:27 -0400
Subject: [Distutils] Forcing buildout to exclude packages from
	site-packages
In-Reply-To: <AANLkTim1f+oNKb22pQTbdKBoX-AE2FiD8yaFetJf+2+6@mail.gmail.com>
References: <AANLkTim1f+oNKb22pQTbdKBoX-AE2FiD8yaFetJf+2+6@mail.gmail.com>
Message-ID: <AANLkTimC8E68+0rD0ZfWmcnEN97X2urqJeZ7h2jUESmc@mail.gmail.com>

On Tue, Oct 12, 2010 at 9:51 AM, Mathieu Leduc-Hamel <marrakis at gmail.com> wrote:
> Hey All
>
> In a debate between virtualenv and buildout at work, i've started to
> search a way to force buildout to exclude packages from any
> site-packages. Do you know if there's to make it work ? like the
> --no-site-package option of virtualenv ?

See http://pypi.python.org/pypi/zc.buildout#working-with-a-system-python,
especially the bits about include-site-packages = false and
exec-sitecustomize = false.
-- 
Benji York

From pje at telecommunity.com  Tue Oct 12 17:46:24 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 12 Oct 2010 11:46:24 -0400
Subject: [Distutils] Recent buildout failure in Mailman 3
In-Reply-To: <20101011114611.24aced8a@mission>
References: <20101008154800.00366c79@mission>
	<20101009033044.7C4153A4114@sparrow.telecommunity.com>
	<20101009033712.C1B6A3A4114@sparrow.telecommunity.com>
	<20101011114611.24aced8a@mission>
Message-ID: <20101012154617.05EDA3A4105@sparrow.telecommunity.com>

At 11:46 AM 10/11/2010 -0400, Barry Warsaw wrote:
>On Oct 08, 2010, at 11:37 PM, P.J. Eby wrote:
>
> >At this point, I'm a bit stumped, as I don't know enough about how tarballs
> >are supposed to work internally; should I just whip up a patch for the
> >situation where the path has no slashes in it (the logilab case), or do I
> >need to do something more sophisticated in the general case?
>
>I don't know, but the fix you did commit fixes the build problem for me.

But it still doesn't fully extract the archive.  The problem should 
now be fixed entirely in the latest snapshot -- in addition to not 
matching the path processing correctly, I had an extraneous argument 
being passed to getmember().


From email at lnowak.com  Fri Oct 15 13:44:53 2010
From: email at lnowak.com (=?UTF-8?Q?=C5=81ukasz_Nowak?=)
Date: Fri, 15 Oct 2010 13:44:53 +0200
Subject: [Distutils] using python generated by buildout to run buildout
In-Reply-To: <AANLkTim5UW0tjcqbQTdKOmCp6F_w5+zqk045jGOhgdhn@mail.gmail.com>
References: <AANLkTinFMnZOu2t3aoUZTx4mY4DnKuv2PYWZ9qXCoSPg@mail.gmail.com>
	<AANLkTim5UW0tjcqbQTdKOmCp6F_w5+zqk045jGOhgdhn@mail.gmail.com>
Message-ID: <AANLkTi=-YXc-csui4gXG8TpFMAFJ0LunAuXT1-XhG=zz@mail.gmail.com>

Hello,

W dniu 6 pa?dziernika 2010 19:20 u?ytkownik Jim Fulton <jim at zope.com> napisa?:
(...)
> 2010/10/6 ?ukasz Nowak <email at lnowak.com>:
>> Running with python /home/luke/pybuildout/parts/python2.4/bin/python2.4
>
> Sounds cool!
>> And so far I am happy, I am using correct python version.
>>
>> What I'd like to develop is to make those steps automatically, so that
>> after running:
>>
>> curl -s http://svn.zope.org/*checkout*/zc.buildout/trunk/bootstrap/bootstrap.py
>> | python -S -
>> bin/buildout
>>
>> Such steps would happen automagically:
>>
>> ?1 normal bootstrap with system python
>> ?2 compile python as needed
>> ?3 install new bin/buildout
>> ?4 re-run bin/buildout
>> ?5 compile python as signature change
>> ?6 install new bin/buildout (which is the same)
>> ?7 re-run bin/buildout
>> ?8 do not compile python, as signature had not changed, continue...
>>
>> Theoretically steps 5,6 and 7 could be avoided, but lets say I am
>> purist -- I do not trust that python compiled by buildout which is
>> running python I do not trust is bad :)
>>
>> Easy question: Is there ability for buildout to do smart rebootstrap
>> in one run, with selecting part which contains python?
>
> No. ?It might be worthwhile to build in support for this use case.
>
>> If no I am ready to develop such thing.
>
> Cool.

So I did. All those steps are done by one extension, available on pypi:

http://pypi.python.org/pypi/slapos.rebootstrap/

Happy hacking :)

>
>> So I read a bit about
>> extensions - I can hook before and after buildout is run. Extensions
>> have access to buildout object, I think they can play with buildout
>> externals a lot, including re-running buildout (I saw in output that
>> buildout re-runs itself after upgrading himself).
>>
>> Are extension the way to go with such thing?
>
> Extensions are a valuable experiment, but they aren't
> really fleshed out.
>
> The mechanism is very simple, which is good. ?To make it more
> valuable would require providing more hook points in buildout itself.
>
> As you point out though, you could convievably use the unload
> extension to cause the buildout to be rerun.

For now, for simplicity, I put everything in load. I need to "feel"
zc.buildout internals more to be sure, what is needed.

>
>> If needed I am even ready to play a bit more with zc.buildout
>> internals code to extend if required, but according to my
>> understanding how buildout works there are enough places I can "hook"
>> to.
>
> You may be right. ?It sounds like a good next step.
>
> You could also propose additional apis that might help your
> extension.

I feel it is too early for me to propose any API to zc.buildout.

For now I focused on producing something dirty, but working well enough.

Maybe in some day, after receiving feedback, I will really know what I need.

For *now* such methods might be cool:

 * isBuildoutUpgradable -- which tells, if it is possible to do
buildout upgrade, to not try to upgrade non local buildout
 * reinstallBuildout -- which reinstalls zc.buildout and
setuptools/distribute, with accepting executable parameter, but
preserving current versions
 * restartBuildout -- which restart buildout process again

But as I said before -- I'd prefer such extension to grow from infant
to kidnergarten before proposing anything.

>
>> Any other comments?
>
> Only that your use case is not that uncommon. ?Perhaps there
> should be a built in option to have buildout build it's own Python.
>
> One thought is that building Python is expensive enough that you'd like
> to share the work accross buildouts. ?*Maybe* it would be interesting
> to point buildout at a separate Python buildout. ?It would look at the
> Python buildout and build *it* if necessary and then use the resulting
> Python.

My extensions supports this case. The part which will be reused to
reboostrap buildout just need executable parameter, that is all. So
then if there would be any simple recipe, and executable pointing to
another python -- it will be picked up.

>
> Jim
>
> --
> Jim Fulton
>



-- 
?ukasz Nowak IT Specialist email at lnowak.com http://lnowak.com/
Skype: Shufla jid: shufla at jabster.pl gg: 1157726
``Use the Source, Luke...''

From manlio.perillo at gmail.com  Sat Oct 16 15:29:20 2010
From: manlio.perillo at gmail.com (Manlio Perillo)
Date: Sat, 16 Oct 2010 15:29:20 +0200
Subject: [Distutils] [Python] setuptools python_requires
Message-ID: <4CB9A8B0.1010605@gmail.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi.

In many of my projects I add a check for the Python version being used,
like:

if sys.version_info < (2, 6):
    raise SystemExit('xxx requires Python 2.6 or higher.')


I think it would be better to specify this requirement as a
`python_requires` parameter for the setup function.

What is the reason why this is not supported?


Thanks  Manlio
_______________________________________________
Python mailing list
Python at lists.python.it
http://lists.python.it/mailman/listinfo/python

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAky5qLAACgkQscQJ24LbaUTy5ACggh4EtKGjzKNACEGyxfY+g7Yr
prsAn02ESKScMxpqh3uxDpWLnv09KJFs
=+Yqt
-----END PGP SIGNATURE-----

From Jonathan.Endy at Gmail.com  Mon Oct 18 15:53:50 2010
From: Jonathan.Endy at Gmail.com (Jonathan Endy)
Date: Mon, 18 Oct 2010 15:53:50 +0200
Subject: [Distutils] easy_install MySQL-python file error
Message-ID: <AANLkTikixVQ9-rnzKTRpaVV5W8s+Vphnt8aDEFCbP-dx@mail.gmail.com>

Hi,
When I am trying to install MySQL-python package
I receive an error, this is the output:


F:\Python>F:\Python\Python25\Scripts\easy_install.exe
> F:\Python\MySQL-python-1.2.3
> Processing MySQL-python-1.2.3
> Running setup.py -q bdist_egg --dist-dir
> F:\Python\MySQL-python-1.2.3\egg-dist-tmp-ly3u1l
> error: The system cannot find the file specified
>

Does anyone have an idea what casing that or how to spot the problem?

TIA
Jonathan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20101018/062b194d/attachment.html>

From Jonathan.Endy at Gmail.com  Mon Oct 18 15:41:40 2010
From: Jonathan.Endy at Gmail.com (Jonathan Endy)
Date: Mon, 18 Oct 2010 15:41:40 +0200
Subject: [Distutils] easy_install MySQL-python file error
Message-ID: <AANLkTik6rRpg9diDqnQJvvixbKarPhkfdpAipBbzWQ=M@mail.gmail.com>

Hi,
When I am trying to install MySQL-python package
I receive an error, this is the output:


F:\Python>F:\Python\Python25\Scripts\easy_install.exe
> F:\Python\MySQL-python-1.2.3
> Processing MySQL-python-1.2.3
> Running setup.py -q bdist_egg --dist-dir
> F:\Python\MySQL-python-1.2.3\egg-dist-tmp-ly3u1l
> error: The system cannot find the file specified
>

Does anyone have an idea what casing that or how to spot the problem?

TIA
Jonathan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20101018/f882ecc9/attachment.html>

From pje at telecommunity.com  Mon Oct 18 18:20:40 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 18 Oct 2010 12:20:40 -0400
Subject: [Distutils] easy_install MySQL-python file error
In-Reply-To: <AANLkTik6rRpg9diDqnQJvvixbKarPhkfdpAipBbzWQ=M@mail.gmail.c
 om>
References: <AANLkTik6rRpg9diDqnQJvvixbKarPhkfdpAipBbzWQ=M@mail.gmail.com>
Message-ID: <20101018162030.92FA93A40DF@sparrow.telecommunity.com>

At 03:41 PM 10/18/2010 +0200, Jonathan Endy wrote:
>Hi,
>When I am trying to install MySQL-python package
>I receive an error, this is the output:
>
>
>F:\Python>F:\Python\Python25\Scripts\easy_install.exe 
>F:\Python\MySQL-python-1.2.3
>Processing MySQL-python-1.2.3
>Running setup.py -q bdist_egg --dist-dir 
>F:\Python\MySQL-python-1.2.3\egg-dist-tmp-ly3u1l
>error: The system cannot find the file specified
>
>
>Does anyone have an idea what casing that or how to spot the problem?

Try this:

    cd F:\Python\MySQL-python-1.2.3
    F:\Python\Python25\python.exe -c "import setuptools; 
execfile('setup.py')" bdist_egg

And send the full output.


From Jonathan.Endy at Gmail.com  Mon Oct 18 19:07:24 2010
From: Jonathan.Endy at Gmail.com (Jonathan Endy)
Date: Mon, 18 Oct 2010 19:07:24 +0200
Subject: [Distutils] easy_install MySQL-python file error
In-Reply-To: <20101018162030.92FA93A40DF@sparrow.telecommunity.com>
References: <AANLkTik6rRpg9diDqnQJvvixbKarPhkfdpAipBbzWQ=M@mail.gmail.com>
	<20101018162030.92FA93A40DF@sparrow.telecommunity.com>
Message-ID: <AANLkTinhYKNBGyejY=cCmnhanV5LM7Cd7kgQqpv3mkZy@mail.gmail.com>

Thanks,
It seems like it was path problem the
               cd F:\Python\MySQL-python-1.2.3
fix it, after I added the
     [build]
     compiler = mingw32
to setup.cfg file.

But I got another problem, I tried your command and it started to work but
stopped on
      "error: command 'gcc' failed: Permission denied"
someone suggested to move it from the cygwin/bin folder, so I copied it o
system32
then I got:
      error: command 'gcc' failed: Invalid argument

So I found the MinGW library and I took the gcc.exe from their
   and now I get the message:
        gcc: /Zl: No such file or directory
the full output is:

>  F:\Python\MySQL-python-1.2.3>F:\Python\Python25\python.exe -c "import
> setuptools; execfile('setup.py')" bdist_egg
> running bdist_egg
> running egg_info
> writing MySQL_python.egg-info\PKG-INFO
> writing top-level names to MySQL_python.egg-info\top_level.txt
> writing dependency_links to MySQL_python.egg-info\dependency_links.txt
> warning: manifest_maker: standard file '-c' not found
> reading manifest file 'MySQL_python.egg-info\SOURCES.txt'
> reading manifest template 'MANIFEST.in'
> warning: no files found matching 'MANIFEST'
> warning: no files found matching 'ChangeLog'
> warning: no files found matching 'GPL'
> writing manifest file 'MySQL_python.egg-info\SOURCES.txt'
> installing library code to build\bdist.win32\egg
> running install_lib
> running build_py
> copying MySQLdb\release.py -> build\lib.win32-2.5\MySQLdb
> running build_ext
> building '_mysql' extension
> C:\WINDOWS\system32\gcc.exe -mno-cygwin -mdll -O -Wall
> -Dversion_info=(1,2,3,'final',0) -D__version__=1.2.3
> -IF:\MySQL\mysql-5.1.51-win32\include -IF:\Python\Python25\include
> -IF:\Python\Python25\PC -c _mysql.c -o build\temp.win32-2.5\Release\_mysql.o
> /Zl
> gcc: /Zl: No such file or directory
> gcc: CreateProcess: No such file or directory
> error: command 'gcc' failed with exit status 1
>


Have you encountered this situation?



On Mon, Oct 18, 2010 at 6:20 PM, P.J. Eby <pje at telecommunity.com> wrote:

> At 03:41 PM 10/18/2010 +0200, Jonathan Endy wrote:
>
>> Hi,
>> When I am trying to install MySQL-python package
>> I receive an error, this is the output:
>>
>>
>> F:\Python>F:\Python\Python25\Scripts\easy_install.exe
>> F:\Python\MySQL-python-1.2.3
>> Processing MySQL-python-1.2.3
>> Running setup.py -q bdist_egg --dist-dir
>> F:\Python\MySQL-python-1.2.3\egg-dist-tmp-ly3u1l
>> error: The system cannot find the file specified
>>
>>
>> Does anyone have an idea what casing that or how to spot the problem?
>>
>
> Try this:
>
>   cd F:\Python\MySQL-python-1.2.3
>   F:\Python\Python25\python.exe -c "import setuptools;
> execfile('setup.py')" bdist_egg
>
> And send the full output.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20101018/8b7f2642/attachment.html>

From pje at telecommunity.com  Tue Oct 19 01:30:01 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 18 Oct 2010 19:30:01 -0400
Subject: [Distutils] easy_install MySQL-python file error
In-Reply-To: <AANLkTinhYKNBGyejY=cCmnhanV5LM7Cd7kgQqpv3mkZy@mail.gmail.c
 om>
References: <AANLkTik6rRpg9diDqnQJvvixbKarPhkfdpAipBbzWQ=M@mail.gmail.com>
	<20101018162030.92FA93A40DF@sparrow.telecommunity.com>
	<AANLkTinhYKNBGyejY=cCmnhanV5LM7Cd7kgQqpv3mkZy@mail.gmail.com>
Message-ID: <20101018232953.831EF3A40DF@sparrow.telecommunity.com>

At 07:07 PM 10/18/2010 +0200, Jonathan Endy wrote:
>Have you encountered this situation?

No, but at least now you know the problem is with your build 
environment.  I personally use cygwin with the MinGW package, 
installed using cygwin's installer.  I then run builds and 
installation from the cygwin command line, but using a Windows 
Python.  I'm afraid I can't help much more than that.

There is supposed to be a free version of the Microsoft compiler you 
can use to do this instead, but I've never tried that.

You may wish to simply use 1.2.2 instead, via:

easy_install -f http://sourceforge.net/projects/mysql-python/files/ 
MySQL-python==1.2.2)

as that version has pre-built Windows binaries for Python 2.5.


From f at rtfs.org  Tue Oct 19 13:19:54 2010
From: f at rtfs.org (Fabian Sturm)
Date: Tue, 19 Oct 2010 13:19:54 +0200
Subject: [Distutils] mercurial update problem with pip
Message-ID: <20101019131954.152755vldvs25log@www.rtfs.org>

Hi,

I am using pip version 0.3.1 on Ubuntu Linux 9.10 to install django-piston
in a virtualenv.

Since I need a specific version of django-piston I have the following  
entry in the requirements.txt file:

-e  
hg+http://bitbucket.org/jespern/django-piston at dc0ee00d3bfc#egg=django-piston

The first time the install works fine with pip but on the second time  
I get the following error:

Command hg fetch -q failed with error code 255
Exception information:
Traceback (most recent call last):
   File "/usr/lib/python2.6/dist-packages/pip.py", line 252, in main
     self.run(options, args)
   File "/usr/lib/python2.6/dist-packages/pip.py", line 408, in run
     requirement_set.install_files(finder, force_root_egg_info=self.bundle)
   File "/usr/lib/python2.6/dist-packages/pip.py", line 1741, in install_files
     req_to_install.update_editable()
   File "/usr/lib/python2.6/dist-packages/pip.py", line 1486, in  
update_editable
     version_control(self.url).obtain(self.source_dir)
   File "/usr/lib/python2.6/dist-packages/pip.py", line 2942, in obtain
     call_subprocess(['hg', 'fetch', '-q'], cwd=dest)
   File "/usr/lib/python2.6/dist-packages/pip.py", line 3543, in  
call_subprocess
     % (command_desc, proc.returncode))
InstallationError: Command hg fetch -q failed with error code 255


If I run "hg fetch -q" manually in the django-piston repository I get:

abort: working dir not at branch tip (use "hg update" to check out branch tip)

mercurial is at version 1.3.1.

Any ideas what is wrong and how I get it to work?

Thanks a lot,
Fabian




----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

From Jonathan.Endy at Gmail.com  Tue Oct 19 18:15:24 2010
From: Jonathan.Endy at Gmail.com (Jonathan Endy)
Date: Tue, 19 Oct 2010 18:15:24 +0200
Subject: [Distutils] easy_install MySQL-python file error
In-Reply-To: <20101018232953.831EF3A40DF@sparrow.telecommunity.com>
References: <AANLkTik6rRpg9diDqnQJvvixbKarPhkfdpAipBbzWQ=M@mail.gmail.com>
	<20101018162030.92FA93A40DF@sparrow.telecommunity.com>
	<AANLkTinhYKNBGyejY=cCmnhanV5LM7Cd7kgQqpv3mkZy@mail.gmail.com>
	<20101018232953.831EF3A40DF@sparrow.telecommunity.com>
Message-ID: <AANLkTikp3rFDEqf07=nimv18of9uhKLBC4Ny8AsJheMM@mail.gmail.com>

Thank you,
this older version worked!

On Tue, Oct 19, 2010 at 1:30 AM, P.J. Eby <pje at telecommunity.com> wrote:

> At 07:07 PM 10/18/2010 +0200, Jonathan Endy wrote:
>
>> Have you encountered this situation?
>>
>
> No, but at least now you know the problem is with your build environment.
>  I personally use cygwin with the MinGW package, installed using cygwin's
> installer.  I then run builds and installation from the cygwin command line,
> but using a Windows Python.  I'm afraid I can't help much more than that.
>
> There is supposed to be a free version of the Microsoft compiler you can
> use to do this instead, but I've never tried that.
>
> You may wish to simply use 1.2.2 instead, via:
>
> easy_install -f http://sourceforge.net/projects/mysql-python/files/MySQL-python==1.2.2)
>
> as that version has pre-built Windows binaries for Python 2.5.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20101019/3b5f3fe7/attachment.html>

From philmorefield at yahoo.com  Tue Oct 19 23:44:13 2010
From: philmorefield at yahoo.com (Phil Morefield)
Date: Tue, 19 Oct 2010 14:44:13 -0700 (PDT)
Subject: [Distutils] win32 installer only works on one machine
Message-ID: <995519.76175.qm@web53407.mail.re2.yahoo.com>

I'm putting together a Windows installer for a module. A pre-built DLL file is 
being included. I use the command: 


"python setup.py bdist_wininst"

with a couple of module specific commands and there are no warning or errors. As 
far as I can tell everything is built correctly. When I execute the installer on 
that same machine, the module imports fine and I'm off and running.

But when I use the same installer on a different machine, the module will not 
import correctly. The error message is:

ImportError: DLL load failed: The specified module could not be found.
?
The DLL is being placed in "C:\Python26\DLLs". That location is in sys.path. I 
see all of the same files on both machines (so far).?I'm totally lost. 


Can anyone throw some ideas out there? Are there any details I can provide that 
might help?


      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20101019/e59695ea/attachment.html>

From philmorefield at yahoo.com  Wed Oct 20 00:26:45 2010
From: philmorefield at yahoo.com (Phil Morefield)
Date: Tue, 19 Oct 2010 15:26:45 -0700 (PDT)
Subject: [Distutils] win32 installer only works on one machine
In-Reply-To: <995519.76175.qm@web53407.mail.re2.yahoo.com>
References: <995519.76175.qm@web53407.mail.re2.yahoo.com>
Message-ID: <183582.33355.qm@web53407.mail.re2.yahoo.com>

Problem solved.

I had been uninstalling and reinstalling unsuccessfully for a while. 
Something?must have been messed up with name spaces, because once I rebooted, 
the installer works fine on all machines.? Yeesh. 

?




________________________________
From: Phil Morefield <philmorefield at yahoo.com>
To: Distutils-SIG at python.org
Sent: Tue, October 19, 2010 5:44:13 PM
Subject: [Distutils] win32 installer only works on one machine


I'm putting together a Windows installer for a module. A pre-built DLL file is 
being included. I use the command: 


"python setup.py bdist_wininst"

with a couple of module specific commands and there are no warning or errors. As 
far as I can tell everything is built correctly. When I execute the installer on 
that same machine, the module imports fine and I'm off and running.

But when I use the same installer on a different machine, the module will not 
import correctly. The error message is:

ImportError: DLL load failed: The specified module could not be found.
?
The DLL is being placed in "C:\Python26\DLLs". That location is in sys.path. I 
see all of the same files on both machines (so far).?I'm totally lost. 


Can anyone throw some ideas out there? Are there any details I can provide that 
might help?



      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20101019/39d5af3e/attachment.html>

From attilaolah at gmail.com  Thu Oct 21 16:10:15 2010
From: attilaolah at gmail.com (=?ISO-8859-1?Q?Attila_Ol=E1h?=)
Date: Thu, 21 Oct 2010 16:10:15 +0200
Subject: [Distutils] [Zope-dev] Buildout syntax question
In-Reply-To: <20101021142218.2f0e33f0@bluedwarf.infrae>
References: <20101021142218.2f0e33f0@bluedwarf.infrae>
Message-ID: <4CC049C7.6090407@gmail.com>

Hi,

On 10/21/10 14:22, Sylvain Viollon wrote:
> Hello,
>
>    That might be a stupid question, but how do I escape ${ in a buildout
>    configuration file so it doesn't look for a buildout option ?

I have managed to do that by assigning "$" to a buildout variable,
something like

[section]
dollar = $
mystr = ${section:dollar}{foo:bar}

Not very nice but works. I can't find the exact example right now, so
maybe you'll have to fiddle with it, like escaping the dollar sign in
the variable, or putting it in quotes or something, but you get the idea.

>    I found nothing in the README, and if I use something like $${ I
>    have as result $${, and I just would like to get ${. Something like
>    \${ doesn't work as well.

I couldn't find it either. Anyone knows how to properly do this?

Attila

From jim at zope.com  Thu Oct 21 16:22:05 2010
From: jim at zope.com (Jim Fulton)
Date: Thu, 21 Oct 2010 10:22:05 -0400
Subject: [Distutils] [Zope-dev] Buildout syntax question
In-Reply-To: <4CC049C7.6090407@gmail.com>
References: <20101021142218.2f0e33f0@bluedwarf.infrae>
	<4CC049C7.6090407@gmail.com>
Message-ID: <AANLkTi=qwp_+a7LAMqetSMO0reiCqamV3wAAp7-76bLX@mail.gmail.com>

On Thu, Oct 21, 2010 at 10:10 AM, Attila Ol?h <attilaolah at gmail.com> wrote:
> Hi,
>
> On 10/21/10 14:22, Sylvain Viollon wrote:
>> Hello,
>>
>> ? ?That might be a stupid question, but how do I escape ${ in a buildout
>> ? ?configuration file so it doesn't look for a buildout option ?
>
> I have managed to do that by assigning "$" to a buildout variable,
> something like
>
> [section]
> dollar = $
> mystr = ${section:dollar}{foo:bar}
>
> Not very nice but works. I can't find the exact example right now, so
> maybe you'll have to fiddle with it, like escaping the dollar sign in
> the variable, or putting it in quotes or something, but you get the idea.
>
>> ? ?I found nothing in the README, and if I use something like $${ I
>> ? ?have as result $${, and I just would like to get ${. Something like
>> ? ?\${ doesn't work as well.
>
> I couldn't find it either. Anyone knows how to properly do this?

This looks like a bug. The buildout documentation says it's using the
string.Template syntax, but it's apparently only partially handling
the escape. It recognizes that $$ is an escape, but it's not
converting it to a single $.

To report a bug, use: https://bugs.launchpad.net/zc.buildout/+bugs

Jim

--
Jim Fulton

From aodagx at gmail.com  Sat Oct 23 10:57:59 2010
From: aodagx at gmail.com (Atsushi Odagiri)
Date: Sat, 23 Oct 2010 17:57:59 +0900
Subject: [Distutils] [PEP376] RECORD line separator
Message-ID: <AANLkTik0C9oUcGrGei8yEP=y8B2vY6YP4ODZABZbWMfU@mail.gmail.com>

Hi

I read pep276, and I have a question.

This PEP says that line separator of RECORD file is `os.separator`.
and 'bdist-*' command will also create RECORD file.
Does it even work with the package which doesn't depend on the OS like
bdist_egg ?

I think that file format should not depend on os environment and good
choice is CRLF.
It's standard line separator as used in email headers.

by aodag

-- 
/*
Atsushi Odagiri
http://blog.aodag.jp
mailto:aodagx at gmail.com
*/

From sylvain at infrae.com  Thu Oct 21 17:08:30 2010
From: sylvain at infrae.com (Sylvain Viollon)
Date: Thu, 21 Oct 2010 17:08:30 +0200
Subject: [Distutils] [Zope-dev] Buildout syntax question
In-Reply-To: <AANLkTi=qwp_+a7LAMqetSMO0reiCqamV3wAAp7-76bLX@mail.gmail.com>
References: <20101021142218.2f0e33f0@bluedwarf.infrae>
	<4CC049C7.6090407@gmail.com>
	<AANLkTi=qwp_+a7LAMqetSMO0reiCqamV3wAAp7-76bLX@mail.gmail.com>
Message-ID: <20101021170830.79813f21@bluedwarf.infrae>

On Thu, 21 Oct 2010 10:22:05 -0400
Jim Fulton <jim at zope.com> wrote:


  Hello,

> This looks like a bug. The buildout documentation says it's using the
> string.Template syntax, but it's apparently only partially handling
> the escape. It recognizes that $$ is an escape, but it's not
> converting it to a single $.
> 
> To report a bug, use: https://bugs.launchpad.net/zc.buildout/+bugs
> 


  I reported the bug,

  Regards,

  Sylvain,



-- 
Sylvain Viollon -- Infrae
t +31 10 243 7051 -- http://infrae.com
Hoevestraat 10 3033GC Rotterdam -- The Netherlands

From sridharr at activestate.com  Mon Oct 25 00:39:51 2010
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Sun, 24 Oct 2010 15:39:51 -0700
Subject: [Distutils] distutils2: depgraph with extras support and
	dynamic modification
In-Reply-To: <AANLkTi=CA9w4Z03CJMFpJzzby0nt4xBv1O6czMohQaAE@mail.gmail.com>
References: <36C98FFD-F466-4A7A-AA41-F42A7776B9C7@activestate.com>
	<AANLkTi=CA9w4Z03CJMFpJzzby0nt4xBv1O6czMohQaAE@mail.gmail.com>
Message-ID: <4CC4B5B7.509@activestate.com>

On 10/7/2010 1:31 AM, Tarek Ziad? wrote:
> On Tue, Oct 5, 2010 at 7:25 PM, Sridhar Ratnakumar
> <sridharr at activestate.com>  wrote:
>> re: http://bitbucket.org/tarek/distutils2/src/tip/distutils2/depgraph.py and http://tarekziade.wordpress.com/2010/10/03/a-quick-glimpse-at-distutils2-alpha3-part-2/
>>
>> Hi Tarek,
>>
>> I recently implemented a new dependency algorithm in PyPM. Incidentally, it is also called depgraph.py :-)
> Hehe
>
>> Our algorithm supports "extras" out of the box. It also enables dynamic readjustment of nodes/edges by incrementally adding requirements. Consider the case when you do:
>>
>>   pypm install fabric
>>
>> First, the depgraph is populated with installed packages. Then fabric's dependencies: paramiko and"pycrypto<2.1" are processed. paramiko depends on pycrypto, so pycrypto-2.3 (latest from pypi) is added. But when traversing back to fabric, we notice that pycrypto<2.1 is a requirement. So our algorithm takes care of such version requirements and adjusts things accordingly (effectively installing pycrypto-2.1).
>>
> Interesting !
>
>> I wonder if this would be of any use in distutils2. I'm sure the "extras" support would be useful to interoperate with legacy metadata. Even pip doesn't support extras right now, so this may be of incentive for users to move to distutils2 (or let pip use the new depgraph algorithm).  If this would be of benefit to distutils2, I can talk to Trent and Jeff about open sourcing our dependency algorithm implementation -- no guarantees though.
> That would be great indeed. We would love to see depgraph a basis for
> several installers. I am cc'ing Alexis who work on that part.
>

Hi Tarek/Alexis,

I've uploaded our MIT-licensed implementation of the algorithm here,
http://github.com/ActiveState/depgraph/blob/master/lib/depgraph.py

It contains the example usage (using fabric) at the bottom of the script.

Obviously, PyPM has the advantage of getting access to the index of 
available packages as a SQLite db[1], which PyPI/distutils2 doesn't -- 
so it will be tricky to integrate `get_available_distributions` into 
distutils2.

-srid

[1] We have our own internal mirror of PyPI complete with even packages 
without uploaded sdists, and metadata for all packages ... making it 
trivial to query any package (or its metadata) using a simple SQL 
statement (other than the index, all sources and metadata are stored as 
plain files). It is possible that we may open-source this 
implementation, provided there is an interest in first place.

From eric.lemoine at camptocamp.com  Mon Oct 25 10:53:55 2010
From: eric.lemoine at camptocamp.com (Eric Lemoine)
Date: Mon, 25 Oct 2010 10:53:55 +0200
Subject: [Distutils] zc.buildout and "System Python"
Message-ID: <AANLkTinHD4CfPJrFRzDJ8jduXmxW07mLRfO46gsnExf3@mail.gmail.com>

Hi

I'm currently switching to zc.buildout 1.5. I've been looking at
having a Python environment that is fully isolated from the main,
system-wide Python environment.

Can you confirm that zc.buildout 1.5 supports that?

Does this imply working with a "System Python"?

The term "System Python" confuses me a bit, as what I want is an
environment that is isolated from the system-wide Python environment?
So could someone please explain what a "System Python" refers to
exactly?

Also, the doc says: "Using a system Python is inherently fragile.
Using a clean, freshly-installed Python without customization in
site-packages is more robust and repeatable". Does this mean using a
"System Python" isn't recommended for production? And how can use a
"clean, freshly-installed Python"? What's a "clean, freshly-installed
Python" anyway?

Again, what I'm after is a Python environment that is fully isolated
from the system-wide Python, just like virtualenv --no-site-packages
provides?

Thanks for any answer,

-- 
Eric Lemoine

Camptocamp France SAS
Savoie Technolac, BP 352
73377 Le Bourget du Lac, Cedex

Tel : 00 33 4 79 44 44 96
Mail : eric.lemoine at camptocamp.com
http://www.camptocamp.com

From techtonik at gmail.com  Tue Oct 26 08:29:09 2010
From: techtonik at gmail.com (anatoly techtonik)
Date: Tue, 26 Oct 2010 09:29:09 +0300
Subject: [Distutils] Unable to write into TMP directory during installation
Message-ID: <AANLkTik9Svu69ZqUjXdqCjZr+6ks-zgJ0oH8p0+NnorX@mail.gmail.com>

Hello,

How come that packages are unable to write into TEMP directory during
installation?

>python -m easy_install gerald
...
Downloading http://pypi.python.org/packages/source/g/gerald/gerald-0.4.1.zip#md5=1b29e97c0077282a26829693fb07dca1
Processing gerald-0.4.1.zip
Running gerald-0.4.1\setup.py -q bdist_egg --dist-dir
c:\users\user\appdata\local\temp\easy_install-1jedvu\gerald-0.4.1\egg-dist-tmp-wg8yyn
error: SandboxViolation:
open('C:\\Users\\user\\AppData\\Local\\Temp\\gerald.log', 'a') {}

The package setup script has attempted to modify files on your system
that are not within the EasyInstall build area, and has been aborted.

This package cannot be safely installed by EasyInstall, and may not
support alternate installation locations even if you run its setup
script by hand.  Please inform the package's author and the EasyInstall
maintainers to find out if a fix or workaround is available.

--
anatoly t.

From pje at telecommunity.com  Tue Oct 26 22:05:11 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 26 Oct 2010 16:05:11 -0400
Subject: [Distutils] Unable to write into TMP directory during
 installation
In-Reply-To: <AANLkTik9Svu69ZqUjXdqCjZr+6ks-zgJ0oH8p0+NnorX@mail.gmail.c
 om>
References: <AANLkTik9Svu69ZqUjXdqCjZr+6ks-zgJ0oH8p0+NnorX@mail.gmail.com>
Message-ID: <20101026200505.C8DBC3A4758@sparrow.telecommunity.com>

At 09:29 AM 10/26/2010 +0300, anatoly techtonik wrote:
>Hello,
>
>How come that packages are unable to write into TEMP directory during
>installation?

Because projects that write anything outside of distutils-defined 
directories are breaking the implicit contract of a distutils-based 
installation, and that means that easy_install cannot guarantee that 
the installation will proceed correctly.  That is why the message 
says the package "cannot be safely installed by EasyInstall".

(I suppose it could say instead that EasyInstall cannot be sure 
whether installation is safe, and is thus refusing the temptation to 
guess, but that would make the message a bit longer.)

In this specific case ("Gerald"), the package has two problems that 
are causing this error: first, it is trying to import the package 
before it is installed, and second, it writes files as a side-effect 
of this importing.  (The author seems to have tried to work around 
the usual consequences of the first problem, but that can't mitigate 
the second problem.)

While neither of these things is technically illegal, they are in the 
first case bad practice for installation, and in the second case, bad 
programming practice for a library.  ("Gerald" appears to be an 
application rather than a library, of course, but even so, an 
application shouldn't be started up before it's been installed -- 
especially not if it's writing files outside the build/install directory.)

All this being said -- a future version of easy_install could (and 
perhaps should) reset TEMP/TMP in the installation sandbox to 
something that's actually usable.


> >python -m easy_install gerald
>...
>Downloading 
>http://pypi.python.org/packages/source/g/gerald/gerald-0.4.1.zip#md5=1b29e97c0077282a26829693fb07dca1
>Processing gerald-0.4.1.zip
>Running gerald-0.4.1\setup.py -q bdist_egg --dist-dir
>c:\users\user\appdata\local\temp\easy_install-1jedvu\gerald-0.4.1\egg-dist-tmp-wg8yyn
>error: SandboxViolation:
>open('C:\\Users\\user\\AppData\\Local\\Temp\\gerald.log', 'a') {}
>
>The package setup script has attempted to modify files on your system
>that are not within the EasyInstall build area, and has been aborted.
>
>This package cannot be safely installed by EasyInstall, and may not
>support alternate installation locations even if you run its setup
>script by hand.  Please inform the package's author and the EasyInstall
>maintainers to find out if a fix or workaround is available.
>
>--
>anatoly t.
>_______________________________________________
>Distutils-SIG maillist  -  Distutils-SIG at python.org
>http://mail.python.org/mailman/listinfo/distutils-sig


From eric.lemoine at camptocamp.com  Thu Oct 28 00:07:21 2010
From: eric.lemoine at camptocamp.com (Eric Lemoine)
Date: Thu, 28 Oct 2010 00:07:21 +0200
Subject: [Distutils] zc.buildout and "System Python"
In-Reply-To: <AANLkTinHD4CfPJrFRzDJ8jduXmxW07mLRfO46gsnExf3@mail.gmail.com>
References: <AANLkTinHD4CfPJrFRzDJ8jduXmxW07mLRfO46gsnExf3@mail.gmail.com>
Message-ID: <AANLkTinmWBRP7_nn4i110PzZDYg2hvByFQKE=BH30thN@mail.gmail.com>

On Mon, Oct 25, 2010 at 10:53 AM, Eric Lemoine
<eric.lemoine at camptocamp.com> wrote:
> Hi
>
> I'm currently switching to zc.buildout 1.5. I've been looking at
> having a Python environment that is fully isolated from the main,
> system-wide Python environment.
>
> Can you confirm that zc.buildout 1.5 supports that?
>
> Does this imply working with a "System Python"?
>
> The term "System Python" confuses me a bit, as what I want is an
> environment that is isolated from the system-wide Python environment?
> So could someone please explain what a "System Python" refers to
> exactly?
>
> Also, the doc says: "Using a system Python is inherently fragile.
> Using a clean, freshly-installed Python without customization in
> site-packages is more robust and repeatable". Does this mean using a
> "System Python" isn't recommended for production? And how can use a
> "clean, freshly-installed Python"? What's a "clean, freshly-installed
> Python" anyway?
>
> Again, what I'm after is a Python environment that is fully isolated
> from the system-wide Python, just like virtualenv --no-site-packages
> provides?


Noone can help me with this question? Thanks,



-- 
Eric Lemoine

Camptocamp France SAS
Savoie Technolac, BP 352
73377 Le Bourget du Lac, Cedex

Tel : 00 33 4 79 44 44 96
Mail : eric.lemoine at camptocamp.com
http://www.camptocamp.com

From marrakis at gmail.com  Thu Oct 28 03:02:08 2010
From: marrakis at gmail.com (Mathieu Leduc-Hamel)
Date: Wed, 27 Oct 2010 21:02:08 -0400
Subject: [Distutils] zc.buildout and "System Python"
In-Reply-To: <AANLkTinmWBRP7_nn4i110PzZDYg2hvByFQKE=BH30thN@mail.gmail.com>
References: <AANLkTinHD4CfPJrFRzDJ8jduXmxW07mLRfO46gsnExf3@mail.gmail.com>
	<AANLkTinmWBRP7_nn4i110PzZDYg2hvByFQKE=BH30thN@mail.gmail.com>
Message-ID: <AANLkTimiDUvZmLYm2HB2SoYY702wnotRVigzFD63=bH3@mail.gmail.com>

Hi Eric,

>From my understanding, it's officially supported by buildout, but the
problem is that the recipe should support it too. Then, depending of
which recipe you are using, you have to make sure it support it.

There's a flag you can use for that, as somebody told on this list:

See http://pypi.python.org/pypi/zc.buildout#working-with-a-system-python,
especially the bits about include-site-packages = false and
exec-sitecustomize = false.

Tell me if it work well on your side...

mathieu

On Wed, Oct 27, 2010 at 6:07 PM, Eric Lemoine
<eric.lemoine at camptocamp.com> wrote:
> On Mon, Oct 25, 2010 at 10:53 AM, Eric Lemoine
> <eric.lemoine at camptocamp.com> wrote:
>> Hi
>>
>> I'm currently switching to zc.buildout 1.5. I've been looking at
>> having a Python environment that is fully isolated from the main,
>> system-wide Python environment.
>>
>> Can you confirm that zc.buildout 1.5 supports that?
>>
>> Does this imply working with a "System Python"?
>>
>> The term "System Python" confuses me a bit, as what I want is an
>> environment that is isolated from the system-wide Python environment?
>> So could someone please explain what a "System Python" refers to
>> exactly?
>>
>> Also, the doc says: "Using a system Python is inherently fragile.
>> Using a clean, freshly-installed Python without customization in
>> site-packages is more robust and repeatable". Does this mean using a
>> "System Python" isn't recommended for production? And how can use a
>> "clean, freshly-installed Python"? What's a "clean, freshly-installed
>> Python" anyway?
>>
>> Again, what I'm after is a Python environment that is fully isolated
>> from the system-wide Python, just like virtualenv --no-site-packages
>> provides?
>
>
> Noone can help me with this question? Thanks,
>
>
>
> --
> Eric Lemoine
>
> Camptocamp France SAS
> Savoie Technolac, BP 352
> 73377 Le Bourget du Lac, Cedex
>
> Tel : 00 33 4 79 44 44 96
> Mail : eric.lemoine at camptocamp.com
> http://www.camptocamp.com
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Mathieu Leduc-Hamel

From jim at zope.com  Thu Oct 28 16:22:58 2010
From: jim at zope.com (Jim Fulton)
Date: Thu, 28 Oct 2010 10:22:58 -0400
Subject: [Distutils] zc.buildout and "System Python"
In-Reply-To: <AANLkTinHD4CfPJrFRzDJ8jduXmxW07mLRfO46gsnExf3@mail.gmail.com>
References: <AANLkTinHD4CfPJrFRzDJ8jduXmxW07mLRfO46gsnExf3@mail.gmail.com>
Message-ID: <AANLkTi=rTRD5CQy6JMRSbn2HxPggAC3QHcN+QTy6VhwQ@mail.gmail.com>

On Mon, Oct 25, 2010 at 4:53 AM, Eric Lemoine
<eric.lemoine at camptocamp.com> wrote:
> Hi
>
> I'm currently switching to zc.buildout 1.5. I've been looking at
> having a Python environment that is fully isolated from the main,
> system-wide Python environment.
>
> Can you confirm that zc.buildout 1.5 supports that?

That depends on what exactly "that" is. The easiest way to have "a
Python environment that is fully isolated from the main, system-wide
Python environment" is to have a separate build.

> Does this imply working with a "System Python"?

buildout 1.5 *tries* to coexist with system Python installations.

> The term "System Python" confuses me a bit, as what I want is an
> environment that is isolated from the system-wide Python environment?
> So could someone please explain what a "System Python" refers to
> exactly?

Maybe we should use a different term, but "System Python" is a
euphemism for a Python that has been packaged by an OS packager.  For
better or worse, these packagers tend to be innovative and deliver
Python installations that differ from a from-source build in
unpredictable ways.  This could involve additions, omissions, or
specialized build options.

Buildout 1.5 tries to coexist with system Pythons by hiding additions
in site-packages (or the innovative equivalent).  Virtualenv does
something similar.

I use the term "clean python" to refer to a Python installation based
on a straightforward build from source, and without any site-local
packages in it's site-packages directory. (Where straightforward is a
euphemism for a build after making sure that certain system libraries,
such as zlib, are available where Python can find them.)  We here at
ZC package these installs as RPMs that are available system wide along
side the "system Python". I tell all my friends to use clean Python's
for anything important.

> Also, the doc says: "Using a system Python is inherently fragile.

Yes.

> Using a clean, freshly-installed Python without customization in
> site-packages is more robust and repeatable". Does this mean using a
> "System Python" isn't recommended for production?

Yes. That's what it means.

> And how can use a
> "clean, freshly-installed Python"? What's a "clean, freshly-installed
> Python" anyway?

"freshly installed" is irrelevent, otherwise see above.

> Again, what I'm after is a Python environment that is fully isolated
> from the system-wide Python, just like virtualenv --no-site-packages
> provides?

The only way to get a Python that is fully isolated from another
Python install is to build one yourself from source.

Note:

- "System-wide" isn't really relevent.  What's important is a
  predictable install, which I choose to define as one you'd get from
  a source build, without special additions or omissions. You can have
  a clean Python that is installed system wide. Theoretically, system
  packages could provide clean Pythons, although I've never seen one.

  It occurs to me that it would be nice if we made clean Python
  packages available for some of the popular Unix platforms.  I'm not
  sure what would be involved in doing that, from a distribution point
  of view.

- "virtualenv --no-site-packages" doesn't give you complete isolation.
  In particular, it doesn't protect you from omissions.

Jim

--
Jim Fulton

From aljosa.mohorovic at gmail.com  Thu Oct 28 17:14:30 2010
From: aljosa.mohorovic at gmail.com (=?UTF-8?B?QWxqb8WhYSBNb2hvcm92acSH?=)
Date: Thu, 28 Oct 2010 17:14:30 +0200
Subject: [Distutils] howto "python setup.py develop" in distutils2
Message-ID: <AANLkTi=+M8i3i8bz=g6o2FT5iH9zSs_hFhPFYQvpAmF3@mail.gmail.com>

how can i get "python setup.py develop" or something similar with distutils2?
i'm just looking for a simple way to include a package into current
python environment while working on that package.

$ pip freeze|grep -i dist
Distutils2==1.0a3

any tips appreciated.

Aljosa Mohorovic

From ziade.tarek at gmail.com  Thu Oct 28 17:21:48 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 28 Oct 2010 08:21:48 -0700
Subject: [Distutils] howto "python setup.py develop" in distutils2
In-Reply-To: <AANLkTi=+M8i3i8bz=g6o2FT5iH9zSs_hFhPFYQvpAmF3@mail.gmail.com>
References: <AANLkTi=+M8i3i8bz=g6o2FT5iH9zSs_hFhPFYQvpAmF3@mail.gmail.com>
Message-ID: <AANLkTikUgZ+oOSZ2A-xAQCFgDeV2KsFfs6oOyOyVJ-Rj@mail.gmail.com>

On Thu, Oct 28, 2010 at 8:14 AM, Aljo?a Mohorovi?
<aljosa.mohorovic at gmail.com> wrote:
> how can i get "python setup.py develop" or something similar with distutils2?
> i'm just looking for a simple way to include a package into current
> python environment while working on that package

The feature is not existing yet in distutils2. In the meantime you can
change your PYTHONPATH to include the package

>
> $ pip freeze|grep -i dist
> Distutils2==1.0a3
>
> any tips appreciated.
>
> Aljosa Mohorovic
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | http://ziade.org

From a.badger at gmail.com  Thu Oct 28 17:47:47 2010
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Thu, 28 Oct 2010 08:47:47 -0700
Subject: [Distutils] zc.buildout and "System Python"
In-Reply-To: <AANLkTi=rTRD5CQy6JMRSbn2HxPggAC3QHcN+QTy6VhwQ@mail.gmail.com>
References: <AANLkTinHD4CfPJrFRzDJ8jduXmxW07mLRfO46gsnExf3@mail.gmail.com>
	<AANLkTi=rTRD5CQy6JMRSbn2HxPggAC3QHcN+QTy6VhwQ@mail.gmail.com>
Message-ID: <20101028154747.GN14816@unaka.lan>

On Thu, Oct 28, 2010 at 10:22:58AM -0400, Jim Fulton wrote:
> 
>   It occurs to me that it would be nice if we made clean Python
>   packages available for some of the popular Unix platforms.  I'm not
>   sure what would be involved in doing that, from a distribution point
>   of view.
> 
If you're talking about a python that is carried by the OS in their package
sets, updatable using the OS tools, etc catch me on IRC (abadger1999 on
irc.freenode.net) and we could talk about this.  Off the top of my head,
I think it would be possible with a few compromises but not easy in the
decision department.  For instance, distributions have rules such as "don't
bundle libraries that are available on the system" that would apply to
things like libffi which are built from within python by default.  Or the
use of wide-unicode which isn't the default in a vanilla upstream build but
is the default on the Linux distributions that I know of.  Or the use of
multilib which makes for a split directory layout for libraries instead of
a single location.

The biggest issue I see is that it wouldn't be possible to fix bugs in
these packages.  Perhaps it would be possible to compromise and fix bugs but
only when the patches are backports from the upstream repository but.... we
presently do that in Fedora for firefox/xulrunner/thunderbird because of
mozilla's trademark agreement and it causes no end of conflicts between
contributors.

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20101028/c9555218/attachment.pgp>

From aljosa.mohorovic at gmail.com  Thu Oct 28 18:07:47 2010
From: aljosa.mohorovic at gmail.com (=?UTF-8?B?QWxqb8WhYSBNb2hvcm92acSH?=)
Date: Thu, 28 Oct 2010 18:07:47 +0200
Subject: [Distutils] howto "python setup.py develop" in distutils2
In-Reply-To: <AANLkTikUgZ+oOSZ2A-xAQCFgDeV2KsFfs6oOyOyVJ-Rj@mail.gmail.com>
References: <AANLkTi=+M8i3i8bz=g6o2FT5iH9zSs_hFhPFYQvpAmF3@mail.gmail.com>
	<AANLkTikUgZ+oOSZ2A-xAQCFgDeV2KsFfs6oOyOyVJ-Rj@mail.gmail.com>
Message-ID: <AANLkTi=5RcnBVi5VTJ31-A9KeMA6pNLdrKk95SEWzjsa@mail.gmail.com>

2010/10/28 Tarek Ziad? <ziade.tarek at gmail.com>:
> The feature is not existing yet in distutils2. In the meantime you can
> change your PYTHONPATH to include the package

is somebody working on this or it's not a priority for next milestone?

Aljosa

From jim at zope.com  Thu Oct 28 18:08:30 2010
From: jim at zope.com (Jim Fulton)
Date: Thu, 28 Oct 2010 12:08:30 -0400
Subject: [Distutils] zc.buildout and "System Python"
In-Reply-To: <20101028154747.GN14816@unaka.lan>
References: <AANLkTinHD4CfPJrFRzDJ8jduXmxW07mLRfO46gsnExf3@mail.gmail.com>
	<AANLkTi=rTRD5CQy6JMRSbn2HxPggAC3QHcN+QTy6VhwQ@mail.gmail.com>
	<20101028154747.GN14816@unaka.lan>
Message-ID: <AANLkTimOGAhpc2kk9MwA83chxn8de0rdnOd=RRvNB_Ze@mail.gmail.com>

On Thu, Oct 28, 2010 at 11:47 AM, Toshio Kuratomi <a.badger at gmail.com> wrote:
> On Thu, Oct 28, 2010 at 10:22:58AM -0400, Jim Fulton wrote:
>>
>> ? It occurs to me that it would be nice if we made clean Python
>> ? packages available for some of the popular Unix platforms. ?I'm not
>> ? sure what would be involved in doing that, from a distribution point
>> ? of view.
>>
> If you're talking about a python that is carried by the OS in their package
> sets, updatable using the OS tools, etc

That would be great. It might be enough to post pre-built packages. <shug>

> catch me on IRC (abadger1999 on
> irc.freenode.net) and we could talk about this. ?Off the top of my head,
> I think it would be possible with a few compromises but not easy in the
> decision department.

Which makes it unattractive. I'm really not interested in getting
embroiled in a political process.

BTW, I really don't care about certain types of innovation (e.g. file
locations, wide unicode) as long as I as a developer don't feel them.
It occurs to me that it would be useful if there was a definition of a
standard Python that provided a baseline that developers could count
on. Today, the closest thing to a standard is the Python distribution.
I suppose that doesn't have to be the standard.  Of course, defining
such a standard might be really painful, especially via email. It might
be a good PyCon discussion/sprint topic.

> The biggest issue I see is that it wouldn't be possible to fix bugs in
> these packages. ?Perhaps it would be possible to compromise and fix bugs but
> only when the patches are backports from the upstream repository

I'm not sure what you mean. Bugs are fixed via Python distributions.
Is this not fast enough?

> but.... we
> presently do that in Fedora for firefox/xulrunner/thunderbird because of
> mozilla's trademark agreement and it causes no end of conflicts between
> contributors.

I assume that wouldn't be a problem for Python, assuming I have a clue
what that is. :)

Jim

--
Jim Fulton

From mailing at franzoni.eu  Thu Oct 28 18:18:51 2010
From: mailing at franzoni.eu (Alan Franzoni)
Date: Thu, 28 Oct 2010 18:18:51 +0200
Subject: [Distutils] pkg_resources: Loading resources in a uniform fashion
Message-ID: <AANLkTimRTD1fsDc_EPtc+HC7Geh60ogJOhos++axtbgd@mail.gmail.com>

Hello,

lately I was thinking about writing a kind of factory method for
parsing an url and returning a resource filename or stream, something
like:

fs_resource = load_resource_filename("file:///etc/software/config.conf")
pkg_resource = load_resource_filename("pkg://package.something")

I'd like the first one to just return the file system path
"/etc/software/config.conf", while I'd like the second one one to do
something roughly like

from pkg_resources import resource_filename
return resource_filename("package", "something")

This way it would be pretty easy to get a consistent way to describe a
resource from a stream.

I've got some questions:

1) is there anything around that already does something like that?
2) would you think it to be a good addition for pkg_resources, or
would it go beyond its scope?
3) can you see some obvious issues?
4) how would you handle requirements string?







-- 
Alan Franzoni
--
contact me at public@[mysurname].eu

From barry at python.org  Thu Oct 28 18:26:51 2010
From: barry at python.org (Barry Warsaw)
Date: Thu, 28 Oct 2010 12:26:51 -0400
Subject: [Distutils] zc.buildout and "System Python"
In-Reply-To: <AANLkTimOGAhpc2kk9MwA83chxn8de0rdnOd=RRvNB_Ze@mail.gmail.com>
References: <AANLkTinHD4CfPJrFRzDJ8jduXmxW07mLRfO46gsnExf3@mail.gmail.com>
	<AANLkTi=rTRD5CQy6JMRSbn2HxPggAC3QHcN+QTy6VhwQ@mail.gmail.com>
	<20101028154747.GN14816@unaka.lan>
	<AANLkTimOGAhpc2kk9MwA83chxn8de0rdnOd=RRvNB_Ze@mail.gmail.com>
Message-ID: <20101028122651.0d0238e4@mission>

On Oct 28, 2010, at 12:08 PM, Jim Fulton wrote:

>BTW, I really don't care about certain types of innovation (e.g. file
>locations, wide unicode) as long as I as a developer don't feel them.
>It occurs to me that it would be useful if there was a definition of a
>standard Python that provided a baseline that developers could count
>on. Today, the closest thing to a standard is the Python distribution.
>I suppose that doesn't have to be the standard.  Of course, defining
>such a standard might be really painful, especially via email. It might
>be a good PyCon discussion/sprint topic.

We should do this.  The "System Python" has too many competing OS-specific
constraints that pretty much ensure it will always be idiosyncratic.  FWIW, in
Debian/Ubuntu, we're at least trying to *document* some of those issues:

    http://wiki.debian.org/Python

I'm supportive of an effort to define "Clean Python" as a separate "thing"
that you can install in parallel and use as a better base for your third party
application deployment.  To be most useful, I think this should be as similar
and predictable as possible across distributions.  Of course, it gets harder
still when you want to extend that to cross-OS/platform.

But maybe there's still something we can do here.  We should put this on the
agenda for Pycon.  Would the language summit be an appropriate forum (at least
as a start)?

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20101028/434b3822/attachment.pgp>

From ziade.tarek at gmail.com  Thu Oct 28 18:37:44 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 28 Oct 2010 09:37:44 -0700
Subject: [Distutils] howto "python setup.py develop" in distutils2
In-Reply-To: <AANLkTi=5RcnBVi5VTJ31-A9KeMA6pNLdrKk95SEWzjsa@mail.gmail.com>
References: <AANLkTi=+M8i3i8bz=g6o2FT5iH9zSs_hFhPFYQvpAmF3@mail.gmail.com>
	<AANLkTikUgZ+oOSZ2A-xAQCFgDeV2KsFfs6oOyOyVJ-Rj@mail.gmail.com>
	<AANLkTi=5RcnBVi5VTJ31-A9KeMA6pNLdrKk95SEWzjsa@mail.gmail.com>
Message-ID: <AANLkTinymKeVqre2sgmbhKdpFEdcsj8z_exYUOMHBx7a@mail.gmail.com>

2010/10/28 Aljo?a Mohorovi? <aljosa.mohorovic at gmail.com>:
> 2010/10/28 Tarek Ziad? <ziade.tarek at gmail.com>:
>> The feature is not existing yet in distutils2. In the meantime you can
>> change your PYTHONPATH to include the package
>
> is somebody working on this or it's not a priority for next milestone?

I'd love to have it in a4 or b1, but I do lack of time so I am not
sure when it will be included. But that's definitely a feature we
would like to have.


-- 
Tarek Ziad? | http://ziade.org

From jim at zope.com  Thu Oct 28 18:44:59 2010
From: jim at zope.com (Jim Fulton)
Date: Thu, 28 Oct 2010 12:44:59 -0400
Subject: [Distutils] An observation on how system packagers and developers
	can be friends
Message-ID: <AANLkTimZLhJ455FQQqubgTtaXcLX7asE8B7PydXuFuCG@mail.gmail.com>

Periodically, in various venues, we discuss the challenges of
deploying applications with or in spite of system packaging of Python
and system packaging philosophies.

(Note that I'm mainly talking g about deploying applications, as opposed
to individual Python packages.)

In my experience, the conversation usually unfolds with developers
(like me:) snarkely saying how annoyed we are and systems packagers
(and administrators) explaining all of the rules they have to follow
and that we should follow too.

In practice, I think the result is usually that developers make an
end-run around system packaging system, either building in production,
or by building system packages that bundle dependencies to get needed
control.  The result is that at least some of the system packagers
goals are subverted.

In my experience, both sides have legitimate goals, but the problem is
rarely, if ever, approached from the point of view of recognizing both
sets of goals coming up with a (probably new) solution that addresses
both sets of goals.  (Note that "doing things the way I've always done
them" is *not* a valid goal. :)

I believe that solutions that address both sets of goals are possible
and even practical, but it has to start with a consideration of the
underlying goals.

Jim

--
Jim Fulton

From merwok at netwok.org  Thu Oct 28 18:52:42 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Thu, 28 Oct 2010 18:52:42 +0200
Subject: [Distutils] howto "python setup.py develop" in distutils2
In-Reply-To: <AANLkTinymKeVqre2sgmbhKdpFEdcsj8z_exYUOMHBx7a@mail.gmail.com>
References: <AANLkTi=+M8i3i8bz=g6o2FT5iH9zSs_hFhPFYQvpAmF3@mail.gmail.com>	<AANLkTikUgZ+oOSZ2A-xAQCFgDeV2KsFfs6oOyOyVJ-Rj@mail.gmail.com>	<AANLkTi=5RcnBVi5VTJ31-A9KeMA6pNLdrKk95SEWzjsa@mail.gmail.com>
	<AANLkTinymKeVqre2sgmbhKdpFEdcsj8z_exYUOMHBx7a@mail.gmail.com>
Message-ID: <4CC9AA5A.8020806@netwok.org>

See http://bugs.python.org/issue8668 for this feature request.

Regards



From a.badger at gmail.com  Thu Oct 28 19:55:18 2010
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Thu, 28 Oct 2010 10:55:18 -0700
Subject: [Distutils] zc.buildout and "System Python"
In-Reply-To: <AANLkTimOGAhpc2kk9MwA83chxn8de0rdnOd=RRvNB_Ze@mail.gmail.com>
References: <AANLkTinHD4CfPJrFRzDJ8jduXmxW07mLRfO46gsnExf3@mail.gmail.com>
	<AANLkTi=rTRD5CQy6JMRSbn2HxPggAC3QHcN+QTy6VhwQ@mail.gmail.com>
	<20101028154747.GN14816@unaka.lan>
	<AANLkTimOGAhpc2kk9MwA83chxn8de0rdnOd=RRvNB_Ze@mail.gmail.com>
Message-ID: <20101028175518.GO14816@unaka.lan>

On Thu, Oct 28, 2010 at 12:08:30PM -0400, Jim Fulton wrote:
> On Thu, Oct 28, 2010 at 11:47 AM, Toshio Kuratomi <a.badger at gmail.com> wrote:
> > On Thu, Oct 28, 2010 at 10:22:58AM -0400, Jim Fulton wrote:
> >>
> >> ? It occurs to me that it would be nice if we made clean Python
> >> ? packages available for some of the popular Unix platforms. ?I'm not
> >> ? sure what would be involved in doing that, from a distribution point
> >> ? of view.
> >>
> > If you're talking about a python that is carried by the OS in their package
> > sets, updatable using the OS tools, etc
> 
> That would be great. It might be enough to post pre-built packages. <shug>
> 
<nod>  There's a few ways to achieve that as well and it would be a lot
simpler.  There's the opensuse build system which lets you build packages
for a variety of distributions.  There's ubuntu ppas and fedora personal
repos that let you host within the distributions namespace but are marked as
being separate... Lots of different options there that might be suitable.

> > catch me on IRC (abadger1999 on
> > irc.freenode.net) and we could talk about this. ?Off the top of my head,
> > I think it would be possible with a few compromises but not easy in the
> > decision department.
> 
> Which makes it unattractive. I'm really not interested in getting
> embroiled in a political process.
> 

<nod>  If going the route of getting a "clean python" into the distributions
themselvs, there's going to be a good deal of politics as there are a lot of
hard questions to answer there and a lot of contrary goals to reconcile.
The idea of simply hosting package repositories for each distribution would
be a lot easier in this area.

> BTW, I really don't care about certain types of innovation (e.g. file
> locations, wide unicode) as long as I as a developer don't feel them.
> It occurs to me that it would be useful if there was a definition of a
> standard Python that provided a baseline that developers could count
> on. Today, the closest thing to a standard is the Python distribution.
> I suppose that doesn't have to be the standard.  Of course, defining
> such a standard might be really painful, especially via email. It might
> be a good PyCon discussion/sprint topic.
> 
<nod>  That could be a productive definition.

> > The biggest issue I see is that it wouldn't be possible to fix bugs in
> > these packages. ?Perhaps it would be possible to compromise and fix bugs but
> > only when the patches are backports from the upstream repository
> 
> I'm not sure what you mean. Bugs are fixed via Python distributions.
> Is this not fast enough?
> 
Correct, it's not fast enoguh.  Many distributions move imuch faster than
python releases.  Even slow moving distributions can simply be
releasing/releasing updates out of sync with when python itself releases.

As an example, Fedora releases a new version every six months.  Each release
has a 13 month lifetime.  During the 13 month lifetime, Fedora releases
updated packages almost daily.  So if someone filed a bug that
python-2.7.1-1 had a segfault or a UnicodeError or some other bug that the
maintainer felt was worthwhile to fix in the released Fedora, they would
ship an updated python package (perhaps with a backport from python-2.x's
tree or perhaps by coding a fix and then submitting the fix upstream
afterwards) and make the update as soon as they felt they had a workable
solution.

> > but.... we
> > presently do that in Fedora for firefox/xulrunner/thunderbird because of
> > mozilla's trademark agreement and it causes no end of conflicts between
> > contributors.
> 
> I assume that wouldn't be a problem for Python, assuming I have a clue
> what that is. :)
> 
Well -- the causitive agent is different but the results are similar.  In
mozilla's case, the issue is adding code that mozilla doesn't endorse as
with their permission you have to abandon the trademarks.  In a "clean
python"'s case, it would be that we want to enforce on ourselves to only
ship what's in upstream.  In both cases, it prevents fixing bugs and making
other changes ahead of an external (to the distribution) schedule.

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20101028/97d78841/attachment-0001.pgp>

From tseaver at palladion.com  Thu Oct 28 20:02:44 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 28 Oct 2010 14:02:44 -0400
Subject: [Distutils] An observation on how system packagers and
 developers can be friends
In-Reply-To: <AANLkTimZLhJ455FQQqubgTtaXcLX7asE8B7PydXuFuCG@mail.gmail.com>
References: <AANLkTimZLhJ455FQQqubgTtaXcLX7asE8B7PydXuFuCG@mail.gmail.com>
Message-ID: <iacds4$nuq$1@dough.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/28/2010 12:44 PM, Jim Fulton wrote:
> Periodically, in various venues, we discuss the challenges of
> deploying applications with or in spite of system packaging of Python
> and system packaging philosophies.
> 
> (Note that I'm mainly talking g about deploying applications, as opposed
> to individual Python packages.)
> 
> In my experience, the conversation usually unfolds with developers
> (like me:) snarkely saying how annoyed we are and systems packagers
> (and administrators) explaining all of the rules they have to follow
> and that we should follow too.
> 
> In practice, I think the result is usually that developers make an
> end-run around system packaging system, either building in production,
> or by building system packages that bundle dependencies to get needed
> control.  The result is that at least some of the system packagers
> goals are subverted.
> 
> In my experience, both sides have legitimate goals, but the problem is
> rarely, if ever, approached from the point of view of recognizing both
> sets of goals coming up with a (probably new) solution that addresses
> both sets of goals.  (Note that "doing things the way I've always done
> them" is *not* a valid goal. :)
> 
> I believe that solutions that address both sets of goals are possible
> and even practical, but it has to start with a consideration of the
> underlying goals.

I like the idea in general, but worry that some conflicts may not be
resolvable.  For instance, I don't know what goal drives system
packagers to specify UCS4 over the default UCS2, but I won't ever be
happy using a Python built that way for long-running, memory-intensive
applications, where I have measured the overhead of UCS4 and found it
unacceptable (e.g., a server app whose steady-state process size is
800Mb under UCS4, compared to 600Mb under UCS2).



Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkzJusQACgkQ+gerLs4ltQ6GcQCgkLKlUnbb85KFaYVMlJpBBE5F
B1MAn0hDSjU7GJ24/zHExDFqg9kkmTXz
=Lztf
-----END PGP SIGNATURE-----


From jim at zope.com  Thu Oct 28 20:47:27 2010
From: jim at zope.com (Jim Fulton)
Date: Thu, 28 Oct 2010 14:47:27 -0400
Subject: [Distutils] zc.buildout and "System Python"
In-Reply-To: <20101028122651.0d0238e4@mission>
References: <AANLkTinHD4CfPJrFRzDJ8jduXmxW07mLRfO46gsnExf3@mail.gmail.com>
	<AANLkTi=rTRD5CQy6JMRSbn2HxPggAC3QHcN+QTy6VhwQ@mail.gmail.com>
	<20101028154747.GN14816@unaka.lan>
	<AANLkTimOGAhpc2kk9MwA83chxn8de0rdnOd=RRvNB_Ze@mail.gmail.com>
	<20101028122651.0d0238e4@mission>
Message-ID: <AANLkTi=3Gjiz7TE=RtcLFWN0L4hgvJPrYQ2ozT8Tw2h+@mail.gmail.com>

On Thu, Oct 28, 2010 at 12:26 PM, Barry Warsaw <barry at python.org> wrote:
> On Oct 28, 2010, at 12:08 PM, Jim Fulton wrote:
>
>>BTW, I really don't care about certain types of innovation (e.g. file
>>locations, wide unicode) as long as I as a developer don't feel them.
>>It occurs to me that it would be useful if there was a definition of a
>>standard Python that provided a baseline that developers could count
>>on. Today, the closest thing to a standard is the Python distribution.
>>I suppose that doesn't have to be the standard. ?Of course, defining
>>such a standard might be really painful, especially via email. It might
>>be a good PyCon discussion/sprint topic.
>
> We should do this.

Let's do it! :)

...

> But maybe there's still something we can do here. ?We should put this on the
> agenda for Pycon.

Not sure what that involves.

>?Would the language summit be an appropriate forum (at least
> as a start)?

I think it would be a good idea to at least mention it there.
I'm not sure all of the interested people will be there.

Jim

-- 
Jim Fulton

From martin at v.loewis.de  Thu Oct 28 20:48:19 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 28 Oct 2010 20:48:19 +0200
Subject: [Distutils] An observation on how system packagers and
 developers can be friends
In-Reply-To: <iacds4$nuq$1@dough.gmane.org>
References: <AANLkTimZLhJ455FQQqubgTtaXcLX7asE8B7PydXuFuCG@mail.gmail.com>
	<iacds4$nuq$1@dough.gmane.org>
Message-ID: <4CC9C573.5080903@v.loewis.de>

> I like the idea in general, but worry that some conflicts may not be
> resolvable.  For instance, I don't know what goal drives system
> packagers to specify UCS4 over the default UCS2

That is easy to say: feature support. You can't really call it "Unicode"
if you can't index it by code point. Some parts of the API (e.g. the
Unicode database) are incomplete in UCS-2 mode.

A minor objective is C library compliance: wchar_t has also four bytes
per character.

> but I won't ever be
> happy using a Python built that way for long-running, memory-intensive
> applications, where I have measured the overhead of UCS4 and found it
> unacceptable (e.g., a server app whose steady-state process size is
> 800Mb under UCS4, compared to 600Mb under UCS2).

I don't think this is really in the field of issues that Jim was talking
about. In any case, if you have long-running memory-intensive
applications, recompiling Python might be just the right thing to do.
In the same direction, I wish people would understand that 64-bit Python
builds just waste memory, and that they were better off with 32-bit
implementations.

>From the developer point of view, I'd wonder whether there is potential
for some even more dramatic savings than 25%.

Regards,
Martin

From barry at python.org  Thu Oct 28 22:50:35 2010
From: barry at python.org (Barry Warsaw)
Date: Thu, 28 Oct 2010 16:50:35 -0400
Subject: [Distutils] An observation on how system packagers and
 developers can be friends
In-Reply-To: <iacds4$nuq$1@dough.gmane.org>
References: <AANLkTimZLhJ455FQQqubgTtaXcLX7asE8B7PydXuFuCG@mail.gmail.com>
	<iacds4$nuq$1@dough.gmane.org>
Message-ID: <20101028165035.7a3faa3a@mission>

On Oct 28, 2010, at 02:02 PM, Tres Seaver wrote:

>I like the idea in general, but worry that some conflicts may not be
>resolvable.  For instance, I don't know what goal drives system
>packagers to specify UCS4 over the default UCS2, but I won't ever be
>happy using a Python built that way for long-running, memory-intensive
>applications, where I have measured the overhead of UCS4 and found it
>unacceptable (e.g., a server app whose steady-state process size is
>800Mb under UCS4, compared to 600Mb under UCS2).

We're getting closer to being able to support this more easily upstream with
all the work for exposing abi build flags in shared library and executable
names.  But that won't help you much because it's Python 3. ;)

But I think to the extent that something like this could be backported into
Python 2, it's probably the general approach to take.

All that aside, understanding specifically what a "clean Python" means is a
great first step. ;)

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20101028/78f88219/attachment.pgp>

From tseaver at palladion.com  Thu Oct 28 23:44:13 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 28 Oct 2010 17:44:13 -0400
Subject: [Distutils] An observation on how system packagers and
 developers can be friends
In-Reply-To: <20101028165035.7a3faa3a@mission>
References: <AANLkTimZLhJ455FQQqubgTtaXcLX7asE8B7PydXuFuCG@mail.gmail.com>	<iacds4$nuq$1@dough.gmane.org>
	<20101028165035.7a3faa3a@mission>
Message-ID: <iacqrd$jo5$1@dough.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/28/2010 04:50 PM, Barry Warsaw wrote:
> On Oct 28, 2010, at 02:02 PM, Tres Seaver wrote:
> 
>> I like the idea in general, but worry that some conflicts may not be
>> resolvable.  For instance, I don't know what goal drives system
>> packagers to specify UCS4 over the default UCS2, but I won't ever be
>> happy using a Python built that way for long-running, memory-intensive
>> applications, where I have measured the overhead of UCS4 and found it
>> unacceptable (e.g., a server app whose steady-state process size is
>> 800Mb under UCS4, compared to 600Mb under UCS2).
> 
> We're getting closer to being able to support this more easily upstream with
> all the work for exposing abi build flags in shared library and executable
> names.  But that won't help you much because it's Python 3. ;)

Doesn't matter to me anyway, as I don't have the issue about trying to
share hierarchies between multiple slightly-differing builds. ;)

> But I think to the extent that something like this could be backported into
> Python 2, it's probably the general approach to take.
> 
> All that aside, understanding specifically what a "clean Python" means is a
> great first step. ;)

Heh, "untar + CMMI into a non-system prefix" works for me. ;)


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkzJ7q0ACgkQ+gerLs4ltQ4T3QCggKh9FSRDXdVqP3l6xmVthLaJ
YOwAn3CSP3oF0tL3c7oweK3Ecz8QDg8j
=ZNIn
-----END PGP SIGNATURE-----


From benji at benjiyork.com  Thu Oct 28 23:54:21 2010
From: benji at benjiyork.com (Benji York)
Date: Thu, 28 Oct 2010 17:54:21 -0400
Subject: [Distutils] An observation on how system packagers and
 developers can be friends
In-Reply-To: <iacqrd$jo5$1@dough.gmane.org>
References: <AANLkTimZLhJ455FQQqubgTtaXcLX7asE8B7PydXuFuCG@mail.gmail.com>
	<iacds4$nuq$1@dough.gmane.org> <20101028165035.7a3faa3a@mission>
	<iacqrd$jo5$1@dough.gmane.org>
Message-ID: <AANLkTin7yHA3unBR4cBsbzg3yuAQgSZ8dTTEe8FHB2EK@mail.gmail.com>

On Thu, Oct 28, 2010 at 5:44 PM, Tres Seaver <tseaver at palladion.com> wrote:
> Heh, "untar + CMMI into a non-system prefix" works for me. ;)

+1 with the small addition of "after making sure the dev dependencies Python
sniffs out to build modules for (zlib, crypto bits, etc.) are available."
-- 
Benji York

From tseaver at palladion.com  Fri Oct 29 00:08:52 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 28 Oct 2010 18:08:52 -0400
Subject: [Distutils] An observation on how system packagers and
 developers can be friends
In-Reply-To: <AANLkTin7yHA3unBR4cBsbzg3yuAQgSZ8dTTEe8FHB2EK@mail.gmail.com>
References: <AANLkTimZLhJ455FQQqubgTtaXcLX7asE8B7PydXuFuCG@mail.gmail.com>	<iacds4$nuq$1@dough.gmane.org>
	<20101028165035.7a3faa3a@mission>	<iacqrd$jo5$1@dough.gmane.org>
	<AANLkTin7yHA3unBR4cBsbzg3yuAQgSZ8dTTEe8FHB2EK@mail.gmail.com>
Message-ID: <iacs9k$qsn$1@dough.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/28/2010 05:54 PM, Benji York wrote:
> On Thu, Oct 28, 2010 at 5:44 PM, Tres Seaver <tseaver at palladion.com> wrote:
>> Heh, "untar + CMMI into a non-system prefix" works for me. ;)
> 
> +1 with the small addition of "after making sure the dev dependencies Python
> sniffs out to build modules for (zlib, crypto bits, etc.) are available."

Heh, agreed.  That bites me on about every third machine I set up for
the first time.  The Usual Suspects (TM) are whatever the local
packaging system calls the following (and their -dev or -devel packages,
if split out):

- - zlib
- - ncurses
- - bz2
- - readline
- - openssl


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkzJ9HQACgkQ+gerLs4ltQ4BawCfWxmQP+HB9atlynYrnMCTt2ka
oJQAn3rk7OskTRP6vzo7kI4KjoGMhYp3
=R7tK
-----END PGP SIGNATURE-----


From stephen.c.waterbury at nasa.gov  Fri Oct 29 00:10:32 2010
From: stephen.c.waterbury at nasa.gov (Stephen Waterbury)
Date: Thu, 28 Oct 2010 18:10:32 -0400
Subject: [Distutils] sh execution of setuptools egg not working
Message-ID: <4CC9F4D8.9010202@nasa.gov>

I'll probably realize what stupid thing I'm doing as soon
as I send this message, but oh well if that's what it takes ... ;)

I want to install setuptools on a centos 4.3 system, for which
the system python is 2.4 (ugh) so I've compiled and installed
python 2.6 into /usr/local.  When I try running the setuptools
egg as a shell script, the following happens:
---------------------------------------------------------------
$ type  python2.6
python2.6 is /usr/local/bin/python2.6
$ sudo sh setuptools-0.6c11-py2.6.egg
[sudo] password for waterbug:
setuptools-0.6c11-py2.6.egg: line 3: exec: python2.6: not found
---------------------------------------------------------------

... which seems odd.  Just as an experiment, I tried running
it without sudo -- it then finds python2.6 but conks out with
a different error:

---------------------------------------------------------------
$ sh setuptools-0.6c11-py2.6.egg prefix=~
Traceback (most recent call last):
   File "<string>", line 1, in <module>
ImportError: No module named setuptools.command.easy_install
---------------------------------------------------------------

Any suggestions appreciated!  Hope this isn't a faq that I
missed ...

Thanks,
Steve

From glyph at twistedmatrix.com  Fri Oct 29 00:49:50 2010
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Thu, 28 Oct 2010 18:49:50 -0400
Subject: [Distutils] An observation on how system packagers and
	developers can be friends
In-Reply-To: <iacs9k$qsn$1@dough.gmane.org>
References: <AANLkTimZLhJ455FQQqubgTtaXcLX7asE8B7PydXuFuCG@mail.gmail.com>	<iacds4$nuq$1@dough.gmane.org>
	<20101028165035.7a3faa3a@mission>	<iacqrd$jo5$1@dough.gmane.org>
	<AANLkTin7yHA3unBR4cBsbzg3yuAQgSZ8dTTEe8FHB2EK@mail.gmail.com>
	<iacs9k$qsn$1@dough.gmane.org>
Message-ID: <B4E9FBB5-9429-4A32-A5AE-E5720D01887E@twistedmatrix.com>

On Oct 28, 2010, at 6:08 PM, Tres Seaver wrote:

> Heh, agreed.  That bites me on about every third machine I set up for
> the first time.  The Usual Suspects (TM) are whatever the local
> packaging system calls the following (and their -dev or -devel packages,
> if split out):
> 
> - - zlib
> - - ncurses
> - - bz2
> - - readline
> - - openssl

On Debian, there's a handy shortcut: 'apt-get build-dep', which will install the build dependencies for any given source package.  So 'apt-get build-dep python' will get you all set to build Python.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20101028/d11ce870/attachment.html>

From pje at telecommunity.com  Fri Oct 29 05:24:33 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 28 Oct 2010 23:24:33 -0400
Subject: [Distutils] pkg_resources: Loading resources in a uniform
 fashion
In-Reply-To: <AANLkTimRTD1fsDc_EPtc+HC7Geh60ogJOhos++axtbgd@mail.gmail.c
 om>
References: <AANLkTimRTD1fsDc_EPtc+HC7Geh60ogJOhos++axtbgd@mail.gmail.com>
Message-ID: <20101029032446.E713A3A408C@sparrow.telecommunity.com>

At 06:18 PM 10/28/2010 +0200, Alan Franzoni wrote:
>Hello,
>
>lately I was thinking about writing a kind of factory method for
>parsing an url and returning a resource filename or stream, something
>like:
>
>fs_resource = load_resource_filename("file:///etc/software/config.conf")
>pkg_resource = load_resource_filename("pkg://package.something")
>
>I'd like the first one to just return the file system path
>"/etc/software/config.conf", while I'd like the second one one to do
>something roughly like
>
>from pkg_resources import resource_filename
>return resource_filename("package", "something")
>
>This way it would be pretty easy to get a consistent way to describe a
>resource from a stream.
>
>I've got some questions:
>
>1) is there anything around that already does something like that?

PEAK does, but it's even less frequently updated than 
setuptools.  ;-)  There may be other things out there as well, I don't know.

>2) would you think it to be a good addition for pkg_resources, or
>would it go beyond its scope?

Way beyond scope.  It makes more sense for it to be part of a url 
management package.

>3) can you see some obvious issues?
>4) how would you handle requirements string?


From pje at telecommunity.com  Fri Oct 29 05:28:48 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 28 Oct 2010 23:28:48 -0400
Subject: [Distutils] sh execution of setuptools egg not working
In-Reply-To: <4CC9F4D8.9010202@nasa.gov>
References: <4CC9F4D8.9010202@nasa.gov>
Message-ID: <20101029032846.D5D8B3A408C@sparrow.telecommunity.com>

At 06:10 PM 10/28/2010 -0400, Stephen Waterbury wrote:
>Any suggestions appreciated!  Hope this isn't a faq that I
>missed ...

I would suggest trying it with the local Python 2.4; if that works, 
then your 2.6 build is likely broken in some way.  (Another thing to 
check would be your PYTHONPATH.)



From mailing at franzoni.eu  Fri Oct 29 12:10:07 2010
From: mailing at franzoni.eu (Alan Franzoni)
Date: Fri, 29 Oct 2010 12:10:07 +0200
Subject: [Distutils] pkg_resources: Loading resources in a uniform
	fashion
In-Reply-To: <20101029032446.E713A3A408C@sparrow.telecommunity.com>
References: <AANLkTimRTD1fsDc_EPtc+HC7Geh60ogJOhos++axtbgd@mail.gmail.com>
	<20101029032446.E713A3A408C@sparrow.telecommunity.com>
Message-ID: <AANLkTikhuOvE+NWVm2O=8EVUDzgfuCZLmnNxWR1183OP@mail.gmail.com>

On Fri, Oct 29, 2010 at 5:24 AM, P.J. Eby <pje at telecommunity.com> wrote:
> PEAK does, but it's even less frequently updated than setuptools. ?;-)
> ?There may be other things out there as well, I don't know.

I peeked through PEAK, but it's quite a large collection of tools.
Would you point me to a more precise subpackage, please?

-- 
Alan Franzoni
--
contact me at public@[mysurname].eu

From tom.h.miller at gmail.com  Thu Oct 28 23:38:28 2010
From: tom.h.miller at gmail.com (Tom Miller)
Date: Thu, 28 Oct 2010 14:38:28 -0700
Subject: [Distutils] easy_install not choosing source package on pypi
Message-ID: <AANLkTi=X6PkHAm0jRyzmGUWpmNrtk8Sitgt5vNF1=9C7@mail.gmail.com>

Hello,

I'm trying to install GoogleCL using easy_install on a Mac. Here's what I
see:

$ easy_install googlecl
Searching for googlecl
Reading http://pypi.python.org/simple/googlecl/
Reading http://code.google.com/p/googlecl
Best match: googlecl 0.9.11-win32
Downloading http://googlecl.googlecode.com/files/googlecl-0.9.11-win32.zip
Processing googlecl-0.9.11-win32.zip
error: Couldn't find a setup script in
/var/folders/++/++3tbk++6+0++4RjPqRgNE+-SNw/-Tmp-/easy_install-3Y5tAg/googlecl-0.9.11-win32.zip

There's a source package on pypi that apparently gets skipped over:
http://pypi.python.org/pypi/googlecl/0.9.11

plus the Download URL field is filled in with a link directly to the correct
file. Instead, easy_install grabs the first download listed in the project
home's download links, which happens to be a .zip with an executable for
Windows users.

What's going on here? Is there a way to indicate the correct file, or does
the package description need to be updated?

Thanks,
 - Tom
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20101028/f4967fcf/attachment.html>

From jim at zope.com  Fri Oct 29 12:44:13 2010
From: jim at zope.com (Jim Fulton)
Date: Fri, 29 Oct 2010 06:44:13 -0400
Subject: [Distutils] Building Python (Re: An observation on how system
 packagers and developers can be friends)
Message-ID: <AANLkTikFtHWik5aZ3+VzYte3aagT+Ybw1ifRyTTEHYEU@mail.gmail.com>

On Thu, Oct 28, 2010 at 6:08 PM, Tres Seaver <tseaver at palladion.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 10/28/2010 05:54 PM, Benji York wrote:
>> On Thu, Oct 28, 2010 at 5:44 PM, Tres Seaver <tseaver at palladion.com> wrote:
>>> Heh, "untar + CMMI into a non-system prefix" works for me. ;)
>>
>> +1 with the small addition of "after making sure the dev dependencies Python
>> sniffs out to build modules for (zlib, crypto bits, etc.) are available."
>
> Heh, agreed. ?That bites me on about every third machine I set up for
> the first time.

An advantage of automating building of clean Python's with system
packaging tools
is that you can make sure the sane dependencies are present. My clean Python
RPM spec file lists the libraries I need before a build, so I can't
accidentally build a
crippled Python due to forgotten libraries. Of course, you also end up
with a binary package that is quicker to install on similar hosts than
building on
each host.

It would be nice if Python's standard build required more dependencies and
had options to turn them off is desired, rather than sniffing to tell
whether they
are there  and silently building badly.

Jim

-- 
Jim Fulton

From pje at telecommunity.com  Fri Oct 29 20:22:19 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 29 Oct 2010 14:22:19 -0400
Subject: [Distutils] pkg_resources: Loading resources in a uniform
 fashion
In-Reply-To: <AANLkTikhuOvE+NWVm2O=8EVUDzgfuCZLmnNxWR1183OP@mail.gmail.c
 om>
References: <AANLkTimRTD1fsDc_EPtc+HC7Geh60ogJOhos++axtbgd@mail.gmail.com>
	<20101029032446.E713A3A408C@sparrow.telecommunity.com>
	<AANLkTikhuOvE+NWVm2O=8EVUDzgfuCZLmnNxWR1183OP@mail.gmail.com>
Message-ID: <20101029182220.378E33A408C@sparrow.telecommunity.com>

At 12:10 PM 10/29/2010 +0200, Alan Franzoni wrote:
>On Fri, Oct 29, 2010 at 5:24 AM, P.J. Eby <pje at telecommunity.com> wrote:
> > PEAK does, but it's even less frequently updated than setuptools. ? ;-)
> > ? There may be other things out there as well, I don't know.
>
>I peeked through PEAK, but it's quite a large collection of tools.
>Would you point me to a more precise subpackage, please?

peak.naming provides lookup services for URLs and other sorts of 
names; peak.binding provides ways to link object attributes to 
automatically look up relevant objects.  There is very little 
documentation, as the  PEAK core packages were a toolkit I and a 
friend developed for use in our "enterprise" jobs.

In any case, this is now off-topic for distutils-sig; if you are 
still interested, you should follow up via the PEAK mailing list here:

    http://www.eby-sarna.com/mailman/listinfo/peak


>--
>Alan Franzoni
>--
>contact me at public@[mysurname].eu


From pje at telecommunity.com  Fri Oct 29 20:25:36 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 29 Oct 2010 14:25:36 -0400
Subject: [Distutils] easy_install not choosing source package on  pypi
In-Reply-To: <AANLkTi=X6PkHAm0jRyzmGUWpmNrtk8Sitgt5vNF1=9C7@mail.gmail.c
 om>
References: <AANLkTi=X6PkHAm0jRyzmGUWpmNrtk8Sitgt5vNF1=9C7@mail.gmail.com>
Message-ID: <20101029182536.55B023A408C@sparrow.telecommunity.com>

At 02:38 PM 10/28/2010 -0700, Tom Miller wrote:
>Hello,
>
>I'm trying to install GoogleCL using easy_install on a Mac. Here's what I see:
>
>$ easy_install googlecl
>Searching for googlecl
>Reading 
><http://pypi.python.org/simple/googlecl/>http://pypi.python.org/simple/googlecl/
>Reading <http://code.google.com/p/googlecl>http://code.google.com/p/googlecl
>Best match: googlecl 0.9.11-win32
>Downloading 
><http://googlecl.googlecode.com/files/googlecl-0.9.11-win32.zip>http://googlecl.googlecode.com/files/googlecl-0.9.11-win32.zip
>Processing googlecl-0.9.11-win32.zip
>error: Couldn't find a setup script in 
>/var/folders/++/++3tbk++6+0++4RjPqRgNE+-SNw/-Tmp-/easy_install-3Y5tAg/googlecl-0.9.11-win32.zip
>
>There's a source package on pypi that apparently gets skipped over: 
><http://pypi.python.org/pypi/googlecl/0.9.11>http://pypi.python.org/pypi/googlecl/0.9.11
>
>plus the Download URL field is filled in with a link directly to the 
>correct file. Instead, easy_install grabs the first download listed 
>in the project home's download links, which happens to be a .zip 
>with an executable for Windows users.
>
>What's going on here?

By the naming conventions setuptools uses, a file named 
"googlecl-0.9.11-win32.zip" is a source package for version 
"0.9.11-win32" of "googlecl", and thus is used in preference to the 
lower-versioned 
"<http://googlecl.googlecode.com/files/googlecl-0.9.11.tar.gz>googlecl-0.9.11.tar.gz".

If you named that file, say, "googlecl-win32-0.9.11.zip" instead, 
then this confusion would not occur.


>  Is there a way to indicate the correct file, or does the package 
> description need to be updated?
>
>Thanks,
>  - Tom
>_______________________________________________
>Distutils-SIG maillist  -  Distutils-SIG at python.org
>http://mail.python.org/mailman/listinfo/distutils-sig


From tom.h.miller at gmail.com  Fri Oct 29 20:58:33 2010
From: tom.h.miller at gmail.com (Tom Miller)
Date: Fri, 29 Oct 2010 11:58:33 -0700
Subject: [Distutils] easy_install not choosing source package on pypi
In-Reply-To: <20101029182536.55B023A408C@sparrow.telecommunity.com>
References: <AANLkTi=X6PkHAm0jRyzmGUWpmNrtk8Sitgt5vNF1=9C7@mail.gmail.com>
	<20101029182536.55B023A408C@sparrow.telecommunity.com>
Message-ID: <AANLkTi=65UZWtv1TXtVhxmO9GMyec=EbD2qOFs-HUpMF@mail.gmail.com>

On Fri, Oct 29, 2010 at 11:25 AM, P.J. Eby <pje at telecommunity.com> wrote:

> At 02:38 PM 10/28/2010 -0700, Tom Miller wrote:
>
>> Hello,
>>
>> I'm trying to install GoogleCL using easy_install on a Mac. Here's what I
>> see:
>>
>> $ easy_install googlecl
>> Searching for googlecl
>> Reading <http://pypi.python.org/simple/googlecl/>
>> http://pypi.python.org/simple/googlecl/
>> Reading <http://code.google.com/p/googlecl>
>> http://code.google.com/p/googlecl
>>
>> Best match: googlecl 0.9.11-win32
>> Downloading <
>> http://googlecl.googlecode.com/files/googlecl-0.9.11-win32.zip>
>> http://googlecl.googlecode.com/files/googlecl-0.9.11-win32.zip
>>
>> Processing googlecl-0.9.11-win32.zip
>> error: Couldn't find a setup script in
>> /var/folders/++/++3tbk++6+0++4RjPqRgNE+-SNw/-Tmp-/easy_install-3Y5tAg/googlecl-0.9.11-win32.zip
>>
>> There's a source package on pypi that apparently gets skipped over: <
>> http://pypi.python.org/pypi/googlecl/0.9.11>
>> http://pypi.python.org/pypi/googlecl/0.9.11
>>
>>
>> plus the Download URL field is filled in with a link directly to the
>> correct file. Instead, easy_install grabs the first download listed in the
>> project home's download links, which happens to be a .zip with an executable
>> for Windows users.
>>
>> What's going on here?
>>
>
> By the naming conventions setuptools uses, a file named
> "googlecl-0.9.11-win32.zip" is a source package for version "0.9.11-win32"
> of "googlecl", and thus is used in preference to the lower-versioned "<
> http://googlecl.googlecode.com/files/googlecl-0.9.11.tar.gz
> >googlecl-0.9.11.tar.gz".
>
> If you named that file, say, "googlecl-win32-0.9.11.zip" instead, then this
> confusion would not occur.
>
>
>   Is there a way to indicate the correct file, or does the package
>> description need to be updated?
>>
>> Thanks,
>>  - Tom
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> http://mail.python.org/mailman/listinfo/distutils-sig
>>
>
>
Great, thanks.  I guess there's no way to tell easy_install to look for a
specific extension?  Or to use the file hosted at pypi?

Thanks again,
 - Tom
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20101029/a61da6b3/attachment-0001.html>

From pje at telecommunity.com  Fri Oct 29 23:17:54 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 29 Oct 2010 17:17:54 -0400
Subject: [Distutils] easy_install not choosing source package on  pypi
In-Reply-To: <AANLkTi=65UZWtv1TXtVhxmO9GMyec=EbD2qOFs-HUpMF@mail.gmail.c
 om>
References: <AANLkTi=X6PkHAm0jRyzmGUWpmNrtk8Sitgt5vNF1=9C7@mail.gmail.com>
	<20101029182536.55B023A408C@sparrow.telecommunity.com>
	<AANLkTi=65UZWtv1TXtVhxmO9GMyec=EbD2qOFs-HUpMF@mail.gmail.com>
Message-ID: <20101029211812.DEA9A3A408C@sparrow.telecommunity.com>

At 11:58 AM 10/29/2010 -0700, Tom Miller wrote:
>Great, thanks.  I guess there's no way to tell easy_install to look 
>for a specific extension?  Or to use the file hosted at pypi?

If you're the person doing the installing, then of course you can 
manually specify a URL, or use "easy_install googlecl==0.9.11" to 
force the right file to be selected.

However, if you're the package author and want to fix it for 
everybody, you can either rename the non-applicable file, or remove 
the Home Page link from the PyPI pages that it's on.  (You'd have to 
remove it from every version you've ever published on PyPI, though, 
even if it's an inactive one.)


From m.van.rees at zestsoftware.nl  Sat Oct 30 02:43:38 2010
From: m.van.rees at zestsoftware.nl (Maurits van Rees)
Date: Sat, 30 Oct 2010 02:43:38 +0200
Subject: [Distutils] An observation on how system packagers and
 developers can be friends
In-Reply-To: <B4E9FBB5-9429-4A32-A5AE-E5720D01887E@twistedmatrix.com>
References: <AANLkTimZLhJ455FQQqubgTtaXcLX7asE8B7PydXuFuCG@mail.gmail.com>	<iacds4$nuq$1@dough.gmane.org>	<20101028165035.7a3faa3a@mission>	<iacqrd$jo5$1@dough.gmane.org>	<AANLkTin7yHA3unBR4cBsbzg3yuAQgSZ8dTTEe8FHB2EK@mail.gmail.com>	<iacs9k$qsn$1@dough.gmane.org>
	<B4E9FBB5-9429-4A32-A5AE-E5720D01887E@twistedmatrix.com>
Message-ID: <iafpnr$kjk$1@dough.gmane.org>

Op 29-10-10 00:49, Glyph Lefkowitz schreef:
> On Oct 28, 2010, at 6:08 PM, Tres Seaver wrote:
>
>> Heh, agreed. That bites me on about every third machine I set up for
>> the first time. The Usual Suspects (TM) are whatever the local
>> packaging system calls the following (and their -dev or -devel packages,
>> if split out):
>>
>> - - zlib
>> - - ncurses
>> - - bz2
>> - - readline
>> - - openssl
>
> On Debian, there's a handy shortcut: 'apt-get build-dep', which will
> install the build dependencies for any given source package. So 'apt-get
> build-dep python' will get you all set to build Python.

I did not know that one yet, thanks.  I tried that on Ubuntu Jaunty.  It 
returns rather a lot:

$ sudo apt-get build-dep --dry-run python
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
   debhelper debiandoc-sgml doc-base dvipdfmx ed gettext ghostscript 
gsfonts html2text intltool-debian lacheck latex-beamer latex-xcolor 
libcompress-raw-zlib-perl
   libcompress-zlib-perl libcroco3 libcups2 libcupsimage2 
libfreezethaw-perl libgs8 libhtml-parser-perl libhtml-tagset-perl 
libhtml-tree-perl libice6
   libio-compress-base-perl libio-compress-zlib-perl libkpathsea4 
libmail-sendmail-perl libmldbm-perl libpaper-utils libpaper1 libpoppler4 
libroman-perl libsgmls-perl
   libsm6 libsp1c2 libsys-hostname-long-perl libtext-format-perl 
libtiff4 libuuid-perl libxaw7 libxext6 libxmu6 libxt6 lmodern pgf 
po-debconf preview-latex-style prosper
   ps2eps psfontmgr python-docutils python-roman sgml-base sgml-data 
sgmlspl sp tex-common texinfo texlive-base texlive-base-bin 
texlive-base-bin-doc texlive-common
   texlive-doc-base texlive-extra-utils texlive-fonts-recommended 
texlive-fonts-recommended-doc texlive-generic-extra 
texlive-generic-recommended texlive-humanities
   texlive-humanities-doc texlive-latex-base texlive-latex-base-doc 
texlive-latex-extra texlive-latex-extra-doc texlive-latex-recommended 
texlive-latex-recommended-doc
   texlive-pictures texlive-pictures-doc texlive-pstricks 
texlive-pstricks-doc tipa xml-core
0 upgraded, 83 newly installed, 0 to remove and 3 not upgraded.


Somehow I doubt that I need TeX to build python.  (La)TeX is wonderful 
but there are limits to what it can do. :-)  I can imagine it might be 
needed for building a .deb package though, so it probably makes sense.



-- 
Maurits van Rees
Programmer, Zest Software


From marius at pov.lt  Sat Oct 30 15:59:19 2010
From: marius at pov.lt (Marius Gedminas)
Date: Sat, 30 Oct 2010 16:59:19 +0300
Subject: [Distutils] An observation on how system packagers and
 developers can be friends
In-Reply-To: <iafpnr$kjk$1@dough.gmane.org>
References: <AANLkTimZLhJ455FQQqubgTtaXcLX7asE8B7PydXuFuCG@mail.gmail.com>
	<iacds4$nuq$1@dough.gmane.org> <20101028165035.7a3faa3a@mission>
	<iacqrd$jo5$1@dough.gmane.org>
	<AANLkTin7yHA3unBR4cBsbzg3yuAQgSZ8dTTEe8FHB2EK@mail.gmail.com>
	<iacs9k$qsn$1@dough.gmane.org>
	<B4E9FBB5-9429-4A32-A5AE-E5720D01887E@twistedmatrix.com>
	<iafpnr$kjk$1@dough.gmane.org>
Message-ID: <20101030135919.GA31215@fridge.pov.lt>

On Sat, Oct 30, 2010 at 02:43:38AM +0200, Maurits van Rees wrote:
> Op 29-10-10 00:49, Glyph Lefkowitz schreef:
> >On Debian, there's a handy shortcut: 'apt-get build-dep', which will
> >install the build dependencies for any given source package. So 'apt-get
> >build-dep python' will get you all set to build Python.
> 
> I did not know that one yet, thanks.  I tried that on Ubuntu Jaunty.

Jaunty went end-of-life a month ago, time to upgrade?

> It returns rather a lot:
> 
> $ sudo apt-get build-dep --dry-run python
> Reading package lists... Done
> Building dependency tree
> Reading state information... Done
> The following NEW packages will be installed:
...
> 0 upgraded, 83 newly installed, 0 to remove and 3 not upgraded.
> 
> 
> Somehow I doubt that I need TeX to build python.

Why is it a build-dep, anyway?  Python's documentation sources are now
ReStructuredText, not LaTeX -- since 2.6, I think.  I don't believe
there are PDFs shipped in python2.x-doc packages.  Is it an obsolete
build dependency from earlier times?

Marius Gedminas
-- 
Favorite MAC error message: "Not enough memory to eject disk!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 190 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20101030/94c87393/attachment.pgp>