From chris at simplistix.co.uk  Tue Feb  2 18:58:56 2010
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 02 Feb 2010 17:58:56 +0000
Subject: [Distutils] Distutils roadmap of refactoring for 2.7
In-Reply-To: <1927.218.214.45.58.1264390716.squirrel@syd-srv02.ezyreg.com>
References: <94bdd2611001230400sed23a0excc626c85cf6e39bb@mail.gmail.com>	<20100123171947.EBCC63A4065@sparrow.telecommunity.com>	<94bdd2611001230929n47ef1945h6c2e4ae3513b614b@mail.gmail.com>	<20100123181126.AA2AE3A4065@sparrow.telecommunity.com>	<94bdd2611001231041i45ec71qae737789cab2bf40@mail.gmail.com>
	<1927.218.214.45.58.1264390716.squirrel@syd-srv02.ezyreg.com>
Message-ID: <4B6867E0.6060003@simplistix.co.uk>

David Lyon wrote:
>> On Sat, Jan 23, 2010 at 7:11 PM, P.J. Eby <pje at telecommunity.com> wrote:
>> Notice that other changes are coming up once
>> the series of PEP we worked on are accepted (which should be before
>> Pycon - at least for 345 and 386)
> 
> ok - so are you going to sort out the confusing (i386 == mac)
> and (x86 == pc) or (i586 == pc) or (i686 == pc) ?
> 
> in PEP-345? or leave it as downright confusing?

David,

If you're going to hijack a thread to bang on about your pet peeve, 
please at least attribute the mail you're quote to the right person...

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
             - http://www.simplistix.co.uk


From reinout at vanrees.org  Tue Feb  2 21:44:44 2010
From: reinout at vanrees.org (Reinout van Rees)
Date: Tue, 02 Feb 2010 21:44:44 +0100
Subject: [Distutils] nailed-down versions by having custom index
Message-ID: <hka2rr$18f$1@ger.gmane.org>

Hi all,

I'd like to do a bit of experimenting with a custom index with only the 
right packages in just the right versions. Some possible advantages I'm 
hoping to see for my usecase:

- Isolation from one of various servers being down (pypi, sourceforge).

- Guaranteed versions even when not using buildout. (Stuff needs "python 
setup.py install" in the system python, apparently, for arcgis/arcview 
on windows to be able to use it).

Personally, I'd just let everyone use buildout, but I think I won't be 
able to get that working with that "install it in site-packages" 
requirement.  (Ideas welcome, of course).


Question: how to maintain such a custom index?

iirc chrism has a script to build an apache-served folder structure from 
a bunch of eggs/tarballs, so just having the tarballs is enough. 
Probably just putting those tarballs in one location is enough anyway.

But how to get the right collection of tarballs in there? Is that a 
by-hand approach? Can you automate it?


Question 2: I thought you could only have one index.  So installing 
something extra from pypi seems out.  How do people handle that?


I'm totally used to buildout and its [versions] list and 
buildout.dumppickedversions and KGS lists, so I need to reset my 
thinking on this a little.  I need some input :-)



Reinout

-- 
Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org
Programmer/advisor at http://www.nelen-schuurmans.nl
"Military engineers build missiles. Civil engineers build targets"


From tseaver at palladion.com  Tue Feb  2 22:25:26 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 02 Feb 2010 16:25:26 -0500
Subject: [Distutils] nailed-down versions by having custom index
In-Reply-To: <hka2rr$18f$1@ger.gmane.org>
References: <hka2rr$18f$1@ger.gmane.org>
Message-ID: <hka586$bi9$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Reinout van Rees wrote:
> Hi all,
> 
> I'd like to do a bit of experimenting with a custom index with only the 
> right packages in just the right versions. Some possible advantages I'm 
> hoping to see for my usecase:
> 
> - Isolation from one of various servers being down (pypi, sourceforge).
> 
> - Guaranteed versions even when not using buildout. (Stuff needs "python 
> setup.py install" in the system python, apparently, for arcgis/arcview 
> on windows to be able to use it).
> 
> Personally, I'd just let everyone use buildout, but I think I won't be 
> able to get that working with that "install it in site-packages" 
> requirement.  (Ideas welcome, of course).
> 
> 
> Question: how to maintain such a custom index?
> 
> iirc chrism has a script to build an apache-served folder structure from 
> a bunch of eggs/tarballs, so just having the tarballs is enough. 
> Probably just putting those tarballs in one location is enough anyway.

You need the 'simple' bookkeeping files:  the attached script builds
them from that directory full of tarballs.  Run it via:

  $ cd /path/to/directory/full/of/tarballs
  $ /path/to/python /path/to/makeindex.py *.{gz,tgz,zip,egg}
  # move aside previous 'simple' first
  $ mv index simple

> But how to get the right collection of tarballs in there? Is that a 
> by-hand approach? Can you automate it?

You might look at compoze:  it is designed to allow fetching sources and
building an index based on the current workingset:

  http://svn.repoze.org/compoze/trunk/

> Question 2: I thought you could only have one index.  So installing 
> something extra from pypi seems out.  How do people handle that?

Fetch it from pypi and put it in your index.

> I'm totally used to buildout and its [versions] list and 
> buildout.dumppickedversions and KGS lists, so I need to reset my 
> thinking on this a little.  I need some input :-)


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktomEYACgkQ+gerLs4ltQ43DwCgkPpva6OsQ6xf/en8B2mSVIvv
RDgAnRG1pHEreyJmTk/5c6Z2BM4zYV7J
=tkVM
-----END PGP SIGNATURE-----
-------------- next part --------------
A non-text attachment was scrubbed...
Name: makeindex.py
Type: application/x-httpd-cgi
Size: 4389 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20100202/87196e4b/attachment.bin>

From carl at dirtcircle.com  Tue Feb  2 22:38:28 2010
From: carl at dirtcircle.com (Carl Meyer)
Date: Tue, 02 Feb 2010 16:38:28 -0500
Subject: [Distutils] nailed-down versions by having custom index
In-Reply-To: <hka2rr$18f$1@ger.gmane.org>
References: <hka2rr$18f$1@ger.gmane.org>
Message-ID: <4B689B54.6040904@dirtcircle.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Reinout,

Reinout van Rees wrote:
> I'd like to do a bit of experimenting with a custom index with only the
> right packages in just the right versions. Some possible advantages I'm
> hoping to see for my usecase:

I use chishop for this with success: http://github.com/ask/chishop

It emulates PyPI's API, so you can get the packages in there easily
enough by adding your chishop url in .pypirc and (using Python 2.6)
"python setup.py sdist upload -r my_server".

> Question 2: I thought you could only have one index.  So installing
> something extra from pypi seems out.  How do people handle that?

I use pip (and pip requirements files) for installation, and pip has
both the -i/--index-url option (for replacing PyPI entirely) and the
- --extra-index-url option (for layering another index in addition to
PyPI). I don't know how one would do that with other tools.

Carl

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iD8DBQFLaJtU1j/fhc23WEARAlDzAJ9oxm78s54v0BoeyFseSzNRBczACQCgtzrV
W7zFxpPR/7SayS0RmWzGsJU=
=oS95
-----END PGP SIGNATURE-----

From rich at noir.com  Tue Feb  2 22:39:24 2010
From: rich at noir.com (K. Richard Pixley)
Date: Tue, 02 Feb 2010 13:39:24 -0800
Subject: [Distutils] stdeb vs dependencies?
Message-ID: <4B689B8C.6080203@noir.com>

What is the expected work flow around package dependencies between 
python packages and debs in stdeb?

I was expecting it to declare any deb dependencies found, then translate 
any python dependencies into the names they would have if those pypi 
packages were translated using stdeb.  What I'm seeing with stdeb-0.5.1 
is that no dependencies are declared at all, but instead, I'm seeing 
unresolved macro references to ${python:Depends}.

--rich

From pje at telecommunity.com  Wed Feb  3 00:31:24 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 02 Feb 2010 18:31:24 -0500
Subject: [Distutils] nailed-down versions by having custom index
In-Reply-To: <hka2rr$18f$1@ger.gmane.org>
References: <hka2rr$18f$1@ger.gmane.org>
Message-ID: <20100202233120.A6C453A40A0@sparrow.telecommunity.com>

At 09:44 PM 2/2/2010 +0100, Reinout van Rees wrote:
>Question: how to maintain such a custom index?

Create an Apache-served folder structure such that for each project 
you want to serve, indexroot/project/ is a directory listing of 
downloadables.  Alternately, make a indexroot/project/index.html for 
that project, that that links directly to your download targets 
(which you need not directly serve).


>But how to get the right collection of tarballs in there? Is that a 
>by-hand approach? Can you automate it?

Presumably, if you have something particular in mind, you could write 
a script to do it.  Perhaps you could even use buildout to do it in 
some fashion.  ;-)


>Question 2: I thought you could only have one index.  So installing 
>something extra from pypi seems out.

You can always add a mod_rewrite rule in your index root, that checks 
for the nonexistence of a projectname, and then automatically 
redirects to the PyPI index for that page.  easy_install will not be 
confused by this - it will treat the PyPI page as if it had been 
served by your index. (But of course it will correctly interpret any 
relative links found as relative to the PyPI page, rather than as 
relative to the URL it tried to fetch from your index.  If you don't 
understand any of this, don't worry...  it Just Works.)


From david.lyon at pythontest.org  Wed Feb  3 00:25:45 2010
From: david.lyon at pythontest.org (David Lyon)
Date: Wed, 3 Feb 2010 10:25:45 +1100 (EST)
Subject: [Distutils] Distutils roadmap of refactoring for 2.7
In-Reply-To: <4B6867E0.6060003@simplistix.co.uk>
References: <94bdd2611001230400sed23a0excc626c85cf6e39bb@mail.gmail.com>
	<20100123171947.EBCC63A4065@sparrow.telecommunity.com>
	<94bdd2611001230929n47ef1945h6c2e4ae3513b614b@mail.gmail.com>
	<20100123181126.AA2AE3A4065@sparrow.telecommunity.com>
	<94bdd2611001231041i45ec71qae737789cab2bf40@mail.gmail.com>
	<1927.218.214.45.58.1264390716.squirrel@syd-srv02.ezyreg.com>
	<4B6867E0.6060003@simplistix.co.uk>
Message-ID: <1237.218.214.45.58.1265153145.squirrel@syd-srv02.ezyreg.com>

> Chris wrote:
>>> On Sat, Jan 23, 2010 at 7:11 PM, P.J. Eby <pje at telecommunity.com>
>>> wrote:
>>> Notice that other changes are coming up once
>>> the series of PEP we worked on are accepted (which should be before
>>> Pycon - at least for 345 and 386)
>>
>> ok - so are you going to sort out the confusing (i386 == mac)
>> and (x86 == pc) or (i586 == pc) or (i686 == pc) ?
>>
>> in PEP-345? or leave it as downright confusing?
>
> David,
>
> If you're going to hijack a thread to bang on about your pet peeve,
> please at least attribute the mail you're quote to the right person...

ok. I agree.

But it isn't a pet peave..

Introducing 'code' into the metadata will mean that PKG_INFO
files will become hated just as much by developers as what
.PTH have been.

Let me remind you what happened, .PTH files were supposed to
be data files (like metadata files).

Then a bright spark said, 'Let them contain code..'. And then
they did..

Then another bright spark said 'I shalt introduce code that
nobody else cannot understand or change'.

Then regular users found the whole thing such a nightmare
they came to loath the entire concept of a .PTH file.

So now, the exact same dreaded mechanism, is going to
be introduced into PKG_INFO files and I can't see why
it won't be also hijacked to make system administrators
lives a misery.

Giving a normal system administrator the job of knowing
that an 'i386' is actually a mac is going to drive them
bananas figuring out why the package isn't installing
properly on a pc.

Nine out of ten system administrators would guess that
'i386' is actually a pc. It should be but it isn't.

Quoting http://en.wikipedia.org/wiki/Python_%28programming_language%29:

'Python's philosophy rejects the Perl "there is more than one way to do
it" approach to language design in favor of "there should be one?and
preferably only one?obvious way to do it".[17]'

PEP-345 platform 'markers' throws this tradition away. Going instead for
inclusion of code in data files making them like .PTH files. Which will
when implemented, result in total confusion for sysadmins and audible
screaming.

Whatever...

David





From strawman at astraw.com  Wed Feb  3 01:19:49 2010
From: strawman at astraw.com (Andrew Straw)
Date: Tue, 02 Feb 2010 16:19:49 -0800
Subject: [Distutils] stdeb vs dependencies?
In-Reply-To: <4B689B8C.6080203@noir.com>
References: <4B689B8C.6080203@noir.com>
Message-ID: <4B68C125.9010005@astraw.com>

K. Richard Pixley wrote:
> What is the expected work flow around package dependencies between
> python packages and debs in stdeb?
>
> I was expecting it to declare any deb dependencies found, then
> translate any python dependencies into the names they would have if
> those pypi packages were translated using stdeb.  
It will attempt to translate any dependencies specified with
setuptools/distribute's install_requires kwarg to setup(). This can be
disabled with --ignore-install-requires. Otherwise, create an stdeb.cfg
file (or a [stdeb] section in setup.cfg) that has a "Depends:" line for
the Debian package names you want.
> What I'm seeing with stdeb-0.5.1 is that no dependencies are declared
> at all, but instead, I'm seeing unresolved macro references to
> ${python:Depends}.
What exactly is the error? Can you post the traceback or other error?
python-support should resolve this reference at .deb build time.

-Andrew

From rich at noir.com  Wed Feb  3 01:47:07 2010
From: rich at noir.com (K. Richard Pixley)
Date: Tue, 02 Feb 2010 16:47:07 -0800
Subject: [Distutils] stdeb vs dependencies?
In-Reply-To: <4B68C125.9010005@astraw.com>
References: <4B689B8C.6080203@noir.com> <4B68C125.9010005@astraw.com>
Message-ID: <4B68C78B.9030904@noir.com>

Andrew Straw wrote:
>> What I'm seeing with stdeb-0.5.1 is that no dependencies are declared
>> at all, but instead, I'm seeing unresolved macro references to
>> ${python:Depends}.
>>     
> What exactly is the error? Can you post the traceback or other error?
> python-support should resolve this reference at .deb build time.
I trimmed a bit of cruft from the middle.

--rich

. tg2.1b1-dev/bin/activate && cd ui && python setup.py bdist_deb
running bdist_deb
running sdist_dsc
running egg_info
writing requirements to ui.egg-info/requires.txt
writing ui.egg-info/PKG-INFO
writing top-level names to ui.egg-info/top_level.txt
writing dependency_links to ui.egg-info/dependency_links.txt
writing entry points to ui.egg-info/entry_points.txt
writing paster_plugins to ui.egg-info/paster_plugins.txt
reading manifest template 'MANIFEST.in'
writing manifest file 'ui.egg-info/SOURCES.txt'
I found Debian packages "python-turbogears2" which provides Python 
package "TurboGears2" which does not satisfy our version requirements: 
"TurboGears2>=2.1a1" -- ignoring.
I found no Debian package which provides the required Python package 
"TurboGears2" with version requirements "[('>=', '2.1a1')]".
I found a Debian package which provides the require Python package.  
Python package: "Babel", Debian package: "python-pybabel";  adding 
Depends specifications for the following version(s): "[('>=', '0.9.4')]"
I found a Debian package which provides the require Python package.  
Python package: "zope.sqlalchemy", Debian package: 
"python-zope.sqlalchemy";  adding Depends specifications for the 
following version(s): "[('>=', '0.4')]"
I found a Debian package which provides the require Python package.  
Python package: "repoze.tm2", Debian package: "python-repoze.tm2";  
adding Depends specifications for the following version(s): "[('>=', 
'1.0a4')]"
I found no Debian package which provides the required Python package 
"repoze.what-quickstart" with version requirements "[('>=', '1.0')]".
I found Debian packages "python-tgext.admin" which provides Python 
package "tgext.admin" which does not satisfy our version requirements: 
"tgext.admin>=0.3.3" -- ignoring.
I found no Debian package which provides the required Python package 
"tgext.admin" with version requirements "[('>=', '0.3.3')]".
CALLING dpkg-source -b ui-0.1dev-r254369 ui_0.1dev-r254369.orig.tar.gz 
(in dir deb_dist)
dpkg-source: info: using source format `1.0'
dpkg-source: info: building ui using existing ui_0.1dev-r254369.orig.tar.gz
dpkg-source: info: building ui in ui_0.1dev-r254369-1.diff.gz
dpkg-source: info: building ui in ui_0.1dev-r254369-1.dsc
dpkg-source: warning: extracting unsigned source package 
(ui_0.1dev-r254369-1.dsc)
dpkg-source: info: extracting ui in ui-0.1dev-r254369
dpkg-source: info: unpacking ui_0.1dev-r254369.orig.tar.gz
dpkg-source: info: applying ui_0.1dev-r254369-1.diff.gz
dpkg-buildpackage: set CFLAGS to default value: -g -O2
dpkg-buildpackage: set CPPFLAGS to default value:
dpkg-buildpackage: set LDFLAGS to default value: -Wl,-Bsymbolic-functions
dpkg-buildpackage: set FFLAGS to default value: -g -O2
dpkg-buildpackage: set CXXFLAGS to default value: -g -O2
dpkg-buildpackage: source package ui
dpkg-buildpackage: source version 0.1dev-r254369-1
dpkg-buildpackage: source changed by Rich Pixley <rich.pixley at palm.com>
dpkg-buildpackage: host architecture amd64
 fakeroot debian/rules clean
make[1]: Entering directory 
`/home/rich/projects/v4/ui/deb_dist/ui-0.1dev-r254369'
dh clean
   dh_testdir
   dh_auto_clean
running clean
'build/lib.linux-x86_64-2.6' does not exist -- can't clean it
'build/bdist.linux-x86_64' does not exist -- can't clean it
'build/scripts-2.6' does not exist -- can't clean it
   dh_clean
make[1]: Leaving directory 
`/home/rich/projects/v4/ui/deb_dist/ui-0.1dev-r254369'
 debian/rules build
make[1]: Entering directory 
`/home/rich/projects/v4/ui/deb_dist/ui-0.1dev-r254369'
dh build
   dh_testdir
   dh_auto_configure
   dh_auto_build
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.6
creating build/lib.linux-x86_64-2.6/ui
copying ui/__init__.py -> build/lib.linux-x86_64-2.6/ui

[...]

copying ui/config/deployment.ini_tmpl -> 
build/lib.linux-x86_64-2.6/ui/config
copying ui/controllers/controller.template -> 
build/lib.linux-x86_64-2.6/ui/controllers
   dh_auto_test
make[1]: Leaving directory 
`/home/rich/projects/v4/ui/deb_dist/ui-0.1dev-r254369'
 fakeroot debian/rules binary
make[1]: Entering directory 
`/home/rich/projects/v4/ui/deb_dist/ui-0.1dev-r254369'
dh binary
   dh_testroot
   dh_prep
   dh_installdirs
   dh_auto_install
running install
running build
running build_py
running egg_info
writing requirements to ui.egg-info/requires.txt
writing ui.egg-info/PKG-INFO
writing top-level names to ui.egg-info/top_level.txt
writing dependency_links to ui.egg-info/dependency_links.txt
writing entry points to ui.egg-info/entry_points.txt
writing paster_plugins to ui.egg-info/paster_plugins.txt
reading manifest file 'ui.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'ui.egg-info/SOURCES.txt'
running install_lib
creating 
/home/rich/projects/v4/ui/deb_dist/ui-0.1dev-r254369/debian/python-ui/home
creating 
/home/rich/projects/v4/ui/deb_dist/ui-0.1dev-r254369/debian/python-ui/home/rich
creating 
/home/rich/projects/v4/ui/deb_dist/ui-0.1dev-r254369/debian/python-ui/home/rich/projects
creating 
/home/rich/projects/v4/ui/deb_dist/ui-0.1dev-r254369/debian/python-ui/home/rich/projects/v4

[...]

   dh_makeshlibs
   dh_shlibdeps
   dh_installdeb
   dh_gencontrol
dpkg-gencontrol: warning: unknown substitution variable ${python:Provides}
dpkg-gencontrol: warning: unknown substitution variable ${python:Depends}
dpkg-gencontrol: warning: unknown substitution variable ${python:Provides}
dpkg-gencontrol: warning: unknown substitution variable ${python:Versions}
   dh_md5sums
   dh_builddeb
dpkg-deb: building package `python-ui' in 
`../python-ui_0.1dev-r254369-1_all.deb'.
make[1]: Leaving directory 
`/home/rich/projects/v4/ui/deb_dist/ui-0.1dev-r254369'
 dpkg-genchanges -b >../ui_0.1dev-r254369-1_amd64.changes
dpkg-genchanges: binary-only upload - not including any source code
dpkg-buildpackage: binary only upload (no source included)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20100202/28e86def/attachment.htm>

From strawman at astraw.com  Wed Feb  3 03:35:09 2010
From: strawman at astraw.com (Andrew Straw)
Date: Tue, 02 Feb 2010 18:35:09 -0800
Subject: [Distutils] stdeb vs dependencies?
In-Reply-To: <4B68C78B.9030904@noir.com>
References: <4B689B8C.6080203@noir.com> <4B68C125.9010005@astraw.com>
	<4B68C78B.9030904@noir.com>
Message-ID: <4B68E0DD.9040800@astraw.com>

K. Richard Pixley wrote:
> Andrew Straw wrote:
>>> What I'm seeing with stdeb-0.5.1 is that no dependencies are declared
>>> at all, but instead, I'm seeing unresolved macro references to
>>> ${python:Depends}.
>>>     
>> What exactly is the error? Can you post the traceback or other error?
>> python-support should resolve this reference at .deb build time.
> I trimmed a bit of cruft from the middle.
I take it you're talking about this?

> dpkg-gencontrol: warning: unknown substitution variable ${python:Provides}
> dpkg-gencontrol: warning: unknown substitution variable ${python:Depends}
> dpkg-gencontrol: warning: unknown substitution variable ${python:Provides}
> dpkg-gencontrol: warning: unknown substitution variable ${python:Versions}


What's the output of

dpkg -s python-support | grep Version

and

dpkg -s debhelper | grep Version


From r.ritz at biologie.hu-berlin.de  Wed Feb  3 08:55:12 2010
From: r.ritz at biologie.hu-berlin.de (Raphael Ritz)
Date: Wed, 03 Feb 2010 08:55:12 +0100
Subject: [Distutils] nailed-down versions by having custom index
In-Reply-To: <hka2rr$18f$1@ger.gmane.org>
References: <hka2rr$18f$1@ger.gmane.org>
Message-ID: <hkba4u$9om$1@ger.gmane.org>

Reinout van Rees wrote:
> Hi all,

Hi Reinout,


[..]

> 
> 
> Question: how to maintain such a custom index?


    http://pypi.python.org/pypi/haufe.eggserver


might be another option,

	Raphael


> 
> iirc chrism has a script to build an apache-served folder structure from 
> a bunch of eggs/tarballs, so just having the tarballs is enough. 
> Probably just putting those tarballs in one location is enough anyway.
> 
> But how to get the right collection of tarballs in there? Is that a 
> by-hand approach? Can you automate it?
> 
> 
> Question 2: I thought you could only have one index.  So installing 
> something extra from pypi seems out.  How do people handle that?
> 
> 
> I'm totally used to buildout and its [versions] list and 
> buildout.dumppickedversions and KGS lists, so I need to reset my 
> thinking on this a little.  I need some input :-)
> 
> 
> 
> Reinout
> 


From reinout at vanrees.org  Wed Feb  3 11:22:15 2010
From: reinout at vanrees.org (Reinout van Rees)
Date: Wed, 03 Feb 2010 11:22:15 +0100
Subject: [Distutils] nailed-down versions by having custom index
In-Reply-To: <hka586$bi9$1@ger.gmane.org>
References: <hka2rr$18f$1@ger.gmane.org> <hka586$bi9$1@ger.gmane.org>
Message-ID: <hkbiot$58q$1@ger.gmane.org>

On 02/02/2010 10:25 PM, Tres Seaver wrote:

>> But how to get the right collection of tarballs in there? Is that a
>> by-hand approach? Can you automate it?
>
> You might look at compoze:  it is designed to allow fetching sources and
> building an index based on the current workingset:
>
>    http://svn.repoze.org/compoze/trunk/

Thanks, I'll try that one out.  He he, a README in structuredtext (not 
reST) format if I'm not mistaken.  I still like that format :-)

>> Question 2: I thought you could only have one index.  So installing
>> something extra from pypi seems out.  How do people handle that?
>
> Fetch it from pypi and put it in your index.

My fear is that I end up with quite a lot of project-specific custom 
indexes if I want to keep it rebuildable. Multiply that with a quite a 
number of project releases (so: new indexes) and you're looking at a big 
bucket of bits.

How's that working out in practice for you?


If you have a base "product/framework/whatever"  (a bunch of libraries, 
partly your own and partly from pypi), I can see it is useful to have 
that in such a custom index. But multiple in-house websites will use 
that *and* add their own extra needed libraries to the mix.

=> So you need to add a new index for that, duplicating the 
"product/framework/whatever" release and adding your own extra eggs?



Reinout


-- 
Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org
Programmer/advisor at http://www.nelen-schuurmans.nl
"Military engineers build missiles. Civil engineers build targets"


From reinout at vanrees.org  Wed Feb  3 15:51:09 2010
From: reinout at vanrees.org (Reinout van Rees)
Date: Wed, 03 Feb 2010 15:51:09 +0100
Subject: [Distutils] nailed-down versions by having custom index
In-Reply-To: <20100202233120.A6C453A40A0@sparrow.telecommunity.com>
References: <hka2rr$18f$1@ger.gmane.org>
	<20100202233120.A6C453A40A0@sparrow.telecommunity.com>
Message-ID: <hkc2gs$694$1@ger.gmane.org>

On 02/03/2010 12:31 AM, P.J. Eby wrote:

>> Question 2: I thought you could only have one index. So installing
>> something extra from pypi seems out.
>
> You can always add a mod_rewrite rule in your index root, that checks
> for the nonexistence of a projectname, and then automatically redirects
> to the PyPI index for that page. easy_install will not be confused by
> this - it will treat the PyPI page as if it had been served by your
> index. (But of course it will correctly interpret any relative links
> found as relative to the PyPI page, rather than as relative to the URL
> it tried to fetch from your index. If you don't understand any of this,
> don't worry... it Just Works.)

Hey, great idea.

For another employer I did something similar which looked up the 
"missing" package in a local pypi cache/proxy thingy.  Redirecting... Hm :-)

I *think* I'll use this redirect trick in combination with a local 
sdistmaker-made simple index [1].  Then I'll just use my customary 
buildout techniques to get repeatable builds.

For the arcgis-wants-it-in-sitepackages-on-windows problem I'll have to 
fix up something else. I suspect a custom index (with Tres' compoze to 
manage it) fits that bill just fine.  Ok, I'm off setting stuff up.


Reinout

[1]: http://pypi.python.org/pypi/tha.sdistmaker

-- 
Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org
Programmer/advisor at http://www.nelen-schuurmans.nl
"Military engineers build missiles. Civil engineers build targets"


From rich at noir.com  Wed Feb  3 18:16:59 2010
From: rich at noir.com (K. Richard Pixley)
Date: Wed, 03 Feb 2010 09:16:59 -0800
Subject: [Distutils] stdeb vs dependencies?
In-Reply-To: <4B68E0DD.9040800@astraw.com>
References: <4B689B8C.6080203@noir.com> <4B68C125.9010005@astraw.com>
	<4B68C78B.9030904@noir.com> <4B68E0DD.9040800@astraw.com>
Message-ID: <4B69AF8B.7000102@noir.com>

Andrew Straw wrote:
>> dpkg-gencontrol: warning: unknown substitution variable ${python:Provides}
>> dpkg-gencontrol: warning: unknown substitution variable ${python:Depends}
>> dpkg-gencontrol: warning: unknown substitution variable ${python:Provides}
>> dpkg-gencontrol: warning: unknown substitution variable ${python:Versions}
>>     
> What's the output of
>
> dpkg -s python-support | grep Version
>
> and
>
> dpkg -s debhelper | grep Version
>   
rich at irish> dpkg -s debhelper | grep Version
Version: 7.4.11ubuntu1
rich at irish> dpkg -s python-support | grep Version
Version: 1.0.4ubuntu1
rich at irish> lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu lucid (development branch)
Release:        10.04
Codename:       lucid

Current within the minute.

--rich

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20100203/b3ada06a/attachment.htm>

From ronaldoussoren at mac.com  Wed Feb  3 23:28:13 2010
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Wed, 03 Feb 2010 23:28:13 +0100
Subject: [Distutils] Distutils roadmap of refactoring for 2.7
In-Reply-To: <1237.218.214.45.58.1265153145.squirrel@syd-srv02.ezyreg.com>
References: <94bdd2611001230400sed23a0excc626c85cf6e39bb@mail.gmail.com>
	<20100123171947.EBCC63A4065@sparrow.telecommunity.com>
	<94bdd2611001230929n47ef1945h6c2e4ae3513b614b@mail.gmail.com>
	<20100123181126.AA2AE3A4065@sparrow.telecommunity.com>
	<94bdd2611001231041i45ec71qae737789cab2bf40@mail.gmail.com>
	<1927.218.214.45.58.1264390716.squirrel@syd-srv02.ezyreg.com>
	<4B6867E0.6060003@simplistix.co.uk>
	<1237.218.214.45.58.1265153145.squirrel@syd-srv02.ezyreg.com>
Message-ID: <7AC0FCCA-AD35-42C5-9C88-6428E9CC8D4F@mac.com>


On 3 Feb, 2010, at 0:25, David Lyon wrote:

>> Chris wrote:
>>>> On Sat, Jan 23, 2010 at 7:11 PM, P.J. Eby <pje at telecommunity.com>
>>>> wrote:
>>>> Notice that other changes are coming up once
>>>> the series of PEP we worked on are accepted (which should be before
>>>> Pycon - at least for 345 and 386)
>>> 
>>> ok - so are you going to sort out the confusing (i386 == mac)
>>> and (x86 == pc) or (i586 == pc) or (i686 == pc) ?
>>> 
>>> in PEP-345? or leave it as downright confusing?
>> 
>> David,
>> 
>> If you're going to hijack a thread to bang on about your pet peeve,
>> please at least attribute the mail you're quote to the right person...
> 
> ok. I agree.
> 
> But it isn't a pet peave..
> 
> Introducing 'code' into the metadata will mean that PKG_INFO
> files will become hated just as much by developers as what
> .PTH have been.
> 
> Let me remind you what happened, .PTH files were supposed to
> be data files (like metadata files).
> 
> Then a bright spark said, 'Let them contain code..'. And then
> they did..
> 
> Then another bright spark said 'I shalt introduce code that
> nobody else cannot understand or change'.
> 
> Then regular users found the whole thing such a nightmare
> they came to loath the entire concept of a .PTH file.
> 
> So now, the exact same dreaded mechanism, is going to
> be introduced into PKG_INFO files and I can't see why
> it won't be also hijacked to make system administrators
> lives a misery.
> 
> Giving a normal system administrator the job of knowing
> that an 'i386' is actually a mac is going to drive them
> bananas figuring out why the package isn't installing
> properly on a pc.

Could you please explain what the problem is you're complaining about instead of assuming everyone knows the details by heart? The only reference I saw to 'i386' in PEP 345 is in the section about Environment markers and that only refers to the CPU architecture in a test for CPU architecture (platform.machine == 'i386') which seems reasonable to me.  

You can use "sys.platform" to detect various OS types.

BTW. I guess you mean "Windows" where you write "pc".

> 
> Nine out of ten system administrators would guess that
> 'i386' is actually a pc. It should be but it isn't.

'i386' is the commonly used abbreviation for an instruction set also known as 'x86' or 'ia32' and doesn't suggest any particular OS to me, all of Windows, Linux and MacOSX use or can use 'i386' capable CPUs (to just name 3 commonly used operating systems).

Ronald

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3567 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20100203/67938bac/attachment.bin>

From sridharr at activestate.com  Thu Feb  4 03:01:22 2010
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Wed, 3 Feb 2010 18:01:22 -0800
Subject: [Distutils] nailed-down versions by having custom index
In-Reply-To: <4B689B54.6040904@dirtcircle.com>
References: <hka2rr$18f$1@ger.gmane.org> <4B689B54.6040904@dirtcircle.com>
Message-ID: <6EA5C837-F119-4EFA-B641-473B8920A31C@activestate.com>


On 2010-02-02, at 1:38 PM, Carl Meyer wrote:

>> Question 2: I thought you could only have one index.  So installing
>> something extra from pypi seems out.  How do people handle that?
> 
> I use pip (and pip requirements files) for installation, and pip has
> both the -i/--index-url option (for replacing PyPI entirely) and the
> - --extra-index-url option (for layering another index in addition to
> PyPI). I don't know how one would do that with other tools.

You can pass multiple `-i' arguments to pip.

  $ pip -i http://internal/packages/ -i http://pypi.python.org/simple install internalpackage

Not sure what the `--extra-index-url' is for, then.

-srid

From carl at dirtcircle.com  Thu Feb  4 03:23:17 2010
From: carl at dirtcircle.com (Carl Meyer)
Date: Wed, 03 Feb 2010 21:23:17 -0500
Subject: [Distutils] nailed-down versions by having custom index
In-Reply-To: <6EA5C837-F119-4EFA-B641-473B8920A31C@activestate.com>
References: <hka2rr$18f$1@ger.gmane.org> <4B689B54.6040904@dirtcircle.com>
	<6EA5C837-F119-4EFA-B641-473B8920A31C@activestate.com>
Message-ID: <4B6A2F95.8080904@dirtcircle.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



Sridhar Ratnakumar wrote:
> You can pass multiple `-i' arguments to pip.
> 
>   $ pip -i http://internal/packages/ -i http://pypi.python.org/simple install internalpackage
> 
> Not sure what the `--extra-index-url' is for, then.

So you don't have to manually specify the PyPI index URL if you want to
add to it instead of replacing it.

Carl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iD8DBQFLai+V1j/fhc23WEARAjXqAJ9n9zx8AFeYaxBr7TxNlcYEuNZx/wCfZSbc
rJs1YLLEySIpFFybCli+9A8=
=V5K0
-----END PGP SIGNATURE-----

From riccardo.maria.bianchi at cern.ch  Thu Feb  4 17:53:16 2010
From: riccardo.maria.bianchi at cern.ch (Riccardo-Maria BIANCHI)
Date: Thu, 4 Feb 2010 17:53:16 +0100
Subject: [Distutils] modules in different folders but under same namespace
Message-ID: <4B6AFB7C.1080908@cern.ch>


Hi,

I have a package structured like this:


package/__init__.py
    src/ __init__.py
           mod1.py
    share/__init__.py
             mod2.py


Now I can import them as:
package.src.mod1
package.share.mod2

How can I use Distutils to be able to import both of them under the same 
"package" namespace as:

import package.mod1
import package.mod2

?

Thanks a lot in advance for your kind help!

Best regards,

   Ric.



From ziade.tarek at gmail.com  Thu Feb  4 18:05:45 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 4 Feb 2010 18:05:45 +0100
Subject: [Distutils] modules in different folders but under same
	namespace
In-Reply-To: <4B6AFB7C.1080908@cern.ch>
References: <4B6AFB7C.1080908@cern.ch>
Message-ID: <94bdd2611002040905h3e019bfaob892fbc7d3817ed7@mail.gmail.com>

On Thu, Feb 4, 2010 at 5:53 PM, Riccardo-Maria BIANCHI
<riccardo.maria.bianchi at cern.ch> wrote:
>
> Hi,

Hi,

>
> I have a package structured like this:
>
>
> package/__init__.py
> ? src/ __init__.py
> ? ? ? ? ?mod1.py
> ? share/__init__.py
> ? ? ? ? ? ?mod2.py
>
>
> Now I can import them as:
> package.src.mod1
> package.share.mod2
>
> How can I use Distutils to be able to import both of them under the same
> "package" namespace as:
>
> import package.mod1
> import package.mod2

unrelated to Distutils but,

What you can do is add these line in package/__init__.py :

  from package.src import mod1
  from package.share import mod2

Beware though, that this means the modules will be loaded
even if "import package" is called, which could have some unwanted
side effects depending on the code.

Regards
Tarek
-- 
Tarek Ziad? | http://ziade.org

From robert.kern at gmail.com  Thu Feb  4 18:06:07 2010
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 04 Feb 2010 11:06:07 -0600
Subject: [Distutils] modules in different folders but under same
	namespace
In-Reply-To: <4B6AFB7C.1080908@cern.ch>
References: <4B6AFB7C.1080908@cern.ch>
Message-ID: <hkeuq0$4vq$1@ger.gmane.org>

On 2010-02-04 10:53 AM, Riccardo-Maria BIANCHI wrote:
>
> Hi,
>
> I have a package structured like this:
>
>
> package/__init__.py
> src/ __init__.py
> mod1.py
> share/__init__.py
> mod2.py
>
>
> Now I can import them as:
> package.src.mod1
> package.share.mod2
>
> How can I use Distutils to be able to import both of them under the same
> "package" namespace as:
>
> import package.mod1
> import package.mod2
>
> ?
>
> Thanks a lot in advance for your kind help!

Remove the package/src/__init__.py and package/share/__init__.py . In your 
package/__init__.py, append the src/ and share/ directories to the __path__ 
list. E.g.


   import os
   import pkgutil

   for subdir in ['src', 'share']:
       __path__.append(os.path.join(os.path.dirname(__file__), subdir))

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From robert.kern at gmail.com  Thu Feb  4 18:11:56 2010
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 04 Feb 2010 11:11:56 -0600
Subject: [Distutils] modules in different folders but under same
	namespace
In-Reply-To: <hkeuq0$4vq$1@ger.gmane.org>
References: <4B6AFB7C.1080908@cern.ch> <hkeuq0$4vq$1@ger.gmane.org>
Message-ID: <hkev4r$4vq$3@ger.gmane.org>

On 2010-02-04 11:06 AM, Robert Kern wrote:
> On 2010-02-04 10:53 AM, Riccardo-Maria BIANCHI wrote:
>>
>> Hi,
>>
>> I have a package structured like this:
>>
>>
>> package/__init__.py
>> src/ __init__.py
>> mod1.py
>> share/__init__.py
>> mod2.py
>>
>>
>> Now I can import them as:
>> package.src.mod1
>> package.share.mod2
>>
>> How can I use Distutils to be able to import both of them under the same
>> "package" namespace as:
>>
>> import package.mod1
>> import package.mod2
>>
>> ?
>>
>> Thanks a lot in advance for your kind help!
>
> Remove the package/src/__init__.py and package/share/__init__.py . In
> your package/__init__.py, append the src/ and share/ directories to the
> __path__ list.

See this essay for more documentation on this feature:

   http://www.python.org/doc/essays/packages.html

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From robert.kern at gmail.com  Thu Feb  4 18:11:30 2010
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 04 Feb 2010 11:11:30 -0600
Subject: [Distutils] modules in different folders but under same
	namespace
In-Reply-To: <94bdd2611002040905h3e019bfaob892fbc7d3817ed7@mail.gmail.com>
References: <4B6AFB7C.1080908@cern.ch>
	<94bdd2611002040905h3e019bfaob892fbc7d3817ed7@mail.gmail.com>
Message-ID: <hkev41$4vq$2@ger.gmane.org>

On 2010-02-04 11:05 AM, Tarek Ziad? wrote:
> On Thu, Feb 4, 2010 at 5:53 PM, Riccardo-Maria BIANCHI
> <riccardo.maria.bianchi at cern.ch>  wrote:
>>
>> Hi,
>
> Hi,
>
>>
>> I have a package structured like this:
>>
>>
>> package/__init__.py
>>    src/ __init__.py
>>           mod1.py
>>    share/__init__.py
>>             mod2.py
>>
>>
>> Now I can import them as:
>> package.src.mod1
>> package.share.mod2
>>
>> How can I use Distutils to be able to import both of them under the same
>> "package" namespace as:
>>
>> import package.mod1
>> import package.mod2
>
> unrelated to Distutils but,
>
> What you can do is add these line in package/__init__.py :
>
>    from package.src import mod1
>    from package.share import mod2
>
> Beware though, that this means the modules will be loaded
> even if "import package" is called, which could have some unwanted
> side effects depending on the code.

This does not work. You cannot "import package.mod1" under this scenario.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From riccardo.maria.bianchi at cern.ch  Thu Feb  4 18:23:41 2010
From: riccardo.maria.bianchi at cern.ch (Riccardo-Maria BIANCHI)
Date: Thu, 4 Feb 2010 18:23:41 +0100
Subject: [Distutils] modules in different folders but under
	same	namespace
In-Reply-To: <hkev4r$4vq$3@ger.gmane.org>
References: <4B6AFB7C.1080908@cern.ch> <hkeuq0$4vq$1@ger.gmane.org>
	<hkev4r$4vq$3@ger.gmane.org>
Message-ID: <4B6B029D.4000403@cern.ch>


Thanks Robert and Tarek for your answer,

I implemented the __path__ solution, and it works perfectly. Thanks!

and sorry if I was off-topic: I thought that distutils had options for 
multi-folders setup, for "package_dir" and "packages" variables.

Best regards,

   Ric.



Robert Kern wrote:
> On 2010-02-04 11:06 AM, Robert Kern wrote:
>> On 2010-02-04 10:53 AM, Riccardo-Maria BIANCHI wrote:
>>>
>>> Hi,
>>>
>>> I have a package structured like this:
>>>
>>>
>>> package/__init__.py
>>> src/ __init__.py
>>> mod1.py
>>> share/__init__.py
>>> mod2.py
>>>
>>>
>>> Now I can import them as:
>>> package.src.mod1
>>> package.share.mod2
>>>
>>> How can I use Distutils to be able to import both of them under the 
>>> same
>>> "package" namespace as:
>>>
>>> import package.mod1
>>> import package.mod2
>>>
>>> ?
>>>
>>> Thanks a lot in advance for your kind help!
>>
>> Remove the package/src/__init__.py and package/share/__init__.py . In
>> your package/__init__.py, append the src/ and share/ directories to the
>> __path__ list.
>
> See this essay for more documentation on this feature:
>
>   http://www.python.org/doc/essays/packages.html
>


From ziade.tarek at gmail.com  Thu Feb  4 18:24:43 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 4 Feb 2010 18:24:43 +0100
Subject: [Distutils] modules in different folders but under same
	namespace
In-Reply-To: <hkev41$4vq$2@ger.gmane.org>
References: <4B6AFB7C.1080908@cern.ch>
	<94bdd2611002040905h3e019bfaob892fbc7d3817ed7@mail.gmail.com>
	<hkev41$4vq$2@ger.gmane.org>
Message-ID: <94bdd2611002040924t23ee43c8rd2397dcbc881e884@mail.gmail.com>

On Thu, Feb 4, 2010 at 6:11 PM, Robert Kern <robert.kern at gmail.com> wrote:
> On 2010-02-04 11:05 AM, Tarek Ziad? wrote:
>>
>> On Thu, Feb 4, 2010 at 5:53 PM, Riccardo-Maria BIANCHI
>> <riccardo.maria.bianchi at cern.ch> ?wrote:
>>>
>>> Hi,
>>
>> Hi,
>>
>>>
>>> I have a package structured like this:
>>>
>>>
>>> package/__init__.py
>>> ? src/ __init__.py
>>> ? ? ? ? ?mod1.py
>>> ? share/__init__.py
>>> ? ? ? ? ? ?mod2.py
>>>
>>>
>>> Now I can import them as:
>>> package.src.mod1
>>> package.share.mod2
>>>
>>> How can I use Distutils to be able to import both of them under the same
>>> "package" namespace as:
>>>
>>> import package.mod1
>>> import package.mod2
>>
>> unrelated to Distutils but,
>>
>> What you can do is add these line in package/__init__.py :
>>
>> ? from package.src import mod1
>> ? from package.share import mod2
>>
>> Beware though, that this means the modules will be loaded
>> even if "import package" is called, which could have some unwanted
>> side effects depending on the code.
>
> This does not work. You cannot "import package.mod1" under this scenario.

Right, only "from package import mod1" works.

Not sure why you want to remove the __init__.py files in sub and share
in your solution though.

-- 
Tarek Ziad? | http://ziade.org

From robert.kern at gmail.com  Thu Feb  4 18:28:06 2010
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 04 Feb 2010 11:28:06 -0600
Subject: [Distutils] modules in different folders but under same
	namespace
In-Reply-To: <94bdd2611002040924t23ee43c8rd2397dcbc881e884@mail.gmail.com>
References: <4B6AFB7C.1080908@cern.ch>	<94bdd2611002040905h3e019bfaob892fbc7d3817ed7@mail.gmail.com>	<hkev41$4vq$2@ger.gmane.org>
	<94bdd2611002040924t23ee43c8rd2397dcbc881e884@mail.gmail.com>
Message-ID: <hkf035$9ps$1@ger.gmane.org>

On 2010-02-04 11:24 AM, Tarek Ziad? wrote:

> Not sure why you want to remove the __init__.py files in sub and share
> in your solution though.

It prevents accidentally importing both package.src.mod1 and package.mod1 and 
helps readers of the code understand that only package.mod1 is correct.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From floris.bruynooghe at gmail.com  Thu Feb  4 18:37:45 2010
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Thu, 4 Feb 2010 17:37:45 +0000
Subject: [Distutils] modules in different folders but under same
 namespace
In-Reply-To: <4B6AFB7C.1080908@cern.ch>
References: <4B6AFB7C.1080908@cern.ch>
Message-ID: <20100204173745.GA16402@laurie.devork>

On Thu, Feb 04, 2010 at 05:53:16PM +0100, Riccardo-Maria BIANCHI wrote:
> Now I can import them as:
> package.src.mod1
> package.share.mod2
> 
> How can I use Distutils to be able to import both of them under the
> same "package" namespace as:
> 
> import package.mod1
> import package.mod2

In package/__init__.py:

import src.mod1 as mod1
import share.mod2 as mod2

And then you can use just one "import package" statement.  If you want
src and share to be separately installable then it gets more
complicated (you'd need namespace packages etc), but why complicated
when simple works?

Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From robert.kern at gmail.com  Thu Feb  4 18:54:02 2010
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 04 Feb 2010 11:54:02 -0600
Subject: [Distutils] modules in different folders but under same
	namespace
In-Reply-To: <20100204173745.GA16402@laurie.devork>
References: <4B6AFB7C.1080908@cern.ch> <20100204173745.GA16402@laurie.devork>
Message-ID: <hkf1jo$fqe$1@ger.gmane.org>

On 2010-02-04 11:37 AM, Floris Bruynooghe wrote:
> On Thu, Feb 04, 2010 at 05:53:16PM +0100, Riccardo-Maria BIANCHI wrote:
>> Now I can import them as:
>> package.src.mod1
>> package.share.mod2
>>
>> How can I use Distutils to be able to import both of them under the
>> same "package" namespace as:
>>
>> import package.mod1
>> import package.mod2
>
> In package/__init__.py:
>
> import src.mod1 as mod1
> import share.mod2 as mod2
>
> And then you can use just one "import package" statement.  If you want
> src and share to be separately installable then it gets more
> complicated (you'd need namespace packages etc), but why complicated
> when simple works?

Because simple doesn't work. You cannot do "import package.mod1" or "from 
package.mod1 import foo" with that scheme.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From pje at telecommunity.com  Thu Feb  4 18:56:55 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 04 Feb 2010 12:56:55 -0500
Subject: [Distutils] modules in different folders but under same
 namespace
In-Reply-To: <4B6AFB7C.1080908@cern.ch>
References: <4B6AFB7C.1080908@cern.ch>
Message-ID: <20100204175658.914483A40A0@sparrow.telecommunity.com>

At 05:53 PM 2/4/2010 +0100, Riccardo-Maria BIANCHI wrote:
>Hi,
>
>I have a package structured like this:
>
>
>package/__init__.py
>    src/ __init__.py
>           mod1.py
>    share/__init__.py
>             mod2.py
>
>Now I can import them as:
>package.src.mod1
>package.share.mod2
>
>How can I use Distutils to be able to import both of them under the 
>same "package" namespace as:
>
>import package.mod1
>import package.mod2

Move mod1.py and mod2.py under package/, and delete the 
subdirectories.  If you need backward compatibility, leave the old 
subdirectories and files in place, but have them import the new ones 
instead of containing any code.  Then delete them later.

Munging __path__ or using the package_dir options to setup() are both 
bad ideas in this case, unless there are other requirements you haven't stated.

If you use the  __path__ approach, you're going to make it more 
difficult for the distutils to figure out what to include in your 
source and binary distributions and installs, and if you use 
package_dir, you're going to end up with a different installation 
layout than your source layout.  Both will likely lead to hassles 
down the road, even if they appear to work in the short term.


From ziade.tarek at gmail.com  Fri Feb  5 01:55:22 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 5 Feb 2010 01:55:22 +0100
Subject: [Distutils] PEP 376 -- new round
Message-ID: <94bdd2611002041655p212aac84r98986dd9bd67cace@mail.gmail.com>

Hello,

As mentioned on python-dev, I would like to do a new round of work on
PEP 376 before Pycon,

We had a short off-list conversation with Guido on PEP 376, I'd like
to share here so we can discuss the point,
it's about the REQUESTED file:

Guido's question:
 I have a question about REQUESTED. It seems that the absence of this
 file means that the package is installed as a dependency of another
 package, so that it can be removed if that other package is removed .
 But since there is no record of the requesting package, it would seem
 problematic to find out when a dependent package can be removed, since
 there could possibly be multiple packages depending on it. Maybe it
 would be better to keep track of a list of packages that depend on a
 given package, adding a special "root" or "explicit" line for packages
 that were explicitly requested by the user?

My answer:
 Distutils doesn't provide a feature where dependencies gets installed
 or removed. It works at the package level only, and provides tools to
install or
 uninstall one package at a time.

 So we didn't want to provide a record of a full dependency graph
 through this file because we have considered that this work is up to
a full-featured
 install/uninstall tool that wants to install and remove dependencies.
 This tool can calculate this graph when working, as it does when
 installing dependencies, since everything is available in the metadata
 once we have PEP 345 -- with the "requires-dist" field.

 IOW:
 1/ an orphaned package is a package that doesn't have a REQUESTED file,
 *and* that is not used by another package anymore. So this can only be
 known by a uninstaller tool that deals with dependencies.

 2/ a requested package is a package that has the REQUESTED file

 Now for Distutils, the REQUESTED file is useful to mark the packages
 that were explicitely wanted. For example to develop against them.  vs
 things that where installed as a dependency.

 Of course the caveat is that a full-featured uninstall tool might be
 slow to compute the full dependency graph when trying to remove
 dependencies, because it will have to take a look at all other
 REQUESTED packages.

 Installers/uninstallers that are built on the top of Distutils could
 probably create their own dependency
 records inside each .egg-info placeholders, to speed up the work. But
 I am not sure it's such a big deal to look at all metadata files when
 performing an uninstallation. It's ok for this operation to be kind of
 slow I guess. And I am also not sure how this could work if the user
uses different kind
 of installers.

 This REQUESTED file is at the boundary of what we want to provide in
 Distutils. Going further seemed dangerous at this point because as we
 said last year at the summit, we want to make distutils a toolkit that
 provides some standards and a common ground for installers out there,
 rather than a full-featured package manager. IOW: we don't want to
 tell the tools out there how they should deal with the dependencies
 installation / uninstallation process, but give them tools to deal
 efficiently with one package at a time.

 I realize this is not clear enough in the PEP. Let me know if what
 I've explained makes sense to you. If so I'll work on adding details
 on these points. If not, it means we need more work on this.



So Guido wonders if adding this file is really necessary since higher-level
installers/uninstallers will have to build the graph anyways.

I think it's still useful, because it points the root packages that
can be removed
safely without breaking the system -- even if leaving orphaned packages behind.

Any opinion ?

Regards,
Tarek

-- 
Tarek Ziad? | http://ziade.org

From lac at openend.se  Fri Feb  5 02:09:53 2010
From: lac at openend.se (Laura Creighton)
Date: Fri, 05 Feb 2010 02:09:53 +0100
Subject: [Distutils] PEP 376 -- new round
In-Reply-To: Message from =?ISO-8859-1?Q?Tarek_Ziad=E9?=
	<ziade.tarek@gmail.com> of "Fri, 05 Feb 2010 01:55:22 +0100."
	<94bdd2611002041655p212aac84r98986dd9bd67cace@mail.gmail.com> 
References: <94bdd2611002041655p212aac84r98986dd9bd67cace@mail.gmail.com> 
Message-ID: <201002050109.o1519rwc006957@theraft.openend.se>


You need to think about dependencies more, I think.

Having worked on building such a thing, the hellish problem comes
when there are more than one way to support a package .. i.e. this
is my new testing plugin, it works with either py.test or nose.

Laura


From pje at telecommunity.com  Fri Feb  5 02:29:21 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 04 Feb 2010 20:29:21 -0500
Subject: [Distutils] PEP 376 -- new round
In-Reply-To: <94bdd2611002041655p212aac84r98986dd9bd67cace@mail.gmail.co
 m>
References: <94bdd2611002041655p212aac84r98986dd9bd67cace@mail.gmail.com>
Message-ID: <20100205012926.41F813A4768@sparrow.telecommunity.com>

At 01:55 AM 2/5/2010 +0100, Tarek Ziad? wrote:
>I think it's still useful, because it points the root packages that
>can be removed
>safely without breaking the system -- even if leaving orphaned 
>packages behind.
>
>Any opinion ?

I think it's a good idea to have a way to tell what packages were 
*not* installed to satisfy dependencies.

Merely plotting the dependency graph doesn't tell you this, because 
you could have a non-root orphan - i.e., something that was installed 
to fill a dependency, but the depender(s) have now vanished.

I guess what I'm saying is, a thing that is not needed by anything 
else could either be an orphan (due to other uninstalls) *or* a root 
(manually chosen for install), and there is no way to tell them apart 
just by following the graph.


From carl at dirtcircle.com  Fri Feb  5 06:23:40 2010
From: carl at dirtcircle.com (Carl Meyer)
Date: Fri, 05 Feb 2010 00:23:40 -0500
Subject: [Distutils] PEP 376 -- new round
In-Reply-To: <94bdd2611002041655p212aac84r98986dd9bd67cace@mail.gmail.com>
References: <94bdd2611002041655p212aac84r98986dd9bd67cace@mail.gmail.com>
Message-ID: <4B6BAB5C.4010105@dirtcircle.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



Tarek Ziad? wrote:
> So Guido wonders if adding this file is really necessary since higher-level
> installers/uninstallers will have to build the graph anyways.

I think you (Tarek) summarized the situation correctly. Higher-level
tools will need to calculate the dependency graph to do anything useful
with dependencies. But calculating the dep graph would STILL not allow
them to do any handling of orphaned dependencies unless they have access
to the bit of information provided by REQUESTED. And that bit of
information is only useful if it is standardized between all tools.

Carl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iD8DBQFLa6tc1j/fhc23WEARAh+aAKCrZ2gmrDX9fyHqV/rxAdv5h2jzuwCgsTtb
txjKkSOOcu5n1hmtvGtGxaY=
=ohx/
-----END PGP SIGNATURE-----

From ziade.tarek at gmail.com  Fri Feb  5 10:58:27 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 5 Feb 2010 10:58:27 +0100
Subject: [Distutils] PEP 376 -- new round
In-Reply-To: <201002050109.o1519rwc006957@theraft.openend.se>
References: <ziade.tarek@gmail.com>
	<94bdd2611002041655p212aac84r98986dd9bd67cace@mail.gmail.com>
	<201002050109.o1519rwc006957@theraft.openend.se>
Message-ID: <94bdd2611002050158g39a58901of649f2d0357bda14@mail.gmail.com>

On Fri, Feb 5, 2010 at 2:09 AM, Laura Creighton <lac at openend.se> wrote:
>
> You need to think about dependencies more, I think.
>
> Having worked on building such a thing, the hellish problem comes
> when there are more than one way to support a package .. i.e. this
> is my new testing plugin, it works with either py.test or nose.

Do you have a link on the relevant thing so I can take a look ? thanks

>
> Laura
>
>



-- 
Tarek Ziad? | http://ziade.org

From reinout at vanrees.org  Fri Feb  5 10:26:57 2010
From: reinout at vanrees.org (Reinout van Rees)
Date: Fri, 05 Feb 2010 10:26:57 +0100
Subject: [Distutils] PEP 376 -- new round
In-Reply-To: <20100205012926.41F813A4768@sparrow.telecommunity.com>
References: <94bdd2611002041655p212aac84r98986dd9bd67cace@mail.gmail.com>
	<94bdd2611002041655p212aac84r98986dd9bd67cace@mail.gmail.co m>
	<20100205012926.41F813A4768@sparrow.telecommunity.com>
Message-ID: <hkgo90$vsl$1@ger.gmane.org>

On 02/05/2010 02:29 AM, P.J. Eby wrote:
> At 01:55 AM 2/5/2010 +0100, Tarek Ziad? wrote:
>> I think it's still useful, because it points the root packages that
>> can be removed
>> safely without breaking the system -- even if leaving orphaned
>> packages behind.
>>
>> Any opinion ?
>
> I think it's a good idea to have a way to tell what packages were *not*
> installed to satisfy dependencies.

Debian/ubuntu has a mechanism to clean up packages that were once 
installed to satisfy dependencies but that aren't needed anymore.

The handiest course of action is probably looking up how they do it.
(/me doesn't know now it works internally, btw)


Reinout

-- 
Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org
Programmer/advisor at http://www.nelen-schuurmans.nl
"Military engineers build missiles. Civil engineers build targets"


From ziade.tarek at gmail.com  Fri Feb  5 12:28:43 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 5 Feb 2010 12:28:43 +0100
Subject: [Distutils] PEP 376 -- new round
In-Reply-To: <20100205012926.41F813A4768@sparrow.telecommunity.com>
References: <94bdd2611002041655p212aac84r98986dd9bd67cace@mail.gmail.com>
	<20100205012926.41F813A4768@sparrow.telecommunity.com>
Message-ID: <94bdd2611002050328s4c16120x16e7dbc35e407376@mail.gmail.com>

2010/2/5 P.J. Eby <pje at telecommunity.com>:
> At 01:55 AM 2/5/2010 +0100, Tarek Ziad? wrote:
>>
>> I think it's still useful, because it points the root packages that
>> can be removed
>> safely without breaking the system -- even if leaving orphaned packages
>> behind.
>>
>> Any opinion ?
>
> I think it's a good idea to have a way to tell what packages were *not*
> installed to satisfy dependencies.
>
> Merely plotting the dependency graph doesn't tell you this, because you
> could have a non-root orphan - i.e., something that was installed to fill a
> dependency, but the depender(s) have now vanished.
>
> I guess what I'm saying is, a thing that is not needed by anything else
> could either be an orphan (due to other uninstalls) *or* a root (manually
> chosen for install), and there is no way to tell them apart just by
> following the graph.

I am wondering *when* the depender(s) may vanish like that, leaving
behind them orphaned dependencies.

I guess this can happen when:

1/ a package (*looks like we can't help it calling a distribution a
package after all...*) is removed by another
   tool that it was installed with.

2/ something goes wrong during uninstallation

So I wonder : will 1/ really happen that often ? and shouldn't 2/ be
taken care by the high-level uninstaller ?

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org

From carl at dirtcircle.com  Fri Feb  5 14:30:26 2010
From: carl at dirtcircle.com (Carl Meyer)
Date: Fri, 05 Feb 2010 08:30:26 -0500
Subject: [Distutils] PEP 376 -- new round
In-Reply-To: <94bdd2611002050328s4c16120x16e7dbc35e407376@mail.gmail.com>
References: <94bdd2611002041655p212aac84r98986dd9bd67cace@mail.gmail.com>	<20100205012926.41F813A4768@sparrow.telecommunity.com>
	<94bdd2611002050328s4c16120x16e7dbc35e407376@mail.gmail.com>
Message-ID: <4B6C1D72.2040203@dirtcircle.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



Tarek Ziad? wrote:
> I am wondering *when* the depender(s) may vanish like that, leaving
> behind them orphaned dependencies.
> 
> I guess this can happen when:
> 
> 1/ a package (*looks like we can't help it calling a distribution a
> package after all...*) is removed by another
>    tool that it was installed with.
> 
> 2/ something goes wrong during uninstallation
> 
> So I wonder : will 1/ really happen that often ? and shouldn't 2/ be
> taken care by the high-level uninstaller ?


This is assuming a particular mode of uninstaller operation: immediate
removal of all no-longer-needed dependencies at the same moment when the
depender is uninstalled. It was clear during the original discussion of
REQUESTED several months back that this is not the only use case. Many
may prefer to have orphan-removal be a separate process run at a later
time (like deborphan).

In any case, regardless of how infrequently we _think_ two different
tools will be used to manage packages on a system, it seems
irresponsible to choose to prevent them from interoperating cleanly
around orphaned dependencies by choosing not to store the one bit of
metadata that would easily allow them to do so. What's the downside?

Carl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iD8DBQFLbB1y1j/fhc23WEARAtCeAKDHG1+jH1yRN0F48qWJgdx2Ezh2jACgk7TA
Gti73zRZ2lYH8f5iaJjzylY=
=l1ds
-----END PGP SIGNATURE-----

From carl at dirtcircle.com  Fri Feb  5 15:22:36 2010
From: carl at dirtcircle.com (Carl Meyer)
Date: Fri, 05 Feb 2010 09:22:36 -0500
Subject: [Distutils] PEP 376 -- new round
In-Reply-To: <hkgo90$vsl$1@ger.gmane.org>
References: <94bdd2611002041655p212aac84r98986dd9bd67cace@mail.gmail.com>	<94bdd2611002041655p212aac84r98986dd9bd67cace@mail.gmail.co
	m>	<20100205012926.41F813A4768@sparrow.telecommunity.com>
	<hkgo90$vsl$1@ger.gmane.org>
Message-ID: <4B6C29AC.6050106@dirtcircle.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



Reinout van Rees wrote:
> Debian/ubuntu has a mechanism to clean up packages that were once
> installed to satisfy dependencies but that aren't needed anymore.
>
> The handiest course of action is probably looking up how they do it.
> (/me doesn't know now it works internally, btw)

Apt has a file, /var/lib/apt/extended_states, which stores an
"Auto-Installed" flag with same semantics as the REQUESTED flag
(inverted). Though there are issues with the flag not being set
correctly for base-system packages, and not all tools take advantage of
the flag, the stored metadata is the same.

Carl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iD8DBQFLbCms1j/fhc23WEARApmwAJ41SEmj7suUyLEYLnZaCHv/4rsrxACgh8HW
UqGoxwfCGzXUardtOJL5xsA=
=tAzV
-----END PGP SIGNATURE-----

From rrr at ronadam.com  Fri Feb  5 14:37:20 2010
From: rrr at ronadam.com (Ron Adam)
Date: Fri, 05 Feb 2010 07:37:20 -0600
Subject: [Distutils] PEP 376 -- new round
In-Reply-To: <94bdd2611002050328s4c16120x16e7dbc35e407376@mail.gmail.com>
References: <94bdd2611002041655p212aac84r98986dd9bd67cace@mail.gmail.com>	<20100205012926.41F813A4768@sparrow.telecommunity.com>
	<94bdd2611002050328s4c16120x16e7dbc35e407376@mail.gmail.com>
Message-ID: <hkh6ug$vof$1@ger.gmane.org>



Tarek Ziad? wrote:
> 2010/2/5 P.J. Eby <pje at telecommunity.com>:
>> At 01:55 AM 2/5/2010 +0100, Tarek Ziad? wrote:
>>> I think it's still useful, because it points the root packages that
>>> can be removed
>>> safely without breaking the system -- even if leaving orphaned packages
>>> behind.
>>>
>>> Any opinion ?
>> I think it's a good idea to have a way to tell what packages were *not*
>> installed to satisfy dependencies.
>>
>> Merely plotting the dependency graph doesn't tell you this, because you
>> could have a non-root orphan - i.e., something that was installed to fill a
>> dependency, but the depender(s) have now vanished.
>>
>> I guess what I'm saying is, a thing that is not needed by anything else
>> could either be an orphan (due to other uninstalls) *or* a root (manually
>> chosen for install), and there is no way to tell them apart just by
>> following the graph.
> 
> I am wondering *when* the depender(s) may vanish like that, leaving
> behind them orphaned dependencies.
> 
> I guess this can happen when:
> 
> 1/ a package (*looks like we can't help it calling a distribution a
> package after all...*) is removed by another
>    tool that it was installed with.
> 
> 2/ something goes wrong during uninstallation
> 
> So I wonder : will 1/ really happen that often ? and shouldn't 2/ be
> taken care by the high-level uninstaller ?

If eggs were limited to installing their files in one location things would 
be much simpler.

Because files from eggs can be installed in multiple directories, and some 
of those are in arbitrary locations, I am wandering if it might be better 
to use a logging approach as files get installed.

An uninstall tool could then walk though the log to build a dependency 
table.  Or maybe break that into two tools, one that builds a dependency 
table, then another to uninstall files by reading the table.

Regards,
    Ron



















From ziade.tarek at gmail.com  Fri Feb  5 16:57:46 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 5 Feb 2010 16:57:46 +0100
Subject: [Distutils] PEP 376 -- new round
In-Reply-To: <4B6C29AC.6050106@dirtcircle.com>
References: <94bdd2611002041655p212aac84r98986dd9bd67cace@mail.gmail.com>
	<20100205012926.41F813A4768@sparrow.telecommunity.com>
	<hkgo90$vsl$1@ger.gmane.org> <4B6C29AC.6050106@dirtcircle.com>
Message-ID: <94bdd2611002050757s7f66208t9b65a5e64c4f0f4e@mail.gmail.com>

On Fri, Feb 5, 2010 at 3:22 PM, Carl Meyer <carl at dirtcircle.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
>
> Reinout van Rees wrote:
>> Debian/ubuntu has a mechanism to clean up packages that were once
>> installed to satisfy dependencies but that aren't needed anymore.
>>
>> The handiest course of action is probably looking up how they do it.
>> (/me doesn't know now it works internally, btw)
>
> Apt has a file, /var/lib/apt/extended_states, which stores an
> "Auto-Installed" flag with same semantics as the REQUESTED flag
> (inverted). Though there are issues with the flag not being set
> correctly for base-system packages, and not all tools take advantage of
> the flag, the stored metadata is the same.
>
> Carl

FYI  we agreed with Carl on IRC that he would work on a diff for the
REQUESTED section,
to build a small Rationale chapter. So the need becomes more obvious
from Distutils PoV

Tarek

From ziade.tarek at gmail.com  Fri Feb  5 17:13:39 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 5 Feb 2010 17:13:39 +0100
Subject: [Distutils] PEP 376 - RECORD file / Data files + pip feedback ?
Message-ID: <94bdd2611002050813r38b1e96eqa11415274f803041@mail.gmail.com>

Hi,

The part that still requires some work is how to handle files prefixes
in the RECORD file.

Last time we said that we could benefit of having a PREFIXES file.
Then Wolojda started to work on a much more complete solution to track
data file locations :
http://mail.python.org/pipermail/distutils-sig/2009-November/014424.html

But IIRC he's not able to work on this at this time. (CCing him in case..)

What I suggest is that we leave this new work for a future enhanced
version of PEP 376, and try to finish a version that we would want to
have in Python 2.7 (that's coming very soon), where RECORD contains
what we want to make it possible to remove installed files + provide
the query API.

As far as I am concerned, the RECORD thingie needs more live examples
on various platforms (that's what I am working on now),

But since Pip has now a uninstaller for a few months, the question is:
what do you Pip guys think about this RECORD file ? (cc'ing Ian as
well - I know the other involved in that are listening here :))

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Fri Feb  5 10:51:10 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 5 Feb 2010 10:51:10 +0100
Subject: [Distutils] PEP 386 - suggest_normalized version
Message-ID: <94bdd2611002050151r1a6b1940tdee09f82faf6e078@mail.gmail.com>

Hi,

Find attached the latest state of PEP 386 suggest_normalized() over PyPI.

Summary:

Total Packages  :  9058
Already Match   :  8055.0 (88.93%)
Have Suggestion :  792.0 (8.74%)
No Suggestion   :  211.0 (2.33%)

-> the attached file contains the details on the suggestion made, and
the versions that couldn't be treated

I am sure it still can be improved but I think it is already quite good.

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org
-------------- next part --------------
Loading saved pypi data...
Loading saved version info...
Results:
--------

Suggestions
-----------

0.1.0dev -> 0.1.0.dev0
0.8.1dev -> 0.8.1.dev0
1.0dev -> 1.0.dev0
0.99b -> 0.99b0
1.1-20051010 -> 1.1.post20051010
1.01 -> 1.1
1.0a -> 1.0a0
1.0a -> 1.0a0
0.1dev-r11 -> 0.1.dev11
0.01 -> 0.1
1.01 -> 1.1
0.1-a1 -> 0.1a1
0.1.2dev -> 0.1.2.dev0
0.1dev-r4 -> 0.1.dev4
0.4.1dev -> 0.4.1.dev0
3.0_beta_4 -> 3.0b4
0.6.1a -> 0.6.1a0
1.0beta1 -> 1.0b1
1.0.0beta2 -> 1.0.0b2
1.0.0beta1 -> 1.0.0b1
1.0beta4 -> 1.0b4
1.42a -> 1.42a0
0.1.0dev -> 0.1.0.dev0
0.2dev-r288 -> 0.2.dev288
0.1a -> 0.1a0
0.6.9a -> 0.6.9a0
0.1.3-r0 -> 0.1.3.post0
0.1dev -> 0.1.dev0
0.1.2dev -> 0.1.2.dev0
v 1.0 -> 1.0
1.0-beta1 -> 1.0b1
1.0-beta5 -> 1.0b5
1.0-beta1 -> 1.0b1
v1.0.0 -> 1.0.0
0.1dev -> 0.1.dev0
1.0.1-r11801 -> 1.0.1.post11801
1.0dev-r162 -> 1.0.dev162
0.1.0-dev1 -> 0.1.0.dev1
0.4.3b -> 0.4.3b0
0.02 -> 0.2
0.4a -> 0.4a0
0.2.1dev -> 0.2.1.dev0
0.1.1dev -> 0.1.1.dev0
0.02.04 -> 0.2.4
0.1dev -> 0.1.dev0
1.00-721 -> 1.0.post721
1.00-721 -> 1.0.post721
1.00-721 -> 1.0.post721
1.00-721 -> 1.0.post721
0.0.1dev-r41 -> 0.0.1.dev41
1.9b -> 1.9b0
2.0alpha2 -> 2.0a2
0.1dev -> 0.1.dev0
1.01 -> 1.1
0.4a0.dev-r1054 -> 0.4a0.dev1054
0.4a0.dev-r1056 -> 0.4a0.dev1056
0.4beta -> 0.4b0
0.5beta -> 0.5b0
0.1.1dev -> 0.1.1.dev0
0.0.0dev-r6768 -> 0.0.0.dev6768
0.1dev-r6418 -> 0.1.dev6418
09.12.07 -> 9.12.7
0.0dev -> 0.0.dev0
0.1a -> 0.1a0
0.0.1dev-r5450 -> 0.0.1.dev5450
0.97b -> 0.97b0
1.0.2-20050302 -> 1.0.2.post20050302
0.4r2 -> 0.4.post2
v01_00 -> 1.0
0.8-r16144 -> 0.8.post16144
0.8-r16144 -> 0.8.post16144
0.8-r16144 -> 0.8.post16144
0.8-r16321 -> 0.8.post16321
1.0-r16584 -> 1.0.post16584
0.8-r1761 -> 0.8.post1761
0.001 alpha -> 0.1a0
0.0.1a -> 0.0.1a0
2.0.0-final -> 2.0.0
0.4dev -> 0.4.dev0
0.8.3a -> 0.8.3a0
0.093 -> 0.93
0.4dev-r47 -> 0.4.dev47
1.002 -> 1.2
0.0.1a -> 0.0.1a0
1.0a -> 1.0a0
0.4.1dev -> 0.4.1.dev0
1.0.3-beta -> 1.0.3b0
1.3.2-2 -> 1.3.2.post2
1.1dev -> 1.1.dev0
0.1dev -> 0.1.dev0
0.0.3a -> 0.0.3a0
0.1-alpha-4 -> 0.1a4
0.3c -> 0.3c0
0.1.1-alpha-2 -> 0.1.1a2
0.1.5-alpha-1 -> 0.1.5a1
0.6.3-2 -> 0.6.3.post2
1.1.0rc -> 1.1.0c0
0.06 -> 0.6
0.2.11dev -> 0.2.11.dev0
0.11dev-r82630 -> 0.11.dev82630
0.1-20090814 -> 0.1.post20090814
2.2.0-final -> 2.2.0
0.1dev-r81015 -> 0.1.dev81015
0.3beta -> 0.3b0
1.01 -> 1.1
0.1beta -> 0.1b0
0.1dev-r59928 -> 0.1.dev59928
0.1a -> 0.1a0
0.1dev-r77946 -> 0.1.dev77946
1.0dev-r82790 -> 1.0.dev82790
1.0.5dev-r3574 -> 1.0.5.dev3574
3.0-final -> 3.0
1.0-final -> 1.0
1.3.2.1dev-r84292 -> 1.3.2.1.dev84292
0.1.0dev -> 0.1.0.dev0
1.0.0-rc1 -> 1.0.0c1
1.0-final -> 1.0
1.1.1dev-r97462 -> 1.1.1.dev97462
2.1.3-final -> 2.1.3
0.2dev-r58081 -> 0.2.dev58081
0.1beta -> 0.1b0
1.0dev-r75856 -> 1.0.dev75856
0.2dev-r57320 -> 0.2.dev57320
1.0-beta1 -> 1.0b1
1.0-beta1 -> 1.0b1
1.0dev-r110094 -> 1.0.dev110094
0.1.0-3 -> 0.1.0.post3
1.0.1-final -> 1.0.1
1.1a -> 1.1a0
0.2dev-r80219 -> 0.2.dev80219
9.06.01 -> 9.6.1
9.04 -> 9.4
9.06.02 -> 9.6.2
1.0.5beta -> 1.0.5b0
1.0.2beta -> 1.0.2b0
1.0.5beta -> 1.0.5b0
1.0.2beta -> 1.0.2b0
0.1a -> 0.1a0
1.03 -> 1.3
0.1b1dev -> 0.1b1.dev0
2.1.3-final -> 2.1.3
0.1.4dev -> 0.1.4.dev0
0.1a1.dev-r2011 -> 0.1a1.dev2011
0.2dev -> 0.2.dev0
0.1dev -> 0.1.dev0
2.00 -> 2.0
0.7a1.dev -> 0.7a1.dev0
0.3.r1340 -> 0.3.post1340
0.5.4a -> 0.5.4a0
0.6dev -> 0.6.dev0
0.2dev -> 0.2.dev0
0.1-r2147 -> 0.1.post2147
1.05 -> 1.5
0.1b -> 0.1b0
0.1.1beta -> 0.1.1b0
1.0b.54 -> 1.0b54
1.0dev -> 1.0.dev0
0.1dev -> 0.1.dev0
1.0dev -> 1.0.dev0
1.0dev -> 1.0.dev0
0.1dev -> 0.1.dev0
5.01 -> 5.1
1.04 -> 1.4
0.1alpha3 -> 0.1a3
0.5.4a -> 0.5.4a0
1.0a -> 1.0a0
0.1-alpha -> 0.1a0
0.0dev-r37 -> 0.0.dev37
0.01.001dev -> 0.1.1.dev0
0.8.90dev -> 0.8.90.dev0
0.2dev -> 0.2.dev0
0.1.0.dev-r330 -> 0.1.0.dev330
0.1_beta5 -> 0.1b5
0.5dev-r417 -> 0.5.dev417
1.03dev-r12 -> 1.3.dev12
1.1b -> 1.1b0
0.1dev -> 0.1.dev0
0.9.1b -> 0.9.1b0
0.1.1dev-r8 -> 0.1.1.dev8
0.1.0STABLE -> 0.1.0
0.1a1.dev-r610 -> 0.1a1.dev610
0.3.6dev -> 0.3.6.dev0
1.6a -> 1.6a0
0.1-alpha-1 -> 0.1a1
0.1.1-rc1 -> 0.1.1c1
1.0-0 -> 1.0.post0
0.1a -> 0.1a0
0.9-pre -> 0.9c0
0.1.0-dev1 -> 0.1.0.dev1
0.1.4a -> 0.1.4a0
0.3.1beta -> 0.3.1b0
0.2-r0 -> 0.2.post0
0.1.3a -> 0.1.3a0
1.0-alpha-1 -> 1.0a1
1.0.0.final -> 1.0.0
1.0.1-final -> 1.0.1
0.1c -> 0.1c0
0.1.0-pre2 -> 0.1.0c2
1.3-2 -> 1.3.post2
1.4-0 -> 1.4.post0
0.1.6-alpha-1 -> 0.1.6a1
0.1.6-pre2 -> 0.1.6c2
1.1.49-stable -> 1.1.49
0.8-alpha-1 -> 0.8a1
0.1.0-pre3 -> 0.1.0c3
0.1.9.beta -> 0.1.9b0
0.1-1 -> 0.1.post1
0.1-1 -> 0.1.post1
0.1.0-rc1 -> 0.1.0c1
0.1.1-final -> 0.1.1
0.03 -> 0.3
1.3-2 -> 1.3.post2
0.6.0beta -> 0.6.0b0
0.1-beta -> 0.1b0
0.1.5-pre1 -> 0.1.5c1
0.1dev-r23 -> 0.1.dev23
0.1final -> 0.1
1.5dev -> 1.5.dev0
0.1.dev -> 0.1.dev0
0.3.dev-r2016 -> 0.3.dev2016
0.0.1a -> 0.0.1a0
0.5dev-r16676 -> 0.5.dev16676
1.3pre1 -> 1.3c1
0.4a3-r45 -> 0.4a3.post45
0.4b1-r54 -> 0.4b1.post54
1.0dev-r8 -> 1.0.dev8
1.0dev-r10 -> 1.0.dev10
1.02dev-r7 -> 1.2.dev7
1.0.0-pre14 -> 1.0.0c14
0.1dev -> 0.1.dev0
0.02 -> 0.2
0.2.0 alpha -> 0.2.0a0
0.4dev-r2383 -> 0.4.dev2383
0.1-20041009 -> 0.1.post20041009
0.6.1b -> 0.6.1b0
0.1.3dev -> 0.1.3.dev0
0.3-20031123 -> 0.3.post20031123
1.0-20050212 -> 1.0.post20050212
1.2.6-20050316 -> 1.2.6.post20050316
0.1.dev-r149 -> 0.1.dev149
0.0dev-r12 -> 0.0.dev12
0.3dev-r2314 -> 0.3.dev2314
0.1.11dev -> 0.1.11.dev0
0.6a -> 0.6a0
3.2.1-final -> 3.2.1
1.1.0-final -> 1.1.0
1.0-rc2 -> 1.0c2
1.0.1-final -> 1.0.1
0.2a -> 0.2a0
1.0.0dev -> 1.0.0.dev0
0.01 -> 0.1
1.2-20041012 -> 1.2.post20041012
1.0dev-r77925 -> 1.0.dev77925
2.45.241_1926 -> 2.45.241.1926
1.0beta -> 1.0b0
1.0.0a -> 1.0.0a0
v0.5.4 -> 0.5.4
0.1.2pre1 -> 0.1.2c1
0.01a1.dev-r36 -> 0.1a1.dev36
1.0dev -> 1.0.dev0
0.1a7.dev-r2292 -> 0.1a7.dev2292
0.3.5p1 -> 0.3.5.post1
1.05 -> 1.5
0.01 -> 0.1
0.1_dev -> 0.1.dev0
0.02 -> 0.2
0.1r1 -> 0.1.post1
0.3dev -> 0.3.dev0
1.0.3.dev-20091021 -> 1.0.3.dev20091021
1.0.3.dev-20091027 -> 1.0.3.dev20091027
0.2dev -> 0.2.dev0
0.0.1dev-r2 -> 0.0.1.dev2
0.1.0dev -> 0.1.0.dev0
0.1.a -> 0.1a0
0.3.0-final -> 0.3.0
1.0.r39 -> 1.0.post39
0.12dev-r16 -> 0.12.dev16
0.2dev -> 0.2.dev0
1.03 -> 1.3
0.15.0.b3 -> 0.15.0b3
1.6a1dev-r4 -> 1.6a1.dev4
v0.1 -> 0.1
0.1.0a -> 0.1.0a0
0.1dev -> 0.1.dev0
0.1.1dev -> 0.1.1.dev0
0.3.9-stable -> 0.3.9
0.1dev -> 0.1.dev0
0.1dev -> 0.1.dev0
0.1b -> 0.1b0
0.1.0-2 -> 0.1.0.post2
1.01 -> 1.1
1.01p3 -> 1.1.post3
1.0dev-r629 -> 1.0.dev629
1.0dev-r627 -> 1.0.dev627
0.04 -> 0.4
0.01 -> 0.1
0.56p4 -> 0.56.post4
0.87p1 -> 0.87.post1
2.0p3 -> 2.0.post3
2.03 -> 2.3
0.57p4 -> 0.57.post4
0.3-7 -> 0.3.post7
0.02 -> 0.2
1.0-dev-r371 -> 1.0.dev371
1.0-dev-r371 -> 1.0.dev371
1.0-dev-r371 -> 1.0.dev371
0.9-dev-r371 -> 0.9.dev371
0.9-dev-r323 -> 0.9.dev323
1.0-dev-r371 -> 1.0.dev371
1.0-dev-r371 -> 1.0.dev371
1.0-dev-r371 -> 1.0.dev371
1.0-dev-r371 -> 1.0.dev371
1.0-dev-r371 -> 1.0.dev371
0.1dev -> 0.1.dev0
1.05 -> 1.5
0.3.dev-r412 -> 0.3.dev412
0.01 -> 0.1
0.1dev-r27686 -> 0.1.dev27686
0.1dev-r27752 -> 0.1.dev27752
1.02 -> 1.2
0.1a0.dev-r1518 -> 0.1a0.dev1518
0.1b -> 0.1b0
0.1.1dev-r1946 -> 0.1.1.dev1946
0.1.1dev-r2 -> 0.1.1.dev2
0.1dev-r1716 -> 0.1.dev1716
1.9.5-1 -> 1.9.5.post1
0.5.beta.2 -> 0.5b2
2.5.1.beta.1 -> 2.5.1b1
0.2.0RC5 -> 0.2.0c5
0.0.1alpha3.1 -> 0.0.1a3.1
3.1.7-r2 -> 3.1.7.post2
3.1.7-r1 -> 3.1.7.post1
1.0.0-RC1 -> 1.0.0c1
0.1.1dev-r6462 -> 0.1.1.dev6462
1.0.0-beta-2 -> 1.0.0b2
0.1pre3 -> 0.1c3
0.1pre2 -> 0.1c2
0.1pre1 -> 0.1c1
0.3.4.dev -> 0.3.4.dev0
0.06 -> 0.6
1.09 -> 1.9
0.1a0.dev-r1765 -> 0.1a0.dev1765
0.1.0dev -> 0.1.0.dev0
0.6.2005.0217 -> 0.6.2005.217
0.1-dev-r8 -> 0.1.dev8
1.0.0-beta1 -> 1.0.0b1
1.0pre11 -> 1.0c11
0.1.0b -> 0.1.0b0
0.02 -> 0.2
0.5.1dev-r1611 -> 0.5.1.dev1611
00.08.07 -> 0.8.7
4.7-beta-3 -> 4.7b3
1.00 -> 1.0
1.0-dev -> 1.0.dev0
0.3.10b -> 0.3.10b0
2.02 -> 2.2
0.3dev -> 0.3.dev0
0.1.0b -> 0.1.0b0
0.1.0-2 -> 0.1.0.post2
0.1.0-3 -> 0.1.0.post3
0.0.4.dev -> 0.0.4.dev0
0.2dev-r61 -> 0.2.dev61
0.2.beta -> 0.2b0
0.1.4.dev-r0 -> 0.1.4.dev0
0.0.0dev -> 0.0.0.dev0
0.1.0dev -> 0.1.0.dev0
0.1dev -> 0.1.dev0
0.1alpha1 -> 0.1a1
0.1.dev-r972 -> 0.1.dev972
0.1.0dev -> 0.1.0.dev0
0.3dev -> 0.3.dev0
0.9.0~rc10 -> 0.9.0c10
0.2.0.final.0 -> 0.2.0.0
0.1dev-r68295 -> 0.1.dev68295
0.9.1.r1489 -> 0.9.1.post1489
0.6.r1452 -> 0.6.post1452
3.0.r1266 -> 3.0.post1266
0.1.r1491 -> 0.1.post1491
1.0.r1505 -> 1.0.post1505
3.0.r1305 -> 3.0.post1305
0.12b -> 0.12b0
0.2a4.dev-r642 -> 0.2a4.dev642
0.1-beta-0 -> 0.1b0
0.1-r0 -> 0.1.post0
0.1dev -> 0.1.dev0
0.4.20dev -> 0.4.20.dev0
0.1b -> 0.1b0
0.1.2dev -> 0.1.2.dev0
1.0beta -> 1.0b0
0.01 -> 0.1
0.6.4-r3 -> 0.6.4.post3
0.9b -> 0.9b0
2.2.07 -> 2.2.7
1.2.1dev-r24558 -> 1.2.1.dev24558
1.3dev -> 1.3.dev0
1.0dev -> 1.0.dev0
1.1.dev-r2137 -> 1.1.dev2137
0.6.4dev -> 0.6.4.dev0
0.6dev -> 0.6.dev0
0.1.2a -> 0.1.2a0
0.0.1.dev-r737 -> 0.0.1.dev737
1.0.0-release -> 1.0.0
0.2dev-r3746 -> 0.2.dev3746
0.2dev -> 0.2.dev0
1.0dev -> 1.0.dev0
0.6dev -> 0.6.dev0
1.1dev -> 1.1.dev0
0.4pre -> 0.4c0
0.2a -> 0.2a0
0.1.dev-r4 -> 0.1.dev4
0.2.5.dev-r468 -> 0.2.5.dev468
0.01 -> 0.1
0.1a0.dev-r2593 -> 0.1a0.dev2593
0.1.2dev -> 0.1.2.dev0
0.2.2.dev-r854 -> 0.2.2.dev854
0.2.2.dev-r854 -> 0.2.2.dev854
0.2.2.dev-r854 -> 0.2.2.dev854
0.2.2.dev-r854 -> 0.2.2.dev854
0.2.2.dev-r854 -> 0.2.2.dev854
0.2.2.dev-r854 -> 0.2.2.dev854
0.2.2.dev-r854 -> 0.2.2.dev854
0.2.0.dev-r854 -> 0.2.0.dev854
0.2.0.dev-r854 -> 0.2.0.dev854
0.2.1.dev-r854 -> 0.2.1.dev854
0.2.2.dev-r854 -> 0.2.2.dev854
0.2.2.dev-r859 -> 0.2.2.dev859
0.1.6dev-r1 -> 0.1.6.dev1
1.0dev-r781 -> 1.0.dev781
0.1dev-r2148 -> 0.1.dev2148
0.1dev-r2200 -> 0.1.dev2200
0.1dev -> 0.1.dev0
5.0.0-3 -> 5.0.0.post3
5.0.0-3 -> 5.0.0.post3
0.1-20090228 -> 0.1.post20090228
1.1-beta -> 1.1b0
1.0dev-r12908 -> 1.0.dev12908
1.2-devel -> 1.2.dev0
0.2dev -> 0.2.dev0
0.2dev -> 0.2.dev0
0.5.0dev -> 0.5.0.dev0
0.1dev-r1550 -> 0.1.dev1550
3.2.1_2.10.1 -> 3.2.1.2.10.1
1.1 beta -> 1.1b0
0.1-dev-r79 -> 0.1.dev79
1.0alpha -> 1.0a0
v 1.0 -> 1.0
0.05 -> 0.5
0.5a1.dev -> 0.5a1.dev0
1.01 -> 1.1
1.2.10-63 -> 1.2.10.post63
0.02 -> 0.2
0.9.0b -> 0.9.0b0
1.07 -> 1.7
1.07 -> 1.7
1.0dev2 -> 1.0.dev2
1.0dev -> 1.0.dev0
0.6beta3 -> 0.6b3
0.0dev -> 0.0.dev0
1.0b2-20040224 -> 1.0b2.post20040224
0.3dev -> 0.3.dev0
0.07 -> 0.7
0.02 -> 0.2
2.0.0dev -> 2.0.0.dev0
0.2dev-r0 -> 0.2.dev0
1.0-beta2-1 -> 1.0b2.post1
1.0-beta4 -> 1.0b4
0.1dev -> 0.1.dev0
0.1.1dev-r60306 -> 0.1.1.dev60306
0.1dev-r60739 -> 0.1.dev60739
1.0dev -> 1.0.dev0
1.0dev-r584 -> 1.0.dev584
0.1dev-r57547 -> 0.1.dev57547
0.12dev-r60731 -> 0.12.dev60731
1.0dev -> 1.0.dev0
1.0b4dev -> 1.0b4.dev0
0.1dev-r57597 -> 0.1.dev57597
0.1dev-r60732 -> 0.1.dev60732
0.5a1dev -> 0.5a1.dev0
2.6a -> 2.6a0
0.2dev -> 0.2.dev0
0.2dev -> 0.2.dev0
0.1dev -> 0.1.dev0
1.04 -> 1.4
1.01 -> 1.1
0.0.0dev-r5822 -> 0.0.0.dev5822
2.0dev-r13909 -> 2.0.dev13909
1.1-r55 -> 1.1.post55
1.0dev-r69725 -> 1.0.dev69725
0.9.5stable -> 0.9.5
0.9.6stable -> 0.9.6
0.8.4dev -> 0.8.4.dev0
2.1.3-beta2 -> 2.1.3b2
1.3.0-b3 -> 1.3.0b3
1.1.1dev-r93293 -> 1.1.1.dev93293
1.1dev -> 1.1.dev0
1.2.1b1dev-r158 -> 1.2.1b1.dev158
0.1.beta -> 0.1b0
0.5.1dev-r3 -> 0.5.1.dev3
0.4dev -> 0.4.dev0
0.1dev -> 0.1.dev0
0.2dev -> 0.2.dev0
2.2.1dev-r83772 -> 2.2.1.dev83772
1.0dev -> 1.0.dev0
0.4.0beta -> 0.4.0b0
0.3.0beta -> 0.3.0b0
2.2.1dev-r76215 -> 2.2.1.dev76215
2.7.0-beta2 -> 2.7.0b2
1.0.7beta -> 1.0.7b0
4.0dev-r109999 -> 4.0.dev109999
0.1dev-r75024 -> 0.1.dev75024
0.2.0beta -> 0.2.0b0
0.1dev-r67896 -> 0.1.dev67896
1.3-beta -> 1.3b0
0.1.9a -> 0.1.9a0
1.60.00 -> 1.60.0
1.05 -> 1.5
0.1.1alpha -> 0.1.1a0
0.0.10a -> 0.0.10a0
1.12_1 -> 1.12.1
0.1dev-r11 -> 0.1.dev11
0.1b -> 0.1b0
0.1dev-r920 -> 0.1.dev920
0.5-4 -> 0.5.post4
v4.0 -> 4.0
1.0.2a -> 1.0.2a0
0.10beta2 -> 0.10b2
1.0a -> 1.0a0
0.4.5-6 -> 0.4.5.post6
1.0rc1dev -> 1.0c1.dev0
0.1a -> 0.1a0
0.1 beta -> 0.1b0
0.1.6dev -> 0.1.6.dev0
3.0.rc.8 -> 3.0c8
0.1dev-r42 -> 0.1.dev42
0.1dev-r44 -> 0.1.dev44
0.1dev-r33 -> 0.1.dev33
0.1dev-r35 -> 0.1.dev35
0.996a -> 0.996a0
v0.1 -> 0.1
0.6.4a -> 0.6.4a0
0.1dev -> 0.1.dev0
0.4b -> 0.4b0
0.5-rc2 -> 0.5c2
0.9p2 -> 0.9.post2
0.01 -> 0.1
1.0-RC2 -> 1.0c2
0.03 -> 0.3
0.02dev -> 0.2.dev0
0.0.5.0-Alpha -> 0.0.5.0a0
1.1.0-rc4 -> 1.1.0c4
0.1.3dev -> 0.1.3.dev0
0.1dev -> 0.1.dev0
1.0-alpha -> 1.0a0
0.0dev -> 0.0.dev0
0.1dev -> 0.1.dev0
0.1-r28 -> 0.1.post28
0.9.0beta -> 0.9.0b0
0.1-dev-r117 -> 0.1.dev117
2.1alpha -> 2.1a0
0.01 -> 0.1
0.3-r4247 -> 0.3.post4247
0.3dev -> 0.3.dev0
0.1.1dev -> 0.1.1.dev0
0.1.1dev -> 0.1.1.dev0
0.1dev -> 0.1.dev0
0.1dev -> 0.1.dev0
0.1dev -> 0.1.dev0
0.3.5b-5294 -> 0.3.5b5294
1.2.1-1 -> 1.2.1.post1
0.4-alpha4 -> 0.4a4
0.9.3 final -> 0.9.3
0.1c -> 0.1c0
0.3.4c -> 0.3.4c0
2.5-rc2 -> 2.5c2
0.03 -> 0.3
1.3p3 -> 1.3.post3
4.1.12a -> 4.1.12a0
0.2.8a -> 0.2.8a0
0.0.8a -> 0.0.8a0
0.3.0RC2 -> 0.3.0c2
1.0-beta2 -> 1.0b2
0.3.0dev_r88 -> 0.3.0.dev88
1.76.00-0 -> 1.76.0.post0
0.8.23a -> 0.8.23a0
1.1-dev-r371 -> 1.1.dev371
2.0-20031103 -> 2.0.post20031103
2.1b4-20050325 -> 2.1b4.post20050325
09.02.24 -> 9.2.24
0.0dev -> 0.0.dev0
0.2.dev-r16 -> 0.2.dev16
0.2-final -> 0.2
0.1dev -> 0.1.dev0
0.1a -> 0.1a0
0.1.18pre -> 0.1.18c0
0.6-dev -> 0.6.dev0
1.01 -> 1.1
6.4.1b -> 6.4.1b0
0.1alpha2 -> 0.1a2
1.1-0 -> 1.1.post0
2.04 -> 2.4
3.09 -> 3.9
2.04 -> 2.4
0.1-alpha -> 0.1a0
1.01 -> 1.1
2.0dev-r1802 -> 2.0.dev1802
0.03 -> 0.3
0.1a1.dev-20081205 -> 0.1a1.dev20081205
0.1.0pre -> 0.1.0c0
1.02 -> 1.2
0.5a -> 0.5a0
0.2dev -> 0.2.dev0
0.01 -> 0.1
1.1dev -> 1.1.dev0
0.2beta2 -> 0.2b2
1.0dev-r598 -> 1.0.dev598
1.0dev-r1109 -> 1.0.dev1109
1.0.0dev-r572 -> 1.0.0.dev572
1.0dev -> 1.0.dev0
0.2b2dev-r106 -> 0.2b2.dev106
0.2-r282 -> 0.2.post282
0.1r5 -> 0.1.post5
0.1r3 -> 0.1.post3
0.1.0a -> 0.1.0a0
0.1.0alpha -> 0.1.0a0
0.2.0alpha -> 0.2.0a0
0.0.1dev -> 0.0.1.dev0
1.05 -> 1.5
0.8b -> 0.8b0
0.1dev -> 0.1.dev0
0.5.0 Final -> 0.5.0
0.1a0dev-r21 -> 0.1a0.dev21
2.02 -> 2.2
0.1-rc2 -> 0.1c2
0.2.1 beta -> 0.2.1b0
1.09 -> 1.9
1.1beta3 -> 1.1b3
0.0.1dev -> 0.0.1.dev0
0.1-dev-20070930 -> 0.1.dev20070930
0.1dev-r16 -> 0.1.dev16
0.09 -> 0.9
0.6_0 -> 0.6.0
0.9.12beta -> 0.9.12b0
1.0dev-r12813 -> 1.0.dev12813
0.5.4b -> 0.5.4b0
3.1.0dev -> 3.1.0.dev0
3.1.0dev -> 3.1.0.dev0
3.1.0dev -> 3.1.0.dev0
0.2.dev-r803 -> 0.2.dev803
1.0alpha -> 1.0a0
0.1dev -> 0.1.dev0
0.1dev -> 0.1.dev0
0.1dev -> 0.1.dev0
0.7.3dev -> 0.7.3.dev0
1.0a -> 1.0a0
1.1.1-20040207 -> 1.1.1.post20040207
1.1-20030913 -> 1.1.post20030913
0.3.2c -> 0.3.2c0
0.02 -> 0.2
0.1.0dev -> 0.1.0.dev0
0.1.1-1 -> 0.1.1.post1
0.4dev-r11 -> 0.4.dev11
0.1.2a -> 0.1.2a0
1.0-alpha -> 1.0a0
0.4.0.dev-r23725 -> 0.4.0.dev23725
4.0.0a2dev-r227 -> 4.0.0a2.dev227
1.1-r87689 -> 1.1.post87689
0.1.37dev -> 0.1.37.dev0
0.1.1dev-r196 -> 0.1.1.dev196
0.5.2 Alpha -> 0.5.2a0
0.1-alpha -> 0.1a0
0.03 -> 0.3
1.00 -> 1.0
0.2dev -> 0.2.dev0
1.07.3 -> 1.7.3
0.2b -> 0.2b0
0.8.3.c -> 0.8.3c0
1.0.a3 -> 1.0a3
1.0.0.RELEASE -> 1.0.0
1.0.0.RELEASE -> 1.0.0
v1.3.0 -> 1.3.0
0.1.4.dev-r849 -> 0.1.4.dev849
0.5.0alpha -> 0.5.0a0
2.4c -> 2.4c0
0.6.0-beta2 -> 0.6.0b2
0.2dev-r35 -> 0.2.dev35
0.1.1dev -> 0.1.1.dev0
0.9.1a -> 0.9.1a0
0.2.0dev -> 0.2.0.dev0
0.0.1dev -> 0.0.1.dev0
1.4.8b -> 1.4.8b0
0.6a -> 0.6a0
0.80.0pre1 -> 0.80.0c1
1.1.0-r233 -> 1.1.0.post233
0.2.2-r210 -> 0.2.2.post210
1.0.0-r219 -> 1.0.0.post219
0.2.1-r230 -> 0.2.1.post230
1.0.0-r207 -> 1.0.0.post207
0.1pre2 -> 0.1c2
0.01 -> 0.1
0.1dev -> 0.1.dev0
0.1final -> 0.1
00.00.16 -> 0.0.16
0.06 -> 0.6
1.0Alpha -> 1.0a0
0.4p2 -> 0.4.post2
0.02 -> 0.2
0.2-dev-20070930 -> 0.2.dev20070930
0.2.3b -> 0.2.3b0
0.1dev -> 0.1.dev0
0.1dev -> 0.1.dev0
0.1-r26 -> 0.1.post26
0.1rc3-r47 -> 0.1c3.post47
1.0b -> 1.0b0
0.1a0.dev-r1772 -> 0.1a0.dev1772
1.04 -> 1.4
1.05 -> 1.5
0.1a0.dev-r2681 -> 0.1a0.dev2681
0.1.1-a0 -> 0.1.1a0
0.1.1dev -> 0.1.1.dev0
0.2a -> 0.2a0
1.1-20051211 -> 1.1.post20051211
0.1.0dev-r19469 -> 0.1.0.dev19469
0.1.0dev-r19468 -> 0.1.0.dev19468
0.2dev -> 0.2.dev0
0.2.1dev-r4679 -> 0.2.1.dev4679
0.1-r1 -> 0.1.post1
0.1.4dev-r3578 -> 0.1.4.dev3578
0.1.1dev-r3578 -> 0.1.1.dev3578
0.1.0dev-r3580 -> 0.1.0.dev3580
0.5.15dev-r3581 -> 0.5.15.dev3581
0.2dev-r4489 -> 0.2.dev4489
0.1-r1 -> 0.1.post1
0.11.0.3dev -> 0.11.0.3.dev0
0.8.0alpha -> 0.8.0a0
0.10-beta -> 0.10b0
1.05 -> 1.5
0.01a2 -> 0.1a2
0.1a -> 0.1a0
0.2b -> 0.2b0
0.1a0dev-r1571 -> 0.1a0.dev1571
0.1a1dev-r2279 -> 0.1a1.dev2279
1.0.2a -> 1.0.2a0
0.0.1dev-r39 -> 0.0.1.dev39
0.3dev -> 0.3.dev0
0.01 -> 0.1
0.1a0.dev-r720 -> 0.1a0.dev720
0.1dev -> 0.1.dev0
0.2dev -> 0.2.dev0
1.5dev-r792 -> 1.5.dev792
1.0beta3 -> 1.0b3
0.04.1 -> 0.4.1
0.1rc1-r84937 -> 0.1c1.post84937
0.1rc1-r85726 -> 0.1c1.post85726
0.1rc1-r96721 -> 0.1c1.post96721
0.03 -> 0.3
2.0.2-70 -> 2.0.2.post70
0.42b -> 0.42b0
0.2dev-r6 -> 0.2.dev6
0.8.1c -> 0.8.1c0
0.0dev -> 0.0.dev0
0.1.0dev -> 0.1.0.dev0
0.0.1.dev-r29 -> 0.0.1.dev29
1.1.8alpha1 -> 1.1.8a1
0.002 -> 0.2
0.1dev-r17 -> 0.1.dev17
0.1dev-r5497 -> 0.1.dev5497
0.1dev -> 0.1.dev0
0.1dev -> 0.1.dev0
0.1-1 -> 0.1.post1
0.2dev-r19612 -> 0.2.dev19612
0.3dev -> 0.3.dev0
1.4dev -> 1.4.dev0
0.1dev -> 0.1.dev0
1.5dev -> 1.5.dev0
0.9.0a -> 0.9.0a0
0.5.7a -> 0.5.7a0
00.00.12 -> 0.0.12
0.2dev -> 0.2.dev0
1.1.3beta -> 1.1.3b0
0.02 -> 0.2
0.06 -> 0.6
1.01dev -> 1.1.dev0
0.2.1dev-r100408 -> 0.2.1.dev100408
2.01.1 -> 2.1.1
1.1b -> 1.1b0
0.1b -> 0.1b0
0.01 -> 0.1
0.9-rc3 -> 0.9c3
0.10alpha2 -> 0.10a2
3.4dev -> 3.4.dev0
3.4dev-r73108 -> 3.4.dev73108
2.0-rc3 -> 2.0c3

No suggestions
--------------
0.1dev-20091118
0.3-SNAPSHOT-r1389
121
1.2dev-20091120
0.2dev-20091210
2009.05.01
2009.11.16.py3k.cpp
2010.01.21.ast2src
2
20070503
0.1adev-r1651
T-0.3.4 (BitTornado)
0.1dev-20080914
post-r5
1.11.4p
0.26unleak
2009.10.30
13r18
0.2.0-2009-12-28
1
2009.12.24.py3k.cpp
.99 version 3
0.1-svn
20100130.1
0
20090319
9
1.O
beta 013
0
.024
20070511
1.0-r3067-20090410
1.0-r3509-20090814
1.0-r3067-20090410
1.*
1
0.1.rec68f42a975e
1.6.0.multidb
0.2-grigoropoulos
r5
r6
1-0.3.3-alpha
.000001
0.0.37-beta-r1201
0.0.1-beta-r1066
46691.0
0.9.0_0.9.8l_1
rev452
0.1-alphadev
Version 3
520
Alpha
10
beta .001
20040603
.1
k303-q100 code name: virtual
Milestone1
working proof of concept
bzr14
1.0-rc-II.dev
1.20d
1 (first draft)
4
3
beta
1.2.1-a
r74
20081119
5
1.0d
3
0.7.2-9.00
0.1alpha.svn20080501
2008.11.20
2005
alpha-2
1.x
0.99.0dev-r4259-20091102
1.0.0b1_wrapper2
1.0dev-20080523
6
unreleased.unofficialdev
0.2r58dev-r58
1
2
0.1.alphadev
0.5.2-rf1
1.0.0-a002
0.9.8-svn
2008-03-29_r219
20090612
20080610
200
0.0.1a1.incremental
1.0.0.b090714164121
0308
2009.12.24.py3k.cpp
0.6X
0-6-2
20080712
20071223
c2.04b
0.2.5 (deprecated - see pprocess)
release candidate 3
pre 1.0
20100131
13
0.1.bzr496
4.9.9-git1
3
1.2-beta2beta
0.3.1bdev
0.9.1w3
0.1-step10
0.2-Zope2.9dev
2009.12.30.py3k.cpp
2010.01.01.README
0.98.B-beta
0.98.C-beta
1 to 3
1
20081023
2009.01.03
2007.01
2
0.1a.dev
6-1.0.4
0.5.3-3.8
0.2.1.beta-r100
0.08-alpha4b
gt-3.9.3beta-1.0.8
5
20090105
0.7~pre1
0
0.3.0patch1
2007.01a
0.4.0.Pineapple
20091028
alpha1
2005-08-01
1
0.2.1a.dev
stable 1.20
20080920
0.6d
0.hg18
0.5.beta-r97
1.0a.dev2
9
2010b
20061020
1.0 rv 2007-06-19
210
0.4.1b.dev
1.4r
9
0.1adev
2009.Q1b1
5
r105
20071211
0.3-rc
0.3dev-20090916
0.3dev-20090708
2008.03
1.2.1bugfix-r8
2009c
1.0dev-BZR-r45-panta-elasticworld.org-20091021145839-1oceeh3stpvyl04t
20070613
0.4dev-20081221
1.0dev-BZR-r10-panta-elasticworld.org-20091023132843-vitk6k7e5qlvhej5
0.9dev-BZR-r36-panta-elasticworld.org-20091216090517-3o494s4iearpm0rw
1.0dev-BZR-r90-panta-elasticworld.org-20091124105334-sfii6ytnur3a7aec
0.9dev-BZR-r8-panta-elasticworld.org-20091111104900-tj54rjt5f0biy1hc
0.9dev-BZR-r39-panta-elasticworld.org-20091111113416-ir4wlgfmp5lp6m69
1.0dev-BZR-r101-panta-elasticworld.org-20091118083244-inh0tj0d73fcdz14
1.0dev-BZR-r42-panta-elasticworld.org-20091021153851-6ijlut5dkxndxw1h
20080221
3
r9
0.1.0-test
2003.05.09
Pre-Alpha
r118
2009.10.31b
1.2-0.01
RC
0.3.2dev-20090715
1-alpha2
1.0-pre08
20080906
unknown
0.95w2
0.1a0dev-20080711
0.3dev-20090717
0.1a0dev-20090428
1
20091114
0.19 "Nose Furnace"
v0.alpha1
1
1.01b1+encoding_patch+removed_django_depends
0.6.2-test3
alpha
beta pre csound
3.0m2
3.0m4
3.6a-0.9

Summary:
--------
Total Packages  :  9058
Already Match   :  8055.0 (88.93%)
Have Suggestion :  792.0 (8.74%)
No Suggestion   :  211.0 (2.33%)

From strawman at astraw.com  Fri Feb  5 18:16:21 2010
From: strawman at astraw.com (Andrew Straw)
Date: Fri, 05 Feb 2010 09:16:21 -0800
Subject: [Distutils] stdeb vs dependencies?
In-Reply-To: <4B69AF8B.7000102@noir.com>
References: <4B689B8C.6080203@noir.com> <4B68C125.9010005@astraw.com>
	<4B68C78B.9030904@noir.com> <4B68E0DD.9040800@astraw.com>
	<4B69AF8B.7000102@noir.com>
Message-ID: <4B6C5265.2020901@astraw.com>

K. Richard Pixley wrote:
> Andrew Straw wrote:
>>> dpkg-gencontrol: warning: unknown substitution variable ${python:Provides}
>>> dpkg-gencontrol: warning: unknown substitution variable ${python:Depends}
>>> dpkg-gencontrol: warning: unknown substitution variable ${python:Provides}
>>> dpkg-gencontrol: warning: unknown substitution variable ${python:Versions}
>>>     
>> What's the output of
>>
>> dpkg -s python-support | grep Version
>>
>> and
>>
>> dpkg -s debhelper | grep Version
>>   
> rich at irish> dpkg -s debhelper | grep Version
> Version: 7.4.11ubuntu1
> rich at irish> dpkg -s python-support | grep Version
> Version: 1.0.4ubuntu1

With the same versions of debhelper and python-support, I'm not getting
those warnings with stdeb 0.5.1. Can you post the stdeb generated
debian/control file?

Thanks,
Andrew

From pje at telecommunity.com  Fri Feb  5 18:31:22 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 05 Feb 2010 12:31:22 -0500
Subject: [Distutils] PEP 376 -- new round
In-Reply-To: <hkh6ug$vof$1@ger.gmane.org>
References: <94bdd2611002041655p212aac84r98986dd9bd67cace@mail.gmail.com>
	<20100205012926.41F813A4768@sparrow.telecommunity.com>
	<94bdd2611002050328s4c16120x16e7dbc35e407376@mail.gmail.com>
	<hkh6ug$vof$1@ger.gmane.org>
Message-ID: <20100205173126.05BAA3A40A0@sparrow.telecommunity.com>

At 07:37 AM 2/5/2010 -0600, Ron Adam wrote:
>Because files from eggs can be installed in multiple directories, 
>and some of those are in arbitrary locations,

Scripts are the only things that get installed to other directories 
from .eggs.  Everything else is within the .egg file or directory.

It's non-.egg installs that have the problem you describe.


From carl at meyerloewen.net  Fri Feb  5 18:18:17 2010
From: carl at meyerloewen.net (Carl Meyer)
Date: Fri, 05 Feb 2010 12:18:17 -0500
Subject: [Distutils] PEP 376 - RECORD file / Data files + pip feedback ?
In-Reply-To: <94bdd2611002050813r38b1e96eqa11415274f803041@mail.gmail.com>
References: <94bdd2611002050813r38b1e96eqa11415274f803041@mail.gmail.com>
Message-ID: <4B6C52D9.70307@meyerloewen.net>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

hi Tarek,

Tarek Ziad? wrote:
> The part that still requires some work is how to handle files prefixes
> in the RECORD file.
> 
> Last time we said that we could benefit of having a PREFIXES file.
> Then Wolojda started to work on a much more complete solution to track
> data file locations :
> http://mail.python.org/pipermail/distutils-sig/2009-November/014424.html
[snip]
> But since Pip has now a uninstaller for a few months, the question is:
> what do you Pip guys think about this RECORD file ? (cc'ing Ian as
> well - I know the other involved in that are listening here :))

The prefix stuff in RECORD appears to be new since I last looked at PEP
376, and I don't like it.

If RECORD does not contain absolute paths, it becomes a lot trickier to
use it for uninstallation. In particular if prefix placeholders are
used, it's impossible to uninstall unless we can query for the actual
prefix paths used at install time. And we don't have that unless we
finish Wolodja's work.

If we do finish Wolodja's work, the PREFIXES file will necessarily
contain absolute paths anyway. In which case the installation metadata
is once again not relocatable.

So it seems to me that this idea of a relocatable installation is
inherently incompatible with providing metadata that records the actual
files installed on an actual system. Which is the metadata needed for
uninstall.

Are relocatable installations really all that valuable anyway? What's
the use-case? It seems to me that "move an existing installation" is
something a higher-level tool should worry about (rewriting RECORD in
that case).

I think we should remove all of the relative-path and prefix stuff from
RECORD and go back to requiring absolute paths for all files. This keeps
it simple and obvious, drastically reducing the surface area for bugs.
And that makes it orthogonal to Wolodja's work instead of dependent on
it, so we can go ahead with what we have for 2.7.

Carl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iD8DBQFLbFLZFRcxmeyPUXIRAp7gAJ9beyYaNa6fytDihPXJPxTQCkNovwCfdpyu
PA4a2Uzfa0y2g0ieHl835ko=
=SEDF
-----END PGP SIGNATURE-----

From ziade.tarek at gmail.com  Fri Feb  5 18:56:36 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 5 Feb 2010 18:56:36 +0100
Subject: [Distutils] PEP 376 - RECORD file / Data files + pip feedback ?
In-Reply-To: <4B6C52D9.70307@meyerloewen.net>
References: <94bdd2611002050813r38b1e96eqa11415274f803041@mail.gmail.com>
	<4B6C52D9.70307@meyerloewen.net>
Message-ID: <94bdd2611002050956r2f56f879kab0d5c0e278a586c@mail.gmail.com>

On Fri, Feb 5, 2010 at 6:18 PM, Carl Meyer <carl at meyerloewen.net> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> hi Tarek,
>
> Tarek Ziad? wrote:
>> The part that still requires some work is how to handle files prefixes
>> in the RECORD file.
>>
>> Last time we said that we could benefit of having a PREFIXES file.
>> Then Wolojda started to work on a much more complete solution to track
>> data file locations :
>> http://mail.python.org/pipermail/distutils-sig/2009-November/014424.html
> [snip]
>> But since Pip has now a uninstaller for a few months, the question is:
>> what do you Pip guys think about this RECORD file ? (cc'ing Ian as
>> well - I know the other involved in that are listening here :))
>
> The prefix stuff in RECORD appears to be new since I last looked at PEP
> 376, and I don't like it.
>
> If RECORD does not contain absolute paths, it becomes a lot trickier to
> use it for uninstallation. In particular if prefix placeholders are
> used, it's impossible to uninstall unless we can query for the actual
> prefix paths used at install time. And we don't have that unless we
> finish Wolodja's work.

This information is given by the location where the egg info resides.
(sys.prefix/sys.exec_prefix-like)

[..]
>
> Are relocatable installations really all that valuable anyway? What's
> the use-case? It seems to me that "move an existing installation" is
> something a higher-level tool should worry about (rewriting RECORD in
> that case).
>
> I think we should remove all of the relative-path and prefix stuff from
> RECORD and go back to requiring absolute paths for all files. This keeps
> it simple and obvious, drastically reducing the surface area for bugs.
> And that makes it orthogonal to Wolodja's work instead of dependent on
> it, so we can go ahead with what we have for 2.7.

Keeping absolute paths will require extra work in all bdist command because
those use a local "install --root" called to created an binary tree,
then copied on the system target
on installation.

IOW the RECORD file for them would be inaccurate at installation time.
Now for applications, being relocatable
would be a plus but, the real question I see is: how much pain each
one of these solutions will be to implement.

I am not sure we want to change all the binary commands in 2.7...

Tarek

-- 
Tarek Ziad? | http://ziade.org

From carl at dirtcircle.com  Fri Feb  5 19:39:07 2010
From: carl at dirtcircle.com (Carl Meyer)
Date: Fri, 05 Feb 2010 13:39:07 -0500
Subject: [Distutils] PEP 376 - RECORD file / Data files + pip feedback ?
In-Reply-To: <94bdd2611002050956r2f56f879kab0d5c0e278a586c@mail.gmail.com>
References: <94bdd2611002050813r38b1e96eqa11415274f803041@mail.gmail.com>	
	<4B6C52D9.70307@meyerloewen.net>
	<94bdd2611002050956r2f56f879kab0d5c0e278a586c@mail.gmail.com>
Message-ID: <4B6C65CB.7080306@dirtcircle.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Tarek,

I fully admit that I'm not cognizant of all the internal intricacies of
distutils, in particular the bdist_* stuff, so take my thoughts with a
grain of salt. I just have experience trying to make uninstall work in pip.

Tarek Ziad? wrote:
>> If RECORD does not contain absolute paths, it becomes a lot trickier to
>> use it for uninstallation. In particular if prefix placeholders are
>> used, it's impossible to uninstall unless we can query for the actual
>> prefix paths used at install time. And we don't have that unless we
>> finish Wolodja's work.
> 
> This information is given by the location where the egg info resides.
> (sys.prefix/sys.exec_prefix-like)

Presuming the API is now available in distutils for finding PREFIX and
EXEC_PREFIX, given the location of the installed .egg-info, that might
work. But how does this fit with "relocatable"? Doesn't this mean it's
really just "relocatable as long as you're relocating to one of a few
places where distutils expects an installed .egg-info to reside"?

But what about data files? What about all the myriad ways a setup.py
author can munge installation schemes and put things wherever they want?
I already went through the muck once trying to get uninstall working for
the crazy mix of installation schemes we have currently; it's very
unappealing to me to think that relying on complicated prefix-munging is
the best we'll be able to do for uninstall even under PEP 376. I have my
doubts that it will result in reliable uninstall.

> Keeping absolute paths will require extra work in all bdist command because
> those use a local "install --root" called to created an binary tree,
> then copied on the system target
> on installation.
> IOW the RECORD file for them would be inaccurate at installation time.

That is an issue. I can think of two possible solutions, both of which
seem preferable to me to making a mess out of RECORD:

1. Introduce a simple strip filter for adding paths to RECORD that, for
the bdist_* commands, strips the .../build prefix from the paths.

2. Just don't do RECORD for bdist_* at all: if you use bdist_* you're
declaring that you want the target package manager to manage
install/uninstall for you. I have no idea if this meets the needs of all
bdist_* use-cases, but naively it seems to me the whole reason for
bdist_* is because you've got some other package management system
already. Somebody in IRC just said bdist_wininst generates its own
internal record anyway...

> Now for applications, being relocatable
> would be a plus but, the real question I see is: how much pain each
> one of these solutions will be to implement.

I can see the attractiveness of a self-contained, relocatable
installation for applications.

What if RECORD allowed either absolute paths or paths relative to the
location of .egg-info, but nothing else? That would preserve the option
for application authors to make relocatable self-contained
installations, while keeping RECORD simple and useful for uninstall
without relying on prefix-finding magic.

Carl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iD8DBQFLbGXK1j/fhc23WEARAoFmAKCKilm19hI5YfD8Q7CihDLQDcwXtQCgwczN
DGDlgbfHcUKp/XV93ZnGUVg=
=n9Aw
-----END PGP SIGNATURE-----

From pje at telecommunity.com  Fri Feb  5 23:59:39 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 05 Feb 2010 17:59:39 -0500
Subject: [Distutils] PEP 376 - RECORD file / Data files + pip feedback ?
In-Reply-To: <4B6C65CB.7080306@dirtcircle.com>
References: <94bdd2611002050813r38b1e96eqa11415274f803041@mail.gmail.com>
	<4B6C52D9.70307@meyerloewen.net>
	<94bdd2611002050956r2f56f879kab0d5c0e278a586c@mail.gmail.com>
	<4B6C65CB.7080306@dirtcircle.com>
Message-ID: <20100205225940.DFD873A40A0@sparrow.telecommunity.com>

At 01:39 PM 2/5/2010 -0500, Carl Meyer wrote:
>What if RECORD allowed either absolute paths or paths relative to the
>location of .egg-info, but nothing else? That would preserve the option
>for application authors to make relocatable self-contained
>installations, while keeping RECORD simple and useful for uninstall
>without relying on prefix-finding magic.

I'm confused about something here.  There will still be a PREFIXES 
file, right?  So you shouldn't need prefix-finding magic except in 
the case where something has been relocated, right?


From carl at dirtcircle.com  Sat Feb  6 00:07:17 2010
From: carl at dirtcircle.com (Carl Meyer)
Date: Fri, 05 Feb 2010 18:07:17 -0500
Subject: [Distutils] PEP 376 - RECORD file / Data files + pip feedback ?
In-Reply-To: <20100205225940.DFD873A40A0@sparrow.telecommunity.com>
References: <94bdd2611002050813r38b1e96eqa11415274f803041@mail.gmail.com>
	<4B6C52D9.70307@meyerloewen.net>
	<94bdd2611002050956r2f56f879kab0d5c0e278a586c@mail.gmail.com>
	<4B6C65CB.7080306@dirtcircle.com>
	<20100205225940.DFD873A40A0@sparrow.telecommunity.com>
Message-ID: <4B6CA4A5.5050905@dirtcircle.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



P.J. Eby wrote:
> At 01:39 PM 2/5/2010 -0500, Carl Meyer wrote:
>> What if RECORD allowed either absolute paths or paths relative to the
>> location of .egg-info, but nothing else? That would preserve the option
>> for application authors to make relocatable self-contained
>> installations, while keeping RECORD simple and useful for uninstall
>> without relying on prefix-finding magic.
> 
> I'm confused about something here.  There will still be a PREFIXES file,
> right?  So you shouldn't need prefix-finding magic except in the case
> where something has been relocated, right?

Well, that's the third option; if someone has time to finish up the
PREFIXES idea and get it working in time for 2.7. Tarek had said if
Wolodja wasn't available to keep working on it, it might not get done
quickly enough for 2.7 and might need to be postponed. So I'd been
assuming we wouldn't have that... but if we do, that mitigates pretty
much all my concerns.

Carl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iD8DBQFLbKSl1j/fhc23WEARAjceAKCBSKif0+/OW4noIqh9s+4ygo2+FwCg2GAH
cG5GiiDmvMktMEjPIALMNfM=
=osJ5
-----END PGP SIGNATURE-----

From sridharr at activestate.com  Sat Feb  6 01:49:09 2010
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Fri, 5 Feb 2010 16:49:09 -0800
Subject: [Distutils] PEP 376 - RECORD file / Data files + pip feedback ?
In-Reply-To: <94bdd2611002050813r38b1e96eqa11415274f803041@mail.gmail.com>
References: <94bdd2611002050813r38b1e96eqa11415274f803041@mail.gmail.com>
Message-ID: <0A277B9D-10F9-4DAD-AF44-B824A82218D0@activestate.com>


On 2010-02-05, at 8:13 AM, Tarek Ziad? wrote:

> Hi,
> 
> The part that still requires some work is how to handle files prefixes
> in the RECORD file.
> 
> Last time we said that we could benefit of having a PREFIXES file.
> Then Wolojda started to work on a much more complete solution to track
> data file locations :
> http://mail.python.org/pipermail/distutils-sig/2009-November/014424.html
> 
> But IIRC he's not able to work on this at this time. (CCing him in case..)
> 
> What I suggest is that we leave this new work for a future enhanced
> version of PEP 376, and try to finish a version that we would want to
> have in Python 2.7 (that's coming very soon), where RECORD contains
> what we want to make it possible to remove installed files + provide
> the query API.
> 
> As far as I am concerned, the RECORD thingie needs more live examples
> on various platforms (that's what I am working on now),
> 
> But since Pip has now a uninstaller for a few months, the question is:
> what do you Pip guys think about this RECORD file ? (cc'ing Ian as
> well - I know the other involved in that are listening here :))


PyPM stores file paths relative to sys.prefix in the installation database. The binary package file (*.pypm/data.tar.gz) contains installed files relative to sys.prefix - implying that files-outside-sys.prefix are not supported by PyPM.

Do we have a real-world example for installing uninstallable data files outside sys.prefix?

-srid

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20100205/77d4b049/attachment.htm>

From pje at telecommunity.com  Sat Feb  6 01:57:28 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 05 Feb 2010 19:57:28 -0500
Subject: [Distutils] PEP 376 - RECORD file / Data files + pip feedback ?
In-Reply-To: <0A277B9D-10F9-4DAD-AF44-B824A82218D0@activestate.com>
References: <94bdd2611002050813r38b1e96eqa11415274f803041@mail.gmail.com>
	<0A277B9D-10F9-4DAD-AF44-B824A82218D0@activestate.com>
Message-ID: <20100206005729.306E83A40A0@sparrow.telecommunity.com>

At 04:49 PM 2/5/2010 -0800, Sridhar Ratnakumar wrote:
>Do we have a real-world example for installing uninstallable data 
>files outside sys.prefix?

Scripts aren't always in sys.prefix, and some packages install fixed 
data (e.g. web pages, message catalogs, etc.) to /usr/share or some such.

Also make sure you're not forgetting e.g. shared hosting scenarios 
without a virtualenv, unless you're planning to make virtualenvs an 
officially-supported thing.  In such cases, the paths involved won't 
necessarily fall into a single root.

Also don't forget sys.exec_prefix.  (Isn't packaging fun?)



From sridharr at activestate.com  Sat Feb  6 03:10:43 2010
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Fri, 5 Feb 2010 18:10:43 -0800
Subject: [Distutils] PEP 376 - RECORD file / Data files + pip feedback ?
In-Reply-To: <20100206005729.306E83A40A0@sparrow.telecommunity.com>
References: <94bdd2611002050813r38b1e96eqa11415274f803041@mail.gmail.com>
	<0A277B9D-10F9-4DAD-AF44-B824A82218D0@activestate.com>
	<20100206005729.306E83A40A0@sparrow.telecommunity.com>
Message-ID: <CEF2BDD8-F6EC-4914-8D65-02B773B2D370@activestate.com>


On 2010-02-05, at 4:57 PM, P.J. Eby wrote:

> At 04:49 PM 2/5/2010 -0800, Sridhar Ratnakumar wrote:
>> Do we have a real-world example for installing uninstallable data files outside sys.prefix?
> 
> Scripts aren't always in sys.prefix,

Scripts *usually* go to sys.prefix/bin (or sys.prefix/Scripts or site.USER_BASE/Scripts). Are their packages that actually put them elsewhere?

> and some packages install fixed data (e.g. web pages, message catalogs, etc.) to /usr/share or some such.

This seems like a possible case. Does this only happen with Linux specific packages (eg: GNOME apps)?

> Also make sure you're not forgetting e.g. shared hosting scenarios without a virtualenv, unless you're planning to make virtualenvs an officially-supported thing.  In such cases, the paths involved won't necessarily fall into a single root.

PyPM is an ActivePython tool which installs itself into, say, /opt/ActivePython-2.6 - a single root directory. So we never had to worry about other layouts. PyPM supports virtualenvs too, so "pypm -E /tmp/myenv install lxml" would install into the specified virtualenv.

> Also don't forget sys.exec_prefix.  (Isn't packaging fun?)

In ActivePython, sys.exec_prefix == sys.prefix.

-srid

From pje at telecommunity.com  Sat Feb  6 05:33:08 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 05 Feb 2010 23:33:08 -0500
Subject: [Distutils] PEP 376 - RECORD file / Data files + pip feedback ?
In-Reply-To: <CEF2BDD8-F6EC-4914-8D65-02B773B2D370@activestate.com>
References: <94bdd2611002050813r38b1e96eqa11415274f803041@mail.gmail.com>
	<0A277B9D-10F9-4DAD-AF44-B824A82218D0@activestate.com>
	<20100206005729.306E83A40A0@sparrow.telecommunity.com>
	<CEF2BDD8-F6EC-4914-8D65-02B773B2D370@activestate.com>
Message-ID: <20100206043310.39D8C3A40A0@sparrow.telecommunity.com>

At 06:10 PM 2/5/2010 -0800, Sridhar Ratnakumar wrote:

>On 2010-02-05, at 4:57 PM, P.J. Eby wrote:
>
> > At 04:49 PM 2/5/2010 -0800, Sridhar Ratnakumar wrote:
> >> Do we have a real-world example for installing uninstallable 
> data files outside sys.prefix?
> >
> > Scripts aren't always in sys.prefix,
>
>Scripts *usually* go to sys.prefix/bin (or sys.prefix/Scripts or 
>site.USER_BASE/Scripts). Are their packages that actually put them elsewhere?

It's not the package that determines installation locations, it's the 
user's command line options and configuration file settings that 
determine the locations for libraries, scripts, and data files.


> > Also make sure you're not forgetting e.g. shared hosting 
> scenarios without a virtualenv, unless you're planning to make 
> virtualenvs an officially-supported thing.  In such cases, the 
> paths involved won't necessarily fall into a single root.
>
>PyPM is an ActivePython tool which installs itself into, say, 
>/opt/ActivePython-2.6 - a single root directory. So we never had to 
>worry about other layouts. PyPM supports virtualenvs too, so "pypm 
>-E /tmp/myenv install lxml" would install into the specified virtualenv.
>
> > Also don't forget sys.exec_prefix.  (Isn't packaging fun?)
>
>In ActivePython, sys.exec_prefix == sys.prefix.

Yes, well, as I said, isn't packaging (in the wider Python world, 
where few such simplifications can be had) fun?  ;-)


From cournape at gmail.com  Sat Feb  6 05:42:50 2010
From: cournape at gmail.com (David Cournapeau)
Date: Sat, 6 Feb 2010 13:42:50 +0900
Subject: [Distutils] PEP 376 - RECORD file / Data files + pip feedback ?
In-Reply-To: <20100206043310.39D8C3A40A0@sparrow.telecommunity.com>
References: <94bdd2611002050813r38b1e96eqa11415274f803041@mail.gmail.com>
	<0A277B9D-10F9-4DAD-AF44-B824A82218D0@activestate.com>
	<20100206005729.306E83A40A0@sparrow.telecommunity.com>
	<CEF2BDD8-F6EC-4914-8D65-02B773B2D370@activestate.com>
	<20100206043310.39D8C3A40A0@sparrow.telecommunity.com>
Message-ID: <5b8d13221002052042h5f61bbes558440be5ed52113@mail.gmail.com>

On Sat, Feb 6, 2010 at 1:33 PM, P.J. Eby <pje at telecommunity.com> wrote:
> At 06:10 PM 2/5/2010 -0800, Sridhar Ratnakumar wrote:
>
>> On 2010-02-05, at 4:57 PM, P.J. Eby wrote:
>>
>> > At 04:49 PM 2/5/2010 -0800, Sridhar Ratnakumar wrote:
>> >> Do we have a real-world example for installing uninstallable data files
>> >> outside sys.prefix?
>> >
>> > Scripts aren't always in sys.prefix,
>>
>> Scripts *usually* go to sys.prefix/bin (or sys.prefix/Scripts or
>> site.USER_BASE/Scripts). Are their packages that actually put them
>> elsewhere?
>
> It's not the package that determines installation locations, it's the user's
> command line options and configuration file settings that determine the
> locations for libraries, scripts, and data files.
>
>
>> > Also make sure you're not forgetting e.g. shared hosting scenarios
>> > without a virtualenv, unless you're planning to make virtualenvs an
>> > officially-supported thing. ?In such cases, the paths involved won't
>> > necessarily fall into a single root.
>>
>> PyPM is an ActivePython tool which installs itself into, say,
>> /opt/ActivePython-2.6 - a single root directory. So we never had to worry
>> about other layouts. PyPM supports virtualenvs too, so "pypm -E /tmp/myenv
>> install lxml" would install into the specified virtualenv.
>>
>> > Also don't forget sys.exec_prefix. ?(Isn't packaging fun?)
>>
>> In ActivePython, sys.exec_prefix == sys.prefix.
>
> Yes, well, as I said, isn't packaging (in the wider Python world, where few
> such simplifications can be had) fun? ?;-)

May I remind of my recent suggestion of a build manifest ? It was
precisely designed to solve this kind of issues, and a similar format
is used by Cabal for Haskell, which supports a wide variety of install
scenario as well. It supports relocation.

I am rather suspicious about attempts of designing such things from
scratch without looking at how other packaging tools do similar
things.

cheers,

David

From ziade.tarek at gmail.com  Sat Feb  6 11:16:21 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 6 Feb 2010 11:16:21 +0100
Subject: [Distutils] PEP 376 - RECORD file / Data files + pip feedback ?
In-Reply-To: <20100206043310.39D8C3A40A0@sparrow.telecommunity.com>
References: <94bdd2611002050813r38b1e96eqa11415274f803041@mail.gmail.com>
	<0A277B9D-10F9-4DAD-AF44-B824A82218D0@activestate.com>
	<20100206005729.306E83A40A0@sparrow.telecommunity.com>
	<CEF2BDD8-F6EC-4914-8D65-02B773B2D370@activestate.com>
	<20100206043310.39D8C3A40A0@sparrow.telecommunity.com>
Message-ID: <94bdd2611002060216p23b98b3cyf04931412871a351@mail.gmail.com>

On Sat, Feb 6, 2010 at 5:33 AM, P.J. Eby <pje at telecommunity.com> wrote:
> At 06:10 PM 2/5/2010 -0800, Sridhar Ratnakumar wrote:
>
>> On 2010-02-05, at 4:57 PM, P.J. Eby wrote:
>>
>> > At 04:49 PM 2/5/2010 -0800, Sridhar Ratnakumar wrote:
>> >> Do we have a real-world example for installing uninstallable data files
>> >> outside sys.prefix?
>> >
>> > Scripts aren't always in sys.prefix,
>>
>> Scripts *usually* go to sys.prefix/bin (or sys.prefix/Scripts or
>> site.USER_BASE/Scripts). Are their packages that actually put them
>> elsewhere?
>
> It's not the package that determines installation locations, it's the user's
> command line options and configuration file settings that determine the
> locations for libraries, scripts, and data files.

IOW some paths in the PREFIXES file (or the RECORD file as long as we are able
to find-and-replace paths in there) *need* to be updated at
installation time no matter
what we decide to do here.

So, it seems to me that we need to keep unexpanded paths [1] in the RECORD file
and let the install command expand them using installation option .
But we have to avoid this expansion
when doing binary releases.

It seems feasible to me in both cases. And Wolojda could work later on
adding more paths definitions
and how to configure them through distutils options.

[1] See _INSTALL_SCHEME in
http://svn.python.org/view/python/trunk/Lib/sysconfig.py?view=markup
     that can be read through API (expanded or unexpanded)

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Sat Feb  6 12:26:28 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 6 Feb 2010 12:26:28 +0100
Subject: [Distutils] PEP 376 - RECORD file / Data files + pip feedback ?
In-Reply-To: <5b8d13221002052042h5f61bbes558440be5ed52113@mail.gmail.com>
References: <94bdd2611002050813r38b1e96eqa11415274f803041@mail.gmail.com>
	<0A277B9D-10F9-4DAD-AF44-B824A82218D0@activestate.com>
	<20100206005729.306E83A40A0@sparrow.telecommunity.com>
	<CEF2BDD8-F6EC-4914-8D65-02B773B2D370@activestate.com>
	<20100206043310.39D8C3A40A0@sparrow.telecommunity.com>
	<5b8d13221002052042h5f61bbes558440be5ed52113@mail.gmail.com>
Message-ID: <94bdd2611002060326r799b79bcx5dae362481668716@mail.gmail.com>

On Sat, Feb 6, 2010 at 5:42 AM, David Cournapeau <cournape at gmail.com> wrote:
> On Sat, Feb 6, 2010 at 1:33 PM, P.J. Eby <pje at telecommunity.com> wrote:
>> At 06:10 PM 2/5/2010 -0800, Sridhar Ratnakumar wrote:
>>
>>> On 2010-02-05, at 4:57 PM, P.J. Eby wrote:
>>>
>>> > At 04:49 PM 2/5/2010 -0800, Sridhar Ratnakumar wrote:
>>> >> Do we have a real-world example for installing uninstallable data files
>>> >> outside sys.prefix?
>>> >
>>> > Scripts aren't always in sys.prefix,
>>>
>>> Scripts *usually* go to sys.prefix/bin (or sys.prefix/Scripts or
>>> site.USER_BASE/Scripts). Are their packages that actually put them
>>> elsewhere?
>>
>> It's not the package that determines installation locations, it's the user's
>> command line options and configuration file settings that determine the
>> locations for libraries, scripts, and data files.
>>
>>
>>> > Also make sure you're not forgetting e.g. shared hosting scenarios
>>> > without a virtualenv, unless you're planning to make virtualenvs an
>>> > officially-supported thing. ?In such cases, the paths involved won't
>>> > necessarily fall into a single root.
>>>
>>> PyPM is an ActivePython tool which installs itself into, say,
>>> /opt/ActivePython-2.6 - a single root directory. So we never had to worry
>>> about other layouts. PyPM supports virtualenvs too, so "pypm -E /tmp/myenv
>>> install lxml" would install into the specified virtualenv.
>>>
>>> > Also don't forget sys.exec_prefix. ?(Isn't packaging fun?)
>>>
>>> In ActivePython, sys.exec_prefix == sys.prefix.
>>
>> Yes, well, as I said, isn't packaging (in the wider Python world, where few
>> such simplifications can be had) fun? ?;-)
>
> May I remind of my recent suggestion of a build manifest ? It was
> precisely designed to solve this kind of issues, and a similar format
> is used by Cabal for Haskell, which supports a wide variety of install
> scenario as well. It supports relocation.
>
> I am rather suspicious about attempts of designing such things from
> scratch without looking at how other packaging tools do similar
> things.

I still have that in mind. In fact one of the sprints task in Pycon
could be to create
a configure command for distutils (like what 4suite has) (I've pointed
you to the repo where I am
have started a prototype for this in a recent mail -- we sure could
use your help/knowledge on this if you want
to contribute)

But that's a lot of work that will not happen in 2.7. Here, the
attempt is not to correct distutils design
flaw about building things (e.g. the fact that install is also a build
command in some way), but rather
to find the simplest way to define the RECORD file, whichars is the
latest missing bits to have a PEP that
can be accepted for inclusion in PEP 376 --

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org

From wentland at cl.uni-heidelberg.de  Sat Feb  6 12:21:57 2010
From: wentland at cl.uni-heidelberg.de (Wolodja Wentland)
Date: Sat, 6 Feb 2010 12:21:57 +0100
Subject: [Distutils] PEP 376 - RECORD file / Data files + pip feedback ?
In-Reply-To: <94bdd2611002060216p23b98b3cyf04931412871a351@mail.gmail.com>
References: <94bdd2611002050813r38b1e96eqa11415274f803041@mail.gmail.com>
	<0A277B9D-10F9-4DAD-AF44-B824A82218D0@activestate.com>
	<20100206005729.306E83A40A0@sparrow.telecommunity.com>
	<CEF2BDD8-F6EC-4914-8D65-02B773B2D370@activestate.com>
	<20100206043310.39D8C3A40A0@sparrow.telecommunity.com>
	<94bdd2611002060216p23b98b3cyf04931412871a351@mail.gmail.com>
Message-ID: <20100206112157.GA2508@kinakuta.local>

On Sat, Feb 06, 2010 at 11:16 +0100, Tarek Ziad? wrote:

Hi all,

sorry for joining in so late. I am pretty busy with my thesis and
examinations, which is why I can't work on this right now.

I am still subscribed to python-distutils and read the mails regularly,
so there is no need to CC me.

> IOW some paths in the PREFIXES file (or the RECORD file as long as we
> are able to find-and-replace paths in there) *need* to be updated at
> installation time no matter what we decide to do here.

Exactly!

> So, it seems to me that we need to keep unexpanded paths [1] in the
> RECORD file and let the install command expand them using installation
> option .  But we have to avoid this expansion when doing binary
> releases.

I still think that a PREFIX file, that contains *all* prefixes which 
are configurable during installation right now, should make it to PEP
376 and to Python 2.7/3.2 . These prefixes could be kept unexpanded in
the RECORD file and it would therefore only be necessary to change them
within the PREFIX file if one wants to relocate an installation. (Is
this really a common use case?)

The PEP lists $PREFIX and $EXEC_PREFIX, but the set of prefixes that are
configurable during installation comprises:

Prefix              Default value       Command line option
-----------------------------------------------------------
PREFIX              sys.prefix          --install-purelib
EXEC_PREFIX         sys.exec_prefix     --install-platlib
SCRIPTS_PREFIX      $PREFIX/bin         --install-scripts
DATA_PREFIX         $PREFIX/share       --install-data                    

I think that it'll help Linux distribution developers *and* Python
developers a lot if these prefixes where available during runtime or
could be obtained by parsing the PREFIX file. This would mean the end of
__file__ magic to find data files and Python packagers could adapt their
tools to copy, for example, a configuration files from
$DATA_PREFIX/foo.cfg to a location that complies with the FHS (i.e.
/etc/foo.cfg).

> It seems feasible to me in both cases. And Wolojda could work later on
> adding more paths definitions and how to configure them through
> distutils options.

Will do :-)

Yours sincerely
-- 
  .''`.     Wolodja Wentland    <wentland at cl.uni-heidelberg.de> 
 : :'  :    
 `. `'`     4096R/CAF14EFC 
   `-       081C B7CD FF04 2BA9 94EA  36B2 8B7F 7D30 CAF1 4EFC
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20100206/2ff54a98/attachment.pgp>

From floris.bruynooghe at gmail.com  Sun Feb  7 18:32:50 2010
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Sun, 7 Feb 2010 17:32:50 +0000
Subject: [Distutils] PEP 376 - RECORD file / Data files + pip feedback ?
In-Reply-To: <94bdd2611002060326r799b79bcx5dae362481668716@mail.gmail.com>
References: <94bdd2611002050813r38b1e96eqa11415274f803041@mail.gmail.com>
	<0A277B9D-10F9-4DAD-AF44-B824A82218D0@activestate.com>
	<20100206005729.306E83A40A0@sparrow.telecommunity.com>
	<CEF2BDD8-F6EC-4914-8D65-02B773B2D370@activestate.com>
	<20100206043310.39D8C3A40A0@sparrow.telecommunity.com>
	<5b8d13221002052042h5f61bbes558440be5ed52113@mail.gmail.com>
	<94bdd2611002060326r799b79bcx5dae362481668716@mail.gmail.com>
Message-ID: <20100207173250.GA4685@laurie.devork>

On Sat, Feb 06, 2010 at 12:26:28PM +0100, Tarek Ziad? wrote:
> I still have that in mind. In fact one of the sprints task in Pycon
> could be to create
> a configure command for distutils (like what 4suite has) (I've pointed
> you to the repo where I am
> have started a prototype for this in a recent mail -- we sure could
> use your help/knowledge on this if you want
> to contribute)

Either this was in private or I missed this.  If possible could you
share this pointer again?  I'd be interested in looking at it too.

Cheers
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From rich at noir.com  Tue Feb  9 02:24:01 2010
From: rich at noir.com (K. Richard Pixley)
Date: Mon, 08 Feb 2010 17:24:01 -0800
Subject: [Distutils] stdeb vs dependencies?
In-Reply-To: <4B6C5265.2020901@astraw.com>
References: <4B689B8C.6080203@noir.com> <4B68C125.9010005@astraw.com>
	<4B68C78B.9030904@noir.com> <4B68E0DD.9040800@astraw.com>
	<4B69AF8B.7000102@noir.com> <4B6C5265.2020901@astraw.com>
Message-ID: <4B70B931.8080101@noir.com>

Andrew Straw wrote:
> K. Richard Pixley wrote:
>   
>> Andrew Straw wrote:
>>     
>>>> dpkg-gencontrol: warning: unknown substitution variable ${python:Provides}
>>>> dpkg-gencontrol: warning: unknown substitution variable ${python:Depends}
>>>> dpkg-gencontrol: warning: unknown substitution variable ${python:Provides}
>>>> dpkg-gencontrol: warning: unknown substitution variable ${python:Versions}
>>>>     
>>>>         
>>> What's the output of
>>>
>>> dpkg -s python-support | grep Version
>>>
>>> and
>>>
>>> dpkg -s debhelper | grep Version
>>>   
>>>       
>> rich at irish> dpkg -s debhelper | grep Version
>> Version: 7.4.11ubuntu1
>> rich at irish> dpkg -s python-support | grep Version
>> Version: 1.0.4ubuntu1
>>     
>
> With the same versions of debhelper and python-support, I'm not getting
> those warnings with stdeb 0.5.1. Can you post the stdeb generated
> debian/control file?
Source: ui
Maintainer: Rich Pixley <rich.pixley at palm.com>
Section: python
Priority: optional
Build-Depends: python-setuptools (>= 0.6b3), debhelper (>= 7), 
python-support (>= 0.8.4)
Standards-Version: 3.7.2

Package: python-ui
Architecture: all
Depends: ${python:Depends}, python-pybabel (>= 0.9.4), 
python-zope.sqlalchemy (>= 0.4), python-repoze.tm2 (>= 1.0a4)
XB-Python-Version: ${python:Versions}
Provides: ${python:Provides}
Description: UNKNOWN


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20100208/c5f1f5b2/attachment.htm>

From david.lyon at pythontest.org  Tue Feb  9 02:22:49 2010
From: david.lyon at pythontest.org (David Lyon)
Date: Tue, 9 Feb 2010 12:22:49 +1100 (EST)
Subject: [Distutils] Distutils roadmap of refactoring for 2.7
In-Reply-To: <7AC0FCCA-AD35-42C5-9C88-6428E9CC8D4F@mac.com>
References: <94bdd2611001230400sed23a0excc626c85cf6e39bb@mail.gmail.com>
	<20100123171947.EBCC63A4065@sparrow.telecommunity.com>
	<94bdd2611001230929n47ef1945h6c2e4ae3513b614b@mail.gmail.com>
	<20100123181126.AA2AE3A4065@sparrow.telecommunity.com>
	<94bdd2611001231041i45ec71qae737789cab2bf40@mail.gmail.com>
	<1927.218.214.45.58.1264390716.squirrel@syd-srv02.ezyreg.com>
	<4B6867E0.6060003@simplistix.co.uk>
	<1237.218.214.45.58.1265153145.squirrel@syd-srv02.ezyreg.com>
	<7AC0FCCA-AD35-42C5-9C88-6428E9CC8D4F@mac.com>
Message-ID: <2372.218.214.45.58.1265678569.squirrel@syd-srv02.ezyreg.com>

> Could you please explain what the problem is you're complaining about
> instead of assuming everyone knows the details by heart? The only
> reference I saw to 'i386' in PEP 345 is in the section about Environment
> markers and that only refers to the CPU architecture in a test for CPU
> architecture (platform.machine == 'i386') which seems reasonable to me.

Ronald,

What I'm showing is a snippet of what the file
will look like.

What we will have is different ways of referencing
the one operating system.

There's one set of constants for Platform, and another
set of constants for requires-dist.

That breaks the 'one way of doing things' rule.

---------- Metadata snippet ------------------------
Platform: MacOS
Platform: Microsoft Windows
Requires-Dist: pywin32 (>1.0); sys.platform == 'win32'
----------------------------------------------------

But, if it has to be, then it has to be.

It's just an inconsistent way of doing this, using
different constants in the one file to have the
same meaning.

David



From cournape at gmail.com  Tue Feb  9 11:20:53 2010
From: cournape at gmail.com (David Cournapeau)
Date: Tue, 9 Feb 2010 19:20:53 +0900
Subject: [Distutils] PEP 376 - RECORD file / Data files + pip feedback ?
In-Reply-To: <94bdd2611002060326r799b79bcx5dae362481668716@mail.gmail.com>
References: <94bdd2611002050813r38b1e96eqa11415274f803041@mail.gmail.com>
	<0A277B9D-10F9-4DAD-AF44-B824A82218D0@activestate.com>
	<20100206005729.306E83A40A0@sparrow.telecommunity.com>
	<CEF2BDD8-F6EC-4914-8D65-02B773B2D370@activestate.com>
	<20100206043310.39D8C3A40A0@sparrow.telecommunity.com>
	<5b8d13221002052042h5f61bbes558440be5ed52113@mail.gmail.com>
	<94bdd2611002060326r799b79bcx5dae362481668716@mail.gmail.com>
Message-ID: <5b8d13221002090220h3f377daco19b6330b8768547d@mail.gmail.com>

On Sat, Feb 6, 2010 at 8:26 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> On Sat, Feb 6, 2010 at 5:42 AM, David Cournapeau <cournape at gmail.com> wrote:
>> On Sat, Feb 6, 2010 at 1:33 PM, P.J. Eby <pje at telecommunity.com> wrote:
>>> At 06:10 PM 2/5/2010 -0800, Sridhar Ratnakumar wrote:
>>>
>>>> On 2010-02-05, at 4:57 PM, P.J. Eby wrote:
>>>>
>>>> > At 04:49 PM 2/5/2010 -0800, Sridhar Ratnakumar wrote:
>>>> >> Do we have a real-world example for installing uninstallable data files
>>>> >> outside sys.prefix?
>>>> >
>>>> > Scripts aren't always in sys.prefix,
>>>>
>>>> Scripts *usually* go to sys.prefix/bin (or sys.prefix/Scripts or
>>>> site.USER_BASE/Scripts). Are their packages that actually put them
>>>> elsewhere?
>>>
>>> It's not the package that determines installation locations, it's the user's
>>> command line options and configuration file settings that determine the
>>> locations for libraries, scripts, and data files.
>>>
>>>
>>>> > Also make sure you're not forgetting e.g. shared hosting scenarios
>>>> > without a virtualenv, unless you're planning to make virtualenvs an
>>>> > officially-supported thing. ?In such cases, the paths involved won't
>>>> > necessarily fall into a single root.
>>>>
>>>> PyPM is an ActivePython tool which installs itself into, say,
>>>> /opt/ActivePython-2.6 - a single root directory. So we never had to worry
>>>> about other layouts. PyPM supports virtualenvs too, so "pypm -E /tmp/myenv
>>>> install lxml" would install into the specified virtualenv.
>>>>
>>>> > Also don't forget sys.exec_prefix. ?(Isn't packaging fun?)
>>>>
>>>> In ActivePython, sys.exec_prefix == sys.prefix.
>>>
>>> Yes, well, as I said, isn't packaging (in the wider Python world, where few
>>> such simplifications can be had) fun? ?;-)
>>
>> May I remind of my recent suggestion of a build manifest ? It was
>> precisely designed to solve this kind of issues, and a similar format
>> is used by Cabal for Haskell, which supports a wide variety of install
>> scenario as well. It supports relocation.
>>
>> I am rather suspicious about attempts of designing such things from
>> scratch without looking at how other packaging tools do similar
>> things.
>
> I still have that in mind. In fact one of the sprints task in Pycon
> could be to create
> a configure command for distutils

Could you expand on that ? I don't understand the link between
configure and the topic at hand ?

>
> But that's a lot of work that will not happen in 2.7. Here, the
> attempt is not to correct distutils design
> flaw about building things (e.g. the fact that install is also a build
> command in some way), but rather
> to find the simplest way to define the RECORD file

Maybe we are not speaking about the same thing. I was talking about this thread

http://mail.python.org/pipermail/distutils-sig/2009-December/014994.html

Having a workable specification for this should not require any heavy
distutils refactoring. As it is mostly reimplementing a known and
already-used design, it should not be a lot of work.

David

From ziade.tarek at gmail.com  Tue Feb  9 12:00:18 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 9 Feb 2010 12:00:18 +0100
Subject: [Distutils] PEP 376 - RECORD file / Data files + pip feedback ?
In-Reply-To: <5b8d13221002090220h3f377daco19b6330b8768547d@mail.gmail.com>
References: <94bdd2611002050813r38b1e96eqa11415274f803041@mail.gmail.com>
	<0A277B9D-10F9-4DAD-AF44-B824A82218D0@activestate.com>
	<20100206005729.306E83A40A0@sparrow.telecommunity.com>
	<CEF2BDD8-F6EC-4914-8D65-02B773B2D370@activestate.com>
	<20100206043310.39D8C3A40A0@sparrow.telecommunity.com>
	<5b8d13221002052042h5f61bbes558440be5ed52113@mail.gmail.com>
	<94bdd2611002060326r799b79bcx5dae362481668716@mail.gmail.com>
	<5b8d13221002090220h3f377daco19b6330b8768547d@mail.gmail.com>
Message-ID: <94bdd2611002090300o3b63ddf7uee845081a3f25bdd@mail.gmail.com>

On Tue, Feb 9, 2010 at 11:20 AM, David Cournapeau <cournape at gmail.com> wrote:
[..]
>>
>> But that's a lot of work that will not happen in 2.7. Here, the
>> attempt is not to correct distutils design
>> flaw about building things (e.g. the fact that install is also a build
>> command in some way), but rather
>> to find the simplest way to define the RECORD file
>
> Maybe we are not speaking about the same thing. I was talking about this thread
>
> http://mail.python.org/pipermail/distutils-sig/2009-December/014994.html
>
> Having a workable specification for this should not require any heavy
> distutils refactoring. As it is mostly reimplementing a known and
> already-used design, it should not be a lot of work.

ok, I thought you were mentioning the creation of a configure command,
that would create
a description file that can be used at build step.

The build manifest you are mentioning (as I said back then) seems like
a good idea,  and is
complementary in some ways with what Wolojda has started.

Now, since it seems a bit similar to the RECORD file (a list of files
+ variables) , maybe you could describe
the differences (keeping the target file, file groups) by taking an
example of RECORD file from PEP 376,
and rewriting it, so we can compare and understand it better,


Tarek

-- 
Tarek Ziad? | http://ziade.org

From skip at pobox.com  Tue Feb  9 02:45:14 2010
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 8 Feb 2010 19:45:14 -0600
Subject: [Distutils] Building a platform-independent dist?
Message-ID: <19312.48682.998288.606737@montanaro.dyndns.org>


It's been several months since I created a distribution using the bdist
command.  Now I can't seem to build a simple gztar distribution.  My
MANIFEST.in looks like this:

    include README RELEASE-NOTES LICENSE MANIFEST
    include lockfile.py setup.py ACKS
    recursive-include doc *.rst conf.py Makefile

MANIFEST is:

    ACKS
    LICENSE
    MANIFEST
    README
    RELEASE-NOTES
    lockfile.py
    setup.py
    doc/glossary.rst
    doc/index.rst
    doc/lockfile.rst

and I'm executing

    python setup.py bdist --formats=gztar

(where python is 2.7a2+).  The output is a .tar.gz file but it includes
".macosx-10.4-i386" in the filename and the tar file itself has more
structure and fewer files than I think it should have:

    % tar tfz dist/lockfile-0.8.macosx-10.4-i386.tar.gz 
    ./
    ./Users/
    ./Users/skip/
    ./Users/skip/local/
    ./Users/skip/local/lib/
    ./Users/skip/local/lib/python2.7/
    ./Users/skip/local/lib/python2.7/site-packages/
    ./Users/skip/local/lib/python2.7/site-packages/lockfile-0.8-py2.7.egg-info
    ./Users/skip/local/lib/python2.7/site-packages/lockfile.py
    ./Users/skip/local/lib/python2.7/site-packages/lockfile.pyc

setup.py is simple:

    from distutils.core import setup
    setup(name='lockfile',
          ... bunch of kwd args elided ...
         )

Where are the missing files?  How do I create a platform-independent
distribution?

-- 
Skip Montanaro - skip at pobox.com - http://www.smontanaro.net/

From ziade.tarek at gmail.com  Tue Feb  9 21:06:05 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 9 Feb 2010 21:06:05 +0100
Subject: [Distutils] Building a platform-independent dist?
In-Reply-To: <19312.48682.998288.606737@montanaro.dyndns.org>
References: <19312.48682.998288.606737@montanaro.dyndns.org>
Message-ID: <94bdd2611002091206k1f63446et557afa9e57f976d6@mail.gmail.com>

On Tue, Feb 9, 2010 at 2:45 AM,  <skip at pobox.com> wrote:
[..]
> (where python is 2.7a2+). ?The output is a .tar.gz file but it includes
> ".macosx-10.4-i386" in the filename and the tar file itself has more
> structure and fewer files than I think it should have:

The macosx-10.4-i386 bit is normal since you are building a binary distribution.

>
> ? ?% tar tfz dist/lockfile-0.8.macosx-10.4-i386.tar.gz
> ? ?./
> ? ?./Users/
> ? ?./Users/skip/
> ? ?./Users/skip/local/
> ? ?./Users/skip/local/lib/
> ? ?./Users/skip/local/lib/python2.7/
> ? ?./Users/skip/local/lib/python2.7/site-packages/
> ? ?./Users/skip/local/lib/python2.7/site-packages/lockfile-0.8-py2.7.egg-info
> ? ?./Users/skip/local/lib/python2.7/site-packages/lockfile.py
> ? ?./Users/skip/local/lib/python2.7/site-packages/lockfile.pyc


That's a strange output, it should build a tree corresponding to the
mac platform installation
scheme, but here it seems to use a user tree.

so is should be:

./Library/Frameworks/Python.framework/....
or

./usr/local/lib/python2.7/...

depending on how you have installed Python; instead of  ./Users/skip/...

>
> setup.py is simple:
>
> ? ?from distutils.core import setup
> ? ?setup(name='lockfile',
> ? ? ? ? ?... bunch of kwd args elided ...
> ? ? ? ? )
>
> Where are the missing files?

I'd need to see the full setup.py options.


> How do I create a platform-independent distribution?

By using the "sdist" command. bdist will compile and generate platform-specific
distributions.

Tarek

-- 
Tarek Ziad? | http://ziade.org

From tseaver at palladion.com  Tue Feb  9 21:34:47 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 09 Feb 2010 15:34:47 -0500
Subject: [Distutils] Building a platform-independent dist?
In-Reply-To: <19312.48682.998288.606737@montanaro.dyndns.org>
References: <19312.48682.998288.606737@montanaro.dyndns.org>
Message-ID: <hksgt6$9lf$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

skip at pobox.com wrote:

> It's been several months since I created a distribution using the bdist
> command.  Now I can't seem to build a simple gztar distribution.  My
> MANIFEST.in looks like this:
> 
>     include README RELEASE-NOTES LICENSE MANIFEST
>     include lockfile.py setup.py ACKS
>     recursive-include doc *.rst conf.py Makefile
> 
> MANIFEST is:
> 
>     ACKS
>     LICENSE
>     MANIFEST
>     README
>     RELEASE-NOTES
>     lockfile.py
>     setup.py
>     doc/glossary.rst
>     doc/index.rst
>     doc/lockfile.rst
> 
> and I'm executing
> 
>     python setup.py bdist --formats=gztar
> 
> (where python is 2.7a2+).  The output is a .tar.gz file but it includes
> ".macosx-10.4-i386" in the filename and the tar file itself has more
> structure and fewer files than I think it should have:

Why do you want to make a "dumb" bdist rather than an sdist?


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktxxuIACgkQ+gerLs4ltQ64FACfYiXLsAs/i6DnrRyhLbn14VQy
qzYAn1eQWzD+Z/y0LjQrMqSc/vk6wnXF
=tQjw
-----END PGP SIGNATURE-----


From skip at pobox.com  Tue Feb  9 23:11:53 2010
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 9 Feb 2010 16:11:53 -0600
Subject: [Distutils] Building a platform-independent dist?
In-Reply-To: <94bdd2611002091206k1f63446et557afa9e57f976d6@mail.gmail.com>
References: <19312.48682.998288.606737@montanaro.dyndns.org>
	<94bdd2611002091206k1f63446et557afa9e57f976d6@mail.gmail.com>
Message-ID: <19313.56745.529211.533642@montanaro.dyndns.org>


    >> How do I create a platform-independent distribution?

    Tarek> By using the "sdist" command. bdist will compile and generate
    Tarek> platform-specific distributions.

*dope slap*

Thanks,

Skip

From ziade.tarek at gmail.com  Tue Feb  9 23:46:05 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 9 Feb 2010 23:46:05 +0100
Subject: [Distutils] Building a platform-independent dist?
In-Reply-To: <19313.56745.529211.533642@montanaro.dyndns.org>
References: <19312.48682.998288.606737@montanaro.dyndns.org>
	<94bdd2611002091206k1f63446et557afa9e57f976d6@mail.gmail.com>
	<19313.56745.529211.533642@montanaro.dyndns.org>
Message-ID: <94bdd2611002091446v161538fdpd549d6c0b571c2ff@mail.gmail.com>

On Tue, Feb 9, 2010 at 11:11 PM,  <skip at pobox.com> wrote:
>
> ? ?>> How do I create a platform-independent distribution?
>
> ? ?Tarek> By using the "sdist" command. bdist will compile and generate
> ? ?Tarek> platform-specific distributions.
>
> *dope slap*
>
> Thanks,


Notice that I'm interested in understanding how you got that "local"
in your bdist...

Is your python installed in "./Users/skip/local" ?

Tarek
-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Wed Feb 10 00:33:54 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 10 Feb 2010 00:33:54 +0100
Subject: [Distutils] Fwd: PEP 345 and PEP 386
In-Reply-To: <ca471dc21002091529j5ad1bb5fn70b719fb8441f115@mail.gmail.com>
References: <ca471dc21002041120g69a0b683ic1e3da0eee077a3@mail.gmail.com>
	<94bdd2611002041455w69a50000v74f989d1235d9b2a@mail.gmail.com>
	<ca471dc21002091529j5ad1bb5fn70b719fb8441f115@mail.gmail.com>
Message-ID: <94bdd2611002091533u761ba008vaf2b329a0eb92960@mail.gmail.com>

FYI, PEP 345 and 386 are now accepted.

Thanks to all the people that helped in building them !


---------- Forwarded message ----------
From: Guido van Rossum <guido at python.org>
Date: Wed, Feb 10, 2010 at 12:29 AM
Subject: Re: PEP 345 and PEP 386
To: Tarek Ziad? <ziade.tarek at gmail.com>
Cc: Python-Dev <python-dev at python.org>


On Thu, Feb 4, 2010 at 2:55 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> On Thu, Feb 4, 2010 at 8:20 PM, Guido van Rossum <guido at python.org> wrote:
> [..]
>> I have one comment on PEP 345: Why is author-email mandatory? I'm sure
>> there are plenty of cases where either the author doesn't want their
>> email address published, or their last know email address is no longer
>> valid. (Tarek responded off-line that it isn't all that mandatory; I
>> propose to say so in the PEP.)
>
> Yes, I propose to remove the mandatory flag from that field.

Since this is done I now approve both PEP 345 and PEP 386 (which is
not to say that small editorial changes to the text couldn't be made).

--
--Guido van Rossum (python.org/~guido)



-- 
Tarek Ziad? | http://ziade.org

From skip at pobox.com  Wed Feb 10 02:25:56 2010
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 9 Feb 2010 19:25:56 -0600
Subject: [Distutils] Building a platform-independent dist?
In-Reply-To: <94bdd2611002091446v161538fdpd549d6c0b571c2ff@mail.gmail.com>
References: <19312.48682.998288.606737@montanaro.dyndns.org>
	<94bdd2611002091206k1f63446et557afa9e57f976d6@mail.gmail.com>
	<19313.56745.529211.533642@montanaro.dyndns.org>
	<94bdd2611002091446v161538fdpd549d6c0b571c2ff@mail.gmail.com>
Message-ID: <19314.2852.426173.627978@montanaro.dyndns.org>


    Tarek> Notice that I'm interested in understanding how you got that
    Tarek> "local" in your bdist...

    Tarek> Is your python installed in "./Users/skip/local" ?

Well, /Users/skip/local.  I never install Python of any variety from source
anywhere close to the system locations (not even /usr/local).


From cournape at gmail.com  Wed Feb 10 05:35:53 2010
From: cournape at gmail.com (David Cournapeau)
Date: Wed, 10 Feb 2010 13:35:53 +0900
Subject: [Distutils] PEP 376 - RECORD file / Data files + pip feedback ?
In-Reply-To: <94bdd2611002090300o3b63ddf7uee845081a3f25bdd@mail.gmail.com>
References: <94bdd2611002050813r38b1e96eqa11415274f803041@mail.gmail.com>
	<0A277B9D-10F9-4DAD-AF44-B824A82218D0@activestate.com>
	<20100206005729.306E83A40A0@sparrow.telecommunity.com>
	<CEF2BDD8-F6EC-4914-8D65-02B773B2D370@activestate.com>
	<20100206043310.39D8C3A40A0@sparrow.telecommunity.com>
	<5b8d13221002052042h5f61bbes558440be5ed52113@mail.gmail.com>
	<94bdd2611002060326r799b79bcx5dae362481668716@mail.gmail.com>
	<5b8d13221002090220h3f377daco19b6330b8768547d@mail.gmail.com>
	<94bdd2611002090300o3b63ddf7uee845081a3f25bdd@mail.gmail.com>
Message-ID: <5b8d13221002092035o2466cc91u51018e3cdbdefba7@mail.gmail.com>

On Tue, Feb 9, 2010 at 8:00 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

> Now, since it seems a bit similar to the RECORD file (a list of files
> + variables) , maybe you could describe
> the differences (keeping the target file, file groups) by taking an
> example of RECORD file from PEP 376,
> and rewriting it, so we can compare and understand it better,

Is there anything besides what is described in PEP 376 for the RECORD
syntax ? It would also be useful for me to see the actual
implementation for the syntax (or is everything described in PEP 376
already in the main distribute hg repo  ?)

thanks,

David

From tmohr at s.netic.de  Wed Feb 10 18:35:36 2010
From: tmohr at s.netic.de (Torsten Mohr)
Date: Wed, 10 Feb 2010 18:35:36 +0100
Subject: [Distutils] Distributing __only__ *.pyc
Message-ID: <201002101835.36517.tmohr@s.netic.de>

Hi,

not sure if this is possible, i'd like to distribute a pure python module as 
.pyc because there is some code in there that idon't want to spread yet.
I plan to release the code as open source later, but it is not possible for 
now.

I know that something like disassembling is possible but for i don't expect 
that somebody will take that effort.


Is it possible to distribute a module.pyc using distutils?

How would that setup script look like?


Thanks for any hints,
Torsten.

From bradallen137 at gmail.com  Wed Feb 10 20:02:58 2010
From: bradallen137 at gmail.com (Brad Allen)
Date: Wed, 10 Feb 2010 13:02:58 -0600
Subject: [Distutils] Distributing __only__ *.pyc
In-Reply-To: <201002101835.36517.tmohr@s.netic.de>
References: <201002101835.36517.tmohr@s.netic.de>
Message-ID: <4957f1ef1002101102m3171b94ja1a2d5364195e944@mail.gmail.com>

On Wed, Feb 10, 2010 at 11:35 AM, Torsten Mohr <tmohr at s.netic.de> wrote:
> Hi,
>
> not sure if this is possible, i'd like to distribute a pure python module as
> .pyc because there is some code in there that idon't want to spread yet.
> I plan to release the code as open source later, but it is not possible for
> now.
>
> I know that something like disassembling is possible but for i don't expect
> that somebody will take that effort.
>
>
> Is it possible to distribute a module.pyc using distutils?
>
> How would that setup script look like?
>

I don't know about with distutils, but setuptools has an
--exclude-source-files option for creating an egg, used like this:

python setup.py bdist_egg --exclude-source-files

From aljosa.mohorovic at gmail.com  Wed Feb 10 22:15:16 2010
From: aljosa.mohorovic at gmail.com (=?UTF-8?B?QWxqb8WhYSBNb2hvcm92acSH?=)
Date: Wed, 10 Feb 2010 22:15:16 +0100
Subject: [Distutils] setup.py#package_data - howto include entire folder
	when installing
Message-ID: <87d364ab1002101315n634ef9ebn58743c7669c1e800@mail.gmail.com>

i've created MANIFEST.in and setup.py and when i call "python setup.py
sdist" archive is created normally and archive contains all files.
when i call "python setup.py install" data files are not installed and
i have no idea why.
i've been searching for a solution but every combination i've tried failed.

i usually have only html templates so something like:
package_data = {'mypkg': ['templates/*.html',]}
works as expected but now i have a folder with different types of
files and nested folder structure.

my question is:
- how can i define in setup.py to include folder (let's call it
"data") and everything in data so that when "python setup.py install"
is called that folder is copied with python code?

something like "package_data={'mypkg': ['data/*']},"?

Aljosa Mohorovic

From ziade.tarek at gmail.com  Thu Feb 11 00:15:40 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 11 Feb 2010 00:15:40 +0100
Subject: [Distutils] setup.py#package_data - howto include entire folder
	when installing
In-Reply-To: <87d364ab1002101315n634ef9ebn58743c7669c1e800@mail.gmail.com>
References: <87d364ab1002101315n634ef9ebn58743c7669c1e800@mail.gmail.com>
Message-ID: <94bdd2611002101515s5b8eac5bxbcc5672e684e7aa0@mail.gmail.com>

On Wed, Feb 10, 2010 at 10:15 PM, Aljo?a Mohorovi?
<aljosa.mohorovic at gmail.com> wrote:
> i've created MANIFEST.in and setup.py and when i call "python setup.py
> sdist" archive is created normally and archive contains all files.
> when i call "python setup.py install" data files are not installed and
> i have no idea why.
> i've been searching for a solution but every combination i've tried failed.
>
> i usually have only html templates so something like:
> package_data = {'mypkg': ['templates/*.html',]}
> works as expected but now i have a folder with different types of
> files and nested folder structure.
>
> my question is:
> - how can i define in setup.py to include folder (let's call it
> "data") and everything in data so that when "python setup.py install"
> is called that folder is copied with python code?
>
> something like "package_data={'mypkg': ['data/*']},"?

How is your project organized precisely ?
Depending on the layout you might need to use the "package_dir" option,

Tarek

>
> Aljosa Mohorovic
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | http://ziade.org

From jvporter at wisc.edu  Thu Feb 11 02:56:45 2010
From: jvporter at wisc.edu (James Porter)
Date: Wed, 10 Feb 2010 19:56:45 -0600
Subject: [Distutils] Distributing (complicated) C extensions
Message-ID: <hkvo4u$2n9$1@ger.gmane.org>

Before I get started with the meat of this message (it's pretty long - 
for me, anyway), I should give a brief overview. I'm developing Python 
bindings for a set of interfaces called ITAPS, which contains a few 
independent libraries. If you want to look at my setup.py file (it's 
pretty scary), you can download the source at 
<http://pypi.python.org/pypi/PyTAPS/0.9.1>.

Basically, I have three somewhat-related questions, which I wrote a big 
pile of words about below. The short version is:

* Are multiple distributions inherently better/saner than one 
distribution with a handful of optional features?
* What's the best way to detect whether libraries exist to turn said 
features on/off?
* How do I install header files when using setuptools (or easy_install)?

Ok, now for the long version:

First, this distribution is fairly complicated, with a handful of 
setuptools Features controlling the various libraries I want to work 
with. I've waffled on whether this should be one distribution or 
several, but the former is a little problematic because the "core" 
module doesn't actually correlate to an independent thing on the C side. 
It really just piggybacks on each of the libraries. So basically there's 
no reason for the "core" module to exist in isolation, and it would 
actually be impossible to compile unless one of the "real" libraries was 
present. Is rolling this all together a terrible idea? Or a good one?

Second, having a single distribution with multiple features that can be 
enabled/disabled obviously requires a bit of work for configuration. 
Right now, it tries to detect if each library is installed, and if not, 
it disables that feature automatically. I mostly do this to make 
easy_install "just work" even if you don't have everything.

I should explain how I do configuration now: each library has a Makefile 
fragment that defines useful variables, e.g. $MYLIB_INCLUDEDIRS, that I 
read in and add into the appropriate places in the Extension. Each of my 
Features has a global --feature-path option that lets you point to the 
Makefile fragment. If you don't specify that, it steps through some 
environment variables and tries to find it. If it still can't find it, 
it'll assume you specified all the stuff you need as options to 
build_ext. Then I try to compile a simple file with the necessary header 
to make sure I got the paths right.

All of this work is done in an overrided version of build_ext (grabbing 
the Makefile fragment happens during finalize_options and the 
compilation test happens during run). This seems pretty hacky to me, 
especially since there's a config command in distutils that does some of 
what I want. However, I'm not sure how to actually *use* this. I assume 
I override the command, but what would it look like from a user's 
perspective? How do I connect it up to the build process?

Finally, I'd like to be able to install header files from my 
distribution. Looking through the distutils docs, there's an 
install_headers command that gets run as a part of install, and I can 
just specify headers=['foo.h'] as an option to setup. However, this 
doesn't seem to work with setuptools/distribute; it looks like they 
hijack the install command and, instead of actually running it directly, 
they manually run install_lib, install_data, etc.

This is made worse by the fact that, even if I stuck with vanilla 
distutils, easy_install appears to inject its version of Distribution 
into my setup.py anyway. (There's a way to work around that, but it's 
really messy. Think yanking the distutils.core Distribution class - 
"_Distribution" in the code - out of setuptools.dist. Barf.)

If you got all the way here, thanks for reading! Hopefully someone with 
a little more experience than me in this can give me a few pointers.

- Jim


From regebro at gmail.com  Thu Feb 11 08:16:08 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 11 Feb 2010 08:16:08 +0100
Subject: [Distutils] Distributing __only__ *.pyc
In-Reply-To: <4957f1ef1002101102m3171b94ja1a2d5364195e944@mail.gmail.com>
References: <201002101835.36517.tmohr@s.netic.de>
	<4957f1ef1002101102m3171b94ja1a2d5364195e944@mail.gmail.com>
Message-ID: <319e029f1002102316ic12b71ehbb73f61d5331f471@mail.gmail.com>

On Wed, Feb 10, 2010 at 20:02, Brad Allen <bradallen137 at gmail.com> wrote:
> On Wed, Feb 10, 2010 at 11:35 AM, Torsten Mohr <tmohr at s.netic.de> wrote:
>> Hi,
>>
>> not sure if this is possible, i'd like to distribute a pure python module as
>> .pyc because there is some code in there that idon't want to spread yet.
>> I plan to release the code as open source later, but it is not possible for
>> now.
>>
>> I know that something like disassembling is possible but for i don't expect
>> that somebody will take that effort.
>>
>>
>> Is it possible to distribute a module.pyc using distutils?
>>
>> How would that setup script look like?
>>
>
> I don't know about with distutils, but setuptools has an
> --exclude-source-files option for creating an egg, used like this:
>
> python setup.py bdist_egg --exclude-source-files

That said, a decompiled Python files is so close to a source file that
you won't prevent anyone from seeing your code anyway.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From ziade.tarek at gmail.com  Thu Feb 11 10:44:59 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 11 Feb 2010 10:44:59 +0100
Subject: [Distutils] Distributing __only__ *.pyc
In-Reply-To: <319e029f1002102316ic12b71ehbb73f61d5331f471@mail.gmail.com>
References: <201002101835.36517.tmohr@s.netic.de>
	<4957f1ef1002101102m3171b94ja1a2d5364195e944@mail.gmail.com>
	<319e029f1002102316ic12b71ehbb73f61d5331f471@mail.gmail.com>
Message-ID: <94bdd2611002110144s6a16c363ybff11815a4c3aada@mail.gmail.com>

On Thu, Feb 11, 2010 at 8:16 AM, Lennart Regebro <regebro at gmail.com> wrote:
> On Wed, Feb 10, 2010 at 20:02, Brad Allen <bradallen137 at gmail.com> wrote:
>> On Wed, Feb 10, 2010 at 11:35 AM, Torsten Mohr <tmohr at s.netic.de> wrote:
>>> Hi,
>>>
>>> not sure if this is possible, i'd like to distribute a pure python module as
>>> .pyc because there is some code in there that idon't want to spread yet.
>>> I plan to release the code as open source later, but it is not possible for
>>> now.
>>>
>>> I know that something like disassembling is possible but for i don't expect
>>> that somebody will take that effort.
>>>
>>>
>>> Is it possible to distribute a module.pyc using distutils?
>>>
>>> How would that setup script look like?
>>>
>>
>> I don't know about with distutils, but setuptools has an
>> --exclude-source-files option for creating an egg, used like this:
>>
>> python setup.py bdist_egg --exclude-source-files
>
> That said, a decompiled Python files is so close to a source file that
> you won't prevent anyone from seeing your code anyway.

Semi-related:
I have met someone at Rupy that is using a crypted egg format, with a
special importer.

They use this to distribute their applications with sensitive code to
their clients (banks).

Tarek
-- 
Tarek Ziad? | http://ziade.org

From floris.bruynooghe at gmail.com  Thu Feb 11 11:42:57 2010
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Thu, 11 Feb 2010 10:42:57 +0000
Subject: [Distutils] Distributing __only__ *.pyc
In-Reply-To: <94bdd2611002110144s6a16c363ybff11815a4c3aada@mail.gmail.com>
References: <201002101835.36517.tmohr@s.netic.de>
	<4957f1ef1002101102m3171b94ja1a2d5364195e944@mail.gmail.com>
	<319e029f1002102316ic12b71ehbb73f61d5331f471@mail.gmail.com>
	<94bdd2611002110144s6a16c363ybff11815a4c3aada@mail.gmail.com>
Message-ID: <20100211104257.GA27622@laurie.devork>

Definitely off-topic by now, sorry

On Thu, Feb 11, 2010 at 10:44:59AM +0100, Tarek Ziad? wrote:
> Semi-related:
> I have met someone at Rupy that is using a crypted egg format, with a
> special importer.

I've wondered about this before but surely you still have the importer
outside of your encrypted archive as well as the key, so in the end
it is only another simple step in the chain to discover the source.


Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From ziade.tarek at gmail.com  Thu Feb 11 11:57:47 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 11 Feb 2010 11:57:47 +0100
Subject: [Distutils] Distributing __only__ *.pyc
In-Reply-To: <20100211104257.GA27622@laurie.devork>
References: <201002101835.36517.tmohr@s.netic.de>
	<4957f1ef1002101102m3171b94ja1a2d5364195e944@mail.gmail.com>
	<319e029f1002102316ic12b71ehbb73f61d5331f471@mail.gmail.com>
	<94bdd2611002110144s6a16c363ybff11815a4c3aada@mail.gmail.com>
	<20100211104257.GA27622@laurie.devork>
Message-ID: <94bdd2611002110257q1e996c76q632e5346065c9345@mail.gmail.com>

On Thu, Feb 11, 2010 at 11:42 AM, Floris Bruynooghe
<floris.bruynooghe at gmail.com> wrote:
> Definitely off-topic by now, sorry
>
> On Thu, Feb 11, 2010 at 10:44:59AM +0100, Tarek Ziad? wrote:
>> Semi-related:
>> I have met someone at Rupy that is using a crypted egg format, with a
>> special importer.
>
> I've wondered about this before but surely you still have the importer
> outside of your encrypted archive as well as the key, so in the end
> it is only another simple step in the chain to discover the source.

I don't know about the protocol in details. I think eggs acts as
crypted plugins that are dropped in a specific folder
users can access, and the rest of the system is unreachable.

But yeah, as long as you have a hand on the interpreter, you can break
it I guess.

I can ask that person if you are interested.

Tarek

-- 
Tarek Ziad? | http://ziade.org

From gustavonarea at 2degreesnetwork.com  Thu Feb 11 18:37:21 2010
From: gustavonarea at 2degreesnetwork.com (Gustavo Narea)
Date: Thu, 11 Feb 2010 17:37:21 +0000
Subject: [Distutils] Buildout: How to include installed packages in sys.path?
Message-ID: <4B744051.30303@2degreesnetwork.com>

Hello.

I've written a Buildout recipe which relies on a function defined in a
package whose name is set in the recipe's options. That package must
have already been installed.

According to the documentation, I should use
zc.buildout.easy_install:install() like this:
    install(["mydistribution"], None)

But if I use None, I get an exception because Buildout tries to use
.startswith() with None. Then, if instead of None, I use
buildout['buildout']['directory'], it tries to install the distribution
pulling all its dependencies (ignoring the fact I set a download cache
in ~/.buildout/default.cfg).

I just want to make a package available in sys.path. How should I do
that from a Buildout recipe?

Note I really just want to add it sys.path, not get the actual
function/module, because it's going to be used by a function called from
my recipe, not by my recipe.

The following code illustrates how the recipe will look like:

====
class MyRecipe(object):
    def __init__(self, buildout, name, options):
        self.the_package = options['package']

    def install(self):
        add_to_sys_path(self.the_package)
        do_something_useful(self.the_package)
        return ()
====

I'm using zc.buildout 1.4.3.

Thanks in advance.

-- 
Gustavo Narea.
Software Developer.
2degrees, Ltd. <http://www.2degreesnetwork.com/>.


From jim at zope.com  Thu Feb 11 23:48:52 2010
From: jim at zope.com (Jim Fulton)
Date: Thu, 11 Feb 2010 17:48:52 -0500
Subject: [Distutils] Buildout: How to include installed packages in
	sys.path?
In-Reply-To: <4B744051.30303@2degreesnetwork.com>
References: <4B744051.30303@2degreesnetwork.com>
Message-ID: <1099b90b1002111448u7b1d5dbqdc0081b089ffef79@mail.gmail.com>

On Thu, Feb 11, 2010 at 12:37 PM, Gustavo Narea
<gustavonarea at 2degreesnetwork.com> wrote:
> Hello.
>
> I've written a Buildout recipe which relies on a function defined in a
> package whose name is set in the recipe's options. That package must
> have already been installed.
>
> According to the documentation, I should use
> zc.buildout.easy_install:install() like this:
> ? ?install(["mydistribution"], None)
>
> But if I use None, I get an exception because Buildout tries to use
> .startswith() with None. Then, if instead of None, I use
> buildout['buildout']['directory'], it tries to install the distribution
> pulling all its dependencies (ignoring the fact I set a download cache
> in ~/.buildout/default.cfg).
>
> I just want to make a package available in sys.path. How should I do
> that from a Buildout recipe?
>
> Note I really just want to add it sys.path, not get the actual
> function/module, because it's going to be used by a function called from
> my recipe, not by my recipe.

The short answer is that buildout doesn't support this. You can do it, but
with a lot of work.  As the documentation for install,
http://pypi.python.org/pypi/zc.buildout#distribution-installation,
says, if the second argument is None, it will fail unless the package
is already installed.  You need to specify the place to
install to.

It might be easier to use the zc.recipe.egg recipe from your recipe:

  http://pypi.python.org/pypi/zc.recipe.egg#egg-recipe-api-for-other-recipes

After using that recipe to install the needed package, you'd need to use
pkg_resources APIs to load the distribution into the running process.

You might also look at the code buildout uses to load recipes.

Jim

-- 
Jim Fulton

From aljosa.mohorovic at gmail.com  Fri Feb 12 04:16:38 2010
From: aljosa.mohorovic at gmail.com (=?UTF-8?B?QWxqb8WhYSBNb2hvcm92acSH?=)
Date: Fri, 12 Feb 2010 04:16:38 +0100
Subject: [Distutils] setup.py#package_data - howto include entire folder
	when installing
In-Reply-To: <94bdd2611002101515s5b8eac5bxbcc5672e684e7aa0@mail.gmail.com>
References: <87d364ab1002101315n634ef9ebn58743c7669c1e800@mail.gmail.com>
	<94bdd2611002101515s5b8eac5bxbcc5672e684e7aa0@mail.gmail.com>
Message-ID: <87d364ab1002111916v61353dc1udc1f17e7c883ea8b@mail.gmail.com>

2010/2/11 Tarek Ziad? <ziade.tarek at gmail.com>:
> How is your project organized precisely ?
> Depending on the layout you might need to use the "package_dir" option,

ROOT
 | - > mypkg/
 | - > data/
 | - > MANIFEST.in
 | - > setup.py

$ cat MANIFEST.in
recursive-include data *.css *.js *.jpg *.gif *.png *.html

$ cat setup.py
#!/usr/bin/env python

from setuptools import setup, find_packages

setup(
    name='test-pkg',
    packages = find_packages(),
    include_package_data = True,
    package_data = {
        '': ['data'],
    },
)

From gustavonarea at 2degreesnetwork.com  Fri Feb 12 12:48:43 2010
From: gustavonarea at 2degreesnetwork.com (Gustavo Narea)
Date: Fri, 12 Feb 2010 11:48:43 +0000
Subject: [Distutils] Buildout: How to include installed packages in
	sys.path?
In-Reply-To: <1099b90b1002111448u7b1d5dbqdc0081b089ffef79@mail.gmail.com>
References: <4B744051.30303@2degreesnetwork.com>
	<1099b90b1002111448u7b1d5dbqdc0081b089ffef79@mail.gmail.com>
Message-ID: <4B75401B.2020801@2degreesnetwork.com>

Hello, Jim.

Thank you very much for your response.

I modified by recipe to use zc.recipe.egg like this:
====
from zc.recipe.egg import Scripts
# ...
egg = Scripts(buildout, name, options)
egg.working_set((distribution_name,))
# ...
do_something_useful(distribution_name)
====

But still do_something_useful() is failing (distribution_name is not in
sys.path) and I think that's because I didn't use pkg_resources as you
suggested. I tried to do it, and read the documentation and source code
for pkg_resources, but I was not able to find the functionality I should
use.

I'll elaborate on what the recipe actually does, just in case: It loads
options from an external INI file into the parts section, so other parts
can use such variables. To be precise, it loads the variables from a
PasteDeploy configuration file.

Here's its code (it's just ~30 LOC):
http://bitbucket.org/2degrees/deployrecipes/src/tip/deployrecipes.py

And here's its documentation:
http://packages.python.org/deployrecipes/

The tricky thing is, these files contain an option which is the string
representation of a function. I don't need to load the actual function,
all I want is to get all those options as strings. But PasteDeploy will
always attempt to resolve that option and thus I must make it available.

I hope it's not that hard to do with zc.recipe.eggs.

Cheers,

 - Gustavo.

On 11/02/10 22:48, Jim Fulton wrote:
> The short answer is that buildout doesn't support this. You can do it, but
> with a lot of work.  As the documentation for install,
> http://pypi.python.org/pypi/zc.buildout#distribution-installation,
> says, if the second argument is None, it will fail unless the package
> is already installed.  You need to specify the place to
> install to.
>
> It might be easier to use the zc.recipe.egg recipe from your recipe:
>
>   http://pypi.python.org/pypi/zc.recipe.egg#egg-recipe-api-for-other-recipes
>
> After using that recipe to install the needed package, you'd need to use
> pkg_resources APIs to load the distribution into the running process.
>
> You might also look at the code buildout uses to load recipes.
>
>   

-- 
Gustavo Narea.
Software Developer.
2degrees, Ltd. <http://www.2degreesnetwork.com/>.


From jim at zope.com  Fri Feb 12 23:49:33 2010
From: jim at zope.com (Jim Fulton)
Date: Fri, 12 Feb 2010 17:49:33 -0500
Subject: [Distutils] Buildout: How to include installed packages in
	sys.path?
In-Reply-To: <4B75401B.2020801@2degreesnetwork.com>
References: <4B744051.30303@2degreesnetwork.com>
	<1099b90b1002111448u7b1d5dbqdc0081b089ffef79@mail.gmail.com> 
	<4B75401B.2020801@2degreesnetwork.com>
Message-ID: <1099b90b1002121449r25b47676scecd581e760e5ae2@mail.gmail.com>

On Fri, Feb 12, 2010 at 6:48 AM, Gustavo Narea
<gustavonarea at 2degreesnetwork.com> wrote:
> Hello, Jim.
>
> Thank you very much for your response.
>
> I modified by recipe to use zc.recipe.egg like this:
> ====
> from zc.recipe.egg import Scripts
> # ...
> egg = Scripts(buildout, name, options)
> egg.working_set((distribution_name,))
> # ...
> do_something_useful(distribution_name)
> ====
>
> But still do_something_useful() is failing (distribution_name is not in
> sys.path) and I think that's because I didn't use pkg_resources as you
> suggested. I tried to do it, and read the documentation and source code
> for pkg_resources, but I was not able to find the functionality I should
> use.


Change your call to working_set to capture the working set:

  requirements, ws = egg.working_set((distribution_name,))

Then:

  for dist in ws:
      pkg_resources.working_set.add(dist)


Jim

--
Jim Fulton

From michal at michalklich.com  Sun Feb 14 19:19:11 2010
From: michal at michalklich.com (=?utf-8?q?Micha=C5=82_Klich?=)
Date: Sun, 14 Feb 2010 19:19:11 +0100
Subject: [Distutils]  Attaching changelog
Message-ID: <201002141919.12065.michal@michalklich.com>

Hi,

I am trying to attach changelog to spec file using --changelog= as a parameter.
My changelog test file has lines starting with (*) but i keep getting

error: %changelog entries must start with *

when building rpm.
Is it a right way to add changelog to spec file?

Thank you for your help

-- 
Micha? Klich

klich.michal at gmail.com
michal at michalklich.com
http://www.michalklich.com

From gustavonarea at 2degreesnetwork.com  Mon Feb 15 12:04:21 2010
From: gustavonarea at 2degreesnetwork.com (Gustavo Narea)
Date: Mon, 15 Feb 2010 11:04:21 +0000
Subject: [Distutils] Buildout: How to include installed packages in
	sys.path?
In-Reply-To: <1099b90b1002121449r25b47676scecd581e760e5ae2@mail.gmail.com>
References: <4B744051.30303@2degreesnetwork.com>
	<1099b90b1002111448u7b1d5dbqdc0081b089ffef79@mail.gmail.com>
	<4B75401B.2020801@2degreesnetwork.com>
	<1099b90b1002121449r25b47676scecd581e760e5ae2@mail.gmail.com>
Message-ID: <4B792A35.5050807@2degreesnetwork.com>

Thank you, Jim. It worked!

 - Gustavo.

On 12/02/10 22:49, Jim Fulton wrote:
> On Fri, Feb 12, 2010 at 6:48 AM, Gustavo Narea
> <gustavonarea at 2degreesnetwork.com> wrote:
>   
>> Hello, Jim.
>>
>> Thank you very much for your response.
>>
>> I modified by recipe to use zc.recipe.egg like this:
>> ====
>> from zc.recipe.egg import Scripts
>> # ...
>> egg = Scripts(buildout, name, options)
>> egg.working_set((distribution_name,))
>> # ...
>> do_something_useful(distribution_name)
>> ====
>>
>> But still do_something_useful() is failing (distribution_name is not in
>> sys.path) and I think that's because I didn't use pkg_resources as you
>> suggested. I tried to do it, and read the documentation and source code
>> for pkg_resources, but I was not able to find the functionality I should
>> use.
>>     
>
> Change your call to working_set to capture the working set:
>
>   requirements, ws = egg.working_set((distribution_name,))
>
> Then:
>
>   for dist in ws:
>       pkg_resources.working_set.add(dist)
>
>
> Jim
>
> --
> Jim Fulton
>   

-- 
Gustavo Narea.
Software Developer.
2degrees, Ltd. <http://www.2degreesnetwork.com/>.


From sklein at cpcug.org  Mon Feb 15 21:31:42 2010
From: sklein at cpcug.org (Stanley A. Klein)
Date: Mon, 15 Feb 2010 15:31:42 -0500 (EST)
Subject: [Distutils] Changing bdist_rpm locations to conform to other rpm
	packaging
Message-ID: <40877.71.163.205.202.1266265902.squirrel@www.cpcug.org>

I'm trying to build rpms for some Python packages located in an svn
repository.  I would prefer to use the same locations for builds and
results as for non-Python rpm packages.

I can set --rpm-base=$HOME/rpmbuild and some of the process happens in
subdirectories of rpmbuild, but in the end, the source rpms are moved from
rpmbuild/SRPMS to a dist directory under the package home, the binary rpms
are moved from rpmbuild/RPMS to the same directory, and the tar.gz are
similarly moved.  I would prefer to have everything created as part of the
build remain in the rpmbuild directory.

How can I prevent files from being stored back in the svn repository as a
result of the package build?


Stan Klein



From manlio.perillo at gmail.com  Tue Feb 16 14:23:48 2010
From: manlio.perillo at gmail.com (Manlio Perillo)
Date: Tue, 16 Feb 2010 14:23:48 +0100
Subject: [Distutils] setuptools and additional scripts handling
Message-ID: <4B7A9C64.1020905@gmail.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi.

I'm starting to use setuptools for my projects and I have a few questions.

In my web applications, usually I have some scripts that needs to be
installed as cron scripts.

I would like to install them using setuptools, since it is important
that the correct python interpreter is used (I use virtualenv), and
setuptools does this automatically..

Is it possible to declare a custom directory where some of the scripts
needs to be installed?


Thanks  Manlio
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkt6nGQACgkQscQJ24LbaUT8QACgg2tkF6vJWWTiBoFnWtr6tb1X
C7sAnjzu/l/ytRMD44STkTiiVJ/uIxmy
=hSnP
-----END PGP SIGNATURE-----

From pje at telecommunity.com  Tue Feb 16 16:04:05 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 16 Feb 2010 10:04:05 -0500
Subject: [Distutils] setuptools and additional scripts handling
In-Reply-To: <4B7A9C64.1020905@gmail.com>
References: <4B7A9C64.1020905@gmail.com>
Message-ID: <20100216150410.5F26E3A4108@sparrow.telecommunity.com>

At 02:23 PM 2/16/2010 +0100, Manlio Perillo wrote:
>Is it possible to declare a custom directory where some of the scripts
>needs to be installed?

No.  However, you can always create a script whose job is to symlink 
your other scripts to the appropriate cron directory, and ask your 
users to run it after installation.


From manlio.perillo at gmail.com  Tue Feb 16 16:37:49 2010
From: manlio.perillo at gmail.com (Manlio Perillo)
Date: Tue, 16 Feb 2010 16:37:49 +0100
Subject: [Distutils] setuptools and additional scripts handling
In-Reply-To: <20100216150410.5F26E3A4108@sparrow.telecommunity.com>
References: <4B7A9C64.1020905@gmail.com>
	<20100216150410.5F26E3A4108@sparrow.telecommunity.com>
Message-ID: <4B7ABBCD.1050500@gmail.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

P.J. Eby ha scritto:
> At 02:23 PM 2/16/2010 +0100, Manlio Perillo wrote:
>> Is it possible to declare a custom directory where some of the scripts
>> needs to be installed?
> 
> No.  However, you can always create a script whose job is to symlink
> your other scripts to the appropriate cron directory, and ask your users
> to run it after installation.
> 

Can't this be done inside setup.py?


Manlio
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkt6u8wACgkQscQJ24LbaUSVjwCfT3zpQmSG+RoTPqrdOMhnRl31
Y2oAnAlJW48TJ+pPRCAz3s4WdeMtKOx+
=/UQx
-----END PGP SIGNATURE-----

From pje at telecommunity.com  Tue Feb 16 19:38:25 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 16 Feb 2010 13:38:25 -0500
Subject: [Distutils] setuptools and additional scripts handling
In-Reply-To: <4B7ABBCD.1050500@gmail.com>
References: <4B7A9C64.1020905@gmail.com>
	<20100216150410.5F26E3A4108@sparrow.telecommunity.com>
	<4B7ABBCD.1050500@gmail.com>
Message-ID: <20100216183829.680B43A4108@sparrow.telecommunity.com>

At 04:37 PM 2/16/2010 +0100, Manlio Perillo wrote:
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>P.J. Eby ha scritto:
> > At 02:23 PM 2/16/2010 +0100, Manlio Perillo wrote:
> >> Is it possible to declare a custom directory where some of the scripts
> >> needs to be installed?
> >
> > No.  However, you can always create a script whose job is to symlink
> > your other scripts to the appropriate cron directory, and ask your users
> > to run it after installation.
> >
>
>Can't this be done inside setup.py?

You could make a setup command, or customize the "install" command, 
but neither of these things would affect easy_install or pip.

If you allow the scripts to be installed to standard locations, then 
they will contain the correct interpreter, and will only need to be 
symlinked to the cron.hourly or cron.daily or whatever else it is 
you're wanting to do.

(Heck, if what you've got is a singleton application of some sort, 
you could potentially just have it check at startup that the cron 
scripts are symlinked, and let the user know about the problem.)


From manlio.perillo at gmail.com  Tue Feb 16 21:11:45 2010
From: manlio.perillo at gmail.com (Manlio Perillo)
Date: Tue, 16 Feb 2010 21:11:45 +0100
Subject: [Distutils] setuptools and additional scripts handling
In-Reply-To: <20100216183829.680B43A4108@sparrow.telecommunity.com>
References: <4B7A9C64.1020905@gmail.com>
	<20100216150410.5F26E3A4108@sparrow.telecommunity.com>
	<4B7ABBCD.1050500@gmail.com>
	<20100216183829.680B43A4108@sparrow.telecommunity.com>
Message-ID: <4B7AFC01.2040108@gmail.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

P.J. Eby ha scritto:
> At 04:37 PM 2/16/2010 +0100, Manlio Perillo wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> P.J. Eby ha scritto:
>> > At 02:23 PM 2/16/2010 +0100, Manlio Perillo wrote:
>> >> Is it possible to declare a custom directory where some of the scripts
>> >> needs to be installed?
>> >
>> > No.  However, you can always create a script whose job is to symlink
>> > your other scripts to the appropriate cron directory, and ask your
>> users
>> > to run it after installation.
>> >
>>
>> Can't this be done inside setup.py?
> 
> You could make a setup command, or customize the "install" command, but
> neither of these things would affect easy_install or pip.
> 
> [...]
> 
> (Heck, if what you've got is a singleton application of some sort, you
> could potentially just have it check at startup that the cron scripts
> are symlinked, and let the user know about the problem.)
> 

You are right, this is a singleton web application!
So, it will not be installed using pip or easy_install.

I have to decide if it is best to "abuse" setuptools plugin support, in
order to install system services (cron scripts, init.d script) or if it
is better to provide a separate script, as you suggested.

And, no, it is not that easy to do checks at startup, since it is a WSGI
application embedded in Nginx (and when I'm on my PC or testing server,
the cron scripts are not required).


Thanks  Manlio

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEUEARECAAYFAkt6/AEACgkQscQJ24LbaUSkxQCYtpARxMvMABorLqgfqYQxpdls
XgCfStNbPH+WeWmwr40IQLqzFUkZ/m4=
=57or
-----END PGP SIGNATURE-----

From pje at telecommunity.com  Wed Feb 17 01:05:32 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 16 Feb 2010 19:05:32 -0500
Subject: [Distutils] setuptools and additional scripts handling
In-Reply-To: <4B7AFC01.2040108@gmail.com>
References: <4B7A9C64.1020905@gmail.com>
	<20100216150410.5F26E3A4108@sparrow.telecommunity.com>
	<4B7ABBCD.1050500@gmail.com>
	<20100216183829.680B43A4108@sparrow.telecommunity.com>
	<4B7AFC01.2040108@gmail.com>
Message-ID: <20100217000538.216623A4108@sparrow.telecommunity.com>

At 09:11 PM 2/16/2010 +0100, Manlio Perillo wrote:
>You are right, this is a singleton web application!
>So, it will not be installed using pip or easy_install.
>
>I have to decide if it is best to "abuse" setuptools plugin support, in
>order to install system services (cron scripts, init.d script) or if it
>is better to provide a separate script, as you suggested.

You can also extend the "install" command, and this could be made to 
work with plain distutils, as well as setuptools.  (Subclassing 
distutils commands is a standard practice, described in the 
documentation for distutils.)  It just means that programs that don't 
use "install" to do the installation won't work correctly, if at all.

However, if you just want "setup.py install" to do things 
differently, subclassing the install command is the way to go.


From r.ritz at biologie.hu-berlin.de  Wed Feb 17 10:00:03 2010
From: r.ritz at biologie.hu-berlin.de (Raphael Ritz)
Date: Wed, 17 Feb 2010 10:00:03 +0100
Subject: [Distutils] setuptools and additional scripts handling
In-Reply-To: <4B7A9C64.1020905@gmail.com>
References: <4B7A9C64.1020905@gmail.com>
Message-ID: <hlgb6h$a5e$2@ger.gmane.org>

Manlio Perillo wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hi.
> 
> I'm starting to use setuptools for my projects and I have a few questions.
> 
> In my web applications, usually I have some scripts that needs to be
> installed as cron scripts.
> 
> I would like to install them using setuptools, since it is important
> that the correct python interpreter is used (I use virtualenv), and
> setuptools does this automatically..
> 
> Is it possible to declare a custom directory where some of the scripts
> needs to be installed?

A different line of thought.

For people adopting buildout

   http://pypi.python.org/pypi/z3c.recipe.usercrontab

could come in handy at times for such use cases.

Raphael



> 
> 
> Thanks  Manlio
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
> 
> iEYEARECAAYFAkt6nGQACgkQscQJ24LbaUT8QACgg2tkF6vJWWTiBoFnWtr6tb1X
> C7sAnjzu/l/ytRMD44STkTiiVJ/uIxmy
> =hSnP
> -----END PGP SIGNATURE-----
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
> 


From manlio.perillo at gmail.com  Wed Feb 17 14:47:33 2010
From: manlio.perillo at gmail.com (Manlio Perillo)
Date: Wed, 17 Feb 2010 14:47:33 +0100
Subject: [Distutils] setuptools and additional scripts handling
In-Reply-To: <hlgb6h$a5e$2@ger.gmane.org>
References: <4B7A9C64.1020905@gmail.com> <hlgb6h$a5e$2@ger.gmane.org>
Message-ID: <4B7BF375.10909@gmail.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Raphael Ritz ha scritto:
> Manlio Perillo wrote:
> Hi.
> 
> I'm starting to use setuptools for my projects and I have a few
> questions.
> 
> In my web applications, usually I have some scripts that needs to be
> installed as cron scripts.
> 
> I would like to install them using setuptools, since it is important
> that the correct python interpreter is used (I use virtualenv), and
> setuptools does this automatically..
> 
> Is it possible to declare a custom directory where some of the scripts
> needs to be installed?
> 
>> A different line of thought.
> 
>> For people adopting buildout
> 
>>   http://pypi.python.org/pypi/z3c.recipe.usercrontab
> 
>> could come in handy at times for such use cases.
> 

Thanks, there is also a Unix deployment recipe.

However, for my web application there is more work to do.

I use Nginx + wsgi module, so I also need to configure Nginx.
The Nginx configuration file is actually a Mako template, since some
parameters (server name, location paths) depends on the deployment type
(development PC, testing server, production server).

These parameters are defined in a configuration file, and during setup
the configuration file needs to be generated.
Should this be done subclassing the config command?

>> Raphael
> 

Manlio
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkt783UACgkQscQJ24LbaUQj5ACfYxFnEs3GUUi8ILccx1nDKLx9
3bgAn0yXdNn8kI92IhPxxI4AE6NHeINR
=aZvH
-----END PGP SIGNATURE-----

From r.ritz at biologie.hu-berlin.de  Wed Feb 17 16:12:46 2010
From: r.ritz at biologie.hu-berlin.de (Raphael Ritz)
Date: Wed, 17 Feb 2010 16:12:46 +0100
Subject: [Distutils] setuptools and additional scripts handling
In-Reply-To: <4B7BF375.10909@gmail.com>
References: <4B7A9C64.1020905@gmail.com> <hlgb6h$a5e$2@ger.gmane.org>
	<4B7BF375.10909@gmail.com>
Message-ID: <hlh11b$jns$1@ger.gmane.org>

Manlio Perillo wrote:
[..]

> 
> However, for my web application there is more work to do.
> 
> I use Nginx + wsgi module, so I also need to configure Nginx.
> The Nginx configuration file is actually a Mako template, since some
> parameters (server name, location paths) depends on the deployment type
> (development PC, testing server, production server).

Again, if that's your use case I encourage you to look into
using buildout. At least I use it myself in situations that seem
similar to yours and I'm happy with it (thanks Jim!).

One of the many nice things about buildout configuration files
is that you can quite flexibly mix-and-match, meaning you can
have a 'base.cfg' which is then extended by other more specific
cfg files, etc. And it is very easy to selective overwrite individual
settings from those more specialized ones.

In case you want to explore that option, here a few more pointers:

buildout: http://pypi.python.org/pypi/zc.buildout

nginx: http://pypi.python.org/pypi/gocept.nginx


A non-trivial example explicitly spelled out:
http://www.martinaspeli.net/articles/an-uber-buildout-for-a-production-plone-server
(some but not all parts in there are Zope/Plone specific
but still it should help getting the big picture)

http://www.martinaspeli.net/articles/new-uber-buildout-with-repoze-and-deliverance
which points to cfgs in
http://svn.plone.org/svn/collective/buildout/uber/plone3.x-repoze/

would be another example illustrating a multi-platform,
development-and-production setup. See in particular
http://svn.plone.org/svn/collective/buildout/uber/plone3.x-repoze/README.txt
for more.

The last example sets up the Zope server clients to run under
'paster' with a WSGI pipeline.

Again, I'm not suggesting to use Plone here but I happen to know
that stuff ;-) and I think the examples illustrate nicely how
one can use buildout to manage somewhat complex systems quite
comfortably. (and I know some people are doing way more crazy
things with it ...)

YMMV,

	Raphael


> 
> These parameters are defined in a configuration file, and during setup
> the configuration file needs to be generated.
> Should this be done subclassing the config command?
> 
>>> Raphael
> 
> Manlio
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
> 
> iEYEARECAAYFAkt783UACgkQscQJ24LbaUQj5ACfYxFnEs3GUUi8ILccx1nDKLx9
> 3bgAn0yXdNn8kI92IhPxxI4AE6NHeINR
> =aZvH
> -----END PGP SIGNATURE-----
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
> 


From assem.ch at gmail.com  Fri Feb 19 01:34:37 2010
From: assem.ch at gmail.com (Assem Chelli)
Date: Fri, 19 Feb 2010 01:34:37 +0100
Subject: [Distutils] include special filetypes and subdirectories
Message-ID: <b67942fb1002181634h67faa9cbq2c09461cb5b49654@mail.gmail.com>

Hi



how to change setup.py to include special filetypes and subdirectories
recursively in python eggs

From assem.ch at gmail.com  Fri Feb 19 01:09:45 2010
From: assem.ch at gmail.com (Assem Chelli)
Date: Fri, 19 Feb 2010 01:09:45 +0100
Subject: [Distutils] dependencies
Message-ID: <b67942fb1002181609o9e9ca6br9915ec40051b7c8d@mail.gmail.com>

Hi,


how can i add the dependencies of my project in setup.py ,

From pje at telecommunity.com  Fri Feb 19 02:04:30 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 18 Feb 2010 20:04:30 -0500
Subject: [Distutils] include special filetypes and subdirectories
In-Reply-To: <b67942fb1002181634h67faa9cbq2c09461cb5b49654@mail.gmail.co
 m>
References: <b67942fb1002181634h67faa9cbq2c09461cb5b49654@mail.gmail.com>
Message-ID: <20100219010438.2329B3A40A0@sparrow.telecommunity.com>

At 01:34 AM 2/19/2010 +0100, Assem Chelli wrote:
>Hi
>
>
>
>how to change setup.py to include special filetypes and subdirectories
>recursively in python eggs


Option 1: Put them under revision control using CVS, Subversion, or 
any revision control system that has a setuptools plugin (in which 
case, you'll need the plugin installed in your development 
environment).  Then, add 'include_package_data=True' in your setup() call.

Option 2: Use the package_data setup() argument (see the distutils 
docs), and explicitly list the files and directories to 
include.  This is not recursive, but will also work with plain distutils.

Option 3: Use a MANIFEST.in file (see the distutils docs) and specify 
the directories to include, and also place 
'include_package_data=True' in your setup() call.

Of these, option 1 is generally the "best" option since it doesn't 
require you to maintain information in two or more places; the files 
are included as a side effect of your ordinary development practices, 
as long as you're using a revision control system.


From ben+python at benfinney.id.au  Fri Feb 19 02:09:36 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 19 Feb 2010 12:09:36 +1100
Subject: [Distutils] include special filetypes and subdirectories
References: <b67942fb1002181634h67faa9cbq2c09461cb5b49654@mail.gmail.com>
Message-ID: <87wrya9gkv.fsf@benfinney.id.au>

Assem Chelli <assem.ch at gmail.com> writes:

> how to change setup.py to include special filetypes and subdirectories
> recursively in python eggs

The manifest file allows specifying globs and recursive includes
<URL:http://docs.python.org/distutils/sourcedist.html#specifying-the-files-to-distribute>.

-- 
 \      ?Reichel's Law: A body on vacation tends to remain on vacation |
  `\            unless acted upon by an outside force.? ?Carol Reichel |
_o__)                                                                  |
Ben Finney


From pje at telecommunity.com  Fri Feb 19 02:17:16 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 18 Feb 2010 20:17:16 -0500
Subject: [Distutils] include special filetypes and subdirectories
In-Reply-To: <b67942fb1002181711w4b11cb32j190445a183799892@mail.gmail.co
 m>
References: <b67942fb1002181634h67faa9cbq2c09461cb5b49654@mail.gmail.com>
	<20100219010438.2329B3A40A0@sparrow.telecommunity.com>
	<b67942fb1002181711w4b11cb32j190445a183799892@mail.gmail.com>
Message-ID: <20100219011723.2E7003A4114@sparrow.telecommunity.com>

At 02:11 AM 2/19/2010 +0100, Assem Chelli wrote:
>i have eclipse with cvs and pydev plugin's  ,how can i do it,or its
>automatically

If your files are under CVS control (in which case you will have a 
bunch of directories named "CVS" in your project), then all you need 
is to add the include_package_data=True in your setup().


>2010/2/19, P.J. Eby <pje at telecommunity.com>:
> > At 01:34 AM 2/19/2010 +0100, Assem Chelli wrote:
> >>Hi
> >>
> >>
> >>
> >>how to change setup.py to include special filetypes and subdirectories
> >>recursively in python eggs
> >
> >
> > Option 1: Put them under revision control using CVS, Subversion, or
> > any revision control system that has a setuptools plugin (in which
> > case, you'll need the plugin installed in your development
> > environment).  Then, add 'include_package_data=True' in your setup() call.
> >
> > Option 2: Use the package_data setup() argument (see the distutils
> > docs), and explicitly list the files and directories to
> > include.  This is not recursive, but will also work with plain distutils.
> >
> > Option 3: Use a MANIFEST.in file (see the distutils docs) and specify
> > the directories to include, and also place
> > 'include_package_data=True' in your setup() call.
> >
> > Of these, option 1 is generally the "best" option since it doesn't
> > require you to maintain information in two or more places; the files
> > are included as a side effect of your ordinary development practices,
> > as long as you're using a revision control system.
> >
> >


From kiorky at cryptelium.net  Fri Feb 19 15:34:48 2010
From: kiorky at cryptelium.net (kiorky)
Date: Fri, 19 Feb 2010 15:34:48 +0100
Subject: [Distutils] upload size limit on pypi
Message-ID: <4B7EA188.20204@cryptelium.net>

Hello, i am again packaging some webapp as a paster template.
Now, it's turn to hudson to join minitage.paste.extras.

The distribution will be around 50MB.

What is the current distribution limit, and if lower, can we again
increase the size ?

-- 
--
Cordialement,
KiOrKY
GPG Key FingerPrint: 0x1A1194B7681112AF




From petrilli at amber.org  Fri Feb 19 16:24:57 2010
From: petrilli at amber.org (Christopher Petrilli)
Date: Fri, 19 Feb 2010 10:24:57 -0500
Subject: [Distutils] upload size limit on pypi
In-Reply-To: <4B7EA188.20204@cryptelium.net>
References: <4B7EA188.20204@cryptelium.net>
Message-ID: <ea68aec1002190724s2b5f205ck4548b76280f403ef@mail.gmail.com>

On Fri, Feb 19, 2010 at 9:34 AM, kiorky <kiorky at cryptelium.net> wrote:
> Hello, i am again packaging some webapp as a paster template.
> Now, it's turn to hudson to join minitage.paste.extras.
>
> The distribution will be around 50MB.
>
> What is the current distribution limit, and if lower, can we again
> increase the size ?

This is just my view, but it seems like 50MB is awfully large to be
uploading into PyPI. For example, Django is 5.4MB. If you have a bunch
of 3rd party dependencies, like hudson, you really should be
downloading those elsewhere, rather than wrapping up a bunch of 3rd
party libraries in your own distribution.

Perhaps you could break down what you've got inside that 50MB and
let's see what we can do to make it more manageable?

Chris
-- 
| Chris Petrilli
| petrilli at amber.org

From kiorky at cryptelium.net  Fri Feb 19 16:55:56 2010
From: kiorky at cryptelium.net (kiorky)
Date: Fri, 19 Feb 2010 16:55:56 +0100
Subject: [Distutils] upload size limit on pypi
In-Reply-To: <ea68aec1002190724s2b5f205ck4548b76280f403ef@mail.gmail.com>
References: <4B7EA188.20204@cryptelium.net>
	<ea68aec1002190724s2b5f205ck4548b76280f403ef@mail.gmail.com>
Message-ID: <4B7EB48C.1060002@cryptelium.net>

Christopher Petrilli a ?crit :
> On Fri, Feb 19, 2010 at 9:34 AM, kiorky <kiorky at cryptelium.net> wrote:
>   
>> Hello, i am again packaging some webapp as a paster template.
>> Now, it's turn to hudson to join minitage.paste.extras.
>>
>> The distribution will be around 50MB.
>>
>> What is the current distribution limit, and if lower, can we again
>> increase the size ?
>>     
>
> This is just my view, but it seems like 50MB is awfully large to be
> uploading into PyPI. For example, Django is 5.4MB. If you have a bunch
> of 3rd party dependencies, like hudson, you really should be
> downloading those elsewhere, rather than wrapping up a bunch of 3rd
> party libraries in your own distribution.
>   
Yes i know, but i am packaging them, and from the packager pov it's more
simple and i think the best way to to do as it is done right now.
There is no big files but a lot of files more or less large. Im not
dropping hudson as is but configuring things inside the exploded war
when you have asked the paster questions and making wrappers, init
scripts, logrotates & co around, it's a real integration.
I can upload them elsewhere of course but if it can be hosted on pypi i
would have prefered.

> Perhaps you could break down what you've got inside that 50MB and
> let's see what we can do to make it more manageable?
>
> Chris
>   


-- 
--
Cordialement,
KiOrKY
GPG Key FingerPrint: 0x1A1194B7681112AF




From kiorky at cryptelium.net  Fri Feb 19 17:00:00 2010
From: kiorky at cryptelium.net (kiorky)
Date: Fri, 19 Feb 2010 17:00:00 +0100
Subject: [Distutils] upload size limit on pypi
In-Reply-To: <ea68aec1002190724s2b5f205ck4548b76280f403ef@mail.gmail.com>
References: <4B7EA188.20204@cryptelium.net>
	<ea68aec1002190724s2b5f205ck4548b76280f403ef@mail.gmail.com>
Message-ID: <4B7EB580.5000308@cryptelium.net>

And a word around minitage.paste.extras, The thing we like here is to do
here is:
For example for hudson,

paster create -t minitage.instances.hudson myproject (inside_minitage=no
ifyou don't use minitage)
paster create -t minitage.instances.hudson myproject (inside_minitage=no
ifyou don't use minitage)

Will create for you a configured tomcat with a running hudson instance
inside with lot of things configured for you (ssl certificates, init
scripts, logrotate, irc/email notifications and so on in hudson)

-- 
--
Cordialement,
KiOrKY
GPG Key FingerPrint: 0x1A1194B7681112AF




From setuptools at bugs.python.org  Mon Feb 22 17:16:37 2010
From: setuptools at bugs.python.org (Tres Seaver)
Date: Mon, 22 Feb 2010 16:16:37 +0000
Subject: [Distutils] [issue102] setuptools-0.6c11 regression
	in	package_index.PackageIndex.fetch_distribution
In-Reply-To: <1266855397.07.0.421511661273.issue102@psf.upfronthosting.co.za>
Message-ID: <1266855397.07.0.421511661273.issue102@psf.upfronthosting.co.za>


New submission from Tres Seaver <tseaver at palladion.com>:

package_index.PackageIndex.fetch_distribution changed between 0.6c9 and 0.6c11, adding some logging, and returning a clone of the found distribution.  However,
the new code neglects to check that the dist is not None before cloning it.

The attached patch fixes this bug.

----------
assignedto: pje
files: setuptools-0.6c11-find_distribution_miss.patch
messages: 504
nosy: pje, tseaver
priority: bug
status: unread
title: setuptools-0.6c11 regression in package_index.PackageIndex.fetch_distribution
Added file: http://bugs.python.org/setuptools/file66/setuptools-0.6c11-find_distribution_miss.patch

_______________________________________________
Setuptools tracker <setuptools at bugs.python.org>
<http://bugs.python.org/setuptools/issue102>
_______________________________________________
-------------- next part --------------
A non-text attachment was scrubbed...
Name: setuptools-0.6c11-find_distribution_miss.patch
Type: text/x-diff
Size: 873 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20100222/4efbc160/attachment.patch>

From jeandaniel.browne at gmail.com  Mon Feb 22 17:20:01 2010
From: jeandaniel.browne at gmail.com (Jean Daniel)
Date: Mon, 22 Feb 2010 17:20:01 +0100
Subject: [Distutils] using distribute instead of distutils
Message-ID: <b61c19e71002220820o15617d56xfc7e34b51e18d212@mail.gmail.com>

Hello,

I am writing a pure python module called wordish, which is similar to
doctest in that it test snippets of code, parsed from an rst article
and produce a report. wordish is available on Pypi in version 1.0 and
the documentation is available at jdb.github.com/wordish.

So far, I only distributed one file, *wordish.py* and the file was used
like the following::

?? ?python -m wordish my_article.rst

I think I will turn to 'distribute' for the future release so that
console entry points can be created automatically and installed in the
shell path. I would like wordish to be used in this way::

?? ?wordish my_article.rst

I have read the documentation for distutils, distribute but still, but the
overlap that the tools, and the problems that each tool solve. Can
my setup.py support the installation of the following files, or should they
be included in debian/fedora packages instead?

1. few examples files?

2. static html documentation files?

3. man pages for the console scripts?

Also, I upload a source distribution (sdist) to the Python Package
Index

4. should I use an *egg* instead? what does it change for me or
    the users?


Thanks for your help,

From daniel.niccoli at gmail.com  Sun Feb 21 04:13:27 2010
From: daniel.niccoli at gmail.com (Daniel Niccoli)
Date: Sun, 21 Feb 2010 04:13:27 +0100
Subject: [Distutils] setuptools-0.6c11.win32-py2.6.exe
Message-ID: <b9c13f131002201913sa938ed7i70a0643b94fbf65f@mail.gmail.com>

Hello Phillip,

if you install the x86-64 version of python, the windows installer "*
setuptools-0.6c11.win32-py2.6.exe"* does not find any Python installation. I
had to export the registry key "HKEY_LOCAL_MACHINE\SOFTWARE\Python", change
it to "HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Python" and import it
afterwards to be able to install the software. The installer should be
changed to avoid trouble for other people.

Kind regards

Daniel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20100221/71b0fc21/attachment.html>

From pje at telecommunity.com  Mon Feb 22 20:33:09 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 22 Feb 2010 14:33:09 -0500
Subject: [Distutils] using distribute instead of distutils
In-Reply-To: <b61c19e71002220820o15617d56xfc7e34b51e18d212@mail.gmail.co
 m>
References: <b61c19e71002220820o15617d56xfc7e34b51e18d212@mail.gmail.com>
Message-ID: <20100222193304.DB09C3A406A@sparrow.telecommunity.com>

At 05:20 PM 2/22/2010 +0100, Jean Daniel wrote:
>Hello, I am writing a pure python module called wordish, which is 
>similar to doctest in that it test snippets of code, parsed from an 
>rst article and produce a report. wordish is available on Pypi in 
>version 1.0 and the documentation is available at 
>jdb.github.com/wordish. So far, I only distributed one file, 
>*wordish.py* and the file was used like the following:: ? ?  ? 
>python -m wordish my_article.rst I think I will turn to 'distribute' 
>for the future release so that console entry points can be created 
>automatically and installed in the shell path. I would like wordish 
>to be used in this way:: ? ?  ? wordish my_article.rst I have read 
>the documentation for distutils, distribute but still, but the 
>overlap that the tools, and the problems that each tool solve. Can 
>my setup.py support the installation of the following files, or 
>should they be included in debian/fedora packages instead? 1. few 
>examples files? 2. static html documentation files? 3. man pages for 
>the console scripts?

The distutils doesn't really support installing such things well; you 
can certainly include them in your *source* distribution, though.


>Also, I upload a source distribution (sdist) to the Python Package 
>Index 4. should I use an *egg* instead?

No, it's not necessary.  Eggs are mainly useful as a drop-in-and-use 
format for application plugins (such as extensions to Chandler, Trac, 
and other plugin-enabled application environments), and distributing 
precompiled C extensions on certain platforms.  easy_install and pip 
can both operate just fine on source distributions, and if your code 
is only pure Python, you don't need to distribute binaries.


From pje at telecommunity.com  Mon Feb 22 20:37:00 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 22 Feb 2010 14:37:00 -0500
Subject: [Distutils] setuptools-0.6c11.win32-py2.6.exe
In-Reply-To: <b9c13f131002201913sa938ed7i70a0643b94fbf65f@mail.gmail.com
 >
References: <b9c13f131002201913sa938ed7i70a0643b94fbf65f@mail.gmail.com>
Message-ID: <20100222193655.A6CB13A415F@sparrow.telecommunity.com>

At 04:13 AM 2/21/2010 +0100, Daniel Niccoli wrote:
>Hello Phillip,
>
>if you install the x86-64 version of python, the windows installer 
>"setuptools-0.6c11.win32-py2.6.exe" does not find any Python 
>installation. I had to export the registry key 
>"HKEY_LOCAL_MACHINE\SOFTWARE\Python", change it to 
>"HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Python" and import it 
>afterwards to be able to install the software. The installer should 
>be changed to avoid trouble for other people.


This is a *Python* installer issue; Please see:

   http://bugs.python.org/issue6792


Until the issue in Python itself is fixed, the only things I can do 
are document the problem and/or stop distributing an .exe installer. 


From jeandaniel.browne at gmail.com  Mon Feb 22 20:49:09 2010
From: jeandaniel.browne at gmail.com (Jean Daniel)
Date: Mon, 22 Feb 2010 20:49:09 +0100
Subject: [Distutils] using distribute instead of distutils
In-Reply-To: <20100222193304.DB09C3A406A@sparrow.telecommunity.com>
References: <b61c19e71002220820o15617d56xfc7e34b51e18d212@mail.gmail.com>
	<20100222193304.DB09C3A406A@sparrow.telecommunity.com>
Message-ID: <b61c19e71002221149u3764dec9l1b0f02c378339605@mail.gmail.com>

>> Can my setup.py support the
>> installation of the following files, or should they be included in
>> debian/fedora packages instead?
>> 1. few examples files?
>> 2. static html documentation files?
>> 3. man pages for the console scripts?
>
> The distutils doesn't really support installing such things well; you can
> certainly include them in your *source* distribution, though.
>
>
>> Also, I upload a source distribution (sdist) to the Python Package Index
>> 4. should I use an *egg* instead?
>
> No, it's not necessary. ?Eggs are mainly useful as a drop-in-and-use format
> for application plugins (such as extensions to Chandler, Trac, and other
> plugin-enabled application environments), and distributing precompiled C
> extensions on certain platforms. ?easy_install and pip can both operate just
> fine on source distributions, and if your code is only pure Python, you
> don't need to distribute binaries.

Thank you, I got it.

From martin at v.loewis.de  Mon Feb 22 21:59:18 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 22 Feb 2010 21:59:18 +0100
Subject: [Distutils] setuptools-0.6c11.win32-py2.6.exe
In-Reply-To: <20100222193655.A6CB13A415F@sparrow.telecommunity.com>
References: <b9c13f131002201913sa938ed7i70a0643b94fbf65f@mail.gmail.com>
	<20100222193655.A6CB13A415F@sparrow.telecommunity.com>
Message-ID: <4B82F026.50907@v.loewis.de>

> This is a *Python* installer issue; Please see:
> 
>   http://bugs.python.org/issue6792
> 
> 
> Until the issue in Python itself is fixed, the only things I can do are
> document the problem and/or stop distributing an .exe installer.

You could also distribute two versions of the installer (a 32-bit
version, and a 64-bit version).

Regards,
Martin

From setuptools at bugs.python.org  Tue Feb 23 00:48:51 2010
From: setuptools at bugs.python.org (Tres Seaver)
Date: Mon, 22 Feb 2010 23:48:51 +0000
Subject: [Distutils] [issue103] develop command needs a way to install extra
	requirements
In-Reply-To: <1266882531.4.0.246872311842.issue103@psf.upfronthosting.co.za>
Message-ID: <1266882531.4.0.246872311842.issue103@psf.upfronthosting.co.za>


New submission from Tres Seaver <tseaver at palladion.com>:

When running 'python setup.py develop', it would be useful to install one (or
more) of the 'extra' requirements for the package-under-development.  The
attached patch allows installing one at a time:  I don't have the options fu to
make it a repeatable option.

----------
assignedto: pje
files: setuptools-develop_w_extra.patch
messages: 506
nosy: pje, tseaver
priority: feature
status: unread
title: develop command needs a way to install extra requirements
Added file: http://bugs.python.org/setuptools/file67/setuptools-develop_w_extra.patch

_______________________________________________
Setuptools tracker <setuptools at bugs.python.org>
<http://bugs.python.org/setuptools/issue103>
_______________________________________________
-------------- next part --------------
A non-text attachment was scrubbed...
Name: setuptools-develop_w_extra.patch
Type: text/x-diff
Size: 1983 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20100222/14fadc0f/attachment.patch>

From theller at ctypes.org  Tue Feb 23 20:55:11 2010
From: theller at ctypes.org (Thomas Heller)
Date: Tue, 23 Feb 2010 20:55:11 +0100
Subject: [Distutils] from distutils.core import DistutilsOptionError
Message-ID: <hm1bqv$hm5$1@dough.gmane.org>

One of my setup scripts contains:

  from distutils.core import DistutilsOptionError

which doesn't work anymore with Python trunk.  Bug in Python,
or in my setup script?


-- 
Thanks,
Thomas


From ziade.tarek at gmail.com  Tue Feb 23 22:14:18 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 23 Feb 2010 16:14:18 -0500
Subject: [Distutils] from distutils.core import DistutilsOptionError
In-Reply-To: <hm1bqv$hm5$1@dough.gmane.org>
References: <hm1bqv$hm5$1@dough.gmane.org>
Message-ID: <94bdd2611002231314sb3c5277j424f5874582cad87@mail.gmail.com>

On Tue, Feb 23, 2010 at 2:55 PM, Thomas Heller <theller at ctypes.org> wrote:
> One of my setup scripts contains:
>
> ?from distutils.core import DistutilsOptionError
>
> which doesn't work anymore with Python trunk. ?Bug in Python,
> or in my setup script?

This name was imported in distutils.core but never used, so the import
line was removed in trunk, since importing it from core is considered
as a mistake.

you should do :

  from distutils.errors import DistutilsOptionError


Tarek
-- 
Tarek Ziad? | http://ziade.org

From guido at python.org  Tue Feb 23 23:04:48 2010
From: guido at python.org (Guido van Rossum)
Date: Tue, 23 Feb 2010 14:04:48 -0800
Subject: [Distutils] from distutils.core import DistutilsOptionError
In-Reply-To: <94bdd2611002231314sb3c5277j424f5874582cad87@mail.gmail.com>
References: <hm1bqv$hm5$1@dough.gmane.org>
	<94bdd2611002231314sb3c5277j424f5874582cad87@mail.gmail.com>
Message-ID: <ca471dc21002231404l1e440aa4k185a75c6ab6636e7@mail.gmail.com>

On Tue, Feb 23, 2010 at 1:14 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> On Tue, Feb 23, 2010 at 2:55 PM, Thomas Heller <theller at ctypes.org> wrote:
>> One of my setup scripts contains:
>>
>> ?from distutils.core import DistutilsOptionError
>>
>> which doesn't work anymore with Python trunk. ?Bug in Python,
>> or in my setup script?
>
> This name was imported in distutils.core but never used, so the import
> line was removed in trunk, since importing it from core is considered
> as a mistake.
>
> you should do :
>
> ?from distutils.errors import DistutilsOptionError

But you see the problem with "improving" distutils... That setup.py
script worked before. I recommend restoring as much as possible of
distutils in the trunk to its state as of 2.6. (I think some 2.6 micro
releases also broke some distros, but IIRC that was fixed in the
latest.)

-- 
--Guido van Rossum (python.org/~guido)

From ziade.tarek at gmail.com  Tue Feb 23 23:32:57 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 23 Feb 2010 17:32:57 -0500
Subject: [Distutils] from distutils.core import DistutilsOptionError
In-Reply-To: <ca471dc21002231404l1e440aa4k185a75c6ab6636e7@mail.gmail.com>
References: <hm1bqv$hm5$1@dough.gmane.org>
	<94bdd2611002231314sb3c5277j424f5874582cad87@mail.gmail.com>
	<ca471dc21002231404l1e440aa4k185a75c6ab6636e7@mail.gmail.com>
Message-ID: <94bdd2611002231432x3a692bbawdddea86f30683a95@mail.gmail.com>

On Tue, Feb 23, 2010 at 5:04 PM, Guido van Rossum <guido at python.org> wrote:
> On Tue, Feb 23, 2010 at 1:14 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>> On Tue, Feb 23, 2010 at 2:55 PM, Thomas Heller <theller at ctypes.org> wrote:
>>> One of my setup scripts contains:
>>>
>>> ?from distutils.core import DistutilsOptionError
>>>
>>> which doesn't work anymore with Python trunk. ?Bug in Python,
>>> or in my setup script?
>>
>> This name was imported in distutils.core but never used, so the import
>> line was removed in trunk, since importing it from core is considered
>> as a mistake.
>>
>> you should do :
>>
>> ?from distutils.errors import DistutilsOptionError
>
> But you see the problem with "improving" distutils... That setup.py
> script worked before. I recommend restoring as much as possible of
> distutils in the trunk to its state as of 2.6.
 > (I think some 2.6 micro
> releases also broke some distros, but IIRC that was fixed in the
> latest.)

Yes,

Two things here:

1/ the micro release change didn't change the public API but private
elements + a behavior that was not clearly defined in the
documentation. I have/am not chang[ed|ing] anything public in 2.6.x
either added new feature in there.

2/ the work done in 2.7 so far was following the usual deprecation
process for relocated
APIs or removed APIs, but nothing for misplaced import statements. We
did discuss this in the past and said it was fine this way.

*Now*, since we are working in distutils2 I am proposing this strategy:

- I completely revert Distutils 2.7 to the latest 2.6.x state, so
people can safely rely on it when 2.7 comes out.

- distutils at sdtlib is feature-frozen and I don't do anything else but
bug fixes in there,
  that I apply in 2.7 and 2.6 the same way. Any new feature request is
redirected to distutils2.

- we keep the new sysconfig module and use it in distutils2 and in site.py.

we are planning to release a first version of distutils2 by the time
2.7 final is out btw.

Regards,
Tarek

-- 
Tarek Ziad? | http://ziade.org

From pje at telecommunity.com  Tue Feb 23 23:52:28 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 23 Feb 2010 17:52:28 -0500
Subject: [Distutils] from distutils.core import DistutilsOptionError
In-Reply-To: <94bdd2611002231432x3a692bbawdddea86f30683a95@mail.gmail.co
 m>
References: <hm1bqv$hm5$1@dough.gmane.org>
	<94bdd2611002231314sb3c5277j424f5874582cad87@mail.gmail.com>
	<ca471dc21002231404l1e440aa4k185a75c6ab6636e7@mail.gmail.com>
	<94bdd2611002231432x3a692bbawdddea86f30683a95@mail.gmail.com>
Message-ID: <20100223225227.840BD3A411C@sparrow.telecommunity.com>

At 05:32 PM 2/23/2010 -0500, Tarek Ziad? wrote:
>2/ the work done in 2.7 so far was following the usual deprecation
>process for relocated
>APIs or removed APIs, but nothing for misplaced import statements. We
>did discuss this in the past and said it was fine this way.

Btw, imports from distutils.core *are* API; its docstring describes 
it as "The only module that needs to be imported to use the 
Distutils".  It's supposed to be a one-stop shop for importing common 
things needed by setup scripts, including the errors, Extension, 
Command, Distribution, etc.

So, the unused imports in distutils.core are not "misplaced import 
statements", nor is there a good reason (AFAICT) for changing the API 
to require the average setup script author to remember which module 
to import things from.  There should be only one obvious way to do 
it, and right now, the one obvious way to import things in a setup 
script is from distutils.core.


From ziade.tarek at gmail.com  Wed Feb 24 00:05:01 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 23 Feb 2010 18:05:01 -0500
Subject: [Distutils] from distutils.core import DistutilsOptionError
In-Reply-To: <20100223225227.840BD3A411C@sparrow.telecommunity.com>
References: <hm1bqv$hm5$1@dough.gmane.org>
	<94bdd2611002231314sb3c5277j424f5874582cad87@mail.gmail.com>
	<ca471dc21002231404l1e440aa4k185a75c6ab6636e7@mail.gmail.com>
	<94bdd2611002231432x3a692bbawdddea86f30683a95@mail.gmail.com>
	<20100223225227.840BD3A411C@sparrow.telecommunity.com>
Message-ID: <94bdd2611002231505q562102bfuced03cf43db99b2b@mail.gmail.com>

2010/2/23 P.J. Eby <pje at telecommunity.com>:
> At 05:32 PM 2/23/2010 -0500, Tarek Ziad? wrote:
>>
>> 2/ the work done in 2.7 so far was following the usual deprecation
>> process for relocated
>> APIs or removed APIs, but nothing for misplaced import statements. We
>> did discuss this in the past and said it was fine this way.
>
> Btw, imports from distutils.core *are* API; its docstring describes it as
> "The only module that needs to be imported to use the Distutils". ?It's
> supposed to be a one-stop shop for importing common things needed by setup
> scripts, including the errors, Extension, Command, Distribution, etc.

Yes exactly, here's the full docstring:

"""
The only module that needs to be imported to use the Distutils; provides
the 'setup' function (which is to be called from the setup script).  Also
indirectly provides the Distribution and Command classes, although they are
really defined in distutils.dist and distutils.cmd.
"""

And if you look at the code you will see the list of local imports
done on purpose (that I have
kept of course):

  # Mainly import these so setup scripts can "from distutils.core import" them.

So, no exception classes here, which *are* in effect a misplaced
import statement.

Or do you mean that you consider the exception classes located in
Distutils to be a common need
for people that write setup.py scripts ?

I don't think for example that this line that you will find in Distutils 2.6:

   from types import *

Is something we want to keep either. But if the process I've proposed
is accepted then there will
be no more problem.

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org

From pje at telecommunity.com  Wed Feb 24 02:28:42 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 23 Feb 2010 20:28:42 -0500
Subject: [Distutils] from distutils.core import DistutilsOptionError
In-Reply-To: <94bdd2611002231505q562102bfuced03cf43db99b2b@mail.gmail.co
 m>
References: <hm1bqv$hm5$1@dough.gmane.org>
	<94bdd2611002231314sb3c5277j424f5874582cad87@mail.gmail.com>
	<ca471dc21002231404l1e440aa4k185a75c6ab6636e7@mail.gmail.com>
	<94bdd2611002231432x3a692bbawdddea86f30683a95@mail.gmail.com>
	<20100223225227.840BD3A411C@sparrow.telecommunity.com>
	<94bdd2611002231505q562102bfuced03cf43db99b2b@mail.gmail.com>
Message-ID: <20100224012840.A0ABE3A411E@sparrow.telecommunity.com>

At 06:05 PM 2/23/2010 -0500, Tarek Ziad? wrote:
>Or do you mean that you consider the exception classes located in
>Distutils to be a common need
>for people that write setup.py scripts ?

A setup script may want to explicitly throw (or catch) distutils 
errors, and having only one place to import these things from makes 
distutils easier to use.  I don't consider it a *common* need, but I 
do consider it part of distutils' "core API", in the sense that if 
your code is either invoking distutils or being invoked by it, you 
may need to throw or catch those errors.

In general, everything that's a *distutils-defined* symbol (not 
types, os, etc.) in distutils.core, I thought was intended to be 
imported from there, as they are all things that could be considered 
part of the "core API".  (And apparently, at least Thomas thought the 
same thing.)

(If I'd been writing distutils, I'd have named the module 
distutils.api rather than distutils.core, and explicitly created an 
__all__ list for it, but it's pretty much the same thing.)


From ziade.tarek at gmail.com  Wed Feb 24 02:54:16 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 23 Feb 2010 20:54:16 -0500
Subject: [Distutils] from distutils.core import DistutilsOptionError
In-Reply-To: <20100224012840.A0ABE3A411E@sparrow.telecommunity.com>
References: <hm1bqv$hm5$1@dough.gmane.org>
	<94bdd2611002231314sb3c5277j424f5874582cad87@mail.gmail.com>
	<ca471dc21002231404l1e440aa4k185a75c6ab6636e7@mail.gmail.com>
	<94bdd2611002231432x3a692bbawdddea86f30683a95@mail.gmail.com>
	<20100223225227.840BD3A411C@sparrow.telecommunity.com>
	<94bdd2611002231505q562102bfuced03cf43db99b2b@mail.gmail.com>
	<20100224012840.A0ABE3A411E@sparrow.telecommunity.com>
Message-ID: <94bdd2611002231754o13fd4ec3y69ff9f19217b3c75@mail.gmail.com>

On Tue, Feb 23, 2010 at 8:28 PM, P.J. Eby <pje at telecommunity.com> wrote:
> At 06:05 PM 2/23/2010 -0500, Tarek Ziad? wrote:
>>
>> Or do you mean that you consider the exception classes located in
>> Distutils to be a common need
>> for people that write setup.py scripts ?
>
> A setup script may want to explicitly throw (or catch) distutils errors, and
> having only one place to import these things from makes distutils easier to
> use. ?I don't consider it a *common* need, but I do consider it part of
> distutils' "core API", in the sense that if your code is either invoking
> distutils or being invoked by it, you may need to throw or catch those
> errors.
>
> In general, everything that's a *distutils-defined* symbol (not types, os,
> etc.) in distutils.core, I thought was intended to be imported from there,
> as they are all things that could be considered part of the "core API".
> ?(And apparently, at least Thomas thought the same thing.)
> (If I'd been writing distutils, I'd have named the module distutils.api
> rather than distutils.core, and explicitly created an __all__ list for it,
> but it's pretty much the same thing.)

Ok. I don't think this was the intent,  (and from googling, it looks
like most ppl are using distutils.errors), but you are right : __all__
should have been used here in the first place.

I will follow your advice in distutils2, and probably make the call as
simple as setuptools':

    from distutils2 import setup

(if we keep setup.py which is not sure at all)

Regards,
Tarek

From viktor.nagy at gmail.com  Wed Feb 24 12:49:28 2010
From: viktor.nagy at gmail.com (}--o)
Date: Wed, 24 Feb 2010 12:49:28 +0100
Subject: [Distutils] how to use mr.developer?
Message-ID: <8f75d251002240349u20bc481bue264c086592bde34@mail.gmail.com>

hi,

I'm just experimenting with distutils, and would like to move my django apps
under it.
One of my apps uses django-sugar that should be checked out from github.

bin/buildout check out my django-sugar repo under src/, and creates an
egg-link under develop-eggs, but it's not added to sys.path for the bin/*
binaries.
Is there a simple way to do it? I've seen the extra_paths variable, but then
I still have to write each ${buildout:auto-checkout} line again.

this is my buildout.cfg file:

[buildout]
parts = python django-1.1
develop = .
eggs =
    django-contacts
    django-l10n
    django-registration
eggs-directory = /home/nagyv/.buildout/eggs
find-links = ${buildout:eggs-directory}
download-cache = /home/nagyv/.buildout/dlcache

extensions = mr.developer
sources = sources
auto-checkout =
    uni-form
    sugar

[python]
recipe = zc.recipe.egg
interpreter = python
eggs = ${buildout:eggs}

[django-1.1]
recipe = djangorecipe
version = 1.1.1
project = contacts
projectegg = contacts
settings = testsettings
test = contacts
testrunner = test-1.1
eggs = ${buildout:eggs}

[sources]
uni-form = git git://github.com/nagyv/django-uni-form.git
sugar = git git://github.com/nagyv/django-sugar.git

thanks for your help!
Viktor
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20100224/2c12105b/attachment.html>

From viktor.nagy at gmail.com  Wed Feb 24 11:44:33 2010
From: viktor.nagy at gmail.com (}--o)
Date: Wed, 24 Feb 2010 11:44:33 +0100
Subject: [Distutils] how to use mr.developer?
Message-ID: <8f75d251002240244y6e0cb14cg96fae87b1dbc2499@mail.gmail.com>

hi,

I'm just experimenting with distutils, and would like to move my django apps
under it.
One of my apps uses django-sugar that should be checked out from github.

bin/buildout check out my django-sugar repo under src/, and creates an
egg-link under develop-eggs, but it's not added to sys.path for the bin/*
binaries.
Is there a simple way to do it? I've seen the extra_paths variable, but then
I still have to write each ${buildout:auto-checkout} line again.

this is my buildout.cfg file:

[buildout]
parts = python django-1.1
develop = .
eggs =
    django-contacts
    django-l10n
    django-registration
eggs-directory = /home/nagyv/.buildout/eggs
find-links = ${buildout:eggs-directory}
download-cache = /home/nagyv/.buildout/dlcache

extensions = mr.developer
sources = sources
auto-checkout =
    uni-form
    sugar

[python]
recipe = zc.recipe.egg
interpreter = python
eggs = ${buildout:eggs}

[django-1.1]
recipe = djangorecipe
version = 1.1.1
project = contacts
projectegg = contacts
settings = testsettings
test = contacts
testrunner = test-1.1
eggs = ${buildout:eggs}

[sources]
uni-form = git git://github.com/nagyv/django-uni-form.git
sugar = git git://github.com/nagyv/django-sugar.git

thanks for your help!
Viktor
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20100224/b4c3ff5d/attachment.html>

From reinout at vanrees.org  Wed Feb 24 15:40:16 2010
From: reinout at vanrees.org (Reinout van Rees)
Date: Wed, 24 Feb 2010 15:40:16 +0100
Subject: [Distutils] setuptools and additional scripts handling
In-Reply-To: <4B7AFC01.2040108@gmail.com>
References: <4B7A9C64.1020905@gmail.com>	<20100216150410.5F26E3A4108@sparrow.telecommunity.com>	<4B7ABBCD.1050500@gmail.com>	<20100216183829.680B43A4108@sparrow.telecommunity.com>
	<4B7AFC01.2040108@gmail.com>
Message-ID: <hm3doh$r6e$1@dough.gmane.org>

On 02/16/2010 09:11 PM, Manlio Perillo wrote:
>
> You are right, this is a singleton web application!
> So, it will not be installed using pip or easy_install.

Ah, then you also won't be using buildout, I'd guess :-)

If you had, you could have used 
http://pypi.python.org/pypi/z3c.recipe.usercrontab

But also that recipe just points cron at one or more python scripts 
somewhere.


Reinout

-- 
Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org
Programmer at http://www.nelen-schuurmans.nl
"Military engineers build missiles. Civil engineers build targets"


From kiorky at cryptelium.net  Wed Feb 24 17:52:31 2010
From: kiorky at cryptelium.net (kiorky)
Date: Wed, 24 Feb 2010 17:52:31 +0100
Subject: [Distutils] how to use mr.developer?
In-Reply-To: <8f75d251002240349u20bc481bue264c086592bde34@mail.gmail.com>
References: <8f75d251002240349u20bc481bue264c086592bde34@mail.gmail.com>
Message-ID: <4B85594F.5070201@cryptelium.net>

add django-sugar & uni-form to buildout:eggs.

Le 24/02/2010 12:49, }--o a ?crit :
> hi,
> 
> I'm just experimenting with distutils, and would like to move my django
> apps under it.
> One of my apps uses django-sugar that should be checked out from github.
> 
> bin/buildout check out my django-sugar repo under src/, and creates an
> egg-link under develop-eggs, but it's not added to sys.path for the
> bin/* binaries.
> Is there a simple way to do it? I've seen the extra_paths variable, but
> then I still have to write each ${buildout:auto-checkout} line again.
> 
> this is my buildout.cfg file:
> 
> [buildout]
> parts = python django-1.1
> develop = .
> eggs =
>     django-contacts
>     django-l10n
>     django-registration
> eggs-directory = /home/nagyv/.buildout/eggs
> find-links = ${buildout:eggs-directory}
> download-cache = /home/nagyv/.buildout/dlcache
> 
> extensions = mr.developer
> sources = sources
> auto-checkout =
>     uni-form
>     sugar
> 
> [python]
> recipe = zc.recipe.egg
> interpreter = python
> eggs = ${buildout:eggs}
> 
> [django-1.1]
> recipe = djangorecipe
> version = 1.1.1
> project = contacts
> projectegg = contacts
> settings = testsettings
> test = contacts
> testrunner = test-1.1
> eggs = ${buildout:eggs}
> 
> [sources]
> uni-form = git git://github.com/nagyv/django-uni-form.git
> <http://github.com/nagyv/django-uni-form.git>
> sugar = git git://github.com/nagyv/django-sugar.git
> <http://github.com/nagyv/django-sugar.git>
> 
> thanks for your help!
> Viktor
> 
> 
> 
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig

-- 
Cordialement,
KiOrKY
GPG Key FingerPrint: 0x1A1194B7681112AF

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 262 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20100224/e4421919/attachment.pgp>

From jim at zope.com  Wed Feb 24 20:19:35 2010
From: jim at zope.com (Jim Fulton)
Date: Wed, 24 Feb 2010 14:19:35 -0500
Subject: [Distutils] how to use mr.developer?
In-Reply-To: <8f75d251002240244y6e0cb14cg96fae87b1dbc2499@mail.gmail.com>
References: <8f75d251002240244y6e0cb14cg96fae87b1dbc2499@mail.gmail.com>
Message-ID: <1099b90b1002241119o558f48h6c12757b498574ba@mail.gmail.com>

On Wed, Feb 24, 2010 at 5:44 AM, }--o <viktor.nagy at gmail.com> wrote:
> hi,
>
> I'm just experimenting with distutils, and would like to move my django apps
> under it.
> One of my apps uses django-sugar that should be checked out from github.
>
> bin/buildout check out my django-sugar repo under src/, and creates an
> egg-link under develop-eggs, but it's not added to sys.path for the bin/*
> binaries.
> Is there a simple way to do it? I've seen the extra_paths variable, but then
> I still have to write each ${buildout:auto-checkout} line again.

You just need to include the project name in the list of eggs for the
parts that generate the scripts you're interested in.


>
> this is my buildout.cfg file:
>
> [buildout]
> parts = python django-1.1
> develop = .
> eggs =
>     django-contacts
>     django-l10n
>     django-registration

I'm guessing you want to add django-sugar here.

Jim

--
Jim Fulton

From pje at telecommunity.com  Thu Feb 25 18:00:06 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 25 Feb 2010 12:00:06 -0500
Subject: [Distutils] RFC: Barely minimal CI + tests + packaging in
 Python WAS: Guidelines for where to put tests & how to package them
In-Reply-To: <24ea26601002250741n1c42b271oc96b7466853bd970@mail.gmail.co
 m>
References: <24ea26601002250741n1c42b271oc96b7466853bd970@mail.gmail.com>
Message-ID: <20100225170007.3A66D3A406A@sparrow.telecommunity.com>

At 10:41 AM 2/25/2010 -0500, Olemis Lang wrote:
>PS: BTW, how could I trigger easy_install(ation) at a given point
>while implementing a distutils command, and let the command perform
>further actions if deps are installed correctly ?

Setuptools' "Distribution" object has a method for fetching 
dependencies.  See setuptools' "test" command for an example.  (This 
doesn't install the dependencies globally, just drops eggs into the 
build directory.  But they're there and available to be reused for 
installation in a later phase, under normal circumstances.)


From olemis at gmail.com  Thu Feb 25 18:10:06 2010
From: olemis at gmail.com (Olemis Lang)
Date: Thu, 25 Feb 2010 12:10:06 -0500
Subject: [Distutils] RFC: Barely minimal CI + tests + packaging in
	Python WAS: Guidelines for where to put tests & how to package them
In-Reply-To: <20100225170007.3A66D3A406A@sparrow.telecommunity.com>
References: <24ea26601002250741n1c42b271oc96b7466853bd970@mail.gmail.com>
	<20100225170007.3A66D3A406A@sparrow.telecommunity.com>
Message-ID: <24ea26601002250910g7019363ama75f1d53c02461f8@mail.gmail.com>

Thnx for your reply !

On Thu, Feb 25, 2010 at 12:00 PM, P.J. Eby <pje at telecommunity.com> wrote:
> At 10:41 AM 2/25/2010 -0500, Olemis Lang wrote:
>>
>> PS: BTW, how could I trigger easy_install(ation) at a given point
>> while implementing a distutils command, and let the command perform
>> further actions if deps are installed correctly ?
>
> Setuptools' "Distribution" object has a method for fetching dependencies.
> ?See setuptools' "test" command for an example. ?(This doesn't install the
> dependencies globally, just drops eggs into the build directory. ?But
> they're there and available to be reused for installation in a later phase,
> under normal circumstances.)
>

So this means that setuptools `test` command already retrieves dependencies ?

If so, then I shouldn't care about that, because I'd only need to
override `run_tests` method in order to do what I want to do (i.e. use
another test runner ;o)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Support micro-seconds as added by Trac in revision 9210 for upcoming
0.12. Fixes the two inst...  -
http://bitbucket.org/osimons/trac-rpc-mq/changeset/62ffe719a84a/

From pje at telecommunity.com  Thu Feb 25 18:29:33 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 25 Feb 2010 12:29:33 -0500
Subject: [Distutils] RFC: Barely minimal CI + tests + packaging in
 Python WAS: Guidelines for where to put tests & how to package them
In-Reply-To: <24ea26601002250910g7019363ama75f1d53c02461f8@mail.gmail.co
 m>
References: <24ea26601002250741n1c42b271oc96b7466853bd970@mail.gmail.com>
	<20100225170007.3A66D3A406A@sparrow.telecommunity.com>
	<24ea26601002250910g7019363ama75f1d53c02461f8@mail.gmail.com>
Message-ID: <20100225172934.028103A406A@sparrow.telecommunity.com>

At 12:10 PM 2/25/2010 -0500, Olemis Lang wrote:
>Thnx for your reply !
>
>On Thu, Feb 25, 2010 at 12:00 PM, P.J. Eby <pje at telecommunity.com> wrote:
> > At 10:41 AM 2/25/2010 -0500, Olemis Lang wrote:
> >>
> >> PS: BTW, how could I trigger easy_install(ation) at a given point
> >> while implementing a distutils command, and let the command perform
> >> further actions if deps are installed correctly ?
> >
> > Setuptools' "Distribution" object has a method for fetching dependencies.
> >  See setuptools' "test" command for an example.  (This doesn't install the
> > dependencies globally, just drops eggs into the build directory.  But
> > they're there and available to be reused for installation in a later phase,
> > under normal circumstances.)
> >
>
>So this means that setuptools `test` command already retrieves dependencies ?

Yes - it also retrieves any testing-specific dependencies (per the 
"tests_require" setup() argument), so that if you use another testing 
library like py.test or nose, it can be used instead.


>If so, then I shouldn't care about that, because I'd only need to
>override `run_tests` method in order to do what I want to do (i.e. use
>another test runner ;o)

In that case, you may want to simply use the setuptools trunk version 
(setuptools==dev), which supports a test_runner setup() argument to 
the test command.  ;-)


From olemis at gmail.com  Thu Feb 25 18:36:23 2010
From: olemis at gmail.com (Olemis Lang)
Date: Thu, 25 Feb 2010 12:36:23 -0500
Subject: [Distutils] RFC: Barely minimal CI + tests + packaging in
	Python WAS: Guidelines for where to put tests & how to package them
In-Reply-To: <20100225172934.028103A406A@sparrow.telecommunity.com>
References: <24ea26601002250741n1c42b271oc96b7466853bd970@mail.gmail.com>
	<20100225170007.3A66D3A406A@sparrow.telecommunity.com>
	<24ea26601002250910g7019363ama75f1d53c02461f8@mail.gmail.com>
	<20100225172934.028103A406A@sparrow.telecommunity.com>
Message-ID: <24ea26601002250936m26e9ccebkfe5c55e9de4776bb@mail.gmail.com>

On Thu, Feb 25, 2010 at 12:29 PM, P.J. Eby <pje at telecommunity.com> wrote:
> At 12:10 PM 2/25/2010 -0500, Olemis Lang wrote:
>>
>> Thnx for your reply !
>>
>> On Thu, Feb 25, 2010 at 12:00 PM, P.J. Eby <pje at telecommunity.com> wrote:
>> > At 10:41 AM 2/25/2010 -0500, Olemis Lang wrote:
>> >>
>> >> PS: BTW, how could I trigger easy_install(ation) at a given point
>> >> while implementing a distutils command, and let the command perform
>> >> further actions if deps are installed correctly ?
>> >
>> > Setuptools' "Distribution" object has a method for fetching
>> > dependencies.
>> > ?See setuptools' "test" command for an example. ?(This doesn't install
>> > the
>> > dependencies globally, just drops eggs into the build directory. ?But
>> > they're there and available to be reused for installation in a later
>> > phase,
>> > under normal circumstances.)
>> >
>>
>> So this means that setuptools `test` command already retrieves
>> dependencies ?
>
> Yes - it also retrieves any testing-specific dependencies (per the
> "tests_require" setup() argument), so that if you use another testing
> library like py.test or nose, it can be used instead.
>
>
>> If so, then I shouldn't care about that, because I'd only need to
>> override `run_tests` method in order to do what I want to do (i.e. use
>> another test runner ;o)
>
> In that case, you may want to simply use the setuptools trunk version
> (setuptools==dev), which supports a test_runner setup() argument to the test
> command. ?;-)
>

Ooops ! Almost nothing to do then, and I can happily add setuptools to
my list to get a barely minimal build script for CI.

Thnx veeeery veeery much for your veeeery useful reply !

;o)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Re: [Python-Dev] setUpClass and setUpModule in unittest :: ASPN ...  -
http://feedproxy.google.com/~r/TracGViz-full/~3/y2L-XX7-3vQ/3817074

From optilude+lists at gmail.com  Fri Feb 26 11:16:44 2010
From: optilude+lists at gmail.com (Martin Aspeli)
Date: Fri, 26 Feb 2010 18:16:44 +0800
Subject: [Distutils] zc.buildout - order of extends processing
Message-ID: <hm872d$git$1@dough.gmane.org>

Hi,

In my quest to make re-usable buildout components, I've started using a 
pattern like this:

In a server-specific buildout file (e.g. prod-web-master.cfg) I have e.g.:

[buildout]
extends =
     buildout.d/base.cfg
     buildout.d/zope.cfg
     buildout.d/relstorage.cfg
     buildout.d/haproxy.cfg

[ports]
haproxy = 8100

[eggs]
main = my.package

...

Each file in buildout.cfg adds additional functionality, usually by 
defining new parts and doing e.g.

    parts += haproxy-build haproxy-config

    [haproxy-build]
    recipe = zc.recipe.cmmi
    ...

Sometimes, they also extend some shared variables, e.g.

    [eggs]
    main += RelStorage

With this type of setup, the order in which configuration is loaded and 
processed becomes important, in particular for += and -= type operations 
and straightforward overrides. For example, I may have a default 
${ports:haproxy} defined in haproxy.cfg, but overridden in 
prod-web-slave.cfg because the slave node wants it to run on a different 
port, say.

I can't find much documentation about how this is supposed to work. I've 
had situations where I've been surprised by the order in which things 
are processed, in particular if I extend one file that in turn extends 
another. So, a few questions

  - Is the order well-defined? e.g. is it always doing a depth-first or 
breath-first inclusion? Is the load order the same as the execution order?

  - Do settings in the top-level config (the one run with bin/buildout 
-c <file>) always override extended ones?

  - More generally, if A extends B and B extends C, does configuration 
in B always take precedence over configuration in C with the same 
part/option? Does a += in B always extend the options set in C?

  - If I have multiple files all extending a common base, e.g. A extends 
B and C, and both B and C extend a common D, when does the common base 
(D) get processed?

Cheers,
Martin

-- 
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book


From olemis at gmail.com  Fri Feb 26 16:16:27 2010
From: olemis at gmail.com (Olemis Lang)
Date: Fri, 26 Feb 2010 10:16:27 -0500
Subject: [Distutils] RFC: Barely minimal CI + tests + packaging in
	Python WAS: Guidelines for where to put tests & how to package them
In-Reply-To: <24ea26601002250936m26e9ccebkfe5c55e9de4776bb@mail.gmail.com>
References: <24ea26601002250741n1c42b271oc96b7466853bd970@mail.gmail.com>
	<20100225170007.3A66D3A406A@sparrow.telecommunity.com>
	<24ea26601002250910g7019363ama75f1d53c02461f8@mail.gmail.com>
	<20100225172934.028103A406A@sparrow.telecommunity.com>
	<24ea26601002250936m26e9ccebkfe5c55e9de4776bb@mail.gmail.com>
Message-ID: <24ea26601002260716k4f321142t156c38c13d69f064@mail.gmail.com>

On Thu, Feb 25, 2010 at 12:36 PM, Olemis Lang <olemis at gmail.com> wrote:
> On Thu, Feb 25, 2010 at 12:29 PM, P.J. Eby <pje at telecommunity.com> wrote:
>> At 12:10 PM 2/25/2010 -0500, Olemis Lang wrote:
>>>
>>> Thnx for your reply !
>>>
>>> On Thu, Feb 25, 2010 at 12:00 PM, P.J. Eby <pje at telecommunity.com> wrote:
>>> > At 10:41 AM 2/25/2010 -0500, Olemis Lang wrote:
>>> >>
>>> >> PS: BTW, how could I trigger easy_install(ation) at a given point
>>> >> while implementing a distutils command, and let the command perform
>>> >> further actions if deps are installed correctly ?
>>> >
>>> > Setuptools' "Distribution" object has a method for fetching
>>> > dependencies.
>>> > ?See setuptools' "test" command for an example. ?(This doesn't install
>>> > the
>>> > dependencies globally, just drops eggs into the build directory. ?But
>>> > they're there and available to be reused for installation in a later
>>> > phase,
>>> > under normal circumstances.)
>>> >
>>>
>>> So this means that setuptools `test` command already retrieves
>>> dependencies ?
>>
>> Yes - it also retrieves any testing-specific dependencies (per the
>> "tests_require" setup() argument), so that if you use another testing
>> library like py.test or nose, it can be used instead.
>>
>>
>>> If so, then I shouldn't care about that, because I'd only need to
>>> override `run_tests` method in order to do what I want to do (i.e. use
>>> another test runner ;o)
>>
>> In that case, you may want to simply use the setuptools trunk version
>> (setuptools==dev), which supports a test_runner setup() argument to the test
>> command. ?;-)
>>
>
> Ooops ! Almost nothing to do then, and I can happily add setuptools to
> my list to get a barely minimal build script for CI.
>

Well not as happily as I thought in first place. The fact is that
`setuptools` test command creates the runner by calling the class
specified in `test_runner` arg but supplies no parameters. In this
case I'd like to specify `xmlrunner:XMLTestRunner` (is that the
correct syntax ?) to obtain a JUnit XML report, but I need to specify
the path where those reports will be outputted, so that the CI tool
(Hudson in this case) can publish'em and generate the nice graphs we
all love . Considering the current implementation, I still need to
override `run_tests` method as a whole, and repeat 99% of what is
implemented in there, in order to get the exact runner I need

:(

My suggestion (hint, $0.02 ... whatever ;o) is

   1. use a factory method (e.g `create_runner` ) returning an
     instance of  test runner. This method could be overridden by
     subclasses if further options or whatever needs to be considered
     to instanciate the runner
  2. select an instance instead of a class

So far I prefer (1)

Something similar happens with loaders. For instance, if I was using
`dutest` loaders I wouldn't be able to specify parameters to
instantiate those loaders so, similar problem => similar solutions .

The goal is to make the command more flexible (easily extensible ?) to
support further options so as to instantiate runners and loaders

;o)

Comments ?

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Free calculus 1.3.4 Download - mac software  -
http://feedproxy.google.com/~r/TracGViz-full/~3/O5E_yyBKv0U/

From pje at telecommunity.com  Fri Feb 26 16:52:05 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 26 Feb 2010 10:52:05 -0500
Subject: [Distutils] RFC: Barely minimal CI + tests + packaging in
 Python WAS: Guidelines for where to put tests & how to package them
In-Reply-To: <24ea26601002260716k4f321142t156c38c13d69f064@mail.gmail.co
 m>
References: <24ea26601002250741n1c42b271oc96b7466853bd970@mail.gmail.com>
	<20100225170007.3A66D3A406A@sparrow.telecommunity.com>
	<24ea26601002250910g7019363ama75f1d53c02461f8@mail.gmail.com>
	<20100225172934.028103A406A@sparrow.telecommunity.com>
	<24ea26601002250936m26e9ccebkfe5c55e9de4776bb@mail.gmail.com>
	<24ea26601002260716k4f321142t156c38c13d69f064@mail.gmail.com>
Message-ID: <20100226155207.50A643A406A@sparrow.telecommunity.com>

At 10:16 AM 2/26/2010 -0500, Olemis Lang wrote:
>Well not as happily as I thought in first place. The fact is that
>`setuptools` test command creates the runner by calling the class
>specified in `test_runner` arg but supplies no parameters. In this
>case I'd like to specify `xmlrunner:XMLTestRunner` (is that the
>correct syntax ?) to obtain a JUnit XML report, but I need to specify
>the path where those reports will be outputted, so that the CI tool
>(Hudson in this case) can publish'em and generate the nice graphs we
>all love . Considering the current implementation, I still need to
>override `run_tests` method as a whole, and repeat 99% of what is
>implemented in there, in order to get the exact runner I need
>
>:(
>
>My suggestion (hint, $0.02 ... whatever ;o) is
>
>    1. use a factory method (e.g `create_runner` ) returning an
>      instance of  test runner. This method could be overridden by
>      subclasses if further options or whatever needs to be considered
>      to instanciate the runner

I'm confused.  Why don't you just set test_runner to point to your 
factory function, then?  That's what nose does.  (Well, in its case 
it's the test_loader, but same basic idea.)


From olemis at gmail.com  Fri Feb 26 17:19:29 2010
From: olemis at gmail.com (Olemis Lang)
Date: Fri, 26 Feb 2010 11:19:29 -0500
Subject: [Distutils] RFC: Barely minimal CI + tests + packaging in
	Python WAS: Guidelines for where to put tests & how to package them
In-Reply-To: <20100226155207.50A643A406A@sparrow.telecommunity.com>
References: <24ea26601002250741n1c42b271oc96b7466853bd970@mail.gmail.com>
	<20100225170007.3A66D3A406A@sparrow.telecommunity.com>
	<24ea26601002250910g7019363ama75f1d53c02461f8@mail.gmail.com>
	<20100225172934.028103A406A@sparrow.telecommunity.com>
	<24ea26601002250936m26e9ccebkfe5c55e9de4776bb@mail.gmail.com>
	<24ea26601002260716k4f321142t156c38c13d69f064@mail.gmail.com>
	<20100226155207.50A643A406A@sparrow.telecommunity.com>
Message-ID: <24ea26601002260819i6c29b59dwfc112c6517853283@mail.gmail.com>

On Fri, Feb 26, 2010 at 10:52 AM, P.J. Eby <pje at telecommunity.com> wrote:
> At 10:16 AM 2/26/2010 -0500, Olemis Lang wrote:
>>
>> Well not as happily as I thought in first place. The fact is that
>> `setuptools` test command creates the runner by calling the class
>> specified in `test_runner` arg but supplies no parameters. In this
>> case I'd like to specify `xmlrunner:XMLTestRunner` (is that the
>> correct syntax ?) to obtain a JUnit XML report, but I need to specify
>> the path where those reports will be outputted, so that the CI tool
>> (Hudson in this case) can publish'em and generate the nice graphs we
>> all love . Considering the current implementation, I still need to
>> override `run_tests` method as a whole, and repeat 99% of what is
>> implemented in there, in order to get the exact runner I need
>>
>> :(
>>
>> My suggestion (hint, $0.02 ... whatever ;o) is
>>
>> ? 1. use a factory method (e.g `create_runner` ) returning an
>> ? ? instance of ?test runner. This method could be overridden by
>> ? ? subclasses if further options or whatever needs to be considered
>> ? ? to instanciate the runner
>
> I'm confused. ?Why don't you just set test_runner to point to your factory
> function, then? ?That's what nose does. ?(Well, in its case it's the
> test_loader, but same basic idea.)
>

Well in theory it is possible (and did not think about that in the
first place, thnx ;o) but :

  1. How could I know about extra command line
      params (without parsing them once again of course ;o) ?
  2. In the case of the build script, I'm writing a setup script rather than
      a framework (e.g. nose).
     2.1 The XML runner is not always used, but only when running at
           the CI buildenv (I supposed I could just use
           `-r xmlrunner:XMLTestRunner` but I need an extra arg)
           How could I specify output path in cmdline ?
     2.2 So I suppose I should write a function to do that but
            - Under tests pkg? ... (Unit | functional | ...) tests
don't need it
            - In setup.py itself ? ... What should I supply in to -r arg ?
              How could I specify output path in cmdline ?
  3. In the case of `dutest` well ... that could help (depending on
      the answer to 1 ;o)
     3.1 How could I reuse features already stored in `Distribution`
            (especially values set after finalize_xxx ) ?
  4. (Hypothetical so far, so I'm prepared to receive tomatoes ;o)
      What if I want to reuse (subclass) `test` command ?

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Added unit test for general provider error, and fixing a missing
import discovered by test. A...  -
http://bitbucket.org/osimons/trac-rpc-mq/changeset/409eb5ecdbd8/

From pje at telecommunity.com  Fri Feb 26 17:38:53 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 26 Feb 2010 11:38:53 -0500
Subject: [Distutils] RFC: Barely minimal CI + tests + packaging in
 Python WAS: Guidelines for where to put tests & how to package them
In-Reply-To: <24ea26601002260819i6c29b59dwfc112c6517853283@mail.gmail.co
 m>
References: <24ea26601002250741n1c42b271oc96b7466853bd970@mail.gmail.com>
	<20100225170007.3A66D3A406A@sparrow.telecommunity.com>
	<24ea26601002250910g7019363ama75f1d53c02461f8@mail.gmail.com>
	<20100225172934.028103A406A@sparrow.telecommunity.com>
	<24ea26601002250936m26e9ccebkfe5c55e9de4776bb@mail.gmail.com>
	<24ea26601002260716k4f321142t156c38c13d69f064@mail.gmail.com>
	<20100226155207.50A643A406A@sparrow.telecommunity.com>
	<24ea26601002260819i6c29b59dwfc112c6517853283@mail.gmail.com>
Message-ID: <20100226163855.D00DE3A406A@sparrow.telecommunity.com>

At 11:19 AM 2/26/2010 -0500, Olemis Lang wrote:
>On Fri, Feb 26, 2010 at 10:52 AM, P.J. Eby <pje at telecommunity.com> wrote:
> > At 10:16 AM 2/26/2010 -0500, Olemis Lang wrote:
> >>
> >> Well not as happily as I thought in first place. The fact is that
> >> `setuptools` test command creates the runner by calling the class
> >> specified in `test_runner` arg but supplies no parameters. In this
> >> case I'd like to specify `xmlrunner:XMLTestRunner` (is that the
> >> correct syntax ?) to obtain a JUnit XML report, but I need to specify
> >> the path where those reports will be outputted, so that the CI tool
> >> (Hudson in this case) can publish'em and generate the nice graphs we
> >> all love . Considering the current implementation, I still need to
> >> override `run_tests` method as a whole, and repeat 99% of what is
> >> implemented in there, in order to get the exact runner I need
> >>
> >> :(
> >>
> >> My suggestion (hint, $0.02 ... whatever ;o) is
> >>
> >>   1. use a factory method (e.g `create_runner` ) returning an
> >>     instance of  test runner. This method could be overridden by
> >>     subclasses if further options or whatever needs to be considered
> >>     to instanciate the runner
> >
> > I'm confused.  Why don't you just set test_runner to point to your factory
> > function, then?  That's what nose does.  (Well, in its case it's the
> > test_loader, but same basic idea.)
> >
>
>Well in theory it is possible (and did not think about that in the
>first place, thnx ;o) but :
>
>   1. How could I know about extra command line
>       params (without parsing them once again of course ;o) ?
>   2. In the case of the build script, I'm writing a setup script rather than
>       a framework (e.g. nose).
>      2.1 The XML runner is not always used, but only when running at
>            the CI buildenv (I supposed I could just use
>            `-r xmlrunner:XMLTestRunner` but I need an extra arg)
>            How could I specify output path in cmdline ?

An environment variable would solve these problems, at least if you 
have a decent shell.  ;-)

>      2.2 So I suppose I should write a function to do that but
>             - Under tests pkg? ... (Unit | functional | ...) tests
>don't need it
>             - In setup.py itself ? ... What should I supply in to -r arg ?

If you are running setup.py from the command line (as opposed to 
programmatically via easy_install), then it's __main__:whatever.  (It 
won't work if setup.py isn't the "true" __main__, though, e.g., if 
the script is being execfile'd.)

>               How could I specify output path in cmdline ?
>   3. In the case of `dutest` well ... that could help (depending on
>       the answer to 1 ;o)
>      3.1 How could I reuse features already stored in `Distribution`
>             (especially values set after finalize_xxx ) ?

You can't, unless you...


>   4. (Hypothetical so far, so I'm prepared to receive tomatoes ;o)
>       What if I want to reuse (subclass) `test` command ?

Which is perfectly allowable.  You'll just have to duplicate that one 
method.  Of course, if you write a suitable patch to refactor it to 
call another method, I'll happily put it in the trunk, since you're 
working from trunk to get the test_runner feature anyway.

("Suitable" = tested by you, doesn't break or remove existing 
functionality, & other "I'll know it if I see it" criteria ;-) )


From pje at telecommunity.com  Fri Feb 26 21:53:21 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 26 Feb 2010 15:53:21 -0500
Subject: [Distutils] [TIP] Guidelines for where to put tests & how to
 package them
In-Reply-To: <24ea26601002261127n59cccc9dh7fd67199939cb7cd@mail.gmail.co
 m>
References: <20100225050151.GB20448@idyll.org>
	<1267081404.14435.64.camel@lifeless-64>
	<20100225161808.GA5776@idyll.org>
	<4B8819C8.6010808@voidspace.org.uk>
	<24ea26601002261127n59cccc9dh7fd67199939cb7cd@mail.gmail.com>
Message-ID: <20100226205324.12BB13A406A@sparrow.telecommunity.com>

At 02:27 PM 2/26/2010 -0500, Olemis Lang wrote:
>test_suite is ( supported | introduced | defined ) by
>setuptools.comand.test.test class
>
> > Is it required to be a
> > callable
>
>yep

No.  test_suite is to be an object recognized by the test_loader as 
convertible to a unittest TestSuite.  For example, a module, TestCase 
subclass, etc.

You are probably thinking of the test_runner option, not test_suite.



>test command performs test discovery (as a last recourse AFAICR, /me
>cc-ing to P.J. Eby because probably he has further updates ;o) using a
>loader: setuptools.comand.test.xxx

Setuptools' test loader will scan modules and packages, but you still 
have to tell it where your top-level test package or module is.  (The 
test_suite.)


>BTW, I am strongly -1 for using the discovery protocol introduced in
>2.7 (AFAICR), specially because it collides with test_suite and many
>other arguments ... so it would be very nice when defining packaging
>conventions, layout ... not to enforce the use of that protocol, so
>that if another discovery is used then it could still be compatible
>with the more abstract good practices. (all this said from my selfish
>position, I confess)

I'm not sure what you're talking about here; I'm not much in the loop 
for what's going on with distutils updates, and even more so for 
unittest updates.


From ziade.tarek at gmail.com  Fri Feb 26 22:52:29 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 26 Feb 2010 22:52:29 +0100
Subject: [Distutils] Fwd: The fate of Distutils in Python 2.7
In-Reply-To: <94bdd2611002261344q361e702av10fdf99ea5eb9f71@mail.gmail.com>
References: <94bdd2611002261344q361e702av10fdf99ea5eb9f71@mail.gmail.com>
Message-ID: <94bdd2611002261352v5dfa8768p7c2bb266efab1272@mail.gmail.com>

FIY (I didn't cross-post to make it easier for following the threads)


---------- Forwarded message ----------
From: Tarek Ziad? <ziade.tarek at gmail.com>
Date: Fri, Feb 26, 2010 at 10:44 PM
Subject: The fate of Distutils in Python 2.7
To: Python-Dev <python-dev at python.org>


Hello,

This is a follow-up of the Pycon summit + sprints on packaging.

This is what we have planned to do:

1. refactor distutils in a new standalone version called distutils2
[this is done already and we are actively working in the code]
2. completely revert distutils in Lib/ and Doc/ so the code + doc is
the same than the current 2.6.x branch
3. leave the new sysconfig module, that is used by the Makefile and
the site module

The rest of the work will happen in distutils2 and we will try to
release a version asap for Python 2.x and 3.x (2.4 to 3.2), and the
goal
is to put it back in the stdlib in Python 3.3

Distutils in Python will be feature-frozen and I will only do bug
fixes there. ?All feature requests will be redirected to Distutils2.

I think the easiest way to manage this for me and for the feedback of
the community is to add in bugs.python.org a "Distutils2" component,
so I can
start to reorganize the issues in there and reassign new issues to
Distutils2 when it applies.

Regards
Tarek

--
Tarek Ziad? | http://ziade.org



-- 
Tarek Ziad? | http://ziade.org

From jeandaniel.browne at gmail.com  Sun Feb 28 22:03:16 2010
From: jeandaniel.browne at gmail.com (Jean Daniel)
Date: Sun, 28 Feb 2010 22:03:16 +0100
Subject: [Distutils]  layout and setup.py for packaging documentation
Message-ID: <b61c19e71002281303s79c55e61u1eef6c82e07796ed@mail.gmail.com>

Hello,

I am looking for the simplest way to package the html files related to
a pure python module. At this point, I would like to just embed them
in the source tarball generated by distribute and leave it to a debian
or fedora package to install them. I would like a source tarball laid
out as the following:

    wordish-1.0.2/
        setup.py
        wordish.py
        docs/
            index.html
            command-ref.html

So far, I tended to use a single file python module because there was
no need for more and I wanted to keep it simple:

    wordish/
        setup.py
        wordish.py
        test_wordish.py

The setup.py is minimal:  setup( py_modules = [ 'wordish' ], ... )

But If I want to include docs, I must use the package_data command
which expects a package name. So I think that to embed documentation
in the source tarball, I should change the way I lay out the source
repository to something like  :

    wordish/
        setup.py
        wordish
            wordish.py
            __init__.py
            docs/
                index.html
                command-ref.html
       test_wordish.py

The corresponding setup.py would be

setup(
    packages = [ 'wordish' ],
    package_data = {'wordish':['docs/*']}, ...

I am not sure this is the right way to do it, just to include
documentations in the tarball:

- the package_data requires a package name, so I turn my pure python
module into a package just for this reason

- package_data requires the data directory to be inside the package
directory even though the html files are generated from doc sources
located elsewhere,

- there is an extra level of directory to get to the Python sources
module, it is my_virtualenv/wordish/wordish/wordish.py which does not
add much.

Can this be simpler?

Thanks for your help,

From pje at telecommunity.com  Sun Feb 28 23:14:56 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Sun, 28 Feb 2010 17:14:56 -0500
Subject: [Distutils] layout and setup.py for packaging  documentation
In-Reply-To: <b61c19e71002281303s79c55e61u1eef6c82e07796ed@mail.gmail.co
 m>
References: <b61c19e71002281303s79c55e61u1eef6c82e07796ed@mail.gmail.com>
Message-ID: <20100228221502.8CCAD3A40A7@sparrow.telecommunity.com>

At 10:03 PM 2/28/2010 +0100, Jean Daniel wrote:
>Can this be simpler?

Yes.  Don't install docs with your package.  People who want them 
installed locally can just download your source install or use easy_install -e.

Also, if your module is popular enough that people make Linux system 
packages for it, they will make sure the docs get put in a blessed 
install location.  Python doesn't currently have a blessed install 
location for documentation, though perhaps it *should* have one in distutils2.


From fuzzyman at gmail.com  Sun Feb 28 23:39:37 2010
From: fuzzyman at gmail.com (Michael Foord)
Date: Sun, 28 Feb 2010 22:39:37 +0000
Subject: [Distutils] layout and setup.py for packaging documentation
In-Reply-To: <20100228221502.8CCAD3A40A7@sparrow.telecommunity.com>
References: <b61c19e71002281303s79c55e61u1eef6c82e07796ed@mail.gmail.com>
	<20100228221502.8CCAD3A40A7@sparrow.telecommunity.com>
Message-ID: <6f4025011002281439t2dda1538gb488e20e0df6cc6c@mail.gmail.com>

On 28 February 2010 22:14, P.J. Eby <pje at telecommunity.com> wrote:

> At 10:03 PM 2/28/2010 +0100, Jean Daniel wrote:
>
>> Can this be simpler?
>>
>
> Yes.  Don't install docs with your package.  People who want them installed
> locally can just download your source install or use easy_install -e.
>
> Also, if your module is popular enough that people make Linux system
> packages for it, they will make sure the docs get put in a blessed install
> location.  Python doesn't currently have a blessed install location for
> documentation, though perhaps it *should* have one in distutils2.



How to include documentation in a package is a common question, so it would
be great if distutils2 could deal with this issue.

My current solution is, as Phillip suggests, to not include the docs in
distributions available via PyPI and to provide a more 'complete' download
separately.

Michael



>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
http://www.ironpythoninaction.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20100228/8dc2c3fc/attachment.html>