From setuptools at bugs.python.org  Fri Apr  1 17:12:37 2011
From: setuptools at bugs.python.org (Kurt Schwehr)
Date: Fri, 01 Apr 2011 15:12:37 +0000
Subject: [Distutils] [issue125] macosx unittest failure for /tmp
In-Reply-To: <1301670757.77.0.816793675532.issue125@psf.upfronthosting.co.za>
Message-ID: <1301670757.77.0.816793675532.issue125@psf.upfronthosting.co.za>


New submission from Kurt Schwehr <kurtschwehr at yahoo.com>:

On the Mac, /tmp is a symbolic link to /private/tmp.  We are stuck Apple's decision to do this.  Hit this when trying adding code to the Mac OSX Fink distribute package.

-kurt
(goatbar)


FAIL: test_local_index (setuptools.tests.test_easy_install.TestUserInstallTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sw32/src/fink.build/distribute-py27-0.6.15-1/distribute-0.6.15/setuptools/tests/test_easy_install.py", line 234, in test_local_index
    self.assertEquals(res.location, new_location)
AssertionError: '/private/tmp/tmpCjeemJ' != '/tmp/tmpCjeemJ'

----------------------------------------------------------------------
Ran 71 tests in 10.861s

FAILED (failures=1)

----------
messages: 610
nosy: schwehr
priority: bug
status: unread
title: macosx unittest failure for /tmp

_______________________________________________
Setuptools tracker <setuptools at bugs.python.org>
<http://bugs.python.org/setuptools/issue125>
_______________________________________________

From carl at oddbird.net  Fri Apr  1 20:14:29 2011
From: carl at oddbird.net (Carl Meyer)
Date: Fri, 01 Apr 2011 14:14:29 -0400
Subject: [Distutils] pythonv, take two
In-Reply-To: <loom.20110331T090727-413@post.gmane.org>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<4D8172FB.3020109@oddbird.net>	<loom.20110317T112246-771@post.gmane.org>	<4D8233EF.7070205@oddbird.net>	<loom.20110318T004329-895@post.gmane.org>	<4D83A807.30003@oddbird.net>	<loom.20110329T154252-221@post.gmane.org>	<4D920A7D.9040109@oddbird.net>	<loom.20110330T005934-410@post.gmane.org>	<4D9369E5.1050506@oddbird.net>	<loom.20110331T005457-50@post.gmane.org>	<4D941203.3020804@oddbird.net>
	<loom.20110331T090727-413@post.gmane.org>
Message-ID: <4D961605.7050307@oddbird.net>

On 03/31/2011 04:12 AM, Vinay Sajip wrote:
> Carl Meyer <carl <at> oddbird.net> writes:
> 
>> The logical conclusion of that approach, if you want
>> full compatibility with third-party code, is to just copy the entire
>> Python installation to the new location.
> 
> We do want that compatibility, don't we? Using a virtual environment should be a
> no-brainer, as virtualenv currently is (most of the time). Of course, the
> downside of the copying would be around 50MB per virtualenv on Windows, and much
> less where symlinking or hardlinking is available. Not ideal, of course, but
> disk space is cheap ;-)

People tend to create a lot of virtualenvs. I don't think copying the
entire Python installation is viable (it is slow, and it also would mean
that security updates to the base standard library would not be
reflected in the virtualenv). If copying the entire base Python install
were an adequate solution, this entire virtual-python proposal would be
moot - you can do that right now and it works fine.

>> 1) Add sys.virtual_prefix (or perhaps sys.site_prefix would be a better
>> name), and require third-party code to use that (or sysconfig) to find
>> site-packages, if it wants to be virtual-compatible.
> 
> But the success of virtualenv is because you can install just about any package
> in an environment, and packages don't have to do anything to be "virtual
> compatible".

Sure. "virtual-compatible" was a poor choice of terms - in the long run,
there should be no such concept. In the short run, we would be
clarifying the semantics of sys.prefix and introducing a new
sys.site_prefix. And in the short run, the effect of code that used
sys.prefix contrary to those semantics would be failure in a virtualenv.

But, as I mentioned, the fact that easy_install itself relies on finding
site-packages relative to sys.prefix forces the issue - easy_install
(including existing versions of it) needs to just work.

>> 2) Repoint sys.prefix, and require third-party code to understand some
>> new sys.system_prefix or sys.installed_prefix or equivalent to find the
>> rest of the base Python installation.
> 
> I think that the answer is for third party code not to use any prefix directly,
> but rely on an API such as  sysconfig.get_path('purelib'). There's a distutils
> version of this - distutils.sysconfig.get_python_lib() - but this is brittle as
> per issue #6087.

Yes, of course. If all code did this, we wouldn't have this problem; we
would just fix sysconfig.py and site.py and all would be well.

>> 3) Attempt the virtualenv approach of trying to fool third-party code
>> into thinking the virtualenv IS a full Python installation, even though
>> it's not.
> 
>> I think #1 is far preferable to #2, and in the long run preferable to #3
>> even if it involves some short-term pain. But maybe I'm being too
>> idealistic, and #3 is the pragmatic choice even though it's "wrong."
> 
> I'm afraid you may be right, at least until people stop using sys.prefix and
> sys.exec_prefix directly and call functions instead, which can operate
> differently in a virtual environment.

Yup.

Carl

From carl at oddbird.net  Fri Apr  1 20:16:44 2011
From: carl at oddbird.net (Carl Meyer)
Date: Fri, 01 Apr 2011 14:16:44 -0400
Subject: [Distutils] pythonv, take two
In-Reply-To: <loom.20110331T145603-128@post.gmane.org>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<4D8172FB.3020109@oddbird.net>	<loom.20110317T112246-771@post.gmane.org>	<4D8233EF.7070205@oddbird.net>	<loom.20110318T004329-895@post.gmane.org>	<4D83A807.30003@oddbird.net>	<loom.20110329T154252-221@post.gmane.org>	<4D920A7D.9040109@oddbird.net>	<loom.20110330T005934-410@post.gmane.org>	<4D9369E5.1050506@oddbird.net>	<loom.20110331T124928-850@post.gmane.org>
	<loom.20110331T145603-128@post.gmane.org>
Message-ID: <4D96168C.6000105@oddbird.net>



On 03/31/2011 09:21 AM, Vinay Sajip wrote:
> Vinay Sajip <vinay_sajip <at> yahoo.co.uk> writes:
> 
>> I think the problem is occurring because there's incomplete support for running
>> from a source build.
> 
> To confirm this, I installed pythonv - and then running pmv.py produces the
> expected result - an env with a copied Python which allows you to install stuff
> into the env. I packaged the pythonv I used as a .deb using checkinstall, so
> that it's uninstallable:
> 
> http://www.red-dove.com/pythonv_3.3-1_i386.deb
> 
> and having installed it, I re-ran pmv and did some smoke testing, output is at
> 
> http://paste.pocoo.org/show/363186/
> 
> I know we're not done yet, but at least this shows that the getpath.c code seems
> to be working (because sys.path seems correct when running the env's copy of
> Python) and distribute_setup.py installs without errors, followed by
> easy_installing sample packages setuptools-git, ply, Pygments, Jinja2,
> SQLAlchemy and coverage ... all seemingly without problems.
> 
> Of course, this is still using sys.prefix/sys.exec_prefix.

Looks great! I can confirm it works when installed. I'll look into the
source-build issues.

Carl

From vinay_sajip at yahoo.co.uk  Sat Apr  2 12:35:20 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 2 Apr 2011 10:35:20 +0000 (UTC)
Subject: [Distutils] pythonv, take two
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<4D8172FB.3020109@oddbird.net>	<loom.20110317T112246-771@post.gmane.org>	<4D8233EF.7070205@oddbird.net>	<loom.20110318T004329-895@post.gmane.org>	<4D83A807.30003@oddbird.net>	<loom.20110329T154252-221@post.gmane.org>	<4D920A7D.9040109@oddbird.net>	<loom.20110330T005934-410@post.gmane.org>	<4D9369E5.1050506@oddbird.net>	<loom.20110331T005457-50@post.gmane.org>	<4D941203.3020804@oddbird.net>
	<loom.20110331T090727-413@post.gmane.org>
	<4D961605.7050307@oddbird.net>
Message-ID: <loom.20110402T121823-949@post.gmane.org>

Carl Meyer <carl <at> oddbird.net> writes:

> 

> People tend to create a lot of virtualenvs. I don't think copying the
> entire Python installation is viable (it is slow, and it also would mean
> that security updates to the base standard library would not be
> reflected in the virtualenv).

You're probably right - I haven't actually tried it.

> But, as I mentioned, the fact that easy_install itself relies on finding
> site-packages relative to sys.prefix forces the issue - easy_install
> (including existing versions of it) needs to just work.

Presumably this means that we can't use site_prefix in the short term, and just
have it there to allow code to evolve compatibility to it. So everything else
just uses site.prefix and site.exec_prefix as it does currently, and we achieve
parity with virtualenv as far as compatibility is concerned. Internally, we use
sys.installed_prefix and sys.installed_exec_prefix when we need to. We still
benefit from the fact that no copying of the standard lib is needed, we're just
using the stock site.py etc.

BTW you might find this script handy while testing:

https://gist.github.com/896831

Regards,

Vinay Sajip


From pje at telecommunity.com  Sat Apr  2 17:08:29 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Sat, 02 Apr 2011 11:08:29 -0400
Subject: [Distutils] pythonv, take two
In-Reply-To: <4D961605.7050307@oddbird.net>
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>
	<4D8172FB.3020109@oddbird.net>
	<loom.20110317T112246-771@post.gmane.org>
	<4D8233EF.7070205@oddbird.net>
	<loom.20110318T004329-895@post.gmane.org>
	<4D83A807.30003@oddbird.net>
	<loom.20110329T154252-221@post.gmane.org>
	<4D920A7D.9040109@oddbird.net>
	<loom.20110330T005934-410@post.gmane.org>
	<4D9369E5.1050506@oddbird.net>
	<loom.20110331T005457-50@post.gmane.org>
	<4D941203.3020804@oddbird.net>
	<loom.20110331T090727-413@post.gmane.org>
	<4D961605.7050307@oddbird.net>
Message-ID: <20110402150842.2D0493A40A4@sparrow.telecommunity.com>

At 02:14 PM 4/1/2011 -0400, Carl Meyer wrote:
>But, as I mentioned, the fact that easy_install itself relies on finding
>site-packages relative to sys.prefix forces the issue - easy_install
>(including existing versions of it) needs to just work.

No, it doesn't, actually.

easy_install only uses sys.prefix to identify "site" directories 
without having to test them.  Its install paths actually come from 
the distutils -- so as long as the distutils "install" command has 
the right defaults, easy_install will work just fine.  (Assuming 
sys.executable is correct.)

What easy_install does if it's asked to install to a directory that:

1) doesn't look like a site directory (based on sys.prefix), and
2) isn't listed in --site-dirs on the command line or in the config files,

is that it runs a test (using sys.executable) to see if .pth files 
are actually supported in that directory.

So, if sys.prefix leads it to believe that the distutils-supplied 
target directory isn't really a "site" directory, it checks for 
itself whether it functions as a site directory.

In other words, easy_install will run a bit slower in such a scenario 
(due to the extra time taken to test), but still function 
correctly.  To stop it taking the extra time for the test, an 
additional command line option or configuration file option can be given.


From vinay_sajip at yahoo.co.uk  Sun Apr  3 13:02:40 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sun, 3 Apr 2011 11:02:40 +0000 (UTC)
Subject: [Distutils] pythonv, take two
References: <E1Q00w8-0004a6-3l@lettuce.threepines.org>	<4D8172FB.3020109@oddbird.net>	<loom.20110317T112246-771@post.gmane.org>	<4D8233EF.7070205@oddbird.net>	<loom.20110318T004329-895@post.gmane.org>	<4D83A807.30003@oddbird.net>	<loom.20110329T154252-221@post.gmane.org>	<4D920A7D.9040109@oddbird.net>	<loom.20110330T005934-410@post.gmane.org>	<4D9369E5.1050506@oddbird.net>	<loom.20110331T124928-850@post.gmane.org>
	<loom.20110331T145603-128@post.gmane.org>
	<4D96168C.6000105@oddbird.net>
Message-ID: <loom.20110403T125245-924@post.gmane.org>

Carl Meyer <carl <at> oddbird.net> writes:

> Looks great! I can confirm it works when installed. I'll look into the
> source-build issues.

I've also updated pmv.py at

https://gist.github.com/893507

to be closer to what we want for virtualize.py in the stdlib, so that you can do

python3.3 -m virtualize /tmp/testenv

and have it work:

vinay at eta-natty:~/projects/scratch$ python3.3 -m virtualize -h
usage: virtualize.py [-h] [--no-distribute] [--no-site-packages] [--clear]
ENV_DIR [ENV_DIR ...]
 
positional arguments:
ENV_DIR A directory to create the environment in.
 
optional arguments:
-h, --help show this help message and exit
--no-distribute Don't install Distribute in the virtual environment.
--no-site-packages Don't give access to the global site-packages dir to the
virtual environment.
--clear Delete the environment directory if it already exists.
If not specified and the directory exists, an error is
raised.
vinay at eta-natty:~/projects/scratch$ python3.3 -m virtualize /tmp/testenv
Installing distribute
..........................................................................
..........................................................................
..........................................................................
..........................................................................
.............................done.
vinay at eta-natty:~/projects/scratch$


From reinout at vanrees.org  Sun Apr  3 22:05:58 2011
From: reinout at vanrees.org (Reinout van Rees)
Date: Sun, 03 Apr 2011 22:05:58 +0200
Subject: [Distutils] setuptools download error?
Message-ID: <inajv6$ahb$1@dough.gmane.org>

Hi,


My setuptools-using colleagues have problems with downloading 
setuptools.  Initially I suspected the 0.6c12-r12345something releases, 
that old release numbers were disappearing. But that's apparently not 
the case, even though having such version numbers downloaded by a 
standard buildout bootstrap worries me a bit.

I tried using "wget" to download one such release and it also failed. It 
would download 95% of the file and then stall...

Pinning setuptools to 06c11 (the latest stable) worked like a charm: 
that one downloaded just fine (or was already available in the cache, 
probably).

Is there something wrong with the peak server? Should I fix something on 
our end? (I'm using distribute myself, so I don't have this problem).


Reinout


From pje at telecommunity.com  Mon Apr  4 04:17:44 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Sun, 03 Apr 2011 22:17:44 -0400
Subject: [Distutils] setuptools download error?
In-Reply-To: <inajv6$ahb$1@dough.gmane.org>
References: <inajv6$ahb$1@dough.gmane.org>
Message-ID: <20110404021756.C6FA93A4063@sparrow.telecommunity.com>

At 10:05 PM 4/3/2011 +0200, Reinout van Rees wrote:
>Hi,
>
>
>My setuptools-using colleagues have problems with downloading 
>setuptools.  Initially I suspected the 0.6c12-r12345something 
>releases, that old release numbers were disappearing. But that's 
>apparently not the case, even though having such version numbers 
>downloaded by a standard buildout bootstrap worries me a bit.
>
>I tried using "wget" to download one such release and it also 
>failed. It would download 95% of the file and then stall...
>
>Pinning setuptools to 06c11 (the latest stable) worked like a charm: 
>that one downloaded just fine (or was already available in the 
>cache, probably).
>
>Is there something wrong with the peak server? Should I fix 
>something on our end? (I'm using distribute myself, so I don't have 
>this problem).

You're not the only one who has reported this problem, but I've never 
been able to reproduce it.  I have no clue what's going on, unless 
it's a matter of what time of day it works vs. not.  It's a 
bog-standard Apache server serving a simple file, so I have no clue 
what would make it stick like that.  Suggestions welcome.

(If all else fails, remember that the dev snapshots are pulled from 
svn.python.org...  so you can actually use a svn url instead.)


From chris at simplistix.co.uk  Mon Apr  4 19:16:10 2011
From: chris at simplistix.co.uk (Chris Withers)
Date: Mon, 04 Apr 2011 18:16:10 +0100
Subject: [Distutils] Stop using setuptools or distribute in buildout
 (was Re: Fwd: Re: window 64bit madness)
In-Reply-To: <AANLkTinjDh=x7=1pCx+sShfBD12B0BtNO=ZU50SDMyP8@mail.gmail.com>
References: <AANLkTimmkQN8Uzi0GCfFMDBM_RqSOF26LrfdMEPTzsne@mail.gmail.com>	<AANLkTimWrM=4Tt62gZSTmzu5NTr61vdtUL172-+rhzpw@mail.gmail.com>
	<AANLkTinjDh=x7=1pCx+sShfBD12B0BtNO=ZU50SDMyP8@mail.gmail.com>
Message-ID: <4D99FCDA.4020405@simplistix.co.uk>

On 19/03/2011 16:04, Jim Fulton wrote:
> Buildout uses the easy_install command to build, but it would do
> better to just invoke a project's setup script, possibly after
> arranging that certain other required packages are accessible to the
> setup script.

Support for setup_requires would be extremely welcome!

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From agroszer at gmail.com  Tue Apr  5 09:16:48 2011
From: agroszer at gmail.com (Adam GROSZER)
Date: Tue, 05 Apr 2011 09:16:48 +0200
Subject: [Distutils] window 64bit madness
In-Reply-To: <20110316224724.29CC53A4062@sparrow.telecommunity.com>
References: <4D7E28C0.3080204@gmail.com>	<20110314185106.CF9863A4077@sparrow.telecommunity.com>	<4D8090D0.5050902@gmail.com>
	<20110316224724.29CC53A4062@sparrow.telecommunity.com>
Message-ID: <4D9AC1E0.8060601@gmail.com>

Hello,

On 03/16/2011 11:47 PM, P.J. Eby wrote:
> At 11:28 AM 3/16/2011 +0100, Adam GROSZER wrote:
>> On 03/14/2011 07:50 PM, P.J. Eby wrote:
>>> Run "python -c 'import pkg_resources;print
>>> pkg_resources.get_build_platform()'" (with the Python interpreter you're
>>> using.
>>
>> D:\install>c:\Python26_64\python.exe
>> Python 2.6.6 (r266:84297, Aug 24 2010, 18:13:38) [MSC v.1500 64 bit
>> (AMD64)] on win32
>> Type "help", "copyright", "credits" or "license" for more information.
>> >>> import pkg_resources;print pkg_resources.get_build_platform()
>> win-amd64
>> >>>
>
> Hm. What's sys.platform? Is it win32?
>
> I suspect the problem has to do with all the win32-specificness
> scattered through setuptools; I think I know what I need to do to fix it
> all, but it'll be a fairly substantial patch; can you help with the
> testing if I email you such a patch?

Here is a changeset for distribute that should give some idea what seems 
to make it work

https://bitbucket.org/agroszer/distribute-0.6/changeset/1d3923b865a6

-- 
Best regards,
  Adam GROSZER
--
Quote of the day:
Beware of a dark-haired man with a loud tie.

From barry at python.org  Tue Apr  5 16:31:08 2011
From: barry at python.org (Barry Warsaw)
Date: Tue, 5 Apr 2011 10:31:08 -0400
Subject: [Distutils] DRAFT PEP 396 - module version number
In-Reply-To: <4D8C035E.4050306@netwok.org>
References: <20110324141747.6d90382e@neurotica.wooz.org>
	<4D8C035E.4050306@netwok.org>
Message-ID: <20110405103108.77f144a2@neurotica.wooz.org>

Hi ?ric,

Thanks for the feedback, and my apologies for the delay in responding.

On Mar 25, 2011, at 03:52 AM, ?ric Araujo wrote:

>> `distutils2` [1]_) may be adapted to use the conventions defined here.
>Nit: I never rely on docutils/sphinx default role (using a bare `text`
>without explicit :role:), cause I never remember what it is, and the
>produced text may not be what you thought it would.

EIBTI :).  Thanks.

>> Alice is writing a new module, called ``alice.py``, which she wants to
>> share with other Python developers.  ``alice.py`` is a simple module
>> and lives in one file.
>You should call the module alice and the file alice.py to eliminate
>weirdness.  (Ditto for other examples throughout the PEP.)

Excellent point.

>> Bob learns about the Cheeseshop [2]_, and adds some
>> simple packaging using classic distutils so that he can upload *The
>> Bob Package* to the Cheeseshop.
>I know that the referenced PEP 345 call these things ?software
>packages?, but let?s try to use consistent terminology.  See threads
>starting at
>http://mail.python.org/pipermail/distutils-sig/2011-March/017438.html
>and http://mail.python.org/pipermail/distutils-sig/2011-March/017450.html .

I remember that discussion, but I don't actually remember if there was
agreement on terminology. ;)  I'm sympathetic to Jim's Elaboration On PJE's
Glossary.  That would make a worthy PEP for someone else to write <wink>.

In this case I'm changing it to *The Bob Bundle* since that's sufficiently
meaningless as to not be distracting.

>> Carole maintains several namespace packages, each of which are
>> independently developed and distributed.
>I am not sure we should advertise setuptools namespace packages, given
>that standardization is under way (PEP 382).  On one hand it would be
>childish not to acknowledge that setuptools is widely used, on the other
>hand in this particular time and place I think we should wait for
>official namespace packages to be implemented and talk about those.
>That said, even if the mechanism changes, the packaging-related parts
>won?t, so the story is useful.  Make that +0 after all.

Right.  This is just a use case and it doesn't mention setuptools
specifically, even though it does mention setup.py.  Everybody knows what the
latter is, and I can't think of any better way to phrase this, so I guess I'll
leave it.

>> at *sdist* tarball build time.
>Do non-UNIX people understand ?tarball??  You could remove it or replace
>it with ?archive?.

Good point.  I've rewritten this to read:

"...extract the module version number when she builds the *sdist* archive."

>> David maintains a package in the standard library, and also produces
>> standalone versions for other versions of Python.
>Thoughtful hat tip :)

Kind of worked out that way! :)

>> Often, version numbers
>> can be retrieved from a module programmatically, by importing the
>> module and inspecting an attribute.  Classic Python distutils
>> ``setup()`` functions [3]_ describe a ``version`` argument where the
>> package's version number can be specified.
>See above for package vs. distribution terminology.

s/package's version number/release's version number/

>This makes me think: It is obvious to us that Python packages are a type
>of Python modules, but the PEP could mention that near the beginning, to
>make clear that your PEP is relevant to any kind of module, whatever its
>physical incarnation.

Hmmm.  OT1H I can see your point, but OTOH, I can't really find a good place
to squeeze that definition in, and I'm not certain defining this is a goal of
the PEP.  For example, I don't really like adding that to the Abstract, or as
a separate section between Abstract and User Stories.  I kind of think that it
should be fairly self-evident for any experienced Pythonista.

>> of an ``__version__`` module attribute by independent module
>s/an/a/

Heh, yeah I'm old skool so I still pronounce it "under under".  Saying
"dunder" just reminds me too much of The Office to take seriously. :)

(Sadly, I just missed April Fools, or I would have suggested a PEP to add a
__mifflin__ attribute to all objects.  What joy that would bring to Python
conversations.)

>> Another example of version information is the sqlite3 [5]_ library
>the sqlite3 module
>
>> This informational PEP codifies established practice, and recommends
>> standard ways of describing module version numbers, along with some
>> use cases for when -- and when *not* -- to include them.  Its adoption
>> is purely voluntary, but other tools in the Python universe may
>> support the standards defined herein.
>You say ?other tools? without having referred to tools before, I find it
>confusing (I?m not a native speaker, however).  The text could mention
>here that official tools will have opt-in support for the PEP:
>
>  Its adoption is purely voluntary; the packaging tools in the standard
>  library will provide optional support for the standards defined herein
>  (see section XX), and other tools in the Python universe may comply
>  too.

I rewrote this just slightly:

    This informational PEP codifies established practice, and recommends
    standard ways of describing module version numbers, along with some
    use cases for when -- and when *not* -- to include them.  Its adoption
    by module authors is purely voluntary; packaging tools in the standard
    library will provide optional support for the standards defined
    herein, and other tools in the Python universe may comply as well.

>> #. On a case-by-case basis, standard library modules which are also
>>    distributed as standalone packages for other Python versions MAY
>Problematic ?packages? again.  What about this: ?which are also released
>in standalone form for other Python versions??

Looks good, thanks.

>> #. For modules which are also packages, the top level module namespace
>>    SHOULD include the ``__version__`` attribute.
>Just a remark: I don?t remember ever reading the term ?top-level module
>namespace?.  It?s not hard to understand, but it might be helpful to
>some readers if you add ?(i.e. the :file:`{somepackage}/__init__.py`
>file)?.  (The brackets will cause the enclosed text to be marked up as
>replaceable text, just a nicety.)

How about just removing "top-level"?

    #. For modules which are also packages, the module namespace SHOULD
       include the ``__version__`` attribute.

(Aside: using :file:`file.py` produces a warning when the PEP html is
generated:

pep-0396.txt:31: (ERROR/3) Unknown interpreted text role "file".)

>> #. For modules which live inside a namespace package, the sub-package
>>    name SHOULD include the ``__version__`` attribute.
>I think this works with both setuptools and PEP 382 namespace packages,
>which is nice (see above questioning).
>
>> The namespace module itself SHOULD NOT include its own
>> ``__version__`` attribute.
>I guess this makes sense for setuptools namespace packages, but from my
>understanding of PEP 382, it is possible to have a Python package that
>is a namespace package and has submodules.  (I hope Martin will correct
>me if needed.)  This thing (?portion? in PEP 382 lingo) should be
>allowed to declare a version IMO.

Agreed, if that is true.  I kind of hope it's *not* true though!  Anyway, this
is a SHOULD NOT not MUST NOT so I think it's safe.

>> #. The ``__version__`` attribute's value SHOULD be a simple string.
>What?s a non-simple string?  This may be only me, but here I think you
>tried to be simple and had the opposite effect.  I?d just use the simple
>?a string? or the precise ?a string (str object in 3.x)?.

s/simple//

>> #. The ``version`` attribute in a classic distutils ``setup.py`` file,
>>    or the ``Version`` metadata field in a PEP 345 [7]_ SHOULD be
>>    derived from the ``__version__`` field, or vice versa.
>?In a PEP 345? what? :) file, I guess.

Oops, IIUC there really isn't a PEP 345 "thing" so I've rewritten it like
this:

    #. The ``version`` attribute in a classic distutils ``setup.py`` file,
       or the PEP 345 [7]_ ``Version`` metadata field SHOULD be derived
       from the ``__version__`` field, or vice versa.

>I think this part should either take a position, that is choose one way
>instead of vaguely telling about two ways, or be removed, given that the
>section Deriving has the same info.

I'm not sure we can make that recommendation yet though.  I'd prefer to derive
everything from the __version__, but I'm not sure there's agreement on that.
Leaving it this way for now allows for some experimentation.

>> They also need to be specified in either the classic distutils
>> ``setup()`` function, as the ``version`` attribute in a ``setup.py``,
>> or in the distutils2 ``setup.cfg`` file's ``version`` keyword,
>I?d say the setup.py has a version argument and the setup.cfg has a
>version key.
>
>> they need to get into the PEP 345 package metadata, preferably at
>> *sdist* package build-time.
>s/package metadata/metadata/; s/*sdist* package/*sdist*/

    Module version numbers can appear in at least two places, and sometimes
    more.  For example, in accordance with this PEP, they are available
    programmatically on the module's ``__version__`` attribute.  In a classic
    distutils ``setup.py`` file, the ``setup()`` function takes a ``version``
    argument, while the distutils2 ``setup.cfg`` file has a ``version`` key.
    The version number must also get into the PEP 345 metadata, preferably
    when the *sdist* archive is built.  It's desirable for module authors to
    only have to specify the version number once, and have all the other uses
    derive from this single definition.

>> In the PEP author's experience however, this can fail in some cases,
>> such as when the module uses automatic Python 3 conversion via the
>> ``2to3`` program.
>Just to make this 100% explicit, you could add ?because the setup.py
>script is executed by Python 3 before the elle module has been converted?.

I added this, though I'm not sure if flows as well.

>> In that case, it's not much more difficult to write a little code
>> to parse the ``__version__`` from the file rather than
>The function could be simplified (use re.search, hard-code the re), but
>that?s very minor.

Indeed.

>
>> Because the distutils2 style ``setup.cfg`` is declarative, we can't
>> run any code to extract the ``__version__`` attribute, either via
>> import or via parsing.  This PEP suggests a special key be allowed for
>> the ``version`` field in a ``setup.cfg`` file to indicate "get the
>> version from this file".
>With my distutils2 hat on, I recommend a KISS approach similar to what?s
>done for the long description: just define another key in the setup.cfg
>specification:
>
>  [metadata]
>  version-from-file: elle.py

It's an open issue, but we definitely want built-in support from
distutils2/packaging.  This section was one of the main reasons why I posted
here first, because I really wanted to get feedback on how best to support
this in a setup.cfg file.  I've changed the example to your text above, since
it does seem cleaner.  Though if we get more universal adoption of the PEP, I
suspect version-from-file will be much more popular than version, so we might
want to address keeping the common case simple.

>> An alternative is to only define the version number in setup.cfg and
>> use the ``pkgutil`` module [8]_ to make it available
>> programmatically.  E.g. in ``elle.py``::
>> 
>>     from distutils2._backport import pkgutil
>>     __version__ = pkgutil.get_distribution('elle').metadata['version']
>One of the main complaint against setuptools is that having to change
>your application code because of the packaging tool used was not a good
>idea.  Having to use require instead of import or resource_whatever
>instead of open (or get_data, the most sadly underused function in the
>stdlib) because you use setuptools instead of distutils was a bad thing.
>
>As stated in the PEP, having a __version__ attribute in the module is
>common, so my opinion is that making the packaging tool use that info is
>the Right Thing?, and having the dependency in the reverse sense is
>wrong.  I don?t see a problem with having harmless duplication in the
>*installed* system, once in  elle.__version__ and once in the pkgutil
>metadata database.

I'm including this section because at Pycon, some people did express an
interest in deriving the version number in this direction.  I wanted to
capture what that might look like.  Since this is an informational PEP, I
think it makes sense to include alternative approaches, but I tend to agree
with you that it will be much more common to define module.__version__ and
derive the metadata from that.

>(FWIW, I personally think that it?s not crazy to have the version number
>duplicated in the module, setup script, changelog, Sphinx conf.py file
>and so on; other people like Ronny don?t even want to have the version
>number in one file, but make their build process use the VCS tag.  I
>think this PEP addresses a useful middle ground.)

Cool.

>
>> Static egg info
>Is that the distutils 2.5+ generated metadata?  You may want to read PEP
>376 and update the terminology to ?dist-info metadata? <0.2 wink>.

I meant it in more general terms, that any metadata using the derived version
number is better off being derived at build time rather than install time.
PEP 376 doesn't really talk much about the process by which the metadata files
are created, so I'm not sure referencing it is appropriate.  Hmm, let me think
about that section.

(Side note: PEP 376 could probably benefit from a glossary.  Its abstract
starts off by using the term "project distributions" but doesn't provide any
definition of what "project" means or what "distributions" are.  As I
mentioned before, if you want folks to adopt a specific terminology, I think
you're going to need a separate informational PEP that explicitly defines
that terminology.)

Hmm, okay, I took a crack at rewriting this section:

    PEP 376 metadata
    ================

    PEP 376 [9]_ defines a standard for static metadata, but doesn't
    describe the process by which this metadata gets created.  It is
    highly desirable for the derived version information to be placed into
    the PEP 376 ``.dist-info`` metadata at build-time rather than
    install-time.  This way, the metadata will be available for
    introspection even when the code is not installed.


>> For example, a package's metadata can be made available via a simple
>> HTTP GET operation on a URL within the package's presence on the
>> Cheeseshop.
>First, insert usual remark about ?package? here; second, I think
>discussing PyPI is out of scope here.  The rationale that ?the metadata
>will be available for introspection even when the code is not installed?
>is already strong enough.
>
>> .. [1] Distutils2 documentation
>>    (http://distutils2.notmyidea.org/)
>Alexis, is this still the good link to use?

Again, thanks for the feedback.  I'm going to do another pass through the PEP,
then check it in, and post it to python-dev.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110405/fd65982f/attachment.pgp>

From barry at python.org  Tue Apr  5 16:48:38 2011
From: barry at python.org (Barry Warsaw)
Date: Tue, 5 Apr 2011 10:48:38 -0400
Subject: [Distutils] DRAFT PEP 396 - module version number
In-Reply-To: <201103271536.10593.ygingras@ygingras.net>
References: <20110324141747.6d90382e@neurotica.wooz.org>
	<4D8C035E.4050306@netwok.org>
	<201103271536.10593.ygingras@ygingras.net>
Message-ID: <20110405104838.2780609a@neurotica.wooz.org>

On Mar 27, 2011, at 03:36 PM, Yannick Gingras wrote:

>On March 24, 2011, ?ric Araujo wrote:
>> > Bob learns about the Cheeseshop [2]_, and adds some
>> > simple packaging using classic distutils so that he can upload *The
>> > Bob Package* to the Cheeseshop.
>> I know that the referenced PEP 345 call these things ?software
>> packages?, but let?s try to use consistent terminology.  See threads
>> starting at
>> http://mail.python.org/pipermail/distutils-sig/2011-March/017438.html
>> and http://mail.python.org/pipermail/distutils-sig/2011-March/017450.html .
>
>?ric, can you summarize the discussion that took place on these
>threads?  As far as I can tell, no consensus was reached for a new
>terminology and so the old definition still prevails:
>
>  http://docs.python.org/dev/distutils/introduction#concepts-terminology

As I mentioned in my follow up to ?ric, if you really want to encourage
adoption of specific terms, you're probably going to need to write a PEP that
clearly defines them.  Referencing a mailing list thread ain't gonna cut it.

>In that case, a "package" in PEP 396 would refer to:
>
>  a module that contains other modules; typically contained in a
>  directory in the filesystem and distinguished from other directories
>  by the presence of a file __init__.py.
>
>I fell like the usage proposed for *The Bob Package* is pretty good,
>except for the choice of "upload".  Of course, you don't upload the
>package, you upload "distributions" that contain that package.

See my previous follow up.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110405/96100253/attachment.pgp>

From eric at trueblade.com  Tue Apr  5 17:33:20 2011
From: eric at trueblade.com (Eric Smith)
Date: Tue, 05 Apr 2011 11:33:20 -0400
Subject: [Distutils] [buildout] "private" releases
In-Reply-To: <4D8CEBD5.7010903@simplistix.co.uk>
References: <imih2a$uig$1@dough.gmane.org> <4D8CEBD5.7010903@simplistix.co.uk>
Message-ID: <4D9B3640.4090300@trueblade.com>

On 03/25/2011 03:24 PM, Chris Withers wrote:
> Hi Christian,
> 
> On 25/03/2011 16:49, Christian Theune wrote:
>> the German speaking Zope Users Group (DZUG e.V.) organizes a series of 4
>> sprints this year to support feature development within the proximity of
>> the ZTK and solve problems encountered by Zope, Plone and Python
>> developers.
>>
> <snip>
>> Topics
>> ======
>>
>> * Discussing how to deal with "private" releases
> 
> FWIW, I've had no problems with this, here's a sample buildout.cfg:
> 
> [buildout]
> extensions = lovely.buildouthttp
> find-links = https://example.com/password/protected/folder
> 
> ...and just dump the .tgz sdists in that folder.
> 
> Of course, if you don't need password protection such as when you have
> your "egg server" on a private network, you just need the find-links.
> 
> I'm not really sure why people have written a complicated array of "egg
> servers" and the like when a simple http or file system served folder is
> just fine ;-)

Could you elaborate on this? How does buildout/setuptools/distribut
search for an sdist, lacking an index file? Does it look for .tgz only,
or other extensions as well? This would save me a ton of heartburn if I
could get it to work.

Thanks.
Eric.

From pje at telecommunity.com  Tue Apr  5 18:41:47 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 05 Apr 2011 12:41:47 -0400
Subject: [Distutils] DRAFT PEP 396 - module version number
In-Reply-To: <20110405103108.77f144a2@neurotica.wooz.org>
References: <20110324141747.6d90382e@neurotica.wooz.org>
	<4D8C035E.4050306@netwok.org>
	<20110405103108.77f144a2@neurotica.wooz.org>
Message-ID: <20110405164208.BB5DC3A4063@sparrow.telecommunity.com>

At 10:31 AM 4/5/2011 -0400, Barry Warsaw wrote:
>On Mar 25, 2011, at 03:52 AM, ??ric Araujo wrote:
> >I am not sure we should advertise setuptools namespace packages, given
> >that standardization is under way (PEP 382).  On one hand it would be
> >childish not to acknowledge that setuptools is widely used, on the other
> >hand in this particular time and place I think we should wait for
> >official namespace packages to be implemented and talk about those.

Just as an FYI, "official" namespace packages have existed since 
Python 2.3; they are implemented using pkgutil.extend_path() -- 
added, IIRC, back when Guido and Barry were both working at Zope Corp.  ;-)

Setuptools and PEP 382 are simply enhanced implementations of the concept.


> >> #. For modules which live inside a namespace package, the sub-package
> >>    name SHOULD include the ``__version__`` attribute.
> >I think this works with both setuptools and PEP 382 namespace packages,
> >which is nice (see above questioning).
> >
> >> The namespace module itself SHOULD NOT include its own
> >> ``__version__`` attribute.
> >I guess this makes sense for setuptools namespace packages, but from my
> >understanding of PEP 382, it is possible to have a Python package that
> >is a namespace package and has submodules.  (I hope Martin will correct
> >me if needed.)  This thing (???portion??? in PEP 382 lingo) should be
> >allowed to declare a version IMO.
>
>Agreed, if that is true.  I kind of hope it's *not* true though!  Anyway, this
>is a SHOULD NOT not MUST NOT so I think it's safe.

In principle, if you simply *must* have a non-empty __init__ module 
in a namespace package, and you really need to have a __version__ of 
that, then I suppose its __version__ should match the version of the 
project that supplies it.

But yeah, I think it should still be considered a poor practice to 
put stuff in a namespace's __init__, even if/when PEP 382 makes it 
practical to implement non-empty __init__'s.


From suresh_vv at yahoo.com  Tue Apr  5 19:34:56 2011
From: suresh_vv at yahoo.com (Suresh V.)
Date: Tue, 05 Apr 2011 23:04:56 +0530
Subject: [Distutils] passing command-line options to an additional
	python interpreter?
In-Reply-To: <B769ED2A-56B0-4442-A88C-1088382E299F@shotgunsoftware.com>
References: <B769ED2A-56B0-4442-A88C-1088382E299F@shotgunsoftware.com>
Message-ID: <4D9B52C0.2050207@yahoo.com>

cp.recipe.cmd may be helpful


Mark Visser wrote:
> I'm using buildout for managing a project that runs on several different python interpreters, some of which are embedded in other applications. As such, I'm at the whim of those packaging python for those applications, and sometimes they don't set things up... well, optimally.
> 
> The problem I've run into is that one of the exposed non-standard python interpreters (mayapy) is actually a shell script. Thus, you can't use its direct path with a shebang. Normally, I get around this by using #!/usr/bin/env mayapy.
> 
> For the life of me, I can't figure out a way to pass this to buildout. I've tried this:
> 
> [mayapy]
> executable = /usr/bin/env mayapy
> 
> [mayatests]
> python = mayapy
> ... etc
> 
> But buildout doesn't like the space. Is there a way to specify executable options? I tried this to no avail:
> 
> [mayapy]
> executable = /usr/bin/env
> options = mayapy
> 
> Can anyone suggest a solution?
> 
> thanks,
> -Mark
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
> 


From chris at simplistix.co.uk  Wed Apr  6 09:21:36 2011
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 06 Apr 2011 08:21:36 +0100
Subject: [Distutils] [buildout] "private" releases
In-Reply-To: <4D9B3640.4090300@trueblade.com>
References: <imih2a$uig$1@dough.gmane.org> <4D8CEBD5.7010903@simplistix.co.uk>
	<4D9B3640.4090300@trueblade.com>
Message-ID: <4D9C1480.4020208@simplistix.co.uk>

On 05/04/2011 16:33, Eric Smith wrote:
> Could you elaborate on this? How does buildout/setuptools/distribut
> search for an sdist, lacking an index file? Does it look for .tgz only,
> or other extensions as well? This would save me a ton of heartburn if I
> could get it to work.

Point find-links at either a folder on disk, or that sample folder 
served over http by Apache's normal folder serving (ie: index listings on).

Just stick any type of dist into that folder.

The rest "just works".

If it doesn't, explain what you did and we'll see if we can spot what 
went wrong...

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From lucaskt at linux.vnet.ibm.com  Fri Apr  8 15:57:35 2011
From: lucaskt at linux.vnet.ibm.com (Lucas Kannebley Tavares)
Date: Fri, 08 Apr 2011 10:57:35 -0300
Subject: [Distutils] Trouble with bdist_rpm and data_files
Message-ID: <4D9F144F.1090705@linux.vnet.ibm.com>

Hello all,

I wrote a setup.py file and managed to get debhelper to create a working 
deb package out of it. But I am having trouble with creating a binary 
rpm package. Apparently all data files are not found by rpmbuild.

running install_data
error: can't copy 'etc/proj.conf': doesn't exist or not a regular file
error: Bad exit status from /var/tmp/rpm-tmp.drqkKR (%install)

(I changed the file name for proj.conf in the e-mail as to not divoulge 
any corporate information)

But that file does indeed exist. It is at etc/file.conf in the repo, and 
it should be moved to /etc/proj/cli/proj.conf

The BUILDROOT/etc/proj/cli directory is properly created.

There are other data files listed, but none of them are copied properly, 
even if I change their order.

Any help on this would be appreciated.
Thank you.

-- 
Lucas Kannebley Tavares
Infrastructure Team
IBM Linux Technology Center


From lucaskt at linux.vnet.ibm.com  Fri Apr  8 17:10:11 2011
From: lucaskt at linux.vnet.ibm.com (Lucas Kannebley Tavares)
Date: Fri, 08 Apr 2011 12:10:11 -0300
Subject: [Distutils] Trouble with bdist_rpm and data_files
In-Reply-To: <4D9F144F.1090705@linux.vnet.ibm.com>
References: <4D9F144F.1090705@linux.vnet.ibm.com>
Message-ID: <4D9F2553.7050108@linux.vnet.ibm.com>

On 04/08/2011 10:57 AM, Lucas Kannebley Tavares wrote:
> Hello all,
>
> I wrote a setup.py file and managed to get debhelper to create a working
> deb package out of it. But I am having trouble with creating a binary
> rpm package. Apparently all data files are not found by rpmbuild.
>
> running install_data
> error: can't copy 'etc/proj.conf': doesn't exist or not a regular file
> error: Bad exit status from /var/tmp/rpm-tmp.drqkKR (%install)
>
> (I changed the file name for proj.conf in the e-mail as to not divoulge
> any corporate information)
>
> But that file does indeed exist. It is at etc/file.conf in the repo, and
> it should be moved to /etc/proj/cli/proj.conf
>
> The BUILDROOT/etc/proj/cli directory is properly created.
>
> There are other data files listed, but none of them are copied properly,
> even if I change their order.
>
> Any help on this would be appreciated.
> Thank you.
>

Problem solved.
I was missing a proper MANIFEST file.
Thank you for you time.

-- 
Lucas Kannebley Tavares
Infrastructure Team
IBM Linux Technology Center


From hersonls at gmail.com  Fri Apr  8 19:13:56 2011
From: hersonls at gmail.com (Herson)
Date: Fri, 8 Apr 2011 14:13:56 -0300
Subject: [Distutils] An internal error occurred due to a bug in either
 zc.buildout or in a recipe being used
Message-ID: <BANLkTin5kiWf1AmcTrznUU=AYWvxX9T8hg@mail.gmail.com>

Hi,

Well guys, I'm having trouble running buildout on Mac OSX leopard. My
environment consists of:

buildout.cfg:

[buildout]
parts = python
develop = .
eggs = teste

[python]
recipe = zc.recipe.egg
interpreter = python
eggs = ${buildout:eggs}

I run the:

$ python bootstrap.py
Downloading
http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c11-py2.6.egg
Creating directory '/Users/hersonls/Code/Python/zc.Buildout/teste/bin'.
Creating directory '/Users/hersonls/Code/Python/zc.Buildout/teste/parts'.
Creating directory '/Users/hersonls/Code/Python/zc.Buildout/teste/eggs'.
Creating directory
'/Users/hersonls/Code/Python/zc.Buildout/teste/develop-eggs'.
Generated script
'/Users/hersonls/Code/Python/zc.Buildout/teste/bin/buildout'.

And run the ./bin/buildout

$ ./bin/buildout
Develop: '/Users/hersonls/Code/Python/zc.Buildout/teste/.'
While:
  Installing.
  Processing develop directory
'/Users/hersonls/Code/Python/zc.Buildout/teste/.'.

An internal error occurred due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
  File
"/private/var/folders/Jp/Jp9P8zk3E9yRgo+ZL4wnx++++TI/-Tmp-/tmpqdrwgm/zc.buildout-1.5.2-py2.6.egg/zc/buildout/buildout.py",
line 1805, in main
  File
"/private/var/folders/Jp/Jp9P8zk3E9yRgo+ZL4wnx++++TI/-Tmp-/tmpqdrwgm/zc.buildout-1.5.2-py2.6.egg/zc/buildout/buildout.py",
line 446, in install
  File
"/private/var/folders/Jp/Jp9P8zk3E9yRgo+ZL4wnx++++TI/-Tmp-/tmpqdrwgm/zc.buildout-1.5.2-py2.6.egg/zc/buildout/buildout.py",
line 686, in _develop
  File
"/private/var/folders/Jp/Jp9P8zk3E9yRgo+ZL4wnx++++TI/-Tmp-/tmpqdrwgm/zc.buildout-1.5.2-py2.6.egg/zc/buildout/easy_install.py",
line 1191, in develop
  File
"/private/var/folders/Jp/Jp9P8zk3E9yRgo+ZL4wnx++++TI/-Tmp-/tmpqdrwgm/zc.buildout-1.5.2-py2.6.egg/zc/buildout/easy_install.py",
line 1124, in _copyeggs
AssertionError: []

Initially I had no issues with buildout, this problem appeared suddenly in a
new project.

Can someone give me a light that might be?

=== Notes ===
Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin

-- 
Herson Leite, Desenvolvedor Web
www.hersonls.com.br
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110408/cb993e25/attachment.html>

From jim at zope.com  Sat Apr  9 02:02:16 2011
From: jim at zope.com (Jim Fulton)
Date: Fri, 8 Apr 2011 20:02:16 -0400
Subject: [Distutils] An internal error occurred due to a bug in either
 zc.buildout or in a recipe being used
In-Reply-To: <BANLkTin5kiWf1AmcTrznUU=AYWvxX9T8hg@mail.gmail.com>
References: <BANLkTin5kiWf1AmcTrznUU=AYWvxX9T8hg@mail.gmail.com>
Message-ID: <BANLkTikmXxV6-NdGonrCDqBEZD6-U_Nkkg@mail.gmail.com>

On Fri, Apr 8, 2011 at 1:13 PM, Herson <hersonls at gmail.com> wrote:
> Hi,
>
> Well guys, I'm having trouble running buildout on Mac OSX leopard. My
> environment consists of:
>
> buildout.cfg:
>
> [buildout]
> parts = python
> develop = .
> eggs = teste
>
> [python]
> recipe = zc.recipe.egg
> interpreter = python
> eggs = ${buildout:eggs}
>
> I run the:
>
> $ python bootstrap.py

...

> And run the ./bin/buildout
>
> $ ./bin/buildout
> Develop: '/Users/hersonls/Code/Python/zc.Buildout/teste/.'
> While:
> ? Installing.
> ? Processing develop directory
> '/Users/hersonls/Code/Python/zc.Buildout/teste/.'.
>
> An internal error occurred due to a bug in either zc.buildout or in a
> recipe being used:

...

> ? File
> "/private/var/folders/Jp/Jp9P8zk3E9yRgo+ZL4wnx++++TI/-Tmp-/tmpqdrwgm/zc.buildout-1.5.2-py2.6.egg/zc/buildout/easy_install.py",
> line 1191, in develop
> ? File
> "/private/var/folders/Jp/Jp9P8zk3E9yRgo+ZL4wnx++++TI/-Tmp-/tmpqdrwgm/zc.buildout-1.5.2-py2.6.egg/zc/buildout/easy_install.py",
> line 1124, in _copyeggs
> AssertionError: []
>
> Initially I had no issues with buildout, this problem appeared suddenly in a
> new project.
>
> Can someone give me a light that might be?

It's trying to copy egg link files that should have been produced by
running the setup develop command, and didn't find any. Could there be
something wrong/odd with the setup file? It's hard for me to say,
since I can't see the setup file.

Jim

--
Jim Fulton
http://www.linkedin.com/in/jimfulton

From merwok at netwok.org  Sat Apr  9 18:23:20 2011
From: merwok at netwok.org (=?UTF-8?Q?=C3=89ric_Araujo?=)
Date: Sat, 09 Apr 2011 18:23:20 +0200
Subject: [Distutils] DRAFT PEP 396 - module version number
In-Reply-To: <20110405103108.77f144a2@neurotica.wooz.org>
References: "<20110324141747.6d90382e@neurotica.wooz.org>"
	<4D8C035E.4050306@netwok.org>
	<20110405103108.77f144a2@neurotica.wooz.org>
Message-ID: <5c2331c6216e5085bdc02629b9a42986@netwok.org>

 Hi,

 Glad to read my review helped :)

>> I know that the referenced PEP 345 call these things ?software
>> packages?, but let?s try to use consistent terminology.  See threads
>> starting at
>> 
>> http://mail.python.org/pipermail/distutils-sig/2011-March/017438.html
>> and 
>> http://mail.python.org/pipermail/distutils-sig/2011-March/017450.html 
>> .
> I remember that discussion, but I don't actually remember if there 
> was
> agreement on terminology. ;)  I'm sympathetic to Jim's Elaboration On 
> PJE's
> Glossary.  That would make a worthy PEP for someone else to write 
> <wink>.

 Ah, right, I probably have extrapolated the distutils2 consensus.  I?m
 not sure I should be the one to write this PEP, I already have enough 
 of
 a nitpicker reputation.  Maybe if a distutils-sig veteran accepts to be
 a co-writer.

> In this case I'm changing it to *The Bob Bundle* since that's 
> sufficiently
> meaningless as to not be distracting.

 I also think that ?bundle? is a nice term to name what the docs
 currently calls a distribution.

>>> Carole maintains several namespace packages, each of which are
>>> independently developed and distributed.
>> I am not sure we should advertise setuptools namespace packages, 
>> given
>> that standardization is under way (PEP 382).  [...]
> Right.  This is just a use case and it doesn't mention setuptools
> specifically, even though it does mention setup.py.  Everybody knows 
> what the
> latter is, and I can't think of any better way to phrase this, so I 
> guess I'll
> leave it.

 As PJE helpfully corrected, namespaces packages are not tied to
 setuptools, even though in practice they mostly are.  The current PEP
 text is therefore fine.

>>> of an ``__version__`` module attribute by independent module
>> s/an/a/
> Heh, yeah I'm old skool so I still pronounce it "under under".  
> Saying
> "dunder" just reminds me too much of The Office to take seriously. :)

 I write about Python much more than I talk about it, so I pronounce it
 ?version? (or more exactly, I say ?init? when I talk about ?__init__?,
 or ?underscore-underscore-init-underscore-underscore? when I need total
 explicitness).  I never read or heard ?dunder?.

>>> Another example of version information is the sqlite3 [5]_ library
>> the sqlite3 module

 You overlooked that one.

>>> #. For modules which are also packages, the top level module 
>>> namespace
>>>    SHOULD include the ``__version__`` attribute.
>> Just a remark: I don?t remember ever reading the term ?top-level 
>> module
>> namespace?.  It?s not hard to understand, but it might be helpful to
>> some readers if you add ?(i.e. the :file:`{somepackage}/__init__.py`
>> file)?.  (The brackets will cause the enclosed text to be marked up 
>> as
>> replaceable text, just a nicety.)
> How about just removing "top-level"?
>
>     #. For modules which are also packages, the module namespace 
> SHOULD
>        include the ``__version__`` attribute.

 I?m still not sure ?module namespace? will be clear to everyone.

> (Aside: using :file:`file.py` produces a warning when the PEP html is
> generated:

 Fooled again: it?s a Sphinx-specific role, not a docutils one.  Forget
 my advice and use ````.

>>> #. The ``version`` attribute in a classic distutils ``setup.py`` 
>>> file,
>>>    or the ``Version`` metadata field in a PEP 345 [7]_ SHOULD be
>>>    derived from the ``__version__`` field, or vice versa.
>> ?In a PEP 345? what? :) file, I guess.
> Oops, IIUC there really isn't a PEP 345 "thing" so I've rewritten it 
> like
> this:

 There is actually a METADATA file, but your rewrite is good without
 mentioning that.

>>> Because the distutils2 style ``setup.cfg`` is declarative, we can't
>>> run any code to extract the ``__version__`` attribute, either via
>>> import or via parsing.  This PEP suggests a special key be allowed 
>>> for
>>> the ``version`` field in a ``setup.cfg`` file to indicate "get the
>>> version from this file".
>> With my distutils2 hat on, I recommend a KISS approach similar to 
>> what?s
>> done for the long description: just define another key in the 
>> setup.cfg
>> specification:
>>
>>  [metadata]
>>  version-from-file: elle.py
> It's an open issue, but we definitely want built-in support from
> distutils2/packaging.  This section was one of the main reasons why I 
> posted
> here first, because I really wanted to get feedback on how best to 
> support
> this in a setup.cfg file.  I've changed the example to your text 
> above, since
> it does seem cleaner.  Though if we get more universal adoption of 
> the PEP, I
> suspect version-from-file will be much more popular than version, so 
> we might
> want to address keeping the common case simple.

 Tarek already ruled last summer that field names in setup.cfg have to
 have their PEP 345 name.  I proposed to merge author name and email 
 into
 the author field, and to have the description field always refer to a
 file: author and author_email are still separate, and a new
 description_from_file fields has been added.  That?s why I think a new
 field has to be defined.  version-from should be a short enough name.  
 I
 also expect most people to use copy-paste or the interactive setup.cfg
 creation helper, so field name length should not be that big of an 
 issue.

 Now on to defuse FUD on python-dev.

 Regards

From kelsey.hightower at gmail.com  Sun Apr 10 17:12:23 2011
From: kelsey.hightower at gmail.com (Kelsey Hightower)
Date: Sun, 10 Apr 2011 11:12:23 -0400
Subject: [Distutils] Packaging and the dependency on distribute/setuptools
Message-ID: <BANLkTimMFUDCa5YhrMAJLsuKbzQYHVnqDQ@mail.gmail.com>

Re-posting for visibility:
http://groups.google.com/group/the-fellowship-of-the-packaging/browse_thread/thread/670085455d44eed5

While working on the new pysetup commands I ran into some issues when
trying to uninstall projects based on distribute/setuptools. The
current install method used in packaging is to "shell" out and call:

  # projects based on distutils

  python setup.py install --record=/path/to/record_file

  # projects based on setuptools

  python setup.py install --record=/path/to/record_file
--single-version-externally-managed"


This works and gets the project installed. However, this does not give
us PEP 376 compatibility and adds a dependency on
distribute/setuptools. Another side effect from the methods above, the
RECORD file does not end up in the right place. I am sure we can fix
this, but do we really want to?

To get everything working it seems we will have to do a custom
distribute/setuptools install similar to how pip does things.

https://github.com/pypa/pip/blob/master/pip/req.py#L540

Note, we do not have the above issue with packaging (setup.cfg) based projects.

At this point it seems we need to pick one of three options:

 1. Only support packaging based projects (setup.cfg)
 2. Only support packaging and distutils based projects, punt on
projects that require distribute/setuptools.
 3. Support all three, packaging, distutils, and distribute/setuptool
based projects.


Pros and Cons
=============

Option 1
--------
Only support packaging based projects(setup.cfg).

Pros

 * enforce the new standard, setup.cfg and static metadata.
 * clear project scope; one way forward
 * no dependencies on 3rd party modules (distribute)
 * chance to make a clean break; distutils is still in the stdlib and
pip supports Python 3

Cons

 * unable to install most projects day one; will have to defer to 3rd
party tools such as pip and distribute


Option 2
--------
Only support installing packaging and distutils based projects and
punt on projects that require distribute/setuptools.

Pros

 * can install more projects than option 1.
 * no dependencies on 3rd party modules (distribute)
 * promote the new standard with a transition for distutils based projects

Cons

 * still supports setup.py
 * causes confusion; can install some projects with a setup.py and not others
 * may prolong the move to setup.cfg and static metadata
 * miss our chance for a clean break

Option 3
--------
Support all three, packaging, distutils, and distribute/setuptools
based projects.

Pros

 * can install more projects than option 1 and option 2

Cons

 * WILL prolong the move to setup.cfg and static metadata. Most people
will continue to use what they know best.
 * miss our chance for a clean break


I am leaning toward Option 1 myself, as option 2 will only cause
confusion. Option 3 seems like going backward by requiring distribute
and will prolong the move to setup.cfg and static metadata. Going with
option 1 will require a lot of effort to get the community on board
and start converting to a valid setup.cfg, or providing support for
both, a valid setup.cfg and setup.py.

With packaging and pysetup being new tools we should keep them clean
and focused on the future. This is especially true when good
alternatives exist to provide a transition strategy for most projects.

Thoughts?

From kelsey.hightower at gmail.com  Sun Apr 10 17:36:57 2011
From: kelsey.hightower at gmail.com (Kelsey Hightower)
Date: Sun, 10 Apr 2011 11:36:57 -0400
Subject: [Distutils] Packaging and the dependency on distribute/setuptools
Message-ID: <BANLkTi=XR_60y-W3h=e7zX6CyE+F1NuGWQ@mail.gmail.com>

Re-posting for visibility:
http://groups.google.com/group/the-fellowship-of-the-packaging/browse_thread/thread/670085455d44eed5

While working on the new pysetup commands I ran into some issues when
trying to uninstall projects based on distribute/setuptools. The
current install method used in packaging is to "shell" out and call:

  # projects based on distutils

  python setup.py install --record=/path/to/record_file

  # projects based on setuptools

  python setup.py install --record=/path/to/record_file
--single-version-externally-managed"


This works and gets the project installed. However, this does not give
us PEP 376 compatibility and adds a dependency on
distribute/setuptools. Another side effect from the methods above, the
RECORD file does not end up in the right place. I am sure we can fix
this, but do we really want to?

To get everything working it seems we will have to do a custom
distribute/setuptools install similar to how pip does things.

https://github.com/pypa/pip/blob/master/pip/req.py#L540

Note, we do not have the above issue with packaging (setup.cfg) based projects.

At this point it seems we need to pick one of three options:

 1. Only support packaging based projects (setup.cfg)
 2. Only support packaging and distutils based projects, punt on
projects that require distribute/setuptools.
 3. Support all three, packaging, distutils, and distribute/setuptool
based projects.


Pros and Cons
=============

Option 1
--------
Only support packaging based projects(setup.cfg).

Pros

 * enforce the new standard, setup.cfg and static metadata.
 * clear project scope; one way forward
 * no dependencies on 3rd party modules (distribute)
 * chance to make a clean break; distutils is still in the stdlib and
pip supports Python 3

Cons

 * unable to install most projects day one; will have to defer to 3rd
party tools such as pip and distribute


Option 2
--------
Only support installing packaging and distutils based projects and
punt on projects that require distribute/setuptools.

Pros

 * can install more projects than option 1.
 * no dependencies on 3rd party modules (distribute)
 * promote the new standard with a transition for distutils based projects

Cons

 * still supports setup.py
 * causes confusion; can install some projects with a setup.py and not others
 * may prolong the move to setup.cfg and static metadata
 * miss our chance for a clean break

Option 3
--------
Support all three, packaging, distutils, and distribute/setuptools
based projects.

Pros

 * can install more projects than option 1 and option 2

Cons

 * WILL prolong the move to setup.cfg and static metadata. Most people
will continue to use what they know best.
 * miss our chance for a clean break


I am leaning toward Option 1 myself, as option 2 will only cause
confusion. Option 3 seems like going backward by requiring distribute
and will prolong the move to setup.cfg and static metadata. Going with
option 1 will require a lot of effort to get the community on board
and start converting to a valid setup.cfg, or providing support for
both, a valid setup.cfg and setup.py.

With packaging and pysetup being new tools we should keep them clean
and focused on the future. This is especially true when good
alternatives exist to provide a transition strategy for most projects.

Thoughts?

From cool-rr at cool-rr.com  Mon Apr 11 21:27:37 2011
From: cool-rr at cool-rr.com (cool-RR)
Date: Mon, 11 Apr 2011 15:27:37 -0400
Subject: [Distutils] Redirect from packages.python.org/my_project to my doc
	site
Message-ID: <BANLkTim_5jnB6Hbb05J065HYRxMRNa7=EQ@mail.gmail.com>

Hello,

Perhaps *packages.python.org* will become the new go-to place where people
expect to find documentation, like PyPI. I prefer to host my docs on my own
server. Can I set something in my project settings that would cause a
redirect to be created from `packages.python.org/my_project` to my own doc
site?


Thanks,
Ram.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110411/50d0de92/attachment.html>

From techtonik at gmail.com  Tue Apr 12 17:44:08 2011
From: techtonik at gmail.com (anatoly techtonik)
Date: Tue, 12 Apr 2011 18:44:08 +0300
Subject: [Distutils] bdist_wininst on unix and install_lib
Message-ID: <BANLkTimU=Xg1d4GTmH+z6ogfoxZuSUPzSw@mail.gmail.com>

Hi,

I want to build binary wininst distributions of pure python package on
unix. That's pretty possible and works already. But I need to patch
`install_lib` and `install_data` targets with custom platform-specific
modifications. I've noticed that bdist_wininst runs `install_*`
targets during the build. The question is:

How can `install_lib` detect if it is executed to build code for
windows platform when it is executed on unix?

Please, CC.
--
anatoly t.

From barry at python.org  Tue Apr 12 20:23:13 2011
From: barry at python.org (Barry Warsaw)
Date: Tue, 12 Apr 2011 14:23:13 -0400
Subject: [Distutils] DRAFT PEP 396 - module version number
In-Reply-To: <5c2331c6216e5085bdc02629b9a42986@netwok.org>
References: <20110324141747.6d90382e@neurotica.wooz.org>
	<4D8C035E.4050306@netwok.org>
	<20110405103108.77f144a2@neurotica.wooz.org>
	<5c2331c6216e5085bdc02629b9a42986@netwok.org>
Message-ID: <20110412142313.34c79298@neurotica.wooz.org>

On Apr 09, 2011, at 06:23 PM, ?ric Araujo wrote:

> Glad to read my review helped :)

Indeed, thanks.

> I also think that ?bundle? is a nice term to name what the docs
> currently calls a distribution.

At the very least, *bundle* isn't completely overloaded 10x over in Pythonland
yet. :)

>>>> Another example of version information is the sqlite3 [5]_ library
>>> the sqlite3 module
>
> You overlooked that one.

Got it.

>>>> #. For modules which are also packages, the top level module >>> namespace
>>>>    SHOULD include the ``__version__`` attribute.
>>> Just a remark: I don?t remember ever reading the term ?top-level >> module
>>> namespace?.  It?s not hard to understand, but it might be helpful to
>>> some readers if you add ?(i.e. the :file:`{somepackage}/__init__.py`
>>> file)?.  (The brackets will cause the enclosed text to be marked up >> as
>>> replaceable text, just a nicety.)
>> How about just removing "top-level"?
>>
>>     #. For modules which are also packages, the module namespace > SHOULD
>>        include the ``__version__`` attribute.
>
> I?m still not sure ?module namespace? will be clear to everyone.

Really?  We know what a module is, and we know what a namespace is, so given
the context, I think it should be clear.

> Tarek already ruled last summer that field names in setup.cfg have to
> have their PEP 345 name.  I proposed to merge author name and email  into
> the author field, and to have the description field always refer to a
> file: author and author_email are still separate, and a new
> description_from_file fields has been added.  That?s why I think a new
> field has to be defined.  version-from should be a short enough name.   I
> also expect most people to use copy-paste or the interactive setup.cfg
> creation helper, so field name length should not be that big of an  issue.

Perhaps.  In lieu of a better idea it's fine for now.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110412/dc5f923b/attachment.pgp>

From exarkun at twistedmatrix.com  Wed Apr 13 04:52:08 2011
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Wed, 13 Apr 2011 02:52:08 -0000
Subject: [Distutils] Problem uploading an MSI to PyPI
Message-ID: <20110413025208.1992.1121059104.divmod.xquotient.464@localhost.localdomain>

Hello all,

A member of the Tahoe-LAFS community was kind enough to build a Python 
2.6 amd64 MSI installer for pyOpenSSL 0.12 for me (since I am not set up 
to build anything for 64 bit Windows).

The MSI is available at http://twistedmatrix.com/~exarkun/pyOpenSSL-0.12 
.win-amd64-py2.6.msi for the time being.  It might be interesting to 
inspect in considering the question I have.

First I tried uploading it to PyPI using the web interface. 
Unfortunately this resulted in a failed upload (several times) because 
"invalid distribution file".

Then I tried using the distutils upload command (which I had to modify 
to allow uploading a file which was already built, rather than one which 
was being built at the moment).  This produced the same result ("Upload 
failed (400): invalid distribution file").

So... is this a distutils bug?  Is there something wrong with the MSI? 
Is it something wrong with pyOpenSSL's setup.py?  Is it a bug in PyPI 
causing it to reject a valid MSI?

(I hope the cross-posting is okay, since this topic seems to straddle 
these two areas.)

Thanks,
Jean-Paul

From setuptools at bugs.python.org  Fri Apr 15 19:39:51 2011
From: setuptools at bugs.python.org (Mike Cowperthwaite)
Date: Fri, 15 Apr 2011 17:39:51 +0000
Subject: [Distutils] [issue126] Windows: easy_install's 'DOS' window closes
	automatically, error msgs lost
In-Reply-To: <1302889191.06.0.38414267921.issue126@psf.upfronthosting.co.za>
Message-ID: <1302889191.06.0.38414267921.issue126@psf.upfronthosting.co.za>


New submission from Mike Cowperthwaite <mcow at well.com>:

This may be specific to Windows Vista and Windows 7, I think. I am using 64-bit Windows 7; see also this page for similar problem with Vista:
http://catherinedevlin.blogspot.com/2009/08/workaround-easyinstall-windows-vanish.html
It seems that Windows forces easy_install (and pip) to open into a new cmd shell, when run from a command shell. When the install program completes, the cmd shell closes. If there are any problems with the install, the program errors are lost, making troubleshooting difficult.

The hack of elevating to Administrator described there as working under Vista doesn't work for me on W7/64.

I don't expect anyone to figure out how to keep Windows from launching into a new window, but, as a workaround: I'd like a command-line switch I can add to easy_install which would pause at the end of the install and print something like "Hit any key to continue" and then wait for that activity.

----------
messages: 613
nosy: mcow
priority: wish
status: unread
title: Windows: easy_install's 'DOS' window closes automatically, error msgs lost

_______________________________________________
Setuptools tracker <setuptools at bugs.python.org>
<http://bugs.python.org/setuptools/issue126>
_______________________________________________

From arve.knudsen at gmail.com  Mon Apr 18 14:28:47 2011
From: arve.knudsen at gmail.com (Arve Knudsen)
Date: Mon, 18 Apr 2011 14:28:47 +0200
Subject: [Distutils] Need help with a usecase
Message-ID: <BANLkTi=KStXhW2oW-BqSQh1ERCJaxKgrAA@mail.gmail.com>

Hello

I'm new to Buildout and need to figure out how to accomplish a (seemingly)
simple usecase: Install my project from source into an isolated environment.
My project has a Distribute-based setup.py and I'd like to install it and
its dependencies to an isolated environment. I gather this should be doable
using Buildout, but how? I've experimented a bit and read the documentation
on PyPi, but so far I have not succeeded.

Thanks!
Arve
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110418/6361351d/attachment.html>

From jim at zope.com  Mon Apr 18 15:14:24 2011
From: jim at zope.com (Jim Fulton)
Date: Mon, 18 Apr 2011 09:14:24 -0400
Subject: [Distutils] Need help with a usecase
In-Reply-To: <BANLkTi=KStXhW2oW-BqSQh1ERCJaxKgrAA@mail.gmail.com>
References: <BANLkTi=KStXhW2oW-BqSQh1ERCJaxKgrAA@mail.gmail.com>
Message-ID: <BANLkTik4rKXAc03WTdtqPYDuxaKhGXECQA@mail.gmail.com>

On Mon, Apr 18, 2011 at 8:28 AM, Arve Knudsen <arve.knudsen at gmail.com> wrote:
> Hello
> I'm new to Buildout and need to figure out how to accomplish a (seemingly)
> simple usecase: Install my project from source into an isolated environment.
> My project has a Distribute-based setup.py and I'd like to install it and
> its dependencies to an isolated environment. I gather this should be doable
> using Buildout, but how? I've experimented a bit and read the documentation
> on PyPi, but so far I have not succeeded.

See:

  http://pypi.python.org/pypi/zc.sourcerelease

You *might* also find:

  http://www.riversnake.com/pycon2011/full/

helpful, although it is linux oriented.

Jim


-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From arve.knudsen at gmail.com  Mon Apr 18 15:25:24 2011
From: arve.knudsen at gmail.com (Arve Knudsen)
Date: Mon, 18 Apr 2011 15:25:24 +0200
Subject: [Distutils] Need help with a usecase
In-Reply-To: <BANLkTik4rKXAc03WTdtqPYDuxaKhGXECQA@mail.gmail.com>
References: <BANLkTi=KStXhW2oW-BqSQh1ERCJaxKgrAA@mail.gmail.com>
	<BANLkTik4rKXAc03WTdtqPYDuxaKhGXECQA@mail.gmail.com>
Message-ID: <BANLkTi=JHLwOf88o35utm6DVOAdMiRR5hg@mail.gmail.com>

On Mon, Apr 18, 2011 at 3:14 PM, Jim Fulton <jim at zope.com> wrote:

> On Mon, Apr 18, 2011 at 8:28 AM, Arve Knudsen <arve.knudsen at gmail.com>
> wrote:
> > Hello
> > I'm new to Buildout and need to figure out how to accomplish a
> (seemingly)
> > simple usecase: Install my project from source into an isolated
> environment.
> > My project has a Distribute-based setup.py and I'd like to install it and
> > its dependencies to an isolated environment. I gather this should be
> doable
> > using Buildout, but how? I've experimented a bit and read the
> documentation
> > on PyPi, but so far I have not succeeded.
>
> See:
>
>  http://pypi.python.org/pypi/zc.sourcerelease
>
> You *might* also find:
>
>  http://www.riversnake.com/pycon2011/full/
>
> helpful, although it is linux oriented.
>

Thanks for the info, Jim, but I honestly still don't see how to accomplish
my usecase as a whole. The difference I see from the standard Buildout
usecase is to install (with buildout) from a directory rather than to find a
distribution at e.g. PyPI.

Arve
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110418/ef61e270/attachment.html>

From jim at zope.com  Mon Apr 18 15:36:39 2011
From: jim at zope.com (Jim Fulton)
Date: Mon, 18 Apr 2011 09:36:39 -0400
Subject: [Distutils] Need help with a usecase
In-Reply-To: <BANLkTi=JHLwOf88o35utm6DVOAdMiRR5hg@mail.gmail.com>
References: <BANLkTi=KStXhW2oW-BqSQh1ERCJaxKgrAA@mail.gmail.com>
	<BANLkTik4rKXAc03WTdtqPYDuxaKhGXECQA@mail.gmail.com>
	<BANLkTi=JHLwOf88o35utm6DVOAdMiRR5hg@mail.gmail.com>
Message-ID: <BANLkTimejoFBrShtqXCH=0WdGKdQ=gtjTQ@mail.gmail.com>

On Mon, Apr 18, 2011 at 9:25 AM, Arve Knudsen <arve.knudsen at gmail.com> wrote:
> On Mon, Apr 18, 2011 at 3:14 PM, Jim Fulton <jim at zope.com> wrote:
>>
>> On Mon, Apr 18, 2011 at 8:28 AM, Arve Knudsen <arve.knudsen at gmail.com>
>> wrote:
>> > Hello
>> > I'm new to Buildout and need to figure out how to accomplish a
>> > (seemingly)
>> > simple usecase: Install my project from source into an isolated
>> > environment.
>> > My project has a Distribute-based setup.py and I'd like to install it
>> > and
>> > its dependencies to an isolated environment. I gather this should be
>> > doable
>> > using Buildout, but how? I've experimented a bit and read the
>> > documentation
>> > on PyPi, but so far I have not succeeded.
>>
>> See:
>>
>> ?http://pypi.python.org/pypi/zc.sourcerelease
>>
>> You *might* also find:
>>
>> ?http://www.riversnake.com/pycon2011/full/
>>
>> helpful, although it is linux oriented.
>
> Thanks for the info, Jim, but I honestly still don't see how to accomplish
> my usecase as a whole. The difference I see from the standard Buildout
> usecase is to install (with buildout) from a directory rather than to find a
> distribution at e.g. PyPI.

Maybe you should explain your use case more.  I guessed what you
meant. I was obviously wrong.

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From arve.knudsen at gmail.com  Mon Apr 18 16:04:50 2011
From: arve.knudsen at gmail.com (Arve Knudsen)
Date: Mon, 18 Apr 2011 16:04:50 +0200
Subject: [Distutils] Need help with a usecase
In-Reply-To: <BANLkTimejoFBrShtqXCH=0WdGKdQ=gtjTQ@mail.gmail.com>
References: <BANLkTi=KStXhW2oW-BqSQh1ERCJaxKgrAA@mail.gmail.com>
	<BANLkTik4rKXAc03WTdtqPYDuxaKhGXECQA@mail.gmail.com>
	<BANLkTi=JHLwOf88o35utm6DVOAdMiRR5hg@mail.gmail.com>
	<BANLkTimejoFBrShtqXCH=0WdGKdQ=gtjTQ@mail.gmail.com>
Message-ID: <BANLkTik+9TtXkgGZzyKZrw926bYimH6u2Q@mail.gmail.com>

On Mon, Apr 18, 2011 at 3:36 PM, Jim Fulton <jim at zope.com> wrote:

> Maybe you should explain your use case more.  I guessed what you
> meant. I was obviously wrong.


My project (checked out from Git to be specific) comes with a
Distribute-based setup.py, and I'd like to install it into an isolated
Python environment. That is, the equivalent of running 'python
setup.py.install' in my project directory, but into an isolated environment.
I was hoping I could use Buildout for this.

Arve
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110418/27a45de0/attachment.html>

From jim at zope.com  Mon Apr 18 16:32:30 2011
From: jim at zope.com (Jim Fulton)
Date: Mon, 18 Apr 2011 10:32:30 -0400
Subject: [Distutils] Need help with a usecase
In-Reply-To: <BANLkTik+9TtXkgGZzyKZrw926bYimH6u2Q@mail.gmail.com>
References: <BANLkTi=KStXhW2oW-BqSQh1ERCJaxKgrAA@mail.gmail.com>
	<BANLkTik4rKXAc03WTdtqPYDuxaKhGXECQA@mail.gmail.com>
	<BANLkTi=JHLwOf88o35utm6DVOAdMiRR5hg@mail.gmail.com>
	<BANLkTimejoFBrShtqXCH=0WdGKdQ=gtjTQ@mail.gmail.com>
	<BANLkTik+9TtXkgGZzyKZrw926bYimH6u2Q@mail.gmail.com>
Message-ID: <BANLkTinfUTd5oPEL+sy0ev=3nL5kL8jQwA@mail.gmail.com>

On Mon, Apr 18, 2011 at 10:04 AM, Arve Knudsen <arve.knudsen at gmail.com> wrote:
> On Mon, Apr 18, 2011 at 3:36 PM, Jim Fulton <jim at zope.com> wrote:
>>
>> Maybe you should explain your use case more. ?I guessed what you
>> meant. I was obviously wrong.
>
> My project (checked out from Git to be specific) comes with a
> Distribute-based setup.py, and I'd like to install it into an isolated
> Python environment. That is, the equivalent of running 'python
> setup.py.install' in my project directory, but into an isolated environment.
> I was hoping I could use Buildout for this.

Your specific requirements is better matched by virtualenv.

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From arve.knudsen at gmail.com  Mon Apr 18 17:20:42 2011
From: arve.knudsen at gmail.com (Arve Knudsen)
Date: Mon, 18 Apr 2011 17:20:42 +0200
Subject: [Distutils] Need help with a usecase
In-Reply-To: <BANLkTinfUTd5oPEL+sy0ev=3nL5kL8jQwA@mail.gmail.com>
References: <BANLkTi=KStXhW2oW-BqSQh1ERCJaxKgrAA@mail.gmail.com>
	<BANLkTik4rKXAc03WTdtqPYDuxaKhGXECQA@mail.gmail.com>
	<BANLkTi=JHLwOf88o35utm6DVOAdMiRR5hg@mail.gmail.com>
	<BANLkTimejoFBrShtqXCH=0WdGKdQ=gtjTQ@mail.gmail.com>
	<BANLkTik+9TtXkgGZzyKZrw926bYimH6u2Q@mail.gmail.com>
	<BANLkTinfUTd5oPEL+sy0ev=3nL5kL8jQwA@mail.gmail.com>
Message-ID: <BANLkTikujf+0Env5nzgcWOXQ8GAroRuA5w@mail.gmail.com>

On Mon, Apr 18, 2011 at 4:32 PM, Jim Fulton <jim at zope.com> wrote:

> On Mon, Apr 18, 2011 at 10:04 AM, Arve Knudsen <arve.knudsen at gmail.com>
> wrote:
> > On Mon, Apr 18, 2011 at 3:36 PM, Jim Fulton <jim at zope.com> wrote:
> >>
> >> Maybe you should explain your use case more.  I guessed what you
> >> meant. I was obviously wrong.
> >
> > My project (checked out from Git to be specific) comes with a
> > Distribute-based setup.py, and I'd like to install it into an isolated
> > Python environment. That is, the equivalent of running 'python
> > setup.py.install' in my project directory, but into an isolated
> environment.
> > I was hoping I could use Buildout for this.
>
> Your specific requirements is better matched by virtualenv.


Aha. I appreciate the clear answer :)

Arve
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110418/82a36b4f/attachment.html>

From eric at trueblade.com  Mon Apr 18 15:43:36 2011
From: eric at trueblade.com (Eric Smith)
Date: Mon, 18 Apr 2011 09:43:36 -0400 (EDT)
Subject: [Distutils] Need help with a usecase
In-Reply-To: <BANLkTi=JHLwOf88o35utm6DVOAdMiRR5hg@mail.gmail.com>
References: <BANLkTi=KStXhW2oW-BqSQh1ERCJaxKgrAA@mail.gmail.com>
	<BANLkTik4rKXAc03WTdtqPYDuxaKhGXECQA@mail.gmail.com>
	<BANLkTi=JHLwOf88o35utm6DVOAdMiRR5hg@mail.gmail.com>
Message-ID: <9fad9e6eb0ae7d0ac2b4cbff695dd4c3.squirrel@mail.trueblade.com>

> On Mon, Apr 18, 2011 at 3:14 PM, Jim Fulton <jim at zope.com> wrote:
>
>> On Mon, Apr 18, 2011 at 8:28 AM, Arve Knudsen <arve.knudsen at gmail.com>
>> wrote:
>> > Hello
>> > I'm new to Buildout and need to figure out how to accomplish a
>> (seemingly)
>> > simple usecase: Install my project from source into an isolated
>> environment.
>> > My project has a Distribute-based setup.py and I'd like to install it
>> and
>> > its dependencies to an isolated environment. I gather this should be
>> doable
>> > using Buildout, but how? I've experimented a bit and read the
>> documentation
>> > on PyPi, but so far I have not succeeded.
>>
>> See:
>>
>>  http://pypi.python.org/pypi/zc.sourcerelease
>>
>> You *might* also find:
>>
>>  http://www.riversnake.com/pycon2011/full/
>>
>> helpful, although it is linux oriented.
>>
>
> Thanks for the info, Jim, but I honestly still don't see how to accomplish
> my usecase as a whole. The difference I see from the standard Buildout
> usecase is to install (with buildout) from a directory rather than to find
> a
> distribution at e.g. PyPI.

Look at the develop= option in the [buildout] section.

There's a (little) discussion of it here:
http://plone.org/documentation/manual/developer-manual/managing-projects-with-buildout/understanding-buildout.cfg

There are no doubt better references, that was just the first one I came
across.

I use this for testing and developing, but never for deployment.

Eric.


From reinout at vanrees.org  Tue Apr 19 09:46:42 2011
From: reinout at vanrees.org (Reinout van Rees)
Date: Tue, 19 Apr 2011 09:46:42 +0200
Subject: [Distutils] Need help with a usecase
In-Reply-To: <BANLkTinfUTd5oPEL+sy0ev=3nL5kL8jQwA@mail.gmail.com>
References: <BANLkTi=KStXhW2oW-BqSQh1ERCJaxKgrAA@mail.gmail.com>	<BANLkTik4rKXAc03WTdtqPYDuxaKhGXECQA@mail.gmail.com>	<BANLkTi=JHLwOf88o35utm6DVOAdMiRR5hg@mail.gmail.com>	<BANLkTimejoFBrShtqXCH=0WdGKdQ=gtjTQ@mail.gmail.com>	<BANLkTik+9TtXkgGZzyKZrw926bYimH6u2Q@mail.gmail.com>
	<BANLkTinfUTd5oPEL+sy0ev=3nL5kL8jQwA@mail.gmail.com>
Message-ID: <iojel2$ane$1@dough.gmane.org>

On 18-04-11 16:32, Jim Fulton wrote:
>> My project (checked out from Git to be specific) comes with a
>> >  Distribute-based setup.py, and I'd like to install it into an isolated
>> >  Python environment. That is, the equivalent of running 'python
>> >  setup.py.install' in my project directory, but into an isolated environment.
>> >  I was hoping I could use Buildout for this.
> Your specific requirements is better matched by virtualenv.

It *is* possible with buildout in case you want to experiment :-)
Put this in your buildout.cfg:

[buildout]
develop = .
parts = scripts

[scripts]
recipe = zc.recipe.egg
eggs = your-egg-name

Grab a bootstrap.py, run it and run bin/buildout.


The one thing that, to me, still is a big advantage of buildout over 
virtualenv in cases like this: you can just run the scripts in bin/* 
as-is. You do not need to activate the virtualenv beforehand.

For quick installs and try-outs I prefer virtualenv, though :-)


Reinout


From carl at oddbird.net  Tue Apr 19 17:35:49 2011
From: carl at oddbird.net (Carl Meyer)
Date: Tue, 19 Apr 2011 10:35:49 -0500
Subject: [Distutils] Need help with a usecase
In-Reply-To: <iojel2$ane$1@dough.gmane.org>
References: <BANLkTi=KStXhW2oW-BqSQh1ERCJaxKgrAA@mail.gmail.com>	<BANLkTik4rKXAc03WTdtqPYDuxaKhGXECQA@mail.gmail.com>	<BANLkTi=JHLwOf88o35utm6DVOAdMiRR5hg@mail.gmail.com>	<BANLkTimejoFBrShtqXCH=0WdGKdQ=gtjTQ@mail.gmail.com>	<BANLkTik+9TtXkgGZzyKZrw926bYimH6u2Q@mail.gmail.com>	<BANLkTinfUTd5oPEL+sy0ev=3nL5kL8jQwA@mail.gmail.com>
	<iojel2$ane$1@dough.gmane.org>
Message-ID: <4DADABD5.9090909@oddbird.net>

Hi Reinout,

On 04/19/2011 02:46 AM, Reinout van Rees wrote:
> The one thing that, to me, still is a big advantage of buildout over
> virtualenv in cases like this: you can just run the scripts in bin/*
> as-is. You do not need to activate the virtualenv beforehand.

There are good reasons why one might prefer buildout over virtualenv,
but this is not one of them ;-) A virtualenv also has a bin/ directory
with scripts in it (including the interpreter, bin/python). You can run
any of those scripts directly (with no activation) and they will run
within the virtualenv.

Activation is a shell convenience that is entirely unnecessary to the
operation of virtualenv. All it does is put the virtualenv's bin/
directory first on your $PATH, so its scripts will take priority over
globally-installed scripts.

Carl

From jim at zope.com  Tue Apr 19 18:05:52 2011
From: jim at zope.com (Jim Fulton)
Date: Tue, 19 Apr 2011 12:05:52 -0400
Subject: [Distutils] Need help with a usecase
In-Reply-To: <4DADABD5.9090909@oddbird.net>
References: <BANLkTi=KStXhW2oW-BqSQh1ERCJaxKgrAA@mail.gmail.com>
	<BANLkTik4rKXAc03WTdtqPYDuxaKhGXECQA@mail.gmail.com>
	<BANLkTi=JHLwOf88o35utm6DVOAdMiRR5hg@mail.gmail.com>
	<BANLkTimejoFBrShtqXCH=0WdGKdQ=gtjTQ@mail.gmail.com>
	<BANLkTik+9TtXkgGZzyKZrw926bYimH6u2Q@mail.gmail.com>
	<BANLkTinfUTd5oPEL+sy0ev=3nL5kL8jQwA@mail.gmail.com>
	<iojel2$ane$1@dough.gmane.org> <4DADABD5.9090909@oddbird.net>
Message-ID: <BANLkTikA3bwCATG0N0OXhcUG3-rjqLaLoA@mail.gmail.com>

On Tue, Apr 19, 2011 at 11:35 AM, Carl Meyer <carl at oddbird.net> wrote:
> Hi Reinout,
>
> On 04/19/2011 02:46 AM, Reinout van Rees wrote:
>> The one thing that, to me, still is a big advantage of buildout over
>> virtualenv in cases like this: you can just run the scripts in bin/*
>> as-is. You do not need to activate the virtualenv beforehand.
>
> There are good reasons why one might prefer buildout over virtualenv,
> but this is not one of them ;-) A virtualenv also has a bin/ directory
> with scripts in it (including the interpreter, bin/python). You can run
> any of those scripts directly (with no activation) and they will run
> within the virtualenv.
>
> Activation is a shell convenience that is entirely unnecessary to the
> operation of virtualenv. All it does is put the virtualenv's bin/
> directory first on your $PATH, so its scripts will take priority over
> globally-installed scripts.

That's cool. I wish I'd known that. :)

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From merwok at netwok.org  Tue Apr 19 18:18:12 2011
From: merwok at netwok.org (=?UTF-8?Q?=C3=89ric_Araujo?=)
Date: Tue, 19 Apr 2011 18:18:12 +0200
Subject: [Distutils] Need help with a usecase
In-Reply-To: <BANLkTikA3bwCATG0N0OXhcUG3-rjqLaLoA@mail.gmail.com>
References: "\"<BANLkTi=KStXhW2oW-BqSQh1ERCJaxKgrAA@mail.gmail.com>	<BANLkTik4rKXAc03WTdtqPYDuxaKhGXECQA@mail.gmail.com>	<BANLkTi=JHLwOf88o35utm6DVOAdMiRR5hg@mail.gmail.com>	<BANLkTimejoFBrShtqXCH=0WdGKdQ=gtjTQ@mail.gmail.com>	<BANLkTik+9TtXkgGZzyKZrw926bYimH6u2Q@mail.gmail.com>"
	<BANLkTinfUTd5oPEL+sy0ev=3nL5kL8jQwA@mail.gmail.com>"
	<iojel2$ane$1@dough.gmane.org> <4DADABD5.9090909@oddbird.net>
	<BANLkTikA3bwCATG0N0OXhcUG3-rjqLaLoA@mail.gmail.com>
Message-ID: <f24634acfd0d4016d47922a905d61208@netwok.org>

 Le 19/04/2011 18:05, Jim Fulton a ?crit :
> On Tue, Apr 19, 2011 at 11:35 AM, Carl Meyer <carl at oddbird.net> 
> wrote:
>> Activation is a shell convenience that is entirely unnecessary to 
>> the
>> operation of virtualenv. All it does is put the virtualenv's bin/
>> directory first on your $PATH, so its scripts will take priority 
>> over
>> globally-installed scripts.
>
> That's cool. I wish I'd known that. :)

 Carl?s PyCon 2011 talk explains it:
 http://pycon.blip.tv/file/4881525/
 Reverse-engineering Ian Bicking's brain: inside pip and virtualenv

 Regards

From eposse at gmail.com  Tue Apr 19 19:42:00 2011
From: eposse at gmail.com (Ernesto Posse)
Date: Tue, 19 Apr 2011 13:42:00 -0400
Subject: [Distutils] dependencies, pip and non-PyPI-hosted packages
Message-ID: <BANLkTik0+RZVcnGd-B0F2eh0kZ2X2rFJbg@mail.gmail.com>

Hi I have a couple of packages that I don't want to upload to PyPI
just yet but they are hosted some other place, and one (project2)
depends on the other (project1). I've packaged them using the standard
distutils setup.

For project1, setup.py contains:

  setup(name='project1', ..., requires=[], provides=['project1'], ...)

and for project2, setup.py contains:

  setup(name='project2', ..., requires=['project1'], provides=['project2'], ...)

I put them in some host https://my.host.org/repository, but when I run

  pip install -f https://my.host.org/repository project2

installs project2 correctly, but it does not install project1.

So my first question is: is the 'requires' keyword in setup.py
actually used when installing packages? or it only works for projects
hosted on PyPI?

Thanks

-- 
Ernesto Posse

Modelling and Analysis in Software Engineering
School of Computing
Queen's University - Kingston, Ontario, Canada

From carl at oddbird.net  Tue Apr 19 20:26:11 2011
From: carl at oddbird.net (Carl Meyer)
Date: Tue, 19 Apr 2011 13:26:11 -0500
Subject: [Distutils] dependencies, pip and non-PyPI-hosted packages
In-Reply-To: <BANLkTik0+RZVcnGd-B0F2eh0kZ2X2rFJbg@mail.gmail.com>
References: <BANLkTik0+RZVcnGd-B0F2eh0kZ2X2rFJbg@mail.gmail.com>
Message-ID: <4DADD3C3.6010900@oddbird.net>

Hi Ernesto,

On 04/19/2011 12:42 PM, Ernesto Posse wrote:
> Hi I have a couple of packages that I don't want to upload to PyPI
> just yet but they are hosted some other place, and one (project2)
> depends on the other (project1). I've packaged them using the standard
> distutils setup.
> 
> For project1, setup.py contains:
> 
>   setup(name='project1', ..., requires=[], provides=['project1'], ...)
> 
> and for project2, setup.py contains:
> 
>   setup(name='project2', ..., requires=['project1'], provides=['project2'], ...)
> 
> I put them in some host https://my.host.org/repository, but when I run
> 
>   pip install -f https://my.host.org/repository project2
> 
> installs project2 correctly, but it does not install project1.
> 
> So my first question is: is the 'requires' keyword in setup.py
> actually used when installing packages? or it only works for projects
> hosted on PyPI?

The "standard" requires and provides keywords defined in PEP 314 and
accepted by distutils are, AFAIK, not actually used by any existing
tools. They are also deprecated by the newer PEP 345 metadata standard.

(The reason for this, as I understand it, is that those keywords were
defined by PEP 314 to refer to importable module/package names rather
than project names. This makes them almost impossible for an installer
to use, given that we have no global mapping available for translating
an importable module/package name to an installable project name.)

In order to have pip (or any other tool I know of) respect dependencies
specified in setup.py, you need to use setuptools (or the "distribute"
fork) in your setup.py and use its install_requires keyword argument
instead. See
http://peak.telecommunity.com/DevCenter/setuptools#new-and-changed-setup-keywords
for details.

Carl

From marrakis at gmail.com  Tue Apr 19 21:02:15 2011
From: marrakis at gmail.com (Mathieu Leduc-Hamel)
Date: Tue, 19 Apr 2011 15:02:15 -0400
Subject: [Distutils] Support for multi-index ?
Message-ID: <BANLkTimJKSXGsY-BGUPxfYcupH364GpwCA@mail.gmail.com>

Hi all, i'm curious about the status of this "issue", is there any
support in zc.buildout/setuptools for multiple index ?

-- 
Mathieu Leduc-Hamel

From tseaver at palladion.com  Tue Apr 19 21:48:53 2011
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 19 Apr 2011 15:48:53 -0400
Subject: [Distutils] Need help with a usecase
In-Reply-To: <iojel2$ane$1@dough.gmane.org>
References: <BANLkTi=KStXhW2oW-BqSQh1ERCJaxKgrAA@mail.gmail.com>	<BANLkTik4rKXAc03WTdtqPYDuxaKhGXECQA@mail.gmail.com>	<BANLkTi=JHLwOf88o35utm6DVOAdMiRR5hg@mail.gmail.com>	<BANLkTimejoFBrShtqXCH=0WdGKdQ=gtjTQ@mail.gmail.com>	<BANLkTik+9TtXkgGZzyKZrw926bYimH6u2Q@mail.gmail.com>	<BANLkTinfUTd5oPEL+sy0ev=3nL5kL8jQwA@mail.gmail.com>
	<iojel2$ane$1@dough.gmane.org>
Message-ID: <iokov5$cta$1@dough.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 04/19/2011 03:46 AM, Reinout van Rees wrote:
> On 18-04-11 16:32, Jim Fulton wrote:
>>> My project (checked out from Git to be specific) comes with a
>>>>  Distribute-based setup.py, and I'd like to install it into an isolated
>>>>  Python environment. That is, the equivalent of running 'python
>>>>  setup.py.install' in my project directory, but into an isolated environment.
>>>>  I was hoping I could use Buildout for this.
>> Your specific requirements is better matched by virtualenv.
> 
> It *is* possible with buildout in case you want to experiment :-)
> Put this in your buildout.cfg:
> 
> [buildout]
> develop = .
> parts = scripts
> 
> [scripts]
> recipe = zc.recipe.egg
> eggs = your-egg-name
> 
> Grab a bootstrap.py, run it and run bin/buildout.
> 
> 
> The one thing that, to me, still is a big advantage of buildout over 
> virtualenv in cases like this: you can just run the scripts in bin/* 
> as-is. You do not need to activate the virtualenv beforehand.

You don't ever *need* to activate a virtualenv to run its scripts -- you
just type 'bin/foo'  from the VE root, or '/path/to/VE/bin/foo' from
elsewhere.  After using VE for years, I have literally never used
activate, while using multiple VE's daily.


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk2t5yUACgkQ+gerLs4ltQ4KLACguvDEhCAlHdgLGajFZEqjlP+4
Cm8An3gTzHp7E9IkFA+aUpTl2P64n/2S
=tHbE
-----END PGP SIGNATURE-----


From eposse at gmail.com  Tue Apr 19 23:07:46 2011
From: eposse at gmail.com (Ernesto Posse)
Date: Tue, 19 Apr 2011 17:07:46 -0400
Subject: [Distutils] dependencies, pip and non-PyPI-hosted packages
In-Reply-To: <4DADD3C3.6010900@oddbird.net>
References: <BANLkTik0+RZVcnGd-B0F2eh0kZ2X2rFJbg@mail.gmail.com>
	<4DADD3C3.6010900@oddbird.net>
Message-ID: <BANLkTinPe9_PXmEmFb0DqqOh6uJoAZN5nQ@mail.gmail.com>

Thanks. I have tried it with distribute, and it's quite nice, but I
have a couple of questions:

On Tue, Apr 19, 2011 at 2:26 PM, Carl Meyer <carl at oddbird.net> wrote:
> Hi Ernesto,
>
> On 04/19/2011 12:42 PM, Ernesto Posse wrote:
>> Hi I have a couple of packages that I don't want to upload to PyPI
>> just yet but they are hosted some other place, and one (project2)
>> depends on the other (project1). I've packaged them using the standard
>> distutils setup.
>>
>> For project1, setup.py contains:
>>
>> ? setup(name='project1', ..., requires=[], provides=['project1'], ...)
>>
>> and for project2, setup.py contains:
>>
>> ? setup(name='project2', ..., requires=['project1'], provides=['project2'], ...)
>>
>> I put them in some host https://my.host.org/repository, but when I run
>>
>> ? pip install -f https://my.host.org/repository project2
>>
>> installs project2 correctly, but it does not install project1.
>>
>> So my first question is: is the 'requires' keyword in setup.py
>> actually used when installing packages? or it only works for projects
>> hosted on PyPI?
>
> The "standard" requires and provides keywords defined in PEP 314 and
> accepted by distutils are, AFAIK, not actually used by any existing
> tools. They are also deprecated by the newer PEP 345 metadata standard.
>
> (The reason for this, as I understand it, is that those keywords were
> defined by PEP 314 to refer to importable module/package names rather
> than project names. This makes them almost impossible for an installer
> to use, given that we have no global mapping available for translating
> an importable module/package name to an installable project name.)
>
> In order to have pip (or any other tool I know of) respect dependencies
> specified in setup.py, you need to use setuptools (or the "distribute"
> fork) in your setup.py and use its install_requires keyword argument
> instead. See
> http://peak.telecommunity.com/DevCenter/setuptools#new-and-changed-setup-keywords
> for details.

1) using distribute and

  setup(..., install_requires=['project1'],...,
dependency_links=['http://my.host.org/repository/'],...)

installs as expected both project1 and project2, but

  pip uninstall project2

does not uninstall project1. This is quite disappointing, as a user
may be unaware of dependencies automatically installed, and thus, the
uninstall leaves behind something that was installed with the bundle.
I imagine that the idea is that the user may install some other
package that depends on 'project1' and pip takes the conservative
approach (is that the case?) but I would have expected for pip or
distribute or setuptools or distutils to keep some dependency
reference counter. Does any of these tools have something like that?
or is it going to be addressed in distutils2?

2) if install_requires is missing a dependency (project1), the package
gets installed without that dependency, but if I add a dependency and
the user attempts an install (project2) with the updated setup.py
(listing the new dependency) pip will say that the package (project2)
is already installed and won't attempt to install the dependencies. Is
this correct? If so, is there a way to tell pip to install project2's
dependencies?

Thanks



-- 
Ernesto Posse

Modelling and Analysis in Software Engineering
School of Computing
Queen's University - Kingston, Ontario, Canada

From carl at oddbird.net  Tue Apr 19 23:18:20 2011
From: carl at oddbird.net (Carl Meyer)
Date: Tue, 19 Apr 2011 16:18:20 -0500
Subject: [Distutils] dependencies, pip and non-PyPI-hosted packages
In-Reply-To: <BANLkTinPe9_PXmEmFb0DqqOh6uJoAZN5nQ@mail.gmail.com>
References: <BANLkTik0+RZVcnGd-B0F2eh0kZ2X2rFJbg@mail.gmail.com>	<4DADD3C3.6010900@oddbird.net>
	<BANLkTinPe9_PXmEmFb0DqqOh6uJoAZN5nQ@mail.gmail.com>
Message-ID: <4DADFC1C.406@oddbird.net>

On 04/19/2011 04:07 PM, Ernesto Posse wrote:
> 1) using distribute and
> 
>   setup(..., install_requires=['project1'],...,
> dependency_links=['http://my.host.org/repository/'],...)
> 
> installs as expected both project1 and project2, but
> 
>   pip uninstall project2
> 
> does not uninstall project1. This is quite disappointing, as a user
> may be unaware of dependencies automatically installed, and thus, the
> uninstall leaves behind something that was installed with the bundle.
> I imagine that the idea is that the user may install some other
> package that depends on 'project1' and pip takes the conservative
> approach (is that the case?) but I would have expected for pip or
> distribute or setuptools or distutils to keep some dependency
> reference counter. Does any of these tools have something like that?
> or is it going to be addressed in distutils2?

No, there's no current Python packaging tool I know of that keeps a
dependency reference counter and automatically uninstalls orphaned
dependencies. The new standard installation format (defined in PEP 376
and implemented in distutils2) does record whether a package was
installed by explicit user request or as a dependency; combined with
checking dependencies of all other installed packages, this will make it
possible for an uninstaller to implement automatic (or prompted)
dependency uninstalls.

> 2) if install_requires is missing a dependency (project1), the package
> gets installed without that dependency, but if I add a dependency and
> the user attempts an install (project2) with the updated setup.py
> (listing the new dependency) pip will say that the package (project2)
> is already installed and won't attempt to install the dependencies. Is
> this correct? If so, is there a way to tell pip to install project2's
> dependencies?

With a real package release, the setup.py metadata should never change
without a version number change. In which case, if you specify the new
version explicitly or use --upgrade, the previous version will be
uninstalled and the new one installed in its place, with dependencies.
(Actually, if you specify --upgrade and you already have the most recent
version installed, it will still uninstall and reinstall it; but this is
actually considered a bug.)

Carl

From chris at simplistix.co.uk  Wed Apr 20 00:26:54 2011
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 19 Apr 2011 23:26:54 +0100
Subject: [Distutils] dependencies, pip and non-PyPI-hosted packages
In-Reply-To: <4DADFC1C.406@oddbird.net>
References: <BANLkTik0+RZVcnGd-B0F2eh0kZ2X2rFJbg@mail.gmail.com>	<4DADD3C3.6010900@oddbird.net>	<BANLkTinPe9_PXmEmFb0DqqOh6uJoAZN5nQ@mail.gmail.com>
	<4DADFC1C.406@oddbird.net>
Message-ID: <4DAE0C2E.50201@simplistix.co.uk>

On 19/04/2011 22:18, Carl Meyer wrote:
>> does not uninstall project1. This is quite disappointing, as a user
>> may be unaware of dependencies automatically installed, and thus, the
>> uninstall leaves behind something that was installed with the bundle.
>> I imagine that the idea is that the user may install some other
>> package that depends on 'project1' and pip takes the conservative
>> approach (is that the case?) but I would have expected for pip or
>> distribute or setuptools or distutils to keep some dependency
>> reference counter. Does any of these tools have something like that?
>> or is it going to be addressed in distutils2?
>
> No, there's no current Python packaging tool I know of that keeps a
> dependency reference counter and automatically uninstalls orphaned
> dependencies.

I'm pretty sure zc.buildout does this by way of recreating the script 
when necessary.

>> 2) if install_requires is missing a dependency (project1), the package
>> gets installed without that dependency, but if I add a dependency and
>> the user attempts an install (project2) with the updated setup.py
>> (listing the new dependency) pip will say that the package (project2)
>> is already installed and won't attempt to install the dependencies. Is
>> this correct? If so, is there a way to tell pip to install project2's
>> dependencies?
>
> With a real package release, the setup.py metadata should never change
> without a version number change.

I should point out that zc.buildout deals gracefully with this case too, 
by way of regenerating scripts when needed.

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From kai.diefenbach at iqpp.de  Wed Apr 20 08:25:10 2011
From: kai.diefenbach at iqpp.de (Kai Diefenbach)
Date: Wed, 20 Apr 2011 08:25:10 +0200
Subject: [Distutils] Need help with a usecase
References: <BANLkTi=KStXhW2oW-BqSQh1ERCJaxKgrAA@mail.gmail.com>
	<BANLkTik4rKXAc03WTdtqPYDuxaKhGXECQA@mail.gmail.com>
	<BANLkTi=JHLwOf88o35utm6DVOAdMiRR5hg@mail.gmail.com>
	<BANLkTimejoFBrShtqXCH=0WdGKdQ=gtjTQ@mail.gmail.com>
	<BANLkTik+9TtXkgGZzyKZrw926bYimH6u2Q@mail.gmail.com>
	<BANLkTinfUTd5oPEL+sy0ev=3nL5kL8jQwA@mail.gmail.com>
	<iojel2$ane$1@dough.gmane.org>
Message-ID: <iolu7q$rfn$1@dough.gmane.org>

Hi,

On 2011-04-19 09:46:42 +0200, Reinout van Rees said:

> The one thing that, to me, still is a big advantage of buildout over
> virtualenv in cases like this: you can just run the scripts in bin/*
> as-is. You do not need to activate the virtualenv beforehand.

But which is easy with the workon command from virtualenvwrapper.

Kai



From wichert at wiggy.net  Wed Apr 20 10:56:05 2011
From: wichert at wiggy.net (Wichert Akkerman)
Date: Wed, 20 Apr 2011 10:56:05 +0200
Subject: [Distutils] Including symlinked data files in an egg
In-Reply-To: <20101215235214.19D923A408B@sparrow.telecommunity.com>
References: <4D07E55B.3090700@wiggy.net>
	<20101215151601.0BB8B3A4076@sparrow.telecommunity.com>
	<4D08DC43.7040003@wiggy.net>
	<20101215235214.19D923A408B@sparrow.telecommunity.com>
Message-ID: <4DAE9FA5.5060005@wiggy.net>

On 12/16/10 00:52 , P.J. Eby wrote:
> At 04:18 PM 12/15/2010 +0100, Wichert Akkerman wrote:
>> On 12/15/10 16:15 , P.J. Eby wrote:
>>> Most likely, the problem is with the file finder being used. Are you
>>> using a git plugin for setuptools, or simply listing everything in
>>> MANIFEST.in? If the former, it may be that the git plugin you're using
>>> simply doesn't support symlinks. If you're not using a git plugin,
>>> perhaps there is something amiss in the MANIFEST.in, and setuptools'
>>> built-in support for subversion is hiding the problem by picking up the
>>> symlinks.
>>
>> I am not using a git (or other) plugin. I tried listing things in
>> MANIFEST.in, for example using:
>>
>> graft my/package/templates/
>> graft my/package/templates/libraries
>>
>> unfortunately that had no effect: libraries was not included, not as
>> symlink nor as a copy of the symlink target.
>
> I don't know whether MANIFEST.in is supposed to support symlinks or not,
> but the fact that it's working with subversion is strongly indicative
> that setuptools' file finder is picking up stuff that MANIFEST.in is not.
>
> I don't know *why* MANIFEST.in wouldn't pick up symlinks, though.
> Perhaps you could try boiling this down to a simpler test case (e.g.,
> subversion and git versions of the same minimal package containing only
> a symlink referred to by MANIFEST.in) so we can narrow it down some more?

I've finally gotten around to doing this. I have attached test.tar.gz 
with a minimal package that demonstrates this.

Wichert.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: test.tar.gz
Type: application/x-gzip
Size: 682 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110420/85eaf906/attachment.bin>

From reinout at vanrees.org  Wed Apr 20 11:45:30 2011
From: reinout at vanrees.org (Reinout van Rees)
Date: Wed, 20 Apr 2011 11:45:30 +0200
Subject: [Distutils] Need help with a usecase
In-Reply-To: <iokov5$cta$1@dough.gmane.org>
References: <BANLkTi=KStXhW2oW-BqSQh1ERCJaxKgrAA@mail.gmail.com>	<BANLkTik4rKXAc03WTdtqPYDuxaKhGXECQA@mail.gmail.com>	<BANLkTi=JHLwOf88o35utm6DVOAdMiRR5hg@mail.gmail.com>	<BANLkTimejoFBrShtqXCH=0WdGKdQ=gtjTQ@mail.gmail.com>	<BANLkTik+9TtXkgGZzyKZrw926bYimH6u2Q@mail.gmail.com>	<BANLkTinfUTd5oPEL+sy0ev=3nL5kL8jQwA@mail.gmail.com>	<iojel2$ane$1@dough.gmane.org>
	<iokov5$cta$1@dough.gmane.org>
Message-ID: <iom9vq$qs4$1@dough.gmane.org>

On 19-04-11 21:48, Tres Seaver wrote:
>> The one thing that, to me, still is a big advantage of buildout over
>> >  virtualenv in cases like this: you can just run the scripts in bin/*
>> >  as-is. You do not need to activate the virtualenv beforehand.
> You don't ever*need*  to activate a virtualenv to run its scripts -- you
> just type 'bin/foo'  from the VE root, or '/path/to/VE/bin/foo' from
> elsewhere.  After using VE for years, I have literally never used
> activate, while using multiple VE's daily.

That's weird: I never noticed that I didn't need to activate it! It sure 
makes virtualenv much more useful. I feel quite silly for not noticing 
it before...

Reinout


From arve.knudsen at gmail.com  Wed Apr 20 12:03:05 2011
From: arve.knudsen at gmail.com (Arve Knudsen)
Date: Wed, 20 Apr 2011 12:03:05 +0200
Subject: [Distutils] [buildout] getaddrinfo failed
Message-ID: <BANLkTikLj25ji-icm7iYHNZqfLTFt3627A@mail.gmail.com>

Hi

I've got buildout (1.5.2) working for me, but it emits some errors when
installing my distribution (under Python 2.7 on Windows).

$ .\bin\buildout.exe -v -v -v
Installing 'zc.buildout', 'distribute'.
Download error: [Errno 11004] getaddrinfo failed -- Some packages may not be
found!


The same problem does *not* occur on Linux, I've found. Is this due to a bug
in buildout?

Thanks,
Arve
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110420/49037a59/attachment.html>

From pje at telecommunity.com  Wed Apr 20 18:04:40 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 20 Apr 2011 12:04:40 -0400
Subject: [Distutils] Including symlinked data files in an egg
In-Reply-To: <4DAE9FA5.5060005@wiggy.net>
References: <4D07E55B.3090700@wiggy.net>
	<20101215151601.0BB8B3A4076@sparrow.telecommunity.com>
	<4D08DC43.7040003@wiggy.net>
	<20101215235214.19D923A408B@sparrow.telecommunity.com>
	<4DAE9FA5.5060005@wiggy.net>
Message-ID: <20110420160456.6E0B43A409D@sparrow.telecommunity.com>

At 10:56 AM 4/20/2011 +0200, Wichert Akkerman wrote:
>On 12/16/10 00:52 , P.J. Eby wrote:
>>At 04:18 PM 12/15/2010 +0100, Wichert Akkerman wrote:
>>>On 12/15/10 16:15 , P.J. Eby wrote:
>>>>Most likely, the problem is with the file finder being used. Are you
>>>>using a git plugin for setuptools, or simply listing everything in
>>>>MANIFEST.in? If the former, it may be that the git plugin you're using
>>>>simply doesn't support symlinks. If you're not using a git plugin,
>>>>perhaps there is something amiss in the MANIFEST.in, and setuptools'
>>>>built-in support for subversion is hiding the problem by picking up the
>>>>symlinks.
>>>
>>>I am not using a git (or other) plugin. I tried listing things in
>>>MANIFEST.in, for example using:
>>>
>>>graft my/package/templates/
>>>graft my/package/templates/libraries
>>>
>>>unfortunately that had no effect: libraries was not included, not as
>>>symlink nor as a copy of the symlink target.
>>
>>I don't know whether MANIFEST.in is supposed to support symlinks or not,
>>but the fact that it's working with subversion is strongly indicative
>>that setuptools' file finder is picking up stuff that MANIFEST.in is not.
>>
>>I don't know *why* MANIFEST.in wouldn't pick up symlinks, though.
>>Perhaps you could try boiling this down to a simpler test case (e.g.,
>>subversion and git versions of the same minimal package containing only
>>a symlink referred to by MANIFEST.in) so we can narrow it down some more?
>
>I've finally gotten around to doing this. I have attached 
>test.tar.gz with a minimal package that demonstrates this.

Ah.  It appears MANIFEST.in simply doesn't support 
symlinks.  (Distutils has explicit code to filter them out, in 
fact.)  This is a distutils issue, which setuptools can't do anything about.


From wichert at wiggy.net  Wed Apr 20 20:12:56 2011
From: wichert at wiggy.net (Wichert Akkerman)
Date: Wed, 20 Apr 2011 20:12:56 +0200
Subject: [Distutils] Including symlinked data files in an egg
In-Reply-To: <20110420160456.6E0B43A409D@sparrow.telecommunity.com>
References: <4D07E55B.3090700@wiggy.net>
	<20101215151601.0BB8B3A4076@sparrow.telecommunity.com>
	<4D08DC43.7040003@wiggy.net>
	<20101215235214.19D923A408B@sparrow.telecommunity.com>
	<4DAE9FA5.5060005@wiggy.net>
	<20110420160456.6E0B43A409D@sparrow.telecommunity.com>
Message-ID: <4DAF2228.3050604@wiggy.net>

On 4/20/11 18:04 , P.J. Eby wrote:
> At 10:56 AM 4/20/2011 +0200, Wichert Akkerman wrote:
>> On 12/16/10 00:52 , P.J. Eby wrote:
>>> At 04:18 PM 12/15/2010 +0100, Wichert Akkerman wrote:
>>>> On 12/15/10 16:15 , P.J. Eby wrote:
>>>>> Most likely, the problem is with the file finder being used. Are you
>>>>> using a git plugin for setuptools, or simply listing everything in
>>>>> MANIFEST.in? If the former, it may be that the git plugin you're using
>>>>> simply doesn't support symlinks. If you're not using a git plugin,
>>>>> perhaps there is something amiss in the MANIFEST.in, and setuptools'
>>>>> built-in support for subversion is hiding the problem by picking up
>>>>> the
>>>>> symlinks.
>>>>
>>>> I am not using a git (or other) plugin. I tried listing things in
>>>> MANIFEST.in, for example using:
>>>>
>>>> graft my/package/templates/
>>>> graft my/package/templates/libraries
>>>>
>>>> unfortunately that had no effect: libraries was not included, not as
>>>> symlink nor as a copy of the symlink target.
>>>
>>> I don't know whether MANIFEST.in is supposed to support symlinks or not,
>>> but the fact that it's working with subversion is strongly indicative
>>> that setuptools' file finder is picking up stuff that MANIFEST.in is
>>> not.
>>>
>>> I don't know *why* MANIFEST.in wouldn't pick up symlinks, though.
>>> Perhaps you could try boiling this down to a simpler test case (e.g.,
>>> subversion and git versions of the same minimal package containing only
>>> a symlink referred to by MANIFEST.in) so we can narrow it down some
>>> more?
>>
>> I've finally gotten around to doing this. I have attached test.tar.gz
>> with a minimal package that demonstrates this.
>
> Ah. It appears MANIFEST.in simply doesn't support symlinks. (Distutils
> has explicit code to filter them out, in fact.) This is a distutils
> issue, which setuptools can't do anything about.

Is there another way to get this working? I'ld hate to have to resort to 
monkeypatching distutils.

Wichert.

From pje at telecommunity.com  Wed Apr 20 23:38:44 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 20 Apr 2011 17:38:44 -0400
Subject: [Distutils] Including symlinked data files in an egg
In-Reply-To: <4DAF2228.3050604@wiggy.net>
References: <4D07E55B.3090700@wiggy.net>
	<20101215151601.0BB8B3A4076@sparrow.telecommunity.com>
	<4D08DC43.7040003@wiggy.net>
	<20101215235214.19D923A408B@sparrow.telecommunity.com>
	<4DAE9FA5.5060005@wiggy.net>
	<20110420160456.6E0B43A409D@sparrow.telecommunity.com>
	<4DAF2228.3050604@wiggy.net>
Message-ID: <20110420213906.CF15E3A4116@sparrow.telecommunity.com>

At 08:12 PM 4/20/2011 +0200, Wichert Akkerman wrote:
>Is there another way to get this working? I'ld hate to have to 
>resort to monkeypatching distutils.

Well, you could have the symlinks go the other way around, I suppose.  ;-)

Alternately, you could write a "file finder" plugin for setuptools, 
or see whether one of the git plugins for setuptools fixes the problem.

(Note: the plugin would only be needed when building from a direct 
git checkout - building from an "sdist" (source .tar.gz or .zip) will 
not need it.)


From arve.knudsen at gmail.com  Thu Apr 21 10:34:32 2011
From: arve.knudsen at gmail.com (Arve Knudsen)
Date: Thu, 21 Apr 2011 10:34:32 +0200
Subject: [Distutils] Need help with a usecase
In-Reply-To: <iojel2$ane$1@dough.gmane.org>
References: <BANLkTi=KStXhW2oW-BqSQh1ERCJaxKgrAA@mail.gmail.com>
	<BANLkTik4rKXAc03WTdtqPYDuxaKhGXECQA@mail.gmail.com>
	<BANLkTi=JHLwOf88o35utm6DVOAdMiRR5hg@mail.gmail.com>
	<BANLkTimejoFBrShtqXCH=0WdGKdQ=gtjTQ@mail.gmail.com>
	<BANLkTik+9TtXkgGZzyKZrw926bYimH6u2Q@mail.gmail.com>
	<BANLkTinfUTd5oPEL+sy0ev=3nL5kL8jQwA@mail.gmail.com>
	<iojel2$ane$1@dough.gmane.org>
Message-ID: <BANLkTin6RU6a9T08k1Rq+88v+DbcUv2nww@mail.gmail.com>

Hi Reinout

I didn't notice this topic was going on without me :) In the meantime I
received help at StackOverflow:
http://stackoverflow.com/questions/5703456/how-can-i-install-my-project-from-source-with-buildout.
Your solution is identical to what I ended up with, except that I used the
recipe z3c.recipe.scripts, because I understood that it improves on
zc.recipe.egg.

Thanks,
Arve

On Tue, Apr 19, 2011 at 9:46 AM, Reinout van Rees <reinout at vanrees.org>wrote:

> On 18-04-11 16:32, Jim Fulton wrote:
>
>> My project (checked out from Git to be specific) comes with a
>>> >  Distribute-based setup.py, and I'd like to install it into an isolated
>>> >  Python environment. That is, the equivalent of running 'python
>>> >  setup.py.install' in my project directory, but into an isolated
>>> environment.
>>> >  I was hoping I could use Buildout for this.
>>>
>> Your specific requirements is better matched by virtualenv.
>>
>
> It *is* possible with buildout in case you want to experiment :-)
> Put this in your buildout.cfg:
>
> [buildout]
> develop = .
> parts = scripts
>
> [scripts]
> recipe = zc.recipe.egg
> eggs = your-egg-name
>
> Grab a bootstrap.py, run it and run bin/buildout.
>
>
> The one thing that, to me, still is a big advantage of buildout over
> virtualenv in cases like this: you can just run the scripts in bin/* as-is.
> You do not need to activate the virtualenv beforehand.
>
> For quick installs and try-outs I prefer virtualenv, though :-)
>
>
> Reinout
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110421/fb9f4dad/attachment.html>

From wichert at wiggy.net  Thu Apr 21 11:37:43 2011
From: wichert at wiggy.net (Wichert Akkerman)
Date: Thu, 21 Apr 2011 11:37:43 +0200
Subject: [Distutils] Including symlinked data files in an egg
In-Reply-To: <20110420213906.CF15E3A4116@sparrow.telecommunity.com>
References: <4D07E55B.3090700@wiggy.net>
	<20101215151601.0BB8B3A4076@sparrow.telecommunity.com>
	<4D08DC43.7040003@wiggy.net>
	<20101215235214.19D923A408B@sparrow.telecommunity.com>
	<4DAE9FA5.5060005@wiggy.net>
	<20110420160456.6E0B43A409D@sparrow.telecommunity.com>
	<4DAF2228.3050604@wiggy.net>
	<20110420213906.CF15E3A4116@sparrow.telecommunity.com>
Message-ID: <4DAFFAE7.2000809@wiggy.net>

On 4/20/11 23:38 , P.J. Eby wrote:
> At 08:12 PM 4/20/2011 +0200, Wichert Akkerman wrote:
>> Is there another way to get this working? I'ld hate to have to resort
>> to monkeypatching distutils.
>
> Well, you could have the symlinks go the other way around, I suppose. ;-)

In theory I could, but I doubt the designers who work with this 
structure will be able to deal with symlinks.

> Alternately, you could write a "file finder" plugin for setuptools, or
> see whether one of the git plugins for setuptools fixes the problem.

setuptools-git did not support this, but luckily that was easily fixed 
(see 
https://github.com/wichert/setuptools-git/commit/8528d58975c595b64eea9e7e67298517117a7744 
).

Wichert.

From merwok at netwok.org  Thu Apr 21 18:00:31 2011
From: merwok at netwok.org (=?UTF-8?Q?=C3=89ric_Araujo?=)
Date: Thu, 21 Apr 2011 18:00:31 +0200
Subject: [Distutils] Including symlinked data files in an egg
In-Reply-To: <4DAE9FA5.5060005@wiggy.net>
References: "\"<4D07E55B.3090700@wiggy.net>	<20101215151601.0BB8B3A4076@sparrow.telecommunity.com>"
	<4D08DC43.7040003@wiggy.net>"
	<20101215235214.19D923A408B@sparrow.telecommunity.com>
	<4DAE9FA5.5060005@wiggy.net>
Message-ID: <45b7784945782fbe10c07555dba20409@netwok.org>

 Hi,

 Le 20/04/2011 10:56, Wichert Akkerman a ?crit :
> I've finally gotten around to doing this. I have attached test.tar.gz
> with a minimal package that demonstrates this.

 Distutils is frozen except for bug fixes (i.e. discrepancies between 
 the documentation and the behavior), but distutils2 is open to feature 
 requests.  Can you open such a request on bugs.python.org, with a 
 rationale (use cases)?  (Note that if you include files such as your 
 test, one or more files or diffs is preferred to binary archives.) 
 Thanks in advance.

 Regards

From pje at telecommunity.com  Thu Apr 21 22:02:48 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 21 Apr 2011 16:02:48 -0400
Subject: [Distutils] Including symlinked data files in an egg
In-Reply-To: <45b7784945782fbe10c07555dba20409@netwok.org>
References: <"\"<4D07E55B.3090700@wiggy.net>
	<20101215151601.0BB8B3A4076@sparrow.telecommunity.com>
	<4D08DC43.7040003@wiggy.net>
	<20101215235214.19D923A408B@sparrow.telecommunity.com>
	<4DAE9FA5.5060005@wiggy.net>
	<45b7784945782fbe10c07555dba20409@netwok.org>
Message-ID: <20110421200315.1E4683A4080@sparrow.telecommunity.com>

At 06:00 PM 4/21/2011 +0200, ??ric Araujo wrote:
>Note that if you include files such as your test, one or more files 
>or diffs is preferred to binary archives.)

Note that since the issue is symlink support, uploading one or more 
files rather than an archive won't work.  ;-)


From stephane at harobed.org  Fri Apr 22 00:00:44 2011
From: stephane at harobed.org (=?ISO-8859-1?Q?St=E9phane_Klein?=)
Date: Fri, 22 Apr 2011 00:00:44 +0200
Subject: [Distutils] What is the status of "extensions" package ? it's the
 future of packaging / distutils2 "entry points" ?
Message-ID: <4DB0A90C.3040101@harobed.org>

Hi,

I would like to know what is the status of "extensions" package 
(http://pypi.python.org/pypi/extensions/0.4) ?

It's the future of "entry points" of packaging / distutils2 ?

Thanks for your help.

Regards,
Stephane
-- 
St?phane Klein <stephane at harobed.org>
blog: http://stephane-klein.info
Twitter: http://twitter.com/klein_stephane
pro: http://www.is-webdesign.com


From alexis at notmyidea.org  Fri Apr 22 13:28:23 2011
From: alexis at notmyidea.org (=?ISO-8859-1?Q?Alexis_M=E9taireau?=)
Date: Fri, 22 Apr 2011 13:28:23 +0200
Subject: [Distutils] What is the status of "extensions" package ? it's
 the future of packaging / distutils2 "entry points" ?
In-Reply-To: <4DB0A90C.3040101@harobed.org>
References: <4DB0A90C.3040101@harobed.org>
Message-ID: <4DB16657.20500@notmyidea.org>

On 22/04/2011 00:00, St?phane Klein wrote:
> Hi,
>
> I would like to know what is the status of "extensions" package
> (http://pypi.python.org/pypi/extensions/0.4) ?
>
> It's the future of "entry points" of packaging / distutils2 ?

Hi,

The extensions package can be used in replacement to setuptools entry 
points.

However, it is not related to packaging / distutils2 and will not be in 
the future, as there is no reason why this kind of feature would stand 
with packaging related code.

-- 
Alexis

From eposse at gmail.com  Fri Apr 22 18:34:59 2011
From: eposse at gmail.com (Ernesto Posse)
Date: Fri, 22 Apr 2011 12:34:59 -0400
Subject: [Distutils] dependencies, pip and non-PyPI-hosted packages
In-Reply-To: <4DADFC1C.406@oddbird.net>
References: <BANLkTik0+RZVcnGd-B0F2eh0kZ2X2rFJbg@mail.gmail.com>
	<4DADD3C3.6010900@oddbird.net>
	<BANLkTinPe9_PXmEmFb0DqqOh6uJoAZN5nQ@mail.gmail.com>
	<4DADFC1C.406@oddbird.net>
Message-ID: <BANLkTikAxkGd=1cJUiHwOyiGBu0fe2j0zw@mail.gmail.com>

Thanks. I have another question. Now that I'm using distribute, pip
handles the requirements correctly, but if I install with:

python setup.py install

the dependencies are not installed. This is a bit perplexing.

I am trying the scenario where the user has a plain Python
installation with no pip, distribute or setuptools (hence installing
the old fashioned way).

My setup.py looks like this:

# Begin of setup.py

from distribute_setup import use_setuptools
use_setuptools()
from setuptools import setup, find_packages

setup(..., install_requires=['project1'],...,dependency_links=['http://my.host.org/repository/'],...)

# End of setup.py

and I took care of including distribute_setup.py in MANIFEST.in, so it
is part of the .tar.gz.

When attempting to install, I get this

$ sudo python setup.py install
Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.15.tar.gz
Extracting in /tmp/tmp4VDb5n
Now working in /tmp/tmp4VDb5n/distribute-0.6.15
Building a Distribute egg in /home/eposse/Downloads/project2-0.1dev
/home/eposse/Downloads/proyecto3-0.1dev/distribute-0.6.15-py2.6.egg
/usr/lib/python2.6/distutils/dist.py:266: UserWarning: Unknown
distribution option: 'install_requires'
  warnings.warn(msg)
...

And the dependencies are ignored.

I don't understand: I explicitly called use_setuptools in my setup,
and as can be seen, distribute is automatically downloaded (albeit
installed in a temporary directory), and imported setup from
setuptools, but it looks like it is calling the plain distutils setup
!?

Why is it not calling the distribute/setuptools setup?

Is there any way around this?

Thanks

PS: since 'requires' and 'provides' are not used in any tool, why are
they in the official documentation?


On Tue, Apr 19, 2011 at 5:18 PM, Carl Meyer <carl at oddbird.net> wrote:
> On 04/19/2011 04:07 PM, Ernesto Posse wrote:
>> 1) using distribute and
>>
>> ? setup(..., install_requires=['project1'],...,
>> dependency_links=['http://my.host.org/repository/'],...)
>>
>> installs as expected both project1 and project2, but
>>
>> ? pip uninstall project2
>>
>> does not uninstall project1. This is quite disappointing, as a user
>> may be unaware of dependencies automatically installed, and thus, the
>> uninstall leaves behind something that was installed with the bundle.
>> I imagine that the idea is that the user may install some other
>> package that depends on 'project1' and pip takes the conservative
>> approach (is that the case?) but I would have expected for pip or
>> distribute or setuptools or distutils to keep some dependency
>> reference counter. Does any of these tools have something like that?
>> or is it going to be addressed in distutils2?
>
> No, there's no current Python packaging tool I know of that keeps a
> dependency reference counter and automatically uninstalls orphaned
> dependencies. The new standard installation format (defined in PEP 376
> and implemented in distutils2) does record whether a package was
> installed by explicit user request or as a dependency; combined with
> checking dependencies of all other installed packages, this will make it
> possible for an uninstaller to implement automatic (or prompted)
> dependency uninstalls.
>
>> 2) if install_requires is missing a dependency (project1), the package
>> gets installed without that dependency, but if I add a dependency and
>> the user attempts an install (project2) with the updated setup.py
>> (listing the new dependency) pip will say that the package (project2)
>> is already installed and won't attempt to install the dependencies. Is
>> this correct? If so, is there a way to tell pip to install project2's
>> dependencies?
>
> With a real package release, the setup.py metadata should never change
> without a version number change. In which case, if you specify the new
> version explicitly or use --upgrade, the previous version will be
> uninstalled and the new one installed in its place, with dependencies.
> (Actually, if you specify --upgrade and you already have the most recent
> version installed, it will still uninstall and reinstall it; but this is
> actually considered a bug.)
>
> Carl
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Ernesto Posse

Modelling and Analysis in Software Engineering
School of Computing
Queen's University - Kingston, Ontario, Canada

From fdrake at acm.org  Fri Apr 22 18:55:47 2011
From: fdrake at acm.org (Fred Drake)
Date: Fri, 22 Apr 2011 12:55:47 -0400
Subject: [Distutils] dependencies, pip and non-PyPI-hosted packages
In-Reply-To: <BANLkTikAxkGd=1cJUiHwOyiGBu0fe2j0zw@mail.gmail.com>
References: <BANLkTik0+RZVcnGd-B0F2eh0kZ2X2rFJbg@mail.gmail.com>
	<4DADD3C3.6010900@oddbird.net>
	<BANLkTinPe9_PXmEmFb0DqqOh6uJoAZN5nQ@mail.gmail.com>
	<4DADFC1C.406@oddbird.net>
	<BANLkTikAxkGd=1cJUiHwOyiGBu0fe2j0zw@mail.gmail.com>
Message-ID: <BANLkTi=Hv2pm0jo98YjyACji+sOhakcDjA@mail.gmail.com>

On Fri, Apr 22, 2011 at 12:34 PM, Ernesto Posse <eposse at gmail.com> wrote:
> PS: since 'requires' and 'provides' are not used in any tool, why are
> they in the official documentation?

We answered the chicken-and-egg problem incorrectly.

Ignore those.


  -Fred

-- 
Fred L. Drake, Jr.? ? <fdrake at acm.org>
"Give me the luxuries of life and I will willingly do without the necessities."
?? --Frank Lloyd Wright

From merwok at netwok.org  Fri Apr 22 19:20:07 2011
From: merwok at netwok.org (=?UTF-8?Q?=C3=89ric_Araujo?=)
Date: Fri, 22 Apr 2011 19:20:07 +0200
Subject: [Distutils] Including symlinked data files in an egg
In-Reply-To: <20110421200315.1E4683A4080@sparrow.telecommunity.com>
References: "\"<\\\"\\\\\\\"<4D07E55B.3090700@wiggy.net>
	<20101215151601.0BB8B3A4076@sparrow.telecommunity.com>
	<4D08DC43.7040003@wiggy.net>
	<20101215235214.19D923A408B@sparrow.telecommunity.com>
	<4DAE9FA5.5060005@wiggy.net>"
	<45b7784945782fbe10c07555dba20409@netwok.org>"
	<20110421200315.1E4683A4080@sparrow.telecommunity.com>
Message-ID: <b434f0f4521265b7a7074b8de86346da@netwok.org>

> Note that since the issue is symlink support, uploading one or more
> files rather than an archive won't work.  ;-)

 Except if the file is a Mercurial diff :)

From carl at oddbird.net  Fri Apr 22 23:54:08 2011
From: carl at oddbird.net (Carl Meyer)
Date: Fri, 22 Apr 2011 16:54:08 -0500
Subject: [Distutils] dependencies, pip and non-PyPI-hosted packages
In-Reply-To: <BANLkTikAxkGd=1cJUiHwOyiGBu0fe2j0zw@mail.gmail.com>
References: <BANLkTik0+RZVcnGd-B0F2eh0kZ2X2rFJbg@mail.gmail.com>	<4DADD3C3.6010900@oddbird.net>	<BANLkTinPe9_PXmEmFb0DqqOh6uJoAZN5nQ@mail.gmail.com>	<4DADFC1C.406@oddbird.net>
	<BANLkTikAxkGd=1cJUiHwOyiGBu0fe2j0zw@mail.gmail.com>
Message-ID: <4DB1F900.8080800@oddbird.net>

Hi Ernesto,

On 04/22/2011 11:34 AM, Ernesto Posse wrote:
> Thanks. I have another question. Now that I'm using distribute, pip
> handles the requirements correctly, but if I install with:
> 
> python setup.py install
> 
> the dependencies are not installed. This is a bit perplexing.

This is just how it is. "python setup.py install" installs the package
in question. Installer tools like pip or easy_install handle
dependencies. If you want dependencies handled automatically, your users
will need to use pip or easy_install.

> I don't understand: I explicitly called use_setuptools in my setup,
> and as can be seen, distribute is automatically downloaded (albeit
> installed in a temporary directory), and imported setup from
> setuptools, but it looks like it is calling the plain distutils setup
> !?
> 
> Why is it not calling the distribute/setuptools setup?

No, it is calling the distribute setup. If you look at how your package
is installed, you'll find it in an egg - that's a sure sign of
setuptools/distribute. It's just that "python setup.py install" does not
handle dependencies, even with setuptools/distribute.

> Is there any way around this?

Not that I know of.

Carl

From jim at zope.com  Sat Apr 23 00:49:54 2011
From: jim at zope.com (Jim Fulton)
Date: Fri, 22 Apr 2011 18:49:54 -0400
Subject: [Distutils] dependencies, pip and non-PyPI-hosted packages
In-Reply-To: <4DB1F900.8080800@oddbird.net>
References: <BANLkTik0+RZVcnGd-B0F2eh0kZ2X2rFJbg@mail.gmail.com>
	<4DADD3C3.6010900@oddbird.net>
	<BANLkTinPe9_PXmEmFb0DqqOh6uJoAZN5nQ@mail.gmail.com>
	<4DADFC1C.406@oddbird.net>
	<BANLkTikAxkGd=1cJUiHwOyiGBu0fe2j0zw@mail.gmail.com>
	<4DB1F900.8080800@oddbird.net>
Message-ID: <BANLkTi=vAQ44dU=ZAei+ezXabcGd=_Grng@mail.gmail.com>

On Fri, Apr 22, 2011 at 5:54 PM, Carl Meyer <carl at oddbird.net> wrote:
> Hi Ernesto,
>
> On 04/22/2011 11:34 AM, Ernesto Posse wrote:
>> Thanks. I have another question. Now that I'm using distribute, pip
>> handles the requirements correctly, but if I install with:
>>
>> python setup.py install
>>
>> the dependencies are not installed. This is a bit perplexing.
>
> This is just how it is. "python setup.py install" installs the package
> in question. Installer tools like pip or easy_install handle
> dependencies. If you want dependencies handled automatically, your users
> will need to use pip or easy_install.

Or zc.buildout. :)

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From eposse at gmail.com  Sat Apr 23 01:09:14 2011
From: eposse at gmail.com (Ernesto Posse)
Date: Fri, 22 Apr 2011 19:09:14 -0400
Subject: [Distutils] dependencies, pip and non-PyPI-hosted packages
In-Reply-To: <4DB1F900.8080800@oddbird.net>
References: <BANLkTik0+RZVcnGd-B0F2eh0kZ2X2rFJbg@mail.gmail.com>
	<4DADD3C3.6010900@oddbird.net>
	<BANLkTinPe9_PXmEmFb0DqqOh6uJoAZN5nQ@mail.gmail.com>
	<4DADFC1C.406@oddbird.net>
	<BANLkTikAxkGd=1cJUiHwOyiGBu0fe2j0zw@mail.gmail.com>
	<4DB1F900.8080800@oddbird.net>
Message-ID: <BANLkTi=2Wir3nJtaV_zab6QOoP4g4Vy0vA@mail.gmail.com>

On Fri, Apr 22, 2011 at 5:54 PM, Carl Meyer <carl at oddbird.net> wrote:
>> I don't understand: I explicitly called use_setuptools in my setup,
>> and as can be seen, distribute is automatically downloaded (albeit
>> installed in a temporary directory), and imported setup from
>> setuptools, but it looks like it is calling the plain distutils setup
>> !?
>>
>> Why is it not calling the distribute/setuptools setup?
>
> No, it is calling the distribute setup. If you look at how your package
> is installed, you'll find it in an egg - that's a sure sign of
> setuptools/distribute. It's just that "python setup.py install" does not
> handle dependencies, even with setuptools/distribute.

Hmm... OK, I guess I assumed that distribute was extending the
standard distutils install command to install dependencies, rather
than pip being responsible. So essentially to take advantage of
distribute one must use pip?

Anyway, is automatic dependency installation going to be a standard
feature of distutils2?



-- 
Ernesto Posse

Modelling and Analysis in Software Engineering
School of Computing
Queen's University - Kingston, Ontario, Canada

From carl at oddbird.net  Sat Apr 23 01:33:53 2011
From: carl at oddbird.net (Carl Meyer)
Date: Fri, 22 Apr 2011 18:33:53 -0500
Subject: [Distutils] dependencies, pip and non-PyPI-hosted packages
In-Reply-To: <BANLkTi=2Wir3nJtaV_zab6QOoP4g4Vy0vA@mail.gmail.com>
References: <BANLkTik0+RZVcnGd-B0F2eh0kZ2X2rFJbg@mail.gmail.com>	<4DADD3C3.6010900@oddbird.net>	<BANLkTinPe9_PXmEmFb0DqqOh6uJoAZN5nQ@mail.gmail.com>	<4DADFC1C.406@oddbird.net>	<BANLkTikAxkGd=1cJUiHwOyiGBu0fe2j0zw@mail.gmail.com>	<4DB1F900.8080800@oddbird.net>
	<BANLkTi=2Wir3nJtaV_zab6QOoP4g4Vy0vA@mail.gmail.com>
Message-ID: <4DB21061.2000007@oddbird.net>

On 04/22/2011 06:09 PM, Ernesto Posse wrote:
> Hmm... OK, I guess I assumed that distribute was extending the
> standard distutils install command to install dependencies, rather
> than pip being responsible. So essentially to take advantage of
> distribute one must use pip?

It extends distutils to do a number of different things (like install
actual metadata, for instance), and it provides the metadata related to
dependencies - but yes, it's easy_install or pip or zc.buildout (thanks
Jim!) that actually makes use of the dependency info to automatically
install things.

> Anyway, is automatic dependency installation going to be a standard
> feature of distutils2?

Yes, d2 will come with a "pysetup" command similar to pip/easy_install
that will handle dependencies.

Carl

From pje at telecommunity.com  Sat Apr 23 21:00:00 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Sat, 23 Apr 2011 15:00:00 -0400
Subject: [Distutils] dependencies, pip and non-PyPI-hosted packages
In-Reply-To: <4DB1F900.8080800@oddbird.net>
References: <BANLkTik0+RZVcnGd-B0F2eh0kZ2X2rFJbg@mail.gmail.com>
	<4DADD3C3.6010900@oddbird.net>
	<BANLkTinPe9_PXmEmFb0DqqOh6uJoAZN5nQ@mail.gmail.com>
	<4DADFC1C.406@oddbird.net>
	<BANLkTikAxkGd=1cJUiHwOyiGBu0fe2j0zw@mail.gmail.com>
	<4DB1F900.8080800@oddbird.net>
Message-ID: <20110423190003.DD4C13A4080@sparrow.telecommunity.com>

At 04:54 PM 4/22/2011 -0500, Carl Meyer wrote:
>No, it is calling the distribute setup. If you look at how your package
>is installed, you'll find it in an egg - that's a sure sign of
>setuptools/distribute. It's just that "python setup.py install" does not
>handle dependencies, even with setuptools/distribute.

Uh, yes it does, actually.  (At least with setuptools, it does; don't 
know about distribute.)

Ernesto, does it work with the official setuptools?  (i.e., if you 
completely uninstall distribute and install setuptools instead)

If it doesn't work, you can file a bug at 
http://bugs.python.org/setuptools and I'll see what we can do to get 
your project fixed up.

(Just make sure you've *fully* uninstalled Distribute, and use 
ez_setup.py instead of distribute_setup.py, as Distribute does some 
patching that keeps setuptools from installing correctly.)


From carl at oddbird.net  Sat Apr 23 21:12:57 2011
From: carl at oddbird.net (Carl Meyer)
Date: Sat, 23 Apr 2011 14:12:57 -0500
Subject: [Distutils] dependencies, pip and non-PyPI-hosted packages
In-Reply-To: <20110423190003.DD4C13A4080@sparrow.telecommunity.com>
References: <BANLkTik0+RZVcnGd-B0F2eh0kZ2X2rFJbg@mail.gmail.com>
	<4DADD3C3.6010900@oddbird.net>
	<BANLkTinPe9_PXmEmFb0DqqOh6uJoAZN5nQ@mail.gmail.com>
	<4DADFC1C.406@oddbird.net>
	<BANLkTikAxkGd=1cJUiHwOyiGBu0fe2j0zw@mail.gmail.com>
	<4DB1F900.8080800@oddbird.net>
	<20110423190003.DD4C13A4080@sparrow.telecommunity.com>
Message-ID: <4DB324B9.2090300@oddbird.net>



On 04/23/2011 02:00 PM, P.J. Eby wrote:
> At 04:54 PM 4/22/2011 -0500, Carl Meyer wrote:
>> No, it is calling the distribute setup. If you look at how your package
>> is installed, you'll find it in an egg - that's a sure sign of
>> setuptools/distribute. It's just that "python setup.py install" does not
>> handle dependencies, even with setuptools/distribute.
> 
> Uh, yes it does, actually.  (At least with setuptools, it does; don't
> know about distribute.)

Erp. Yeah, you're right; just verified that it works fine with both
setuptools and distribute. Dunno where I got that idea.

Carl

From marc at interunion.ca  Mon Apr 25 20:30:35 2011
From: marc at interunion.ca (Marc Tardif)
Date: Mon, 25 Apr 2011 14:30:35 -0400
Subject: [Distutils] Project with multiple eggs
Message-ID: <20110425183035.GG1955@jrrr.montreal>

Hi folks,

Is there a way to build multiple eggs within the same project? I'm using
distutils with buildout which makes certain assumptions about setup.py
being able to build a single egg. So, I started looking around and found
python-repoze.who-plugins which has a setup.py in the base of the project
calling upon individual setup.py files in subdirectories. I'm not sure
this is recommended though and not sure how it would work with buildout
either. So, I'm wondering if there might be a reasonable example of a
project building multiple eggs from the same project.

-- 
Marc Tardif <marc.tardif at canonical.com>
Freenode: cr3, Jabber: cr3 at jabber.org
1024D/72679CAD 09A9 D871 F7C4 A18F AC08 674D 2B73 740C 7267 9CAD


From wichert at wiggy.net  Tue Apr 26 11:27:52 2011
From: wichert at wiggy.net (Wichert Akkerman)
Date: Tue, 26 Apr 2011 11:27:52 +0200
Subject: [Distutils] symlinks in sources
Message-ID: <4DB69018.1000902@wiggy.net>

I am trying to figure out the intended behaviour of dealing with 
symlinks in package sources. My use case is that I have a number of 
package implementation web things, and they tend to have an interactive 
(static) prototype parts of which are reused directly in the 
application. The filesystem structure for those packages is one of two 
options:

1. Prototype is at the toplevel of the source. For example:

    Prototype/index.html
    Prototyle/style/main.css
    setup.py
    MANIFEST.in
    src/namespace/__init__.py
    src/namespace/templates/index.html
    src/namespace/templates/style -> ../../../Prototype/style

2. Prototype is at outside the source. For example:

    Prototype/index.html
    Prototyle/style/main.css
    buildout/bootstrap.py
    buildout/buildout.cfg
    buildout/src/mypackage/setup.py
    buildout/src/mypackage/MANIFEST.in
    buildout/src/mypackage/src/namespace/__init__.py
    buildout/src/mypackage/src/namespace/templates/index.html
    buildout/src/mypackage/src/namespace/templates/style -> 
../../../../../../Prototype/style

My goal is to be able to create an sdist for my package and upload that 
to pypi, and when people install that namespace/templates/style/main.css 
must exist. Currently this does not appear to be possible. The problems 
I see are:

a) symlinks are skipped by distutils
b) if you use a file finder plugin which traverses the symlink and
    reports the files they end up twice in the sdist: once for the
    Prototype and once inside the package. They second one is stored
    as a link inside the sdist, but that link is not unpacked when
    installing from the sdist and the files go missing.

Trying to analyse things a bit I see a number of possible scenarios for 
symlinks:

i) symlink pointing to file inside the packgage
ii) symlink pointing to directory inside the packgage
iii) symlink pointing to file outside the packgage, but in SCM
iv) symlink pointing to directory outside the packgage, but in SCM
v) symlink pointing to something outside package and not managed by SCM

My expectation is that i and ii work out of the box, iii and iv should 
work if you have a file finder plugin for setuptools that traverses the 
symlink, and v to not work. Is that expectation reasonable?


Wichert.


From eposse at gmail.com  Tue Apr 26 18:02:58 2011
From: eposse at gmail.com (Ernesto Posse)
Date: Tue, 26 Apr 2011 12:02:58 -0400
Subject: [Distutils] dependencies, pip and non-PyPI-hosted packages
In-Reply-To: <4DB324B9.2090300@oddbird.net>
References: <BANLkTik0+RZVcnGd-B0F2eh0kZ2X2rFJbg@mail.gmail.com>
	<4DADD3C3.6010900@oddbird.net>
	<BANLkTinPe9_PXmEmFb0DqqOh6uJoAZN5nQ@mail.gmail.com>
	<4DADFC1C.406@oddbird.net>
	<BANLkTikAxkGd=1cJUiHwOyiGBu0fe2j0zw@mail.gmail.com>
	<4DB1F900.8080800@oddbird.net>
	<20110423190003.DD4C13A4080@sparrow.telecommunity.com>
	<4DB324B9.2090300@oddbird.net>
Message-ID: <BANLkTikKgTGCavziSfkuaRZ=dOeS5wbT9w@mail.gmail.com>

On Sat, Apr 23, 2011 at 3:12 PM, Carl Meyer <carl at oddbird.net> wrote:
>
>
> On 04/23/2011 02:00 PM, P.J. Eby wrote:
>> At 04:54 PM 4/22/2011 -0500, Carl Meyer wrote:
>>> No, it is calling the distribute setup. If you look at how your package
>>> is installed, you'll find it in an egg - that's a sure sign of
>>> setuptools/distribute. It's just that "python setup.py install" does not
>>> handle dependencies, even with setuptools/distribute.
>>
>> Uh, yes it does, actually. ?(At least with setuptools, it does; don't
>> know about distribute.)
>
> Erp. Yeah, you're right; just verified that it works fine with both
> setuptools and distribute. Dunno where I got that idea.
>
> Carl


I'm confused now. I have tried both with setuptools alone (using
ez_setup.py instead of distribute_setup.py) and distribute, and it
doesn't work with distribute alone for me: it doesn't install the
dependencies and gives me those warnings about 'install_requires' not
being recognized.

Any ideas about what could be wrong?



-- 
Ernesto Posse

Modelling and Analysis in Software Engineering
School of Computing
Queen's University - Kingston, Ontario, Canada

From tseaver at palladion.com  Wed Apr 27 01:30:03 2011
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 26 Apr 2011 19:30:03 -0400
Subject: [Distutils] dependencies, pip and non-PyPI-hosted packages
In-Reply-To: <BANLkTikKgTGCavziSfkuaRZ=dOeS5wbT9w@mail.gmail.com>
References: <BANLkTik0+RZVcnGd-B0F2eh0kZ2X2rFJbg@mail.gmail.com>	<4DADD3C3.6010900@oddbird.net>	<BANLkTinPe9_PXmEmFb0DqqOh6uJoAZN5nQ@mail.gmail.com>	<4DADFC1C.406@oddbird.net>	<BANLkTikAxkGd=1cJUiHwOyiGBu0fe2j0zw@mail.gmail.com>	<4DB1F900.8080800@oddbird.net>	<20110423190003.DD4C13A4080@sparrow.telecommunity.com>	<4DB324B9.2090300@oddbird.net>
	<BANLkTikKgTGCavziSfkuaRZ=dOeS5wbT9w@mail.gmail.com>
Message-ID: <ip7khr$52p$1@dough.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 04/26/2011 12:02 PM, Ernesto Posse wrote:
> On Sat, Apr 23, 2011 at 3:12 PM, Carl Meyer <carl at oddbird.net> wrote:
>>
>>
>> On 04/23/2011 02:00 PM, P.J. Eby wrote:
>>> At 04:54 PM 4/22/2011 -0500, Carl Meyer wrote:
>>>> No, it is calling the distribute setup. If you look at how your package
>>>> is installed, you'll find it in an egg - that's a sure sign of
>>>> setuptools/distribute. It's just that "python setup.py install" does not
>>>> handle dependencies, even with setuptools/distribute.
>>>
>>> Uh, yes it does, actually.  (At least with setuptools, it does; don't
>>> know about distribute.)
>>
>> Erp. Yeah, you're right; just verified that it works fine with both
>> setuptools and distribute. Dunno where I got that idea.
>>
>> Carl
> 
> 
> I'm confused now. I have tried both with setuptools alone (using
> ez_setup.py instead of distribute_setup.py) and distribute, and it
> doesn't work with distribute alone for me: it doesn't install the
> dependencies and gives me those warnings about 'install_requires' not
> being recognized.
> 
> Any ideas about what could be wrong?

How are you invoking setup?  Should be something like:

  from setuptools import setup

  setup(name='your.package',
        ...
        install_requires=['other.packaage'],
       )


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk23VXsACgkQ+gerLs4ltQ5O6gCfd8WzYUb6+bWlGvPC/ZUsfcHu
NtIAn1pFAvWtvTV8Slap6Iopl2sT/5UD
=Hi4F
-----END PGP SIGNATURE-----


From eposse at gmail.com  Wed Apr 27 01:45:33 2011
From: eposse at gmail.com (Ernesto Posse)
Date: Tue, 26 Apr 2011 19:45:33 -0400
Subject: [Distutils] dependencies, pip and non-PyPI-hosted packages
In-Reply-To: <ip7khr$52p$1@dough.gmane.org>
References: <BANLkTik0+RZVcnGd-B0F2eh0kZ2X2rFJbg@mail.gmail.com>
	<4DADD3C3.6010900@oddbird.net>
	<BANLkTinPe9_PXmEmFb0DqqOh6uJoAZN5nQ@mail.gmail.com>
	<4DADFC1C.406@oddbird.net>
	<BANLkTikAxkGd=1cJUiHwOyiGBu0fe2j0zw@mail.gmail.com>
	<4DB1F900.8080800@oddbird.net>
	<20110423190003.DD4C13A4080@sparrow.telecommunity.com>
	<4DB324B9.2090300@oddbird.net>
	<BANLkTikKgTGCavziSfkuaRZ=dOeS5wbT9w@mail.gmail.com>
	<ip7khr$52p$1@dough.gmane.org>
Message-ID: <BANLkTimRFzKmFEVMuHODXWOJP-ZaUDsmmQ@mail.gmail.com>

On Tue, Apr 26, 2011 at 7:30 PM, Tres Seaver <tseaver at palladion.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 04/26/2011 12:02 PM, Ernesto Posse wrote:
>> On Sat, Apr 23, 2011 at 3:12 PM, Carl Meyer <carl at oddbird.net> wrote:
>>>
>>>
>>> On 04/23/2011 02:00 PM, P.J. Eby wrote:
>>>> At 04:54 PM 4/22/2011 -0500, Carl Meyer wrote:
>>>>> No, it is calling the distribute setup. If you look at how your package
>>>>> is installed, you'll find it in an egg - that's a sure sign of
>>>>> setuptools/distribute. It's just that "python setup.py install" does not
>>>>> handle dependencies, even with setuptools/distribute.
>>>>
>>>> Uh, yes it does, actually. ?(At least with setuptools, it does; don't
>>>> know about distribute.)
>>>
>>> Erp. Yeah, you're right; just verified that it works fine with both
>>> setuptools and distribute. Dunno where I got that idea.
>>>
>>> Carl
>>
>>
>> I'm confused now. I have tried both with setuptools alone (using
>> ez_setup.py instead of distribute_setup.py) and distribute, and it
>> doesn't work with distribute alone for me: it doesn't install the
>> dependencies and gives me those warnings about 'install_requires' not
>> being recognized.
>>
>> Any ideas about what could be wrong?
>
> How are you invoking setup? ?Should be something like:
>
> ?from setuptools import setup
>
> ?setup(name='your.package',
> ? ? ? ?...
> ? ? ? ?install_requires=['other.packaage'],
> ? ? ? )
>

The version using distribute (which doesn't get the dependencies and
looks as if it was calling the distutils setup) is this:

# Begin of setup.py with distribute

from distribute_setup import use_setuptools
use_setuptools()
from setuptools import setup, find_packages

setup(name="project3",...,
install_requires=['project1','project2'],...,dependency_links=['http://my.host.org/repository/'],...)

# End of setup.py with distribute

The version that uses only setuptools and not distribute (and does
pick the dependencies) is as follows:

# Begin of setup.py with setuptools

from ez_setup import use_setuptools
use_setuptools()
from setuptools import setup, find_packages

setup(name="project3",...,
install_requires=['project1','project2'],...,dependency_links=['http://my.host.org/repository/'],...)

# End of setup.py with setuptools

The only difference is using ez_setup instead of distribute_setup (the
MANIFEST.in file is updated accordingly).

I have tried both in "clean" enviroments, removing both setuptools and
distribute. Of course, if I do use pip instead of "python setup
install" it works just fine.





> Tres.
> - --
> ===================================================================
> Tres Seaver ? ? ? ? ?+1 540-429-0999 ? ? ? ? ?tseaver at palladion.com
> Palladion Software ? "Excellence by Design" ? ?http://palladion.com
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAk23VXsACgkQ+gerLs4ltQ5O6gCfd8WzYUb6+bWlGvPC/ZUsfcHu
> NtIAn1pFAvWtvTV8Slap6Iopl2sT/5UD
> =Hi4F
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Ernesto Posse

Modelling and Analysis in Software Engineering
School of Computing
Queen's University - Kingston, Ontario, Canada

From eric at trueblade.com  Wed Apr 27 01:58:23 2011
From: eric at trueblade.com (Eric Smith)
Date: Tue, 26 Apr 2011 19:58:23 -0400
Subject: [Distutils] [buildout] "private" releases
In-Reply-To: <4D9C1480.4020208@simplistix.co.uk>
References: <imih2a$uig$1@dough.gmane.org> <4D8CEBD5.7010903@simplistix.co.uk>
	<4D9B3640.4090300@trueblade.com>
	<4D9C1480.4020208@simplistix.co.uk>
Message-ID: <4DB75C1F.9020702@trueblade.com>

On 4/6/2011 3:21 AM, Chris Withers wrote:
> On 05/04/2011 16:33, Eric Smith wrote:
>> Could you elaborate on this? How does buildout/setuptools/distribut
>> search for an sdist, lacking an index file? Does it look for .tgz only,
>> or other extensions as well? This would save me a ton of heartburn if I
>> could get it to work.
> 
> Point find-links at either a folder on disk, or that sample folder
> served over http by Apache's normal folder serving (ie: index listings on).
> 
> Just stick any type of dist into that folder.
> 
> The rest "just works".
> 
> If it doesn't, explain what you did and we'll see if we can spot what
> went wrong...

Finally following up on this. Yes, just using "Options +Indexes" works
fine. Thanks.

Eric.

From chris at simplistix.co.uk  Wed Apr 27 13:38:37 2011
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 27 Apr 2011 12:38:37 +0100
Subject: [Distutils] dependencies, pip and non-PyPI-hosted packages
In-Reply-To: <BANLkTimRFzKmFEVMuHODXWOJP-ZaUDsmmQ@mail.gmail.com>
References: <BANLkTik0+RZVcnGd-B0F2eh0kZ2X2rFJbg@mail.gmail.com>	<4DADD3C3.6010900@oddbird.net>	<BANLkTinPe9_PXmEmFb0DqqOh6uJoAZN5nQ@mail.gmail.com>	<4DADFC1C.406@oddbird.net>	<BANLkTikAxkGd=1cJUiHwOyiGBu0fe2j0zw@mail.gmail.com>	<4DB1F900.8080800@oddbird.net>	<20110423190003.DD4C13A4080@sparrow.telecommunity.com>	<4DB324B9.2090300@oddbird.net>	<BANLkTikKgTGCavziSfkuaRZ=dOeS5wbT9w@mail.gmail.com>	<ip7khr$52p$1@dough.gmane.org>
	<BANLkTimRFzKmFEVMuHODXWOJP-ZaUDsmmQ@mail.gmail.com>
Message-ID: <4DB8003D.2050408@simplistix.co.uk>

On 27/04/2011 00:45, Ernesto Posse wrote:
> # Begin of setup.py with distribute
>
> from distribute_setup import use_setuptools
> use_setuptools()
> from setuptools import setup, find_packages
>
> setup(name="project3",...,
> install_requires=['project1','project2'],...,dependency_links=['http://my.host.org/repository/'],...)

Won't solve your problem, but please don't use dependency_links, it's 
just plain evil...

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From jim at zope.com  Sat Apr 30 15:15:54 2011
From: jim at zope.com (Jim Fulton)
Date: Sat, 30 Apr 2011 09:15:54 -0400
Subject: [Distutils] Buildout for Python 3, zc.buildout 2.0.0a1 released
Message-ID: <BANLkTikE4tBv=5AU5fj-7AZq_BdvXoAA=A@mail.gmail.com>

This release is for existing buildout users who want to use buildout
with Python 3.  The main goal for this release is to facilitate
porting projects that use buildout to Python 3.

For more information, see:

  http://pypi.python.org/pypi/zc.buildout/2.0.0a1

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From arfrever.fta at gmail.com  Sat Apr 30 18:22:49 2011
From: arfrever.fta at gmail.com (Arfrever Frehtes Taifersar Arahesis)
Date: Sat, 30 Apr 2011 18:22:49 +0200
Subject: [Distutils] Buildout for Python 3, zc.buildout 2.0.0a1 released
In-Reply-To: <BANLkTikE4tBv=5AU5fj-7AZq_BdvXoAA=A@mail.gmail.com>
References: <BANLkTikE4tBv=5AU5fj-7AZq_BdvXoAA=A@mail.gmail.com>
Message-ID: <201104301823.17878.Arfrever.FTA@gmail.com>

2011-04-30 15:15:54 Jim Fulton napisa?(a):
> This release is for existing buildout users who want to use buildout
> with Python 3.  The main goal for this release is to facilitate
> porting projects that use buildout to Python 3.
> 
> For more information, see:
> 
>   http://pypi.python.org/pypi/zc.buildout/2.0.0a1

Source tarballs are missing for zc.buildout-2.0.0a1 and zc.recipe.egg-2.0.0a1.

-- 
Arfrever Frehtes Taifersar Arahesis
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20110430/27f7a417/attachment.pgp>

From jim at zope.com  Sat Apr 30 18:48:10 2011
From: jim at zope.com (Jim Fulton)
Date: Sat, 30 Apr 2011 12:48:10 -0400
Subject: [Distutils] Buildout for Python 3, zc.buildout 2.0.0a1 released
In-Reply-To: <201104301823.17878.Arfrever.FTA@gmail.com>
References: <BANLkTikE4tBv=5AU5fj-7AZq_BdvXoAA=A@mail.gmail.com>
	<201104301823.17878.Arfrever.FTA@gmail.com>
Message-ID: <BANLkTik=zHAz8UQbZ73JjV59jck9jXDNtg@mail.gmail.com>

On Sat, Apr 30, 2011 at 12:22 PM, Arfrever Frehtes Taifersar Arahesis
<arfrever.fta at gmail.com> wrote:
> 2011-04-30 15:15:54 Jim Fulton napisa?(a):
>> This release is for existing buildout users who want to use buildout
>> with Python 3. ?The main goal for this release is to facilitate
>> porting projects that use buildout to Python 3.
>>
>> For more information, see:
>>
>> ? http://pypi.python.org/pypi/zc.buildout/2.0.0a1
>
> Source tarballs are missing for zc.buildout-2.0.0a1 and zc.recipe.egg-2.0.0a1.

This is intentional.  I don't want these to be accidentally be picked
up by Python 2 users.

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From regebro at gmail.com  Sat Apr 30 20:16:13 2011
From: regebro at gmail.com (Lennart Regebro)
Date: Sat, 30 Apr 2011 20:16:13 +0200
Subject: [Distutils] Buildout for Python 3, zc.buildout 2.0.0a1 released
In-Reply-To: <BANLkTikE4tBv=5AU5fj-7AZq_BdvXoAA=A@mail.gmail.com>
References: <BANLkTikE4tBv=5AU5fj-7AZq_BdvXoAA=A@mail.gmail.com>
Message-ID: <BANLkTimGQ-OfC+=g=JEk0qdXZT7+PtoVOA@mail.gmail.com>

Awesome!

On Sat, Apr 30, 2011 at 15:15, Jim Fulton <jim at zope.com> wrote:
> This release is for existing buildout users who want to use buildout
> with Python 3. ?The main goal for this release is to facilitate
> porting projects that use buildout to Python 3.
>
> For more information, see:
>
> ?http://pypi.python.org/pypi/zc.buildout/2.0.0a1
>
> Jim
>
> --
> Jim Fulton
> http://www.linkedin.com/in/jimfulton
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>