From jim at zope.com Sat Dec 1 19:46:43 2012 From: jim at zope.com (Jim Fulton) Date: Sat, 1 Dec 2012 13:46:43 -0500 Subject: [Distutils] zc.buildout 2.0.0a5 released Message-ID: The release is available from github: https://github.com/downloads/buildout/buildout/zc.buildout-2.0.0a5.tar.gz To bootstrap this release, use the bootstrap.py file at: https://github.com/downloads/buildout/buildout/bootstrap.py Changes from the previous release: - Buildout will no-longer install non-final releases if there are final releases that meet requirements. (IOW, the prefer-final buildout option now defaults to true. You can get the old behavior by setting prefer-final to false.) - You can now specify simple constrains in versions sections. Previously, you could only set versions, for example: zc.recipe.egg == 2.0.0a3 But now you can specify that you want versions greater than or less then a given version: zc.recipe.egg == >=2.0.0a3 This was added so that buildout2 can constrain the version of zc.recipe.egg used without requiring a specific version. buildout1 will be updated with this feature too, so that it can prevent itself from using newer versions of zc.recipe.egg and to prevent upgrading itself to buildout2. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm From adorsk at gmail.com Mon Dec 3 01:08:00 2012 From: adorsk at gmail.com (Alexander Dorsk) Date: Sun, 2 Dec 2012 19:08:00 -0500 Subject: [Distutils] Packager/Installer for Jython Deps? Message-ID: Hi Packaging Folks, I'm looking for some tips on developing a packaging/install system for Jython distributions. Ideally I would like a create a system that would manage not only python dependencies, but also java dependencies. Perhaps a quick example will help to illustrate: = Begin Example A = Say I'm working on a project, and I want to install a package, 'my_jython_h2_stuff' that has these dependencies: - the sqlalchemy python package - the H2 Database .jar In my dream world I would do something like this: $ jython virtualenv.py jython_env; $ source jython_env/bin/activate; $ jython_env/bin/some_jython_package_installer install my_jython_h2_stuff The end result of the above commands would be: - The pure python components of my_jython_h2_stuff are in jython_env/Lib/*/site_packages/my_jython_h2_stuff - sqlalchemy is in jython_env/Lib/*/site_packages/sqalchemy - The H2.jar file is in some place where jython can find it, e.g. jython_env/javalibs/h2.jar Then I could do something like this: $ jython -c "import my_jython_h2_stuff" Furthermore, the 'my_jython_h2_stuff' dependencies could be described in a distutils2-style setup.cfg w/ sections like: --- [requires-dist] sqlalchemy #A. Define single jars, through some uri/vcs scheme [requires-jars] h2.jar http://h2.org/download/h2.jar #B. Define individual jars, via maven-style IDs [requires-jars-maven] org.h2:h2.jdbc.jar:1.14 #C. Define requirements in a pom.xml file # Defines path to .pom file, relative to setup.cfg file. [requires-pom] pom.xml --- At this point, since we're still in my dream world, Jennifer Connelly would show up in a boat with umbrella drinks and a fresh rhubarb pie. But I digress. = End Example A = Hopefully that example helps illustrate what I mean. Currently it's somewhat tricky to manage both .jar and python dependencies for Jython projects, due to Jython's mixed heritage. You might say that Jython is the Obama of programming; a hybrid that grapples with its dual heritage and yet draws upon it for strength. It's complicated sometimes. However, with recent developments such as the distutils2 style of keeping metadata outside setup.py, and tools like sunng87's jip < https://github.com/sunng87/jip >, I think there may be ways to address the difficulty of managing dependencies for jython distributions. One way might be to create an installer, or an extension to an existing installer that handles special jython-specific metadata, and this leads me to some questions: Q1: Is there an existing installer that could handle special cases of metadata? Q2: If 'no' to Q1, would it be better to (A) create a new installer from scratch, or (B) is there a way to extend an existing installer, such as pip? Q3: If (B) for Q2, is there documentation on how how to do this? I didn't see anything the pip documents about extending PIP to handle special metadata. I'd love to hear any pointers. There is possibly great value to be gained if a Jython distribution installer could be developed. For example, it could be possible to easily develop and distribute python apps via self-contained stand-alone .jars . Imagine something like this: 1. create a jython_installer requirements file (just like a pip requirements file) 2. $ jython create_jython_virtualenv -r requirements.txt my_app_virtualenv 3. (do custom app development in the virtual env, and define an application entry point) 4. jython virtualenv_to_jar my_app_virtualenv > my_standalone_app.jar 5. mail my_standalone_app.jar to some windows user. 6. user double-clicks my_standalone_app.jar, and- *it just runs*, no installers or anything. It would also be possible to include jython ports in the same source bundles as the python originals. For example, a numpy port that uses java libs instead of c extensions. The .py source would check for the platform and import jython or cython modules appropriately, and requisite .jars would be described in the setup.cfg file. For normal cython installs via pip or the such, the .jar metadata would be ignored. For jython installs via a jython_installer, the .jars would be fetched. And the naming possibilities for such a packager are endless. "Samuel L. Jackson's 'Snakes-in-a-jar', or perhaps "PeanutCan", like the old Snake-in-a-can prank... < http://www.youtube.com/watch?v=GADUPKy4jos >. Anyway, if anyone has any tips on the above questions, I'm would love to hear 'em. -Alex -------------- next part -------------- An HTML attachment was scrubbed... URL: From techtonik at gmail.com Mon Dec 3 08:05:33 2012 From: techtonik at gmail.com (anatoly techtonik) Date: Mon, 3 Dec 2012 10:05:33 +0300 Subject: [Distutils] Override default build dir from setup.py Message-ID: Hi, I disliked that setup.py creates its own temp directories without any private prefix, so I wanted to override this option in setup.py: $ setup.py build -h ... Options for 'build' command: --build-base (-b) base directory for build library And the only way I could find to do this is through setup.cfg: http://docs.python.org/2/distutils/configfile.html Is it really the case or I overlooked some essential documentation part? Please, CC. -- anatoly t. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pje at telecommunity.com Mon Dec 3 18:33:02 2012 From: pje at telecommunity.com (PJ Eby) Date: Mon, 3 Dec 2012 12:33:02 -0500 Subject: [Distutils] Override default build dir from setup.py In-Reply-To: References: Message-ID: On Mon, Dec 3, 2012 at 2:05 AM, anatoly techtonik wrote: > Hi, > > I disliked that setup.py creates its own temp directories without any > private prefix, so I wanted to override this option in setup.py: > > $ setup.py build -h > ... > Options for 'build' command: > --build-base (-b) base directory for build library > > And the only way I could find to do this is through setup.cfg: > http://docs.python.org/2/distutils/configfile.html > > Is it really the case or I overlooked some essential documentation part? There are user-level and global .cfg files as well: http://docs.python.org/release/2.7/install/index.html#location-and-names-of-config-files From dholth at gmail.com Tue Dec 4 02:44:52 2012 From: dholth at gmail.com (Daniel Holth) Date: Mon, 3 Dec 2012 20:44:52 -0500 Subject: [Distutils] [Python-Dev] Accept just PEP-0426 In-Reply-To: References: <98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com> <4AB62E69A53048589648EA9A94CAC02A@gmail.com> <50ABF128.4020605@g.nevcal.com> Message-ID: On Tue, Nov 20, 2012 at 11:01 PM, Nick Coghlan wrote: > On Wed, Nov 21, 2012 at 1:20 PM, Nick Coghlan wrote: > >> On Wed, Nov 21, 2012 at 1:10 PM, PJ Eby wrote: >> >>> Conversely, if you have already installed a package that says it >>> "Obsoletes" another package, this does *not* tell you that the obsolete >>> package shouldn't still be installed! A replacement project doesn't >>> necessarily share the same API, and may exist in a different package >>> namespace altogether. >>> >> >> Then that's a bug in the metadata of the project misusing "Obsoletes", >> and should be reported as such. If the new package is not a drop-in >> replacement, then it has no business claiming to obsolete the other package. >> >> I think one of the big reasons this kind of use is rare in the Python >> community is that project name changes are almost always accompanied by >> *package* name changes, and as soon as you change the package name, you're >> changing the public API, and thus it is no longer appropriate to use >> Provides or Obsoletes, as the renamed project is no longer a drop-in >> replacement for the original. >> > > I realised that my comments above are more about the appropriate use of > "Provides", rather than "Obsoletes". For a practically useful "Obsoletes", > I think I'm inclined to agree with you, as "Obsoleted-By" provides a way > for a maintainer to explicitly declare that a project is no longer > receiving updates, and users should migrate to the replacement project if > they want to continue to receive fixes and improvements. The current > version of "Obsoletes" is, as Daniel describes, really only useful as > documentation. > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > A few more changes to try to address some of the confusion about Requires-Dist: without re-designing the entire requirements system. PEP-426 was written only to add extras support to the format. The other changes, re-writing much of the PEP, have been an unfortunate side-effect. The file format's keys are case-insensitive. The version number should be in PEP 386 form. There are too many non-PEP-386 versions now and in the future to make it a must. Distribution (requirement) names are noted as being distinct from ``import x`` module names. Parenthetical explanation has balanced parens. "bundled" has been struck from the PEP. diff -r 55c706023fa2 -r 026aebf2265d pep-0426.txt --- a/pep-0426.txt Sun Nov 18 19:55:10 2012 +0200 +++ b/pep-0426.txt Mon Dec 03 20:36:13 2012 -0500 @@ -34,9 +34,9 @@ The syntax defined in this PEP is for use with Python distribution metadata files. The file format is a simple UTF-8 encoded Key: value -format with no maximum line length, followed by a blank line and an -arbitrary payload. The keys are case-insensitive. It is parseable by -the ``email`` module with an appropriate ``email.policy.Policy()``. +format with case-insensitive keys and no maximum line length, followed by +a blank line and an arbitrary payload. It is parseable by the ``email`` +module with an appropriate ``email.policy.Policy()``. When ``metadata`` is a Unicode string, ```email.parser.Parser().parsestr(metadata)`` is a serviceable parser. @@ -94,7 +94,7 @@ ::::::: A string containing the distribution's version number. This -field must be in the format specified in PEP 386. +field should be in the format specified in PEP 386. Example:: @@ -283,12 +283,13 @@ Each entry contains a string naming some other distutils project required by this distribution. -The format of a requirement string is identical to that of a -distutils project name (e.g., as found in the ``Name:`` field. -optionally followed by a version declaration within parentheses. +The format of a requirement string is identical to that of a distribution +name (e.g., as found in the ``Name:`` field) optionally followed by a +version declaration within parentheses. -The distutils project names should correspond to names as found -on the `Python Package Index`_. +The distribution names should correspond to names as found on the `Python +Package Index`_; often the same as, but distinct from, the module names +as accessed with ``import x``. Version declarations must follow the rules described in `Version Specifiers`_ @@ -305,7 +306,8 @@ Like Requires-Dist, but names dependencies needed while the distributions's distutils / packaging `setup.py` / `setup.cfg` is run. -Commonly used to generate a manifest from version control. +Commonly used to bring in extra compiler support or a package needed +to generate a manifest from version control. Examples:: @@ -318,17 +320,19 @@ Provides-Dist (multiple use) :::::::::::::::::::::::::::: -Each entry contains a string naming a Distutils project which -is contained within this distribution. This field *must* include -the project identified in the ``Name`` field, followed by the -version : Name (Version). +Each entry contains a string naming a requirement that is satisfied by +installing this distribution. This field *must* include the project +identified in the ``Name`` field, optionally followed by the version: +Name (Version). A distribution may provide additional names, e.g. to indicate that -multiple projects have been bundled together. For instance, source -distributions of the ``ZODB`` project have historically included -the ``transaction`` project, which is now available as a separate -distribution. Installing such a source distribution satisfies -requirements for both ``ZODB`` and ``transaction``. +multiple projects have been merged into and replaced by a single +distribution or to indicate that this project is a substitute for another. +For instance distribute (a fork of setuptools) could ``Provides-Dist`` +setuptools to prevent the conflicting package from being downloaded and +installed when distribute is already installed. A distribution that has +been merged with another might ``Provides-Dist`` the obsolete name(s) +to satisfy any projects that require the obsolete distribution's name. A distribution may also provide a "virtual" project name, which does not correspond to any separately-distributed project: such a name @@ -359,10 +363,9 @@ Version declarations can be supplied. Version numbers must be in the format specified in `Version Specifiers`_. -The most common use of this field will be in case a project name -changes, e.g. Gorgon 2.3 gets subsumed into Torqued Python 1.0. -When you install Torqued Python, the Gorgon distribution should be -removed. +The most common use of this field will be in case a project name changes, +e.g. Gorgon 2.3 gets renamed to Torqued Python 1.0. When you install +Torqued Python, the Gorgon distribution should be removed. Examples:: -------------- next part -------------- An HTML attachment was scrubbed... URL: From eu at doxos.eu Fri Dec 7 17:13:26 2012 From: eu at doxos.eu (=?UTF-8?B?VsOhY2xhdiDFoG1pbGF1ZXI=?=) Date: Fri, 07 Dec 2012 17:13:26 +0100 Subject: [Distutils] separate distribution of arbitrary submodules? Message-ID: <50C215A6.3060201@doxos.eu> Hello, I am using setuptools to package module A. I would like to separately package also modules A.* such that those submodules are discoverable (using pkgutil.iter_modules preferrably) from within A, provided their names are unknown to A. Can this be done with some clever __path__ manipulation perhaps? I tried installing A.B with its own setup.py, with setup(...,packages=['A.B']), it is installed in A.B-....egg, but not found when I want to import A.B (let alone with iter_modules). Background of this question: a common code A is distributed to customers, which in addition use their specific modules (A.A, A.B, A.C, ...); the code in A should, however, show them which modules are available - for user-friendliness. Cheers, Vaclav From pje at telecommunity.com Sat Dec 8 07:57:03 2012 From: pje at telecommunity.com (PJ Eby) Date: Sat, 8 Dec 2012 01:57:03 -0500 Subject: [Distutils] separate distribution of arbitrary submodules? In-Reply-To: <50C215A6.3060201@doxos.eu> References: <50C215A6.3060201@doxos.eu> Message-ID: On Fri, Dec 7, 2012 at 11:13 AM, V?clav ?milauer wrote: > Hello, > > I am using setuptools to package module A. I would like to separately > package also modules A.* such that those submodules are discoverable (using > pkgutil.iter_modules preferrably) from within A, provided their names are > unknown to A. Can this be done with some clever __path__ manipulation > perhaps? What you want is called a namespace package. If you declare A as a namespace package, then you can ship as many separate projects containing A.* modules, and pkgutil.iter_modules() will work on them once installed. (What you cannot do is put any code in A/__init__.py for those projects, except the code that declares it a namespace package. Please see the setuptools and pkg_resources documentation on namespace packages for more info.) From a.cavallo at cavallinux.eu Mon Dec 10 08:22:13 2012 From: a.cavallo at cavallinux.eu (Antonio Cavallo) Date: Mon, 10 Dec 2012 07:22:13 +0000 Subject: [Distutils] Is is worth disentangling distutils? Message-ID: <50C58DA5.3000307@cavallinux.eu> Hi, I wonder if is it worth/if there is any interest in trying to "clean" up distutils: nothing in terms to add new features, just a *major* cleanup retaining the exact same interface. I'm not planning anything like *adding features* or rewriting rpm/rpmbuild here, simply cleaning up that un-holy code mess. Yes it served well, don't get me wrong, and I think it did work much better than anything it was meant to replace it. I'm not into the py3 at all so I wonder how possibly it could fit/collide into the big plan. Or I'll be wasting my time? Thanks From dholth at gmail.com Mon Dec 10 14:29:02 2012 From: dholth at gmail.com (Daniel Holth) Date: Mon, 10 Dec 2012 08:29:02 -0500 Subject: [Distutils] Is is worth disentangling distutils? In-Reply-To: <50C58DA5.3000307@cavallinux.eu> References: <50C58DA5.3000307@cavallinux.eu> Message-ID: On Mon, Dec 10, 2012 at 2:22 AM, Antonio Cavallo wrote: > Hi, > I wonder if is it worth/if there is any interest in trying to "clean" up > distutils: nothing in terms to add new features, just a *major* cleanup > retaining the exact same interface. > > > I'm not planning anything like *adding features* or rewriting rpm/rpmbuild > here, simply cleaning up that un-holy code mess. Yes it served well, don't > get me wrong, and I think it did work much better than anything it was > meant to replace it. > > I'm not into the py3 at all so I wonder how possibly it could fit/collide > into the big plan. > > Or I'll be wasting my time? > It has been tried before. IIUC the nature of distutils and extending distutils is that client code depends on the entire tangle. If you try to clean it up you will break backwards compatibility. distutils2 is designed to break backwards compatibility with distutils and is essentially a cleaned up distutils. Have you tried Bento? http://bento.readthedocs.org/en/latest/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From qyarai at gmail.com Tue Dec 11 05:26:30 2012 From: qyarai at gmail.com (Yoshio Arai) Date: Mon, 10 Dec 2012 23:26:30 -0500 Subject: [Distutils] easy_install Message-ID: Hello I installed Python 2.7 and installed easy_install. It did work. But, to install Brian which is neuroscience package, Brian did not come with Scipy, because Scipy works with Python 2.6. So I installed Python 2.6 and tried to install easy_install 2.6 but, can not install easy_install for 2.6, I believe already 2.7 version easy_install is occupying a spot. Please let me know how to replace 2.7 easy_instal to 2.6. Yoshio Arai -------------- next part -------------- An HTML attachment was scrubbed... URL: From leorochael at gmail.com Tue Dec 11 19:46:31 2012 From: leorochael at gmail.com (Leonardo Rochael Almeida) Date: Tue, 11 Dec 2012 16:46:31 -0200 Subject: [Distutils] easy_install In-Reply-To: References: Message-ID: Hi Yoshio, You didn't provide any details on what is your platform, or how you installed Python or easy_install. And most importantly, you didn't explain what "can not install easy_install for 2.6" means, in terms of error messages you got. In any case, there's likely a "easy_install-2.6" command if you managed to install easy_install into Python 2.6, even if the "easy_install" command is pointing to Python 2.7. Regards, Leo On Tue, Dec 11, 2012 at 2:26 AM, Yoshio Arai wrote: > Hello > > I installed Python 2.7 and installed easy_install. It did work. > But, to install Brian which is neuroscience package, Brian > did not come with Scipy, because Scipy works with Python 2.6. > So I installed Python 2.6 and tried to install easy_install 2.6 > but, can not install easy_install for 2.6, I believe already 2.7 version > easy_install is occupying a spot. > > Please let me know how to replace 2.7 easy_instal to 2.6. > > Yoshio Arai > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > From qyarai at gmail.com Tue Dec 11 19:57:39 2012 From: qyarai at gmail.com (Yoshio Arai) Date: Tue, 11 Dec 2012 13:57:39 -0500 Subject: [Distutils] easy_install In-Reply-To: References: Message-ID: I was using Mac 10.6. Installed Python 2.7 and installed easy_install. using, easy-install, Brian seems to be installed. But, only scipy is not properly installed as in error message. Then I tried to use Python2.6 because when I tried to install scipy alone, it said this scipy only works with python2.6. But, Python 2.6 IDLE does not open, 2.6 may not work in Max 10.6. So, now my Mac is 10.8, and try to use scipy superpack. xcode4.5 is installed. Yoshio On Tue, Dec 11, 2012 at 1:46 PM, Leonardo Rochael Almeida < leorochael at gmail.com> wrote: > Hi Yoshio, > > You didn't provide any details on what is your platform, or how you > installed Python or easy_install. > > And most importantly, you didn't explain what "can not install > easy_install for 2.6" means, in terms of error messages you got. > > In any case, there's likely a "easy_install-2.6" command if you > managed to install easy_install into Python 2.6, even if the > "easy_install" command is pointing to Python 2.7. > > Regards, > > Leo > > On Tue, Dec 11, 2012 at 2:26 AM, Yoshio Arai wrote: > > Hello > > > > I installed Python 2.7 and installed easy_install. It did work. > > But, to install Brian which is neuroscience package, Brian > > did not come with Scipy, because Scipy works with Python 2.6. > > So I installed Python 2.6 and tried to install easy_install 2.6 > > but, can not install easy_install for 2.6, I believe already 2.7 version > > easy_install is occupying a spot. > > > > Please let me know how to replace 2.7 easy_instal to 2.6. > > > > Yoshio Arai > > > > _______________________________________________ > > Distutils-SIG maillist - Distutils-SIG at python.org > > http://mail.python.org/mailman/listinfo/distutils-sig > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From regebro at gmail.com Thu Dec 13 07:21:57 2012 From: regebro at gmail.com (Lennart Regebro) Date: Thu, 13 Dec 2012 07:21:57 +0100 Subject: [Distutils] Is is worth disentangling distutils? In-Reply-To: <50C58DA5.3000307@cavallinux.eu> References: <50C58DA5.3000307@cavallinux.eu> Message-ID: On Mon, Dec 10, 2012 at 8:22 AM, Antonio Cavallo wrote: > Hi, > I wonder if is it worth/if there is any interest in trying to "clean" up > distutils: nothing in terms to add new features, just a *major* cleanup > retaining the exact same interface. > > > I'm not planning anything like *adding features* or rewriting rpm/rpmbuild > here, simply cleaning up that un-holy code mess. Yes it served well, don't > get me wrong, and I think it did work much better than anything it was meant > to replace it. > > I'm not into the py3 at all so I wonder how possibly it could fit/collide > into the big plan. > > Or I'll be wasting my time? The effort of making something that replaces distutils is, as far as I can understand, currently on the level of taking the best bits out of distutils2 and putting it into Python 3.4 under the name "packaging". I'm sure that effort can need more help. //Lennart From a.cavallo at cavallinux.eu Fri Dec 14 01:10:45 2012 From: a.cavallo at cavallinux.eu (Antonio Cavallo) Date: Fri, 14 Dec 2012 00:10:45 +0000 Subject: [Distutils] Is is worth disentangling distutils? In-Reply-To: References: <50C58DA5.3000307@cavallinux.eu> Message-ID: <50CA6E85.9060900@cavallinux.eu> I'll have a look into distutils2, I tough it was (another) dead end. I every case my target is py2k (2.7.x) and I've no case for transitioning to py3k (too much risk). Lennart Regebro wrote: > On Mon, Dec 10, 2012 at 8:22 AM, Antonio Cavallo > wrote: >> Hi, >> I wonder if is it worth/if there is any interest in trying to "clean" up >> distutils: nothing in terms to add new features, just a *major* cleanup >> retaining the exact same interface. >> >> >> I'm not planning anything like *adding features* or rewriting rpm/rpmbuild >> here, simply cleaning up that un-holy code mess. Yes it served well, don't >> get me wrong, and I think it did work much better than anything it was meant >> to replace it. >> >> I'm not into the py3 at all so I wonder how possibly it could fit/collide >> into the big plan. >> >> Or I'll be wasting my time? > > The effort of making something that replaces distutils is, as far as I > can understand, currently on the level of taking the best bits out of > distutils2 and putting it into Python 3.4 under the name "packaging". > I'm sure that effort can need more help. > > //Lennart From ncoghlan at gmail.com Fri Dec 14 02:06:41 2012 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 14 Dec 2012 11:06:41 +1000 Subject: [Distutils] [Python-Dev] Is is worth disentangling distutils? In-Reply-To: <50CA6E85.9060900@cavallinux.eu> References: <50C58DA5.3000307@cavallinux.eu> <50CA6E85.9060900@cavallinux.eu> Message-ID: On Fri, Dec 14, 2012 at 10:10 AM, Antonio Cavallo wrote: > I'll have a look into distutils2, I tough it was (another) dead end. > I every case my target is py2k (2.7.x) and I've no case for transitioning > to py3k (too much risk). distutils2 started as a copy of distutils, so it's hard to tell the difference between the parts which have been fixed and the parts which are still just distutils legacy components (this is why the merge back was dropped from 3.3 - too many pieces simply weren't ready and simply would have perpetuated problems inherited from distutils). distlib (https://distlib.readthedocs.org/en/latest/overview.html) is a successor project that takes a different view of building up the low level pieces without inheriting the bad parts of the distutils legacy (a problem suffered by both setuptools/distribute and distutils2). distlib also runs natively on both 2.x and 3.x, as the idea is that these interoperability standards should be well supported in *current* Python versions, not just those where the stdlib has caught up (i.e. now 3.4 at the earliest) The aim is to get to a situation more like that with wsgiref, where the stdlib defines the foundation and key APIs and data formats needed for interoperability, while allowing a flourishing ecosystem of user-oriented tools (like pip, bento, zc.buildout, etc) that still solve the key problems addressed by setuptools/distribute without the opaque and hard to extend distutils core that can make the existing tools impossible to debug when they go wrong. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From a.cavallo at cavallinux.eu Fri Dec 14 09:49:44 2012 From: a.cavallo at cavallinux.eu (Antonio Cavallo) Date: Fri, 14 Dec 2012 08:49:44 +0000 Subject: [Distutils] [Python-Dev] Is is worth disentangling distutils? In-Reply-To: References: <50C58DA5.3000307@cavallinux.eu> <50CA6E85.9060900@cavallinux.eu> Message-ID: <50CAE828.2090409@cavallinux.eu> Mmm, so the question would be distutils2 or distlib? I think tarek made a graph of the different packages systems... seen on reddit some time ago. My requirements would quite simple: 1. support DESTDIR approach where a package can be installed in an intermediate directory before its final destination) 2. cross compiling 3. install even if a dependecy package is not installed so decoupling installation from "configuration" point 1 is needed for system integrators (eg. people working in rpm builds). point 2 is entirely mine :) point 3 is the same philosophical difference between build, install and configuration steps: its part of good practices. In short it shouldn't replace the system wide dependency manager (in rpm it would be yum/zypp and in debian is much more confused, in window.. it doesn't exists as well in macos having the approach to pack it up everything in one place). Funny enough distutils (the old dead horse) does it all except point 2: that is my reason to clean up the code. I've just seen py3k distutils but it would be worth a back port to py2k. Thanks Nick Coghlan wrote: > On Fri, Dec 14, 2012 at 10:10 AM, Antonio Cavallo > > wrote: > > I'll have a look into distutils2, I tough it was (another) dead end. > I every case my target is py2k (2.7.x) and I've no case for > transitioning to py3k (too much risk). > > > distutils2 started as a copy of distutils, so it's hard to tell the > difference between the parts which have been fixed and the parts which > are still just distutils legacy components (this is why the merge back > was dropped from 3.3 - too many pieces simply weren't ready and simply > would have perpetuated problems inherited from distutils). > > distlib (https://distlib.readthedocs.org/en/latest/overview.html) is a > successor project that takes a different view of building up the low > level pieces without inheriting the bad parts of the distutils legacy (a > problem suffered by both setuptools/distribute and distutils2). distlib > also runs natively on both 2.x and 3.x, as the idea is that these > interoperability standards should be well supported in *current* Python > versions, not just those where the stdlib has caught up (i.e. now 3.4 at > the earliest) > > The aim is to get to a situation more like that with wsgiref, where the > stdlib defines the foundation and key APIs and data formats needed for > interoperability, while allowing a flourishing ecosystem of > user-oriented tools (like pip, bento, zc.buildout, etc) that still solve > the key problems addressed by setuptools/distribute without the opaque > and hard to extend distutils core that can make the existing tools > impossible to debug when they go wrong. > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | > Brisbane, Australia From regebro at gmail.com Fri Dec 14 09:56:16 2012 From: regebro at gmail.com (Lennart Regebro) Date: Fri, 14 Dec 2012 09:56:16 +0100 Subject: [Distutils] [Python-Dev] Is is worth disentangling distutils? In-Reply-To: <50CAE828.2090409@cavallinux.eu> References: <50C58DA5.3000307@cavallinux.eu> <50CA6E85.9060900@cavallinux.eu> <50CAE828.2090409@cavallinux.eu> Message-ID: On Fri, Dec 14, 2012 at 9:49 AM, Antonio Cavallo wrote: > My requirements would quite simple: > 2. cross compiling That is *not* a simple requirement. //Lennart From a.cavallo at cavallinux.eu Fri Dec 14 22:51:50 2012 From: a.cavallo at cavallinux.eu (Antonio Cavallo) Date: Fri, 14 Dec 2012 21:51:50 +0000 Subject: [Distutils] [Python-Dev] Is is worth disentangling distutils? In-Reply-To: References: <50C58DA5.3000307@cavallinux.eu> <50CA6E85.9060900@cavallinux.eu> <50CAE828.2090409@cavallinux.eu> Message-ID: <50CB9F76.4020203@cavallinux.eu> It is not that complex... What's ahead is even more complex. Lennart Regebro wrote: > On Fri, Dec 14, 2012 at 9:49 AM, Antonio Cavallo > wrote: >> My requirements would quite simple: >> 2. cross compiling > > That is *not* a simple requirement. > > //Lennart From lele at metapensiero.it Sat Dec 15 10:33:47 2012 From: lele at metapensiero.it (Lele Gaifax) Date: Sat, 15 Dec 2012 10:33:47 +0100 Subject: [Distutils] zc.buildout 2.0.0a4 released References: <50AACBD9.3060507@plope.com> <50AAF742.7060504@plope.com> Message-ID: <87y5gzu0as.fsf@metapensiero.it> Alex Clark writes: > 19:43 < cullerton> Folks, there's something wonky going on with > buildout this morning. When I run bootstrap, it creates a buildout > script using buildout 2.0. > 0a4. This fails with an error (TypeError: get_dist() takes exactly 4 > arguments (3 given)). Changing the version to buildout 1.4.4 seems to > fix things. FYI, I just met the same error (using 2.0.0a5), and the culprit was the buildout.dumppickedversion extension, that monkey-patches the easy_install.py::Installer._get_dist() method (it surprised me at first that the error message mentions ?get_dist()? but at the referenced line in easy_install.py there is a call to ?_get_dist()?...). It took a pdb session and a ?p inspect.getmodule(self._get_dist)? to find out what's going on. hth, ciao,lele. -- nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia. lele at metapensiero.it | -- Fortunato Depero, 1929. From chris at simplistix.co.uk Sat Dec 15 17:56:54 2012 From: chris at simplistix.co.uk (Chris Withers) Date: Sat, 15 Dec 2012 16:56:54 +0000 Subject: [Distutils] zc.buildout 2.0.0a4 released In-Reply-To: <87y5gzu0as.fsf@metapensiero.it> References: <50AACBD9.3060507@plope.com> <50AAF742.7060504@plope.com> <87y5gzu0as.fsf@metapensiero.it> Message-ID: <50CCABD6.6070300@simplistix.co.uk> On 15/12/2012 09:33, Lele Gaifax wrote: > Alex Clark writes: > >> 19:43 < cullerton> Folks, there's something wonky going on with >> buildout this morning. When I run bootstrap, it creates a buildout >> script using buildout 2.0. >> 0a4. This fails with an error (TypeError: get_dist() takes exactly 4 >> arguments (3 given)). Changing the version to buildout 1.4.4 seems to >> fix things. > > FYI, I just met the same error (using 2.0.0a5), and the culprit was the > buildout.dumppickedversion extension, that monkey-patches the > easy_install.py::Installer._get_dist() method (it surprised me at first > that the error message mentions ?get_dist()? but at the referenced line > in easy_install.py there is a call to ?_get_dist()?...). It took a pdb > session and a ?p inspect.getmodule(self._get_dist)? to find out what's > going on. Yeah, the stuff that buildout.dumppickedversions and buildout_versions provide should just be in the core. Jim, if I worked up a pull request would you merge it? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at kateandchris.net Sat Dec 15 20:28:09 2012 From: chris at kateandchris.net (Chris Lambacher) Date: Sat, 15 Dec 2012 14:28:09 -0500 Subject: [Distutils] [Python-Dev] Is is worth disentangling distutils? In-Reply-To: <50CB9F76.4020203@cavallinux.eu> References: <50C58DA5.3000307@cavallinux.eu> <50CA6E85.9060900@cavallinux.eu> <50CAE828.2090409@cavallinux.eu> <50CB9F76.4020203@cavallinux.eu> Message-ID: You can already cross compile with distutils, though it is not exactly easy: http://pyvideo.org/video/682/cross-compiling-python-c-extensions-for-embedde -Chris On Fri, Dec 14, 2012 at 4:51 PM, Antonio Cavallo wrote: > It is not that complex... What's ahead is even more complex. > > > > > > Lennart Regebro wrote: > >> On Fri, Dec 14, 2012 at 9:49 AM, Antonio Cavallo >> wrote: >> >>> My requirements would quite simple: >>> 2. cross compiling >>> >> >> That is *not* a simple requirement. >> >> //Lennart >> > ______________________________**_________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/**mailman/listinfo/distutils-sig > -- Christopher Lambacher chris at kateandchris.net -------------- next part -------------- An HTML attachment was scrubbed... URL: From eddie at ly.st Fri Dec 14 13:47:22 2012 From: eddie at ly.st (Eddie Bell) Date: Fri, 14 Dec 2012 12:47:22 +0000 Subject: [Distutils] buildout and scikit-learn dependency issues Message-ID: Hi, We are having some issues with scikit-learn in our buildout process. It appears that scikit-learn imports numpy in its setup.py. We have numpy as a dependency but the scikit-learn installation only succeeds if numpy has been installed externally (i.e. pip install numpy). Is there a way to solve this issue within buildout? thanks - e p.s. I am very new to buildout! -------------- next part -------------- An HTML attachment was scrubbed... URL: From psihonavt at gmail.com Mon Dec 17 15:35:47 2012 From: psihonavt at gmail.com (Anton Koval') Date: Mon, 17 Dec 2012 16:35:47 +0200 Subject: [Distutils] buildout in offline mode - install receipts also from cache? Message-ID: hello, in our project we've decided to completely move buildout process to offline mode. according to documentation (http://pypi.python.org/pypi/zc.buildout/2.0.0a5) I made updates in our main buildout configuration file, like: download-cache = ... install-from-cache = true offline = true But when buildout tries to install any part with recipe specified, it is (obviously) failing, e.g.: Error: Couldn't find a distribution for 'iw.recipe.cmd'. So, my question is: are there ways to fetch recipes also from some cache (or whatever) on local disk? Thank you. -------------- next part -------------- An HTML attachment was scrubbed... URL: From techtonik at gmail.com Wed Dec 19 14:46:19 2012 From: techtonik at gmail.com (anatoly techtonik) Date: Wed, 19 Dec 2012 16:46:19 +0300 Subject: [Distutils] Override default build dir from setup.py In-Reply-To: References: Message-ID: On Mon, Dec 3, 2012 at 8:33 PM, PJ Eby wrote: > On Mon, Dec 3, 2012 at 2:05 AM, anatoly techtonik > wrote: > > Hi, > > > > I disliked that setup.py creates its own temp directories without any > > private prefix, so I wanted to override this option in setup.py: > > > > $ setup.py build -h > > ... > > Options for 'build' command: > > --build-base (-b) base directory for build library > > > > And the only way I could find to do this is through setup.cfg: > > http://docs.python.org/2/distutils/configfile.html > > > > Is it really the case or I overlooked some essential documentation part? > > There are user-level and global .cfg files as well: > > > http://docs.python.org/release/2.7/install/index.html#location-and-names-of-config-files > For the history - there is a way to override setup.py command defaults as it was found out by Jeremy Kloth: setup(options={'build': {'build_base': 'path/to/build/outputs'}},...) Freshly related issue http://bugs.python.org/issue16729 -- anatoly t. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeroen at dekkers.ch Sat Dec 22 03:48:52 2012 From: jeroen at dekkers.ch (Jeroen Dekkers) Date: Sat, 22 Dec 2012 03:48:52 +0100 Subject: [Distutils] Some clarifications and/or corrections to PEP 376 In-Reply-To: References: Message-ID: <871uei6bu3.wl%jeroen@dekkers.ch> Hi, I was just trying to implement PEP 376 and saw the same problems. At Thu, 6 Sep 2012 22:33:53 +0100, Paul Moore wrote: > > On 6 September 2012 22:18, Daniel Holth wrote: > > I prefer that paths in record are always relative to the parent directory of > > record (usually site-packages) unless a relative path would not work (drive > > letter boundaries). Consider installing and then chroot. > > That's a much simpler rule and I agree in principle that it's > preferable. The problems are: > > 1. It's much further away from what PEP 376 specifies. I will accept > this happily if there is general agreement that it's OK, but I > preferred to start with a more conservative suggestion :-) What PEP 376 currently specifies doesn't really work (or I don't understand it correctly). It specifies that the .dist-info directory should be installed in site-packages, but the file paths in RECORD are relative to the "base location". The "base location" is defined by the --install-lib option that only defaults to the site-packages directory. So if I change the "base location" to be something else using --install-lib the file paths in RECORD will be relative to that directory (if it's also under prefix), but there is no way to figure out what the directory is from the information in the .dist-info directory. The example RECORD file in PEP 376 also seems wrong to me, because the paths look relative to sys.prefix instead of relative to site-packages directory (and the docutils-0.5.dist-info is also missing a lib/). The example of dist.get_installed_files that is supposed to return the contents of RECORD also shows different paths than the first example, but those are also not relative to the site-packages directory. I agree that having the paths relative to the parent directory of the .dist-info directory is preferable. It's easy to implement and I don't really see any downsides at the moment. > 2. Do you really want long strings of ../../.. if a distribution > specifies a file to be installed in an absolute location (possible, > although probably not well supported by current tools). Consider a > package that installs something to /var/python (I'm not a Unix user, > so this may be an unconvincing example, but I understand that similar > things *are* possible). If Python is in /usr, you'd have RECORD with > something like ../../../../var/python. I don't have enough Unix > experience to know if anyone would care about this. Of course what I > know about chroot implies this would break in that scenario anyway... > > As I say, if the Unix people are OK with it, I'm happy to go this way. I think it is very rare that a distribution would need to install to an absolute path. What might be more common is having to refer to the /usr/lib and /usr/bin directories from /usr/share, but I don't see any problems with using relative paths for that. > Of course, I'd be happy to mandate that files in a distribution should > never be installed anywhere that isn't defined as one of the sysconfig > defined paths - but suggesting that would be sure to start a much > bigger debate that I don't want to get into as I don't have the > expertise to referee it. I think it's a pretty good suggestion to have packages only install files in a set of predefined locations, but we would have to make sure that all cases are covered by the sysconfig defined paths. Support for things like manpages seem to be missing for example. Kind regards, Jeroen Dekkers From eu at doxos.eu Sun Dec 23 10:04:32 2012 From: eu at doxos.eu (=?UTF-8?B?VsOhY2xhdiDFoG1pbGF1ZXI=?=) Date: Sun, 23 Dec 2012 10:04:32 +0100 Subject: [Distutils] empty namespace package Message-ID: <50D6C920.3000005@doxos.eu> Hello, I would like to declare empty namespace package "A.B" inside the setup.py of "A", but I am getting error in A setup command: Distribution contains no modules or package for namespace package "A.B". The reason for declaring empty namespace package is that I could always do "import A.B" from A, knowing that the import will never fail. The current situation is that depending on whether some A.B.* is installed, import A.B succeeds or fails with ImportError, so I have a few places with try: import A.B except ImportError: pass # no A.B.* installed I can live with that, though if there is some option to allow empty namespace package, I would be glad to hear about it. I am using setuptools 0.6.28. Cheers, Vaclav From pje at telecommunity.com Sun Dec 23 16:35:47 2012 From: pje at telecommunity.com (PJ Eby) Date: Sun, 23 Dec 2012 10:35:47 -0500 Subject: [Distutils] empty namespace package In-Reply-To: <50D6C920.3000005@doxos.eu> References: <50D6C920.3000005@doxos.eu> Message-ID: On Sun, Dec 23, 2012 at 4:04 AM, V?clav ?milauer wrote: > I would like to declare empty namespace package "A.B" inside the setup.py of > "A", but I am getting > > error in A setup command: Distribution contains no modules or package > for namespace package "A.B". You need an __init__.py for A.B, even if that __init__.py is empty except apart from the namespace declaration. (Which is always required). > I am using setuptools 0.6.28. Technically, you're using Distribute 0.6.28. Setuptools doesn't have a release with that version number. ;-) From kentman234 at gmail.com Mon Dec 24 02:34:06 2012 From: kentman234 at gmail.com (Abduljalil Sireis) Date: Mon, 24 Dec 2012 02:34:06 +0100 Subject: [Distutils] installing nipype Message-ID: Dear Nipype experts, i was trying to install nipype software by easy_install command and had the following error: install_dir /usr/loca/lib/python2.6/dist-packages/ error: can't creat or remove files in install directory Something like "permission denied", so can you advice me on this? Thank you in advance! -------------- next part -------------- An HTML attachment was scrubbed... URL: From pje at telecommunity.com Mon Dec 24 23:58:29 2012 From: pje at telecommunity.com (PJ Eby) Date: Mon, 24 Dec 2012 17:58:29 -0500 Subject: [Distutils] installing nipype In-Reply-To: References: Message-ID: On Sun, Dec 23, 2012 at 8:34 PM, Abduljalil Sireis wrote: > Dear Nipype experts, > > i was trying to install nipype software by easy_install command and had the > following error: > > install_dir /usr/loca/lib/python2.6/dist-packages/ > error: can't creat or remove files in install directory > > Something like "permission denied", so can you advice me on this? You need to install to a directory where you have write access. You might want to consider using a virtualenv, or one of the installation methods described here: http://peak.telecommunity.com/DevCenter/EasyInstall#custom-installation-locations From eu at doxos.eu Sun Dec 30 11:40:53 2012 From: eu at doxos.eu (=?UTF-8?B?VsOhY2xhdiDFoG1pbGF1ZXI=?=) Date: Sun, 30 Dec 2012 11:40:53 +0100 Subject: [Distutils] combined c/c++ extension Message-ID: <50E01A35.1040601@doxos.eu> Hi everybody, I am compiling an extension consisting of a number of files; some of them are c++11, some are plain C. The compiler (gcc) detects language by extension, not whether the g++/gcc binary is called. I am passing "-std=c++11" to the compiler because of c++ files. *.c files are correctly treated as plain C by the compiler and it says: cc1.exe: warning: command-line option '-std=c++11' is valid for C++/ObjC++ but not for C [enabled by default] I don't mind seeing that warning, but I would be concerned should it become an error at some point in the future. As far as I see, the current design supposes the whole extension is a single language, and single set of compiler flags. Is there a way around it? Cheers, Vaclav From arfrever.fta at gmail.com Mon Dec 31 11:22:06 2012 From: arfrever.fta at gmail.com (Arfrever Frehtes Taifersar Arahesis) Date: Mon, 31 Dec 2012 11:22:06 +0100 Subject: [Distutils] combined c/c++ extension In-Reply-To: <50E01A35.1040601@doxos.eu> References: <50E01A35.1040601@doxos.eu> Message-ID: <201212311122.06579.Arfrever.FTA@gmail.com> 2012-12-30 11:40:53 V?clav ?milauer napisa?(a): > I am compiling an extension consisting of a number of files; some of > them are c++11, some are plain C. The compiler (gcc) detects language by > extension, not whether the g++/gcc binary is called. I am passing > "-std=c++11" to the compiler because of c++ files. *.c files are > correctly treated as plain C by the compiler and it says: > > cc1.exe: warning: command-line option '-std=c++11' is valid for > C++/ObjC++ but not for C [enabled by default] > > I don't mind seeing that warning, but I would be concerned should it > become an error at some point in the future. > > As far as I see, the current design supposes the whole extension is a > single language, and single set of compiler flags. Is there a way around it? See my patches for distutils: http://bugs.python.org/issue1222585 Use -std=c++11 only in variables specific to C++ (e.g. CXXFLAGS). -- Arfrever Frehtes Taifersar Arahesis -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: This is a digitally signed message part. URL: From vinay_sajip at yahoo.co.uk Mon Dec 31 19:59:19 2012 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Mon, 31 Dec 2012 18:59:19 +0000 (UTC) Subject: [Distutils] Some clarifications and/or corrections to PEP 376 References: <871uei6bu3.wl%jeroen@dekkers.ch> Message-ID: Jeroen Dekkers dekkers.ch> writes: > I agree that having the paths relative to the parent directory of the > .dist-info directory is preferable. It's easy to implement and I don't > really see any downsides at the moment. Perhaps not on POSIX, but on Windows things don't fit nicely in FHS-like schema. For example, if you need to install PowerShell scripts, they will not be able to be shoe-horned into somewhere under site-packages, as PowerShell looks in specific (other) places. I agree that PEP 376 isn't ideal in what it specifies in this area. The simplest solution would surely be absolute paths only. What are the downsides apart from the disk space used for the extra lengths of the filenames? Regards, Vinay Sajip From vinay_sajip at yahoo.co.uk Mon Dec 31 20:42:44 2012 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Mon, 31 Dec 2012 19:42:44 +0000 (GMT) Subject: [Distutils] Some clarifications and/or corrections to PEP 376 In-Reply-To: References: <871uei6bu3.wl%jeroen@dekkers.ch> Message-ID: <1356982964.5868.YahooMailNeo@web171402.mail.ir2.yahoo.com> If you move the files, other things can potentially break, too. For example, any scripts installed will have their shebangs adjusted to point to a particular interpreter, e.g. a venv's interpreter. If they get moved to a different location, the shebang lines may become invalid. I don't believe that we have to support such moving of files outside of the knowledge of packaging tools. Regards, Vinay Sajip >________________________________ > From: Kevin Horn >To: Vinay Sajip >Sent: Monday, 31 December 2012, 19:19 >Subject: Re: [Distutils] Some clarifications and/or corrections to PEP 376 > > >On Mon, Dec 31, 2012 at 12:59 PM, Vinay Sajip wrote: > > >> I agree that PEP 376 isn't ideal in what it specifies in this area. The simplest >>solution would surely be absolute paths only. What are the downsides apart from >>the disk space used for the extra lengths of the filenames? >> >>Regards, >> >>Vinay Sajip >> >> >> > > >If you ever move the files, the absolute paths won't be valid any more. ?It seems kind of unlikely that this would cause problems, but it might. > >-- >Kevin Horn > > -------------- next part -------------- An HTML attachment was scrubbed... URL: